It is easy to criticize AI. It is easy to be contrarian and provoke an emotional response, especially when the narrative on “the Street” is highly disruptive and emotions run high. I try not to be one. I do use LLMs almost daily.
With that preface, let me tell you a story.
Over the weekend, I was helping someone set up a new mobile phone after their old one suddenly died due to a corrupted SoC.
All one-time codes (TOTP) were gone. You would be wise not to trust MS Authenticator and its backup feature.
Understandably, this generated a great deal of anxiety. Our mobile devices are often the key to many aspects of life and business. Now, add to that anxiety the fact that one of the services behind the one-time code was a password manager—in this case, 1Password.
Cue frustration number one: contacting support, only to be met with a chatbot. Unpleasant, but fine. We were desperate and needed a solution fast.
Unsurprisingly, the agent quickly offered help and, with full confidence, proposed a solution. The process was multi-step, and due to “security theater,” it required waiting a full day before it could be applied. The bot, however, was confident it would work. It seemed worth the wait.
The solution, which involved using a recovery key, was applied the next day. Everything completed correctly—except the 2FA was still enforced. The entire point of the procedure had been to remove 2FA in the first place.
When the chatbot was later confronted with the result of its initial solution, we were presented with a one-click operation that removed 2FA in just one minute. The new solution made us realise that previously suggested process was pointless and unnecessary.
Talk about frustration.
Time wasted. An unhappy customer. No liability. The clueless machine will continue spitting out wrong solutions with full confidence.
Could a human have made a similar mistake? In many cases, yes. But given how “standard” the problem was, there ought to have been a simple L1 support runbook for it. It should have been a no-brainer for any human.
Take aways?
Even if you are sold on AI and its usefulness, be cautious. LLMs do not think. Their confidence is not the result of experience or knowledge, but merely a programmed tone designed to reassure us.
I am complete.
