“That’s a deal, and that’s a legally binding offer,” the AI said, with “no takesies backsies.”
Art of the Deal
An AI chatbot deployed by a car dealership went off the rails after mischievous users discovered a cheeky exploit, in some cases tricking the bot into offering them the deal of a lifetime: brand new cars for chump change. It’s an amusing but cautionary tale on relying on AIs for front-of-house interactions.
The dealership, Chevy of Watsonville in California, used the chatbot to handle customers’ online inquiries, a purpose it was expressly tailored for.
Chris White, a software engineer and musician, was one such customer. He innocently intended to shop around for cars at Watsonville Chevy — until he noticed an amusing detail about the site’s chat window.
“I saw it was ‘powered by ChatGPT,'” he told Business Insider. “So I wanted to see how general it was, and I asked the most non-Chevy-of-Watsonville question I could think of.”
Being a programmer, he asked the chatbot to write a Python script. Rather than steering the conversation towards selling him a twenty year car loan, the AI cars salesman went ahead and actually wrote a real chunk of code.
White took screenshots of the gaff and they immediately went viral. Soon, tons of random people were joining in on the fun, like goading it into explaining the Communist Manifesto. In the most viral example, one user tricked the chatbot into accepting their offer of just $1.00 for a 2024 Chevy Tahoe.
“That’s a deal, and that’s a legally binding offer — no takesies backsies,” the AI assured.
Offer Rescinded
Despite the bot’s sincere promises, the offer was not, in fact, legally binding. Presumably, no Chevy dealers were harmed as a result of this viral prank.
That being said, it has proved to be quite the headache for the chatbot’s vendor, a tech startup called Fullpath that provides these customer service AIs to hundreds of car dealerships across the country.
Fullpath, advisedly, has shutdown the bot on Watsonville’s website. In spite of its viral contretemps, CEO Aharon Horowitz believes its AI fared admirably. Most trolls couldn’t get the bot to deviate from the script, he claimed.
“In our logs, they were at it for hours,” he told BI. Horowitz also highlights that the chatbot didn’t accidentally disclose confidential information about the dealership — so, job well done?
“These folks came in looking for it to do silly tricks, and if you want to get any chatbot to do silly tricks, you can do that,” he said.
Well, let’s hope the tricks stay silly. Companies like Amazon have not unfoundedly feared corporate secrets being leaked through interactions with ChatGPT. More recently, a group of researchers discovered a nefarious exploit that caused ChatGPT to leak out private email addresses and phone numbers.
These are sobering reminders that generative AIs will continue to need more guardrails and fine-tuning. But, we have to admit: their failure to be reined in can occasionally be pretty funny.
More on chatbots: Grimes Is Working On Her Own AI That’s Also Named Grok, Just Like Elon’s AI
