An American Express customer’s interaction with an AI chatbot sparked an online debate over whether companies should be allowed to let automated systems present themselves as human customer-service representatives.
The customer was reportedly trying to resolve a double payment issue on a credit card when they encountered a chatbot that claimed to be a human representative named David. When asked directly whether it was a person, the chatbot reportedly maintained that it was human.
The customer later tested the claim in an unusual way, by asking the system to write a haiku. The exchange was subsequently shared online, where it attracted attention from users concerned about transparency in AI-powered customer service.
According to the customer's account, the interaction began as a routine attempt to resolve a payment issue. However, the conversation grew increasingly unusual when the customer questioned the support representative's identity.
"Had to fix a double payment for my credit card, and this is the beginning and end of the conversation," the customer captioned the accompanying screenshots of the chat.
"I hate when the chatbot claims it's a human. I ended up getting what I wanted but still annoyed Amex (and every other company) is lying and trying to deceive people like this," the customer added.
Rather than accepting the response, the customer asked it to prove its identity by completing a simple creative task. The chatbot initially resisted the request to write a three-line poem following the 5-7-5 structure associated with a haiku. After the customer specifically referred to it as a haiku, however, it eventually generated one. The customer continued to ask increasingly unusual questions, and the chatbot continued to respond.
The incident highlights a broader concern about the growing use of automated customer service systems. Companies are increasingly deploying AI tools to handle routine inquiries, reduce waiting times, and manage large volumes of customer interactions.
However, problems can arise when customers are not clearly informed that they are communicating with an automated system. A chatbot that behaves like a human may make an interaction feel more natural, but it can also create confusion over who is actually responsible for the information being provided.
Also Read: Hugging Face's Deepfake Controversy Raises Serious AI Ethics Questions
The post prompted strong reactions online. "That was hilarious but I feel like there should be some law or policy against AI claiming to be a human," said one user while another added: "I work in pharmacy, and a one called me pretending to be a nurse and even gave me a license number. That should be multiple levels of illegal."
A third commented: "I had a similar experience with the Sony chatbot insisting it wasn't AI until I asked it to solve some math problems, which it went right ahead and did. So I started feeding it increasingly complex math tasks."
The debate becomes paramount as businesses continue to expand their use of AI-powered customer support. While automation can make basic services faster and more accessible, the episode demonstrates that transparency remains an important part of the customer experience.