• Nearly one-third dissatisfied with chatbots overall, impacting customer satisfaction
  

• More than 8 in 10 respondents expressed concern about AI bias in chatbots

A study by Applause, a world leader in testing and digital quality, reveals that nearly one-third (30%) of users of chatbots, interactive voice response (IVR) services and conversational assistants are dissatisfied with their experiences. In addition, the study has revealed concerns about inherent bias that can negatively influence engagements and outcomes with AI-driven voice applications.

The results are drawn from Applause’s second annual AI and Voice Applications Survey to gain insights into AI user experiences. Conducted in February and March of 2023, the survey collected responses from nearly 5,200 digital quality testing professionals from around the globe.

ChatGPT is proving to be popular among users, with 74% responding positively about their experience with the recently released generative AI application. When asked if ChatGPT had understood their questions and provided them with helpful answers, more than two-thirds (67%) responded with “yes”. When asked whether ChatGPT or similar technology would help them to complete their work-related tasks, the vast majority responded that it would be “very” (43%) or “somewhat” (37%) helpful.

Conversely, when asked about interactions with chatbots in general, up to 30% said they were dissatisfied with the experience – referring to wrong answers (29%) and a lack of understanding (24%) as reasons for dissatisfaction. Respondents were also frustrated by the failure of chatbots to appreciate language variations or the phrasing of questions. However, one-third indicated that they would use chatbots more if they responded more accurately to questions and voice commands.

Marketing Technology News: MarTech Interview with Doug Huntington, CEO at FatTail

Despite these complaints, 92% of respondents expect companies to have chatbots on their apps or websites, eliminating the need for a call. A similar number (86%) said they expect to be greeted by an IVR system when they call a business. Nearly half of respondents (44%) expect mobile apps to have voice assistants.

The study also looked at sentiment around the inherent biases that can affect interactions with AI powered chatbots and smart voice assistants. Bias occurs when the underlying algorithm has been trained with poor and limited data. This impacts its decision making and leads to negative or unanticipated experiences. When questioned about bias in generative AI technology like ChatGPT, an overwhelming 86% of survey respondents expressed their concern.

“As AI technology becomes ubiquitous and we rely on it more heavily for mission-critical business functions, it is essential that we focus on quality – both from a data collection standpoint, and from a training and testing perspective. For a second year in-a-row, a vast number of survey respondents shared their frustration with current chatbot interactions. Organizations must focus on bringing real world, unbiased insights into their development processes to drive improvements to their AI experiences and meet their customers’ rising expectations,” said Luke Damian, Chief Growth Officer for Applause.

The AI and Voice Applications Survey is the first of a series of digital quality surveys Applause will conduct this year. In May 2023, the company plans to release its second annual State of Digital Quality Report which analyzes a representative sample of its testing data and reports on the most common flaws in digital experiences in several industries including retail, finance, media and telecommunications, and travel and hospitality.

Marketing Technology News: The Movable Middle: Embracing the Discovery Phase