Using ChatGPT for Customer Service

If you’ve landed here looking for a quick answer to the question “Can ChatGPT replace my human customer service team?” then here is the first thing you need to know, directly from ChatGPT’s makers:

“ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.”

So… no. If you want to give your customers accurate, consistently good service experiences, then you can’t have ChatGPT answer them directly without human intervention. Not yet, at least. 

Can ChatGPT (and other AI tools based on large language models) be used to improve your customer service? 

Absolutely, yes.

In fact, they already are. So a more useful question might be, “How can ChatGPT help my team deliver better customer service?” Let’s take a look.

Author’s note: This is not one of those articles where, right at the end, I reveal that ChatGPT generated the entire text for me. I wrote this myself, and any errors are therefore the fault of my meat-based brain and the pale and aging body that carries it around. 

What is ChatGPT?

A while back we wrote about GPT–3, a language model that we tested with real customer service queries, with… mixed results. ChatGPT is built specifically for conversational interaction, using a new language model based on GPT–3.5 (for the free version) and GPT-4 if you pay for ChatGPT Plus.

ChatGPT was fine-tuned using human AI trainers submitting conversations where they “played both sides—the user and an AI assistant.” which feels a bit like being fired and then forced to train your own cheaper replacement before you leave. GPT’s ability to create new outputs based on your questions combined with a vast training set of data is known as “generative AI”, and the results can be genuinely impressive. 

You can ask a question, receive a natural-sounding answer, and then hold a follow-up conversation referring back to earlier information asking for re-phrasings, clarification, summaries, or additional details. 

At its best, ChatGPT can save you time, enhance your knowledge, and provide inspiration for new approaches. When it works, it feels like a sentient sci-fi computer — one of the good ones, not an evil one. Hopefully. 

That is how it feels. That is not what is happening.

What ChatGPT is not

ChatGPT is not a knowledge base or an encyclopedia. It will always sound confident about its answers, even when they are utterly wrong, and it cannot differentiate between “facts” and made up information. That’s why Stack Overflow temporarily banned the use of ChatGPT-generated answers.

We have all been at a party, trapped in a conversation with an overconfident person (let’s be real, it’s almost always a man) who may have qualifications in one field but who holds equally strong opinions on every other topic, all presented as “facts.”

When you know enough about the topic to identify their error, it is irritating. But it’s worse if you can’t spot the error and are left with an understanding that is misleading or outright wrong. Those are not the staff you want on your customer service team, human or AI. 

ChatGPT, the hallucinatory customer service agent

I tested ChatGPT with questions we commonly receive from new Help Scout customers. Note that ChatGPT has no special training in Help Scout and no access to internal information, so it is working from publicly accessible data like our website and knowledge base.

Here is the short version: It worked really well — sometimes. For example, I asked how to get my email from Outlook into Help Scout. The correct answer is to set up a redirect rule in Outlook to send emails to your Help Scout mailbox address. 

ChatGPT is sensitive to quite small changes in questions, and in three attempts asking only slightly varied questions, ChatGPT gave me:

1. The correct instructions for setting up a rule in Outlook.

2. The correct instructions again, but also a confusing and irrelevant reference to using Zapier.

3. Completely incorrect instructions about using IMAP to send outgoing email via Outlook.

Even using the exact same prompt sometimes gave me the correct answer and sometimes an incorrect one. That was enough to confirm that OpenAI’s warning, quoted at the top of this article, was accurate: ChatGPT cannot be relied on for consistently accurate answers to specific questions. 

ChatGPT, during my testing, did better with more carefully crafted questions, but every customer service pro knows that a huge part of their job is translating vague and confusing questions into the actual problem that needs to be solved.

Of course, people are not always 100% reliable either. Every manager knows that part of their role as a people leader is to put their reports into a position where they can be successful. 

Conversational, generative AI tools like ChatGPT can be very helpful tools. Understanding their limits and knowing where best to deploy them is what matters most.  

How ChatGPT and AI can improve your customer service

How might artificial intelligence software and large language models be used behind the scenes to create better customer experiences? 

1. Provide potential answers for human review

We know AI can often predict exactly the right answer, and other times it can offer something that is close to what is needed. Offering up the most likely answers to a customer service person could save a lot of time, but getting to that point would take some real work. ChatGPT itself does not currently allow users to tune the model to their specific content, but you can do so with GPT-3 (though not GPT 4 yet) and other tools.  

There are clear privacy and security concerns to consider before allowing a third party access to the customer and internal data that would be needed. Understanding and controlling the data used to train large language models is a huge concern amongst artists today, and text-based inputs will be no different. 

Even after training, quality outcomes will require team members who can tell the difference between “that sounds right” and “that is actually right.” In the short term, AI tools that offer predictive text to customer service agents (similar to that built into Gmail) may provide some benefit without the need for customization.

2. Speed up onboarding for new team members

Getting started on a customer service team can be hard work. There are so many places to look for answers and so much background context to learn. 

An AI tool could shortcut a lot of searching by suggesting the most useful internal documents and next steps. It may also reduce the burden on the rest of the existing team members who would otherwise be answering questions for new staff. 

In a study of more than 5,000 customer service agents published in April of 2023, the authors introduced a generative AI (conceptually similar to ChatGPT). They found that having access to AI assistance: “increases productivity, as measured by issues resolved per hour, by 14 percent on average, with the greatest impact on novice and low- skilled workers, and minimal impact on experienced and highly skilled workers.”

In this study, AI functioned as a sort of work buddy, sitting alongside a newer team member and showing them the most likely answer in any given situation. In most situations, onboarding and training from a human expert is time limited. Having AI help is more scalable and can amplify and extend the training period.

3. Summarize long discussions

Large language models are really good at summarizing written language. Particularly in complex support conversations, an automatic summary could reduce the time for a human team member to understand the context and keep the conversation moving toward a solution.

4. Categorize and prioritize

Customer service teams spend a lot of time trying to organize incoming questions, sort them into relevant groups, and understand where problems are being created.

AI tools can save a ton of time by learning from past categorizations and suggesting new and more consistent taxonomies. For most companies, the scale of incoming conversations means people can only really review a small portion of the data. AI could review everything, and therefore it may detect patterns and groupings that are not easily visible to people. 

5. Monitor quality

As customer service volume increases, keeping an eye on quality and consistency is challenging. AI tools are already effective in sentiment analysis and could be used to identify the conversations that need to be reviewed and even suggest problems or opportunities to be addressed based on other answers.

6. Proactively offer help

A quick support response time is excellent, but even better is not having to ask for help at all. Applying AI to determine when a customer is stuck (or about to be stuck) and offering the most relevant help article could really improve customer experiences at a scale that’s harder to do manually. 

The future of AI in customer service

“How can AI be used in customer service?” is a perpetual question with an answer that seems to change constantly. The short answer, in the long term, is “It will be used everywhere.”

What is true today about the capabilities (or lack of them) will probably not be true next year. ChatGPT might not be able to accurately provide direct customer support any time soon, but the AI-augmenting-humans model is here to stay.

When considering your own usage of AI, keep these points in mind.

1. What is the value your support offers to your customers? 

ChatGPT is never stumped; it will always generate an answer. But customer service agents know that the best first response is often a follow-up question to clarify the customer’s need or a request for more specific details. 

You offer support because you want your customers to succeed in using your products and services to reach their goals. The more clearly you understand those goals, the more easily you will be able to identify where and how AI will apply best in your situation.

2. Natural conversations make mistakes harder to find

A lot of what makes ChatGPT impressive is how natural it sounds, at least at first glance. A factual error is easier to spot in a bullet list than it is when buried inside three paragraphs of chatty conversation.

The closer we get to human-sounding AI, the more likely it is that mistakes won’t be noticed until it is too late. The first opportunities for AI in customer service are more likely to be AI that makes suggestions for a human to review and AI that measures your quality and effectiveness of support.

3. Trust really matters

When everything is going well, an AI providing the answer is fine. But when things go wrong and mistakes are made, customers will want accountability and assistance. AI does not care about mistakes, and it can’t feel responsible. It can’t feel anything.

Your customers will always want to know there is a person who cares about their experience and wants to make it better. How and where will you apply that most valuable human interaction time with your customers when AI has taken over some of the more repeatable parts of the job?

The thoughtful application of technology

ChatGPT is not going to replace most human customer service staff, though that will not stop some people from trying to make it happen. Continued improvement in language models and the interfaces to them will inevitably become part of the customer service experience for most businesses. 

The companies that succeed with these tools will be those that find ways to use the technology to amplify the things their staff can do best and to add capabilities that people just cannot do (or cannot do at scale).

The more you understand exactly what your customers value, the better decisions you can make about when and where to apply technology in providing customer service.