With AI help, issues resolved rose 14%, and 34% for the least experienced

What the study found

The study was published as a working paper by the National Bureau of Economic Research (NBER). It covered 5,179 customer support agents at a software company. A generative AI assistant for conversations was rolled out at different times, so the researchers could compare agents who had it with agents who did not.

The assistant reads the chat with the customer and suggests replies and relevant internal documents on the spot. The agent decides whether to use them, and the agent remains responsible for the conversation.

In this study, the number of issues resolved per hour rose by 14% on average. For less experienced and lower-skilled agents, the rise was 34%. The effect on experienced and highly skilled agents was small.

The tone of customers’ messages improved, and fewer agents left. The researchers believe the AI spread the practices of the best agents, helping new people get up to speed faster.

Agents who followed the AI’s suggestions more closely improved more. Even during periods when a system outage kept the AI from making suggestions, agents performed better than before it was introduced. They may have been learning from the AI as they used it.

For small teams and sole proprietors

At a small company, new people take time to learn the products and the rules. Meanwhile, customer enquiries tend to pile up on whoever has the most experience. At some companies, the owner is still writing every reply.

In this study, the biggest gains went to the least experienced. If reply suggestions come from the company’s own documents and past answers, new people can learn on the job. They also need to ask colleagues less often.

The same goes for online stores. Questions about shipping, stock and returns tend to bunch up in sales and busy seasons. If someone brought in to help can answer almost like an old hand straight away, the load in busy periods is easier to share.

Sole proprietors and freelancers can think about it the same way. Quote requests, delivery questions and replies to common questions take up much of the day when one person handles them all. Hand the reply drafts to an AI agent, check them yourself before they go, and you have more time for work that needs your judgement.

The finding that customers responded better matters to small companies too. Each customer’s experience easily leads to the next order or a referral.

For mid-sized and larger companies

Large support departments see a constant flow of new hires and transfers. If less experienced people reach a solid standard quickly, the whole department answers more consistently.

In this study, the main reason turnover fell was that new agents became less likely to quit. Given the cost of hiring and training again and again, that is a result worth noting.

Customers also asked less often to speak to a manager. When team leads spend less time taking over escalations, they can spend it on training and improvement.

Training changes too. Instead of long classroom sessions before going live, new people can learn on real enquiries with AI support, while leads check the difficult cases.

On the other hand, the effect on highly experienced agents was small, and for the most skilled the study suggested conversation quality might even fall. Rather than handing everyone the same tool, it pays to think about who needs what kind of support.

The AI was also built to make no suggestion where it lacked training data, leaving the agent to answer alone. Deciding up front what the AI handles and what people take on is essential.

How we use this

First, we organize your company’s knowledge. We gather help articles, internal rules, past answers and product data, and use agentic RAG and knowledge graphs to ground answers in them. The AI agent answers from those materials.

We decide where people make the call. Complaints, refunds and enquiries that need judgement go to a staff member with a summary of the history. Important replies pass through an approval gate (a person always approves important actions) before they are sent.

It also helps new people. Use the same knowledge internally, and new team members have somewhere to check first. Answers your staff correct are added to the shared knowledge and used next time.

We protect customer information. Enquiries are handled in a private environment (data is never used for training). While it runs, our AI agent Polaris (support and monitoring) keeps an eye on how things are going.

We start small and check the numbers. We begin with one channel and the most common enquiries. With the results dashboard (time saved, cost per task and accuracy, checked weekly), we look at the change together. There is also a 20-day money-back guarantee after launch (terms apply).