Applying AI to labor-intensive Customer Service in China

6 min readAug 18, 2020


This is an article written by Customer Service expert Wei Chi based in Beijing and authorized to be republished by Chinapotion.

The application of AI in customer service has experienced explosive growth in past few years.

Since 2017, my previous company has begun to lay out the innovative technology in our BPO (Business Process Outsourcing) business. It is a great honor to participate those excited projects. During this period, I learned a lot and took my lessons. Today, I want to write down what happened and my personal opinion for this excited journey.

First of all, as a BPO company with call center business, our core competence is running the operations with the best staff so we purchased most of the the technologies we use from external suppliers. After I cooperated with some “high-tech” companies in China, I found that only BAT (Baidu, Alibaba and Tencent) and iFlytek really own/have the core technology,while others use the technology provided by BAT and iFlytek to make a specific application in accordance with the specific needs of customers and businesses. Why not go to BAT or iFlytek directly? It’s not worth to purchase the whole package if you are running small/medium size business.

Over the time of three years, we tried OCR recognition (invoice counterfeit checking), outbound robots, chat robots, intelligent QC, intelligent assistants, omni-channel communication, and questionnaire analysis (modelling).

OCR applications have been widely used in converting image of documents from IDs to contracts into automatic data entry.

1: OCR recognition (invoice counterfeit checking)

Application scenario:

Recognize the information from the invoice quickly by batches, convert the pictures into words and save them in word/excel.

Problems encountered in use:

When a customer takes a photo of an invoice, it is easy to have the shadow of the hand or cellphone; other pictures in the background and stamped on invoice which covered some numbers. These reasons caused the OCR accuracy to be only about 82% And this 82% does not mean that 82% of the photos can be recognized accurately, but that average 82% of information from one invoice are correct. This problem leads to the manually check and modify the OCR result after recognition. It does not meet the expectations of customers and us.

Workaround: End user can take a photo as required and let him/her judge whether the information is accurate before upload. It’s easier/free to check one picture by end user than many by BPO company.

Outbound Chatbots are supposed to reduce high cost of human labor by dialling phone numbers on a list.

2: Outbound Chatbot 外呼机器人

Application scenario: the customer has not been contacted for long time (more than half year) or data is not very pleasing (collection)

Problems encountered in use:

The questionnaire cannot be configured by users (BPO or end users). Included but not limited to establishment, update, modification, and reversal of the logic of the questionnaire, all those functions have to be done by supplier, It takes a long time and is inefficient;

The voice-to-text is only about 80% accurate in call center environment and If the user uses hands-free with accent, the accuracy rate is about 50%, because the text is not accurate, NLP is useless. Since the NLP is useless, the most common solution is search by keywords, which is not “smart” at all and it caused customer dissatisfaction;

The dialogue voice is robotic, although it can be recorded with a real person, but when there are variables, (Mr. XXX, or Ms. XXX, or phone number), even for the best supplier in market, you can clearly feel that it is splicing by recordings;

if it supports “interruption”, once there is external noise, it will go to KBS to search the keyword and 99% of time will end with “no keyword” script. If interruption function is not supported, the robot will Ignore any “unplanned” question which asked by customer;

once the questionnaire is long, there are many jumping logics (with this answer questionnaire should go to which question) , and it guarantee will cause error or mistake; national regulations, robots are easy to get complaints and shut down.

Workaround: Collection, don’t care about customer satisfaction at all. Just wants to inform customer a certain message (inviting to the store, or violation of the traffic rules in the middle of the rental). According to the current situation, there are not many actual application scenarios for outbound robots.

When you think yo are talking to a human, think again? Chatbots now can understand up to 100 emotions from the text you sent to create that illusions.

3: Chatbot

Application scenario: Use robots to answer customer questions for Chat business

Problems encountered in use:

Robots cannot “learn” new knowledge by themselves and require manual maintenance/education on daily basis;

many platforms are forbidden to use external software.

workaround: In the early stage of Chat, tell the customer that this is a robot. After answering the question, ask customer if the answer is correct. According to the customer’s answer, manually update and optimize the KBS.

iflytek, the ai company specialised in voice to text transcription has a leading position in the China market and has launched a line of products worldwide.

4: Smart QC, smart assistant

Application scenario: Convert voice into text, query whether customer representative has said forbidden words according to keywords, whether the tone of voice fluctuates, and actively look for suitable FAQs

Problems encountered in use:

Real-time voice-to-text, like outbound robots, the accuracy is one concern and since it’s real-time, doesn’t matter cloud or local deployment, it’s really expensive (server or bandwidth). Moreover, “smart” QC currently matches obscenities words, with limited effect.

In the customer service scenario, 80% of customers ask 20% of frequently used questions, so smart assistants are not very helpful for senior employees.

Workaround: Smart assistants can effectively help new employees find relatively suitable answers faster in order to reduce training time; at the same time, due to the epidemic, intelligent online quality inspection may be used in working from home scenarios.

Customers interact with brands from every device and platforms. AI can be applied to the massive amount of data collected from all the touch-points to identify customers’ needs.

5: Omni-channel communication

Application scenario: The call center system/CRM system no longer a simple one system per channel. Voice, chat, and video can be converted/switched at any time. Communication changes from one-dimensional (voice) to multi-dimensional (pictures, Characters, video).

Problems encountered in use: Customer database integration and many ecosystems are closed.

At present, the most mature product that can be launched if end user agree.

6: Questionnaire analysis (modelling)

Application scenario: Analyze the telemarketing/outbound recording, convert the voice of the questionnaire that is answered but not completed, and then label it, analyze why the answered customers hang up to improve the questionnaire

Problems encountered in use:

Because the voice convert to text is not 100% accurate, it caused the low label accurate; experienced employees can also analyze the recording (low cost or no additional cost). Sometimes, because of the inaccuracy of voice-to-text conversion, BPO company do the data labeled manually, not by machine.

Workaround: purely show “technical strength”, there is no good workaround at present.

In general, everyone knows and heard the market changes due to “technology”, but the fact is the technology is not 100% ready for the market. The “mature” products that can be used for current industry are chat bots, OCR (requires a good UI design) and omni-channel communication systems.

The future of call centers will no longer be a labor-intensive industry, it will have to evolve into operations that fully embrace AI technologies to enable each agent to be smarter. As customers have more and more choices from brands in the market. Whoever that are better at anticipating the problems of callers and providing solutions at the faster speed will be the winners. To achieve that, there is a tremendous room for AI technologies mentioned in this article to be deployed into the specific use cases in order to create proven results for wider adoption.

You may contact the original author of this article Wei Chi at his Linkedin Profile.

Visit ChinaPotion Research Store.




Deep Research on the emerging China Tech Companies and Trends