Chatbots with Machine Learning – Approach & Building blocks
Chatbot is a conversational interface; carry out conversations that communicate knowledge, answer questions, or collect and process information – Intelligently! You must be aware of two key types of chatbots, one powered by a set of rules, and the other powered by machine learning. At ValueFirst, We have successfully launched SURBO, AI Enabled chatbots for Business. There are lot talks around Chatbots / Conversational AI, but lesser information available on bottom-up technical approach for starters.
This blog discusses approach for building Chatbots, and complete journey basis our experience, and also answer some of the questions including Why AI enabled bots are replacing simple text based bots?
Let’s first check how Rule based bots work or can be implemented
Intent is one of core concept in chatbots where multiple keywords are associated with intent. Thereafter you can set the sequence or flow of intents. These intent keywords can be indexed in elasticsearch to search the relevant intent on the basis of User query.
For example, let’s consider a use case for FAQ chatbot for loan service helpdesk
Consider below three FAQ and its corresponding intents:
- Loan eligibility intent: This intent would give the loan eligibility age information. First associate the relevant keywords such as “loan”, ”eligibility”, ”age”, ”what” etc. and then the response of intent would be like “The loan eligibility age is 18 years”
- Rate of interest intent: This intent would give the rate of interest available on loan. Keywords that we might use “rate”, ”interest”, ”loan”, ”gold” etc., and response would be like “The rate of interest on gold loan is 8.9%”
- Type of loan Intent: This intent would provide the types of loan information. Keywords would be like “gold”, ”car”, ”type”, ”loan” etc., and response would be “Type of loan available are gold, car, home”
On chats, expect different users to refer different formats / vocabulary. For example, User enters:
“Please tell me the gold eligibility criteria and other loan types.”
Note that, there are various keywords in user query that lies in different intent. How to decide which intent we should call? What should be the criteria of intent selection if user query matched in different intent keywords? One of the major drawbacks of keywords based bots is that we can’t derive the actual meaning of sentence, what exactly users are trying to say. Suppose User enters:
“Please don’t provide type of loans”.
As it is a keyword based bot, it would give the type of loan as intent response and doesn’t understand the sentiment of sentence. Seems keywords aren’t the solution for chatbots. If keywords aren’t the deciding factor for intent selection on chatbots, then what’s next?
Let’s dive into Machine Learning approach to solve above problem
We have tried multiple approaches and algorithms to reach at maturity and accuracy. But if you are new to machine learning, then should first start classification model, and go for SVM (Support vector Machine), it would give decent results by proper cross validation techniques.
Key Chatbot Building Blocks:
1. Text Pre-processing: Before training of our ML model, essential to clean the phrases and this cleaning of text is called text pre-processing, used different NLP techniques such as Tokenization, Lemmatization, N-grams, Stemming, stopwords removal, punctuation removal etc. These techniques help in creating the relevant feature sets for machine learning model.
(The feature set obtained are then converted to numerical vectors which act as a input to SVM classifier an in next phase used for text similarity percentage calculation. Refer our previous blog for more details – http://www.vfirst.com/blog/techfirst/how-to-make-chatbots-intelligent-using-natural-language-processing-nlp/)
2. Context: Important building block, act as a bot memory, remember all conversations between the user and chatbot to take decision accordingly.
3. Named Entity Recognition (NER) service: The main aim to extract the different entities such as date, time, city, email etc. from the user query. In our system, we have introduced two types of entities i.e. system entities and user entities.
- System entities are built-in entities such as date, city, phone number, email etc. which have some specific pattern.
- User entities are based on use case, that user would make such as employees Id, list of available hotels, flight names etc.
NER would capture the different entity from the text which users have to map with the corresponding intents.
- ML Engine: Create machine learning engine, which helps in training the model, predicting the query in specific intent class and provide the accuracy of model i.e. how model is behaving on different datasets.
From implementation perspective, consider each building block as a separate service forming an intelligent pipeline, once user interacts with the chatbots and enters some query. This query would pass through the different services of pipeline and then would enter into ML engine where prediction of query into specific intent target class would take place. The behaviour of classification model is such that it would definitely predict your input in a plane in some target class, whether it is an outlier or a point lies in some class. So, in order to reduce the impact of outliers during model prediction, introduce a concept of similarity. This similarity concept would try to find the similarity percentage between user query and training phrases. It uses the threshold value (that customer defines for its chatbot between 0 to 1) in the following way:
If similarity percentage >= threshold value then would take model prediction as outcome.
If similarity percentage < threshold value then would consider that point as an outlier.
The key point for ML engine accuracy is Training that users have to perform on platform. Intents have to be trained with the sufficient amount (depend on use case) of relevant phrases so that engine prediction could be improved. We have also built our own Training module which helps the user to improve its chatbot capabilities in order to understand the user queries and provide the relevant responses to users.
ML enabled chatbots are intelligent bots and well trained for a conversation with the human. In Rule based chatbots, you don’t need to train models; chatbot would simply follow the defined sequence of intents.
Once you are comfortable playing with classification mode, then you should move on deep learning neural network, and further add more power and intelligence to your AI conversations.
We have already crossed the complete journey, and reached to mature AI driven conversational platform. In case you need any guidance, do connect with our engineering team.