Automatic intent classification using parts of speech tags and sentence dependency structure (P-0638) (MFN 6791)

Abstract

The year 2016-17 was termed as "The Year of the Chatbots". Chatbots are becoming part of our everyday life fabric. From intelligent personal assistants like Apple's Siri to psychology therapist like WoeBot, they can be seen in every domain and have support from almost all major tech companies. The research on conversational agents or Chatbots has received a lot of attention in the recent years. One of the biggest challenges in developing such agents is to enable them to understand natural language. Natural language is very hard to understand because it is ambiguous and words have multiple meanings. In natural language, the purpose of a sentence is called its "intent". In intelligent systems, an intent represents the mapping between what a user "wants the system to do" and "what action should be taken by the system" in response. When developing Chatbots, developers have to train these agents on large fully-annotated corpuses, using Supervised Machine Learning to automatically classify user intents. These corpuses contain sentences that are tagged or "labelled" to a specific intent. Compiling a 'complete' corpus with all possible sentenceintent mapping is almost impossible and a complete dataset of all sentences and their intents is not available. This is the major drawback and explains why Chatbots have not been able to achieve the success envisioned for such agents. Almost all tech giants like Google, Facebook, IBM, and Microsoft have also introduced frameworks to develop Chatbot but they, again, are dependent on data and require developers to manually enter all possible sentences along with their intended intent. Entering data in this way is a very hectic process and inefficient. Keeping in mind the above problems, we have proposed an algorithm which can classify intents of short sentences automatically without any dependence on a corpus. The algorithm uses "Parts-of-Speech" tags along with "Sentence Dependency Structure" to identify the intents from given input text. The results of our approach are very encouraging as the system is able to generate intents for given inputs automatically and is also able to annotate similar sentence with the same intent. This algorithm is a step towards improving Natural Language Understanding of computers while reducing the dependence on data.