Chatbot Architecture | Pavel Surmenok (2022)

Chatbots are on the rise. Startups are building chatbots, platforms, APIs, tools, analytics. Microsoft, Google, Facebook introduce tools and frameworks, and build smart assistants on top of these frameworks. Multiple blogs, magazines, podcasts report on news in this industry, and chatbot developers gather on meetups and conferences.

I have been working on chatbot software for a while, and I have been looking on what is going on in the industry. See my previous posts:

  • RE-WORK Virtual Assistant Summit Notes
  • 2016 is The Year of Chat Bots
  • Character-level Convolutional Networks for Text Classification

In this article, I will dive into architecture of chatbots.

There are two major types of chatbots: chatbots for entertainment and chatbots for business.

Engineers have been developing chatbots for entertainment for decadessince famous chatbot-psychotherapist ELIZA was introduced in 1966. Creators of these chatbots usually try to make a bot which can look like a human, pass the Turing test. Perhaps all of the bots which participate in Loebner’s prize and similar competitions are in this group. Microsoft’s bots Xiaoice and Tay have similar behavior. The most recent example is “Spock” bot in Skype: “Chat with Spock, second in command of the USS Enterprise, to learn the ways of Vulcan logic!”

Chatbot responses to user messages should be smart enough for user to continue the conversation. A chatbot doesn’t need to understand what user is saying and doesn’t have to remember all the details of the dialogue.

One way to assess an entertainment bot is to compare the bot with a human (Turing test). Other, quantitative, metrics are anaverage length of conversation between the bot and end users or average time spent by a user per week. If conversations are short then the bot is not entertaining enough.

(Video) GLOSERV Workshop: Designing service-oriented chatbots to improve customer experience

Chatbots for business are often transactional, and they have a specific purpose. Conversation is typically focused on user’s needs. Travel chatbot is providing an information about flights, hotels, and tours and helps to find the best package according to user’s criteria. Google Assistant readily provides information requested by the user. Uber bot takes a ride request.

Conversations are typically short, less than 15 minutes. Each conversation has a goal, and quality of the bot can be assessed by how many users get to the goal. Has the user found information she was looking for? Has the user successfully booked a flight and a hotel? Has the user bought products which help to solve the problem at hand? Usually, these metrics are easy to track.

Perhaps some bots don’t fit into this classification, but it should be good enough to work for themajority of bots which are live now.

Another useful classification is based on atype of conversation: one-to-one or one-to-many, if the chatbot is added into a group chat. Dynamics of conversation, use cases, complexity of chatbot software is very different for these cases.

As Denny Britz wrote in “Deep learning for chatbots“, the chatbot can either generate responses from scratch based on machine learning modelsor use some heuristic to select a response from a library of predefined responses.

Generative models are harder to build and train. Typically it requires millions of examples to train a deep learning model to get decent quality of conversation, and still you can’t be totally sure what responses the model will generate. Microsoft Tay is in this category.

Generative models are the future of chatbots, they make bots smarter. This approach is not widely used by chatbot developers, it is mostly in the labs now.

(Video) Chris Currin - Artificial Assistants - How can I help you?

Retrieval-based models are much easier to build. They also provide more predictable results. You probably won’t get 100% accuracy of responses, but at least you know all possible responses and can make sure that there are no inappropriate or grammatically incorrect responses.

Retrieval-based models are more practical at the moment, many algorithms and APIs are readily available for developers.

A chatbot uses the message and context of conversation for selecting the best response from a predefined list of bot messages. The context can include current position in the dialog tree, all previous messages in the conversation, previously saved variables (e.g. username).

If the bot doesn’t use context then it is stateless. It will only respond to the latest user message, disregarding all the history of theconversation.

Heuristics for selecting a response can be engineered in many different ways, from if-else conditional logic to machine learning classifiers. The simplest technology is using a set of rules with patterns as conditions for the rules. This type of models is very popular for entertainment bots. AIML is a widely used language for writing patterns and response templates. Bot developers write code in AIML language, code can include multiple units like this:

<category>
<pattern>WHAT IS YOUR NAME</pattern>
<template>My name is Michael N.S Evanious.</template>
</category>

When the chatbot receives a message, it goes through all the patterns until finds a pattern which matches user message. If the match is found, the chatbot uses thecorresponding template to generate aresponse.

ChatScript is a modern implementation of this idea. It is an open source chatbot engine which allows defining a chatbot in a rule-based language. Each rule contains a pattern and an output:

s: (because [someday "one day"]) That won’t be soon.

ChatScript engine has a powerful natural language processing pipeline and arich pattern language. Using ChatScript you can do much more than with AIML. It will parse user message, tag parts of speech, find synonyms and concepts, and find which rule matches the input. In addition to NLP abilities, ChatScript will keep track of dialog, so that you can design long scripts which cover different topics. It won’t do anything fancy, though. It won’t run machine learning algorithms and won’t access external knowledge bases or 3rd party APIs unless you do all the necessary programming.

The inherent problem of pattern-based heuristics is that patterns should be programmed manually, and it is not an easy task, especially if the chatbot has to correctly distinguish hundreds of intents. Imagine that you are building a customer service bot and the bot should respond to a refund request. Users can express it in hundreds of different ways: “I want a refund”, “Refund my money”, “I need my money back”. At the same time, the bot should respond differently if the same words are used in another context: “Can I request a refund if I don’t like the service?”, “What is your refund policy?”. Humans are not good at writing patterns and rules for natural language understanding, computers are much better at this task.

Machine learning lets us train an intent classification algorithm. You just need a training set of a few hundredor thousands of examples, and it will pick up patterns in the data.

Such algorithms can be built using any popular machine learning library like scikit-learn. Another option is to use one of cloud API: wit.ai, api.ai, Microsoft LUIS. Wit.ai was probably the first machine learning API for chatbots, it was bought by Facebook this year, and it is free.

Patterns or machine learning classification algorithms help to understand what user message means. When the chatbot gets theintent of the message, it shall generate a response. How can the bot do it? The simplest way is just to respond with a static response, one for each intent. Or, perhaps, get a template based on intent and put in some variables. It is what ChatScript based bots and most of other contemporary bots are doing.

How can bots do better? There is no single answer. Response generation mechanism must depend on the task at hand. A medical chatbot will probably use astatistical model of symptoms and conditions to decide which questions to ask to clarify adiagnosis. A question-answering bot will dig into a knowledge graph, generate potential answers and then use other algorithms to score these answers, see how IBM Watson is doing it. A weather bot will just access an API to get aweather forecast for a given location.

The chatbot can express the same message using different words. A weather bot can say “It’s rainy”, or “Probability of rain is 80%” or “Please carry anumbrella today”. Which one will work the best for the user? Different users prefer different styles of response. The bot can analyze previous chats and associated metrics (length of the conversation, probability of sale, rating of customer satisfaction, etc.) to tailor responses for the user.

Chatbot can have separate response generation and response selection modules, as shown in the diagram below.

Chatbot Architecture | Pavel Surmenok (3)

Message processing begins from understanding what the user is talking about. Intent classification module identifies theintent of user message. Typically it isselection of one out of a number of predefined intents, though more sophisticated bots can identify multiple intents from one message. Intent classification can use context information, such as intents of previous messages, user profile, and preferences. Entity recognition module extracts structured bits of information from the message. The weather bot can extract location and date.

The candidate response generator is doing all the domain-specific calculations to process theuser request. It can use different algorithms, call a few external APIs, or even ask a human to help with response generation. The result of these calculations is a list of response candidates. All these responses should be correct according to domain-specific logic, it can’t be just tons of random responses. The response generator must use thecontext of the conversation as well as intent and entities extracted from the last user message, otherwise, it can’t support multi-message conversations.

The response selector just scores all the response candidate and selects a response which should work better for the user.

How are you designing software for your bots, and what algorithms, libraries, and APIs are you using?

FAQs

What is the architecture of chatbot? ›

Most chatbot architectures consist of four pillars, these are typically intents, entities, the dialog flow (State Machine), and scripts. The dialog contains the blocks or states a user navigates between. Each dialog is associated with one or more intents and or entities.

What are the 7 steps to create a chatbot strategy? ›

Let's walk through them in order.
  1. Audience. The first key to a successful strategy is to profile your ideal customers. ...
  2. Goal. To define the purpose or goal for your chatbot strategy, begin with the end in mind. ...
  3. Performance. ...
  4. Key Intents. ...
  5. Storytelling. ...
  6. Platform Strengths: ...
  7. Feedback.

Which model is used for chatbot? ›

Artificial Intelligence Markup Language (AIML) is most popularly used for writing patterns and response in the process of chatbot development. With a natural language processing pipeline and predefined rich pattern, AIML can be used to build a smart chatbot.

Which framework is best for chatbot? ›

Now, let's take a look at some of the best open-source chatbots in 2022, and remember, these are in no particular order!
  1. Microsoft Bot Framework‍ Microsoft Bot Framework (MBF) offers an open-source platform for building bots. ...
  2. Botkit‍ ...
  3. Botpress. ...
  4. Rasa. ...
  5. Wit.ai. ...
  6. OpenDialog. ...
  7. Botonic. ...
  8. Claudia Bot Builder.

What is the component of a chatbot? ›

Essential Components

Receive messages from voice and messaging channels. Check the context of the message in the overall conversational dialog. Interact with the NLP Engine to extract Intents and Entities from user's message. Integrate with Backend Systems of Record to retrieve user-related information.

How chatbots are built? ›

To create your own chatbot:

Design your bot conversation flow by using the right nodes. Test your chatbot and collect messages to get more insights. Use data and feedback from customers to train your bot. Check which conversation routes are the most popular and improve them for a better user experience.

How chatbot works step by step? ›

Put simply, chatbots follow three simple steps: understand, act and respond. In the first step, the chatbot processes what the user sends. Then, it acts according to a series of algorithms that interpret what the user said. And finally, it picks from a series of appropriate responses.

How is NLP used in chatbots? ›

These AI-powered chatbots use a branch of AI called natural language processing (NLP) to provide a better user experience. Often referred to as virtual agents or intelligent virtual assistants, these NLP chatbots help human agents by taking over repetitive and time consuming communications.

Are chatbots AI or machine learning? ›

The Types of Chatbots

Chatbots are often associated with Artificial Intelligence (AI). This happens because AI gives them the ability to handle requests without the need for human intervention. However, some chatbots don't have AI and, as such, are more basic.

What is Microsoft bot framework? ›

Microsoft Bot Framework and Azure Bot Service are a collection of libraries, tools, and services that let you build, test, deploy, and manage intelligent bots. The Bot Framework includes a modular and extensible SDK for building bots and connecting to AI services.

What is AWS chatbot? ›

AWS Chatbot is an interactive agent that makes it easy to monitor, operate, and troubleshoot your AWS workloads in your chat channels. With AWS Chatbot, you can receive alerts, run commands to retrieve diagnostic information, configure AWS resources, and initiate workflows.

What are the limitations of Chatbots? ›

Each of them is strictly connected with a specific limitation of this technology or its bad implementation.
  • Unclear scope of the chatbot and/or too broad purposes of its utilization. ...
  • Setting unrealistic expectations is often the reason why chatbots fail. ...
  • Lack of customer perspective in building the chatbot.
12 Oct 2021

How are Chatbots trained? ›

Quite simply, you choose a common question, train the chatbot to recognize it, then create the answer. Once you set the answer live, the chatbot will reply to every customer who asks a matching question. The more answers you create, the more hours you save your team every week.

How do chatbots work in the backend? ›

A chatbot interacts on a format similar to instant messaging. By artificially replicating the patterns of human interactions in machine learning allows computers to learn by themselves without programming natural language processing.

What is the difference between chatbot and virtual assistant? ›

The main difference between a chatbot and a virtual assistant is design and purpose. While virtual assistants are made to handle a variety of requests, chatbots are typically more specific with their offerings. Both increase the delivery of personal assistance.

What is conversational architecture? ›

A conversation architect designs powerful, strategic conversations. They determine the questions to trigger the conversations and design the processes to convene and host them.

Is Siri a chatbot? ›

Yes! Technologies like Siri, Alexa and Google Assistant that are ubiquitous in every household today are excellent examples of conversational AI. These conversational AI bots are more advanced than regular chatbots that are programmed with answers to certain questions.

How long does it take to build a chatbot? ›

Implementing a chatbot takes 4 to 12 weeks, depending on the bot's scope, the time required to build your knowledge base and its technical complexity.

Do chatbots use neural networks? ›

A generative-based chatbot follows this neural network to understand user queries and to carry forward an intent-based communication. It conceives a sequence of context tokens (Input layer) one at a time and updates its hidden state.

Is chatbot an example of AI? ›

Chatbots, also called chatterbots, is a form of artificial intelligence (AI) used in messaging apps. This tool helps add convenience for customers—they are automated programs that interact with customers like a human would and cost little to nothing to engage with.

How do I make a chat flow? ›

If you want to keep the conversation flowing, make sure to involve other people in the conversation. If you're asking one person a question, make it a point to ask everyone as well. Give people a chance to share their experiences and make it a comfortable space to talk, share ideas, feelings, jokes, and stories.

Can you build a bot without artificial intelligence? ›

Without Artificial Intelligence, we won't be able to create a bot. As a result, it is incorrect. To make the bot adapt to the information and examples, we'll require machine learning. The bot's ability to deduce specific probability on which can be decided must then be tested in the real world.

Are all chatbots based on AI? ›

Not all chatbots are based on AI. The so-called rule-based chatbots are often used for simple applications, standardised questions and uncomplicated issues. By linking commands, rules and keywords, the appropriate answers can be provided for predefined questions.

What is Omnichannel chatbot? ›

An omnichannel chatbot is an AI-enabled chatbot that provides customers with an integrated buying and customer support experience across all channels. You can deploy and manage a single omnichannel chatbot across all devices and communication channels to offer consistent user support.

What are the characteristics of chatbots? ›

Seven characteristics of a great chatbot
  • Conversational maturity. ...
  • Omni-capable. ...
  • Integrates with CRM. ...
  • Emotionally intelligent. ...
  • Free to explore. ...
  • Autonomous reasoning. ...
  • Pre-trained.
15 Jan 2020

What are NLP algorithms? ›

NLP algorithms are typically based on machine learning algorithms. Instead of hand-coding large sets of rules, NLP can rely on machine learning to automatically learn these rules by analyzing a set of examples (i.e. a large corpus, like a book, down to a collection of sentences), and making a statistical inference.

What is the difference between NLP and NLU? ›

The major difference between the NLU and NLP is that NLP focuses on building algorithms to recognize and understand natural language, while NLU focuses on the meaning of a sentence. Another difference is that NLP breaks and processes language, while NLU provides language comprehension.

Which machine learning algorithms is used in chatbots? ›

The main algorithm that's used for making chatbots is the “Multinomial Naive Bayes” algorithm. It is used for text classification and natural language processing (NLP).

What is the difference between chatbot and AI? ›

Conversational AI is all about the tools and programming that allow a computer to mimic and carry out conversational experiences with people. A chatbot is a program that can (but doesn't always) use conversational AI. It's the program that communicates with people.

Can a chatbot learn? ›

Chatbots can learn automatically by analyzing past data and making assumptions on what is right. The other way chatbots learn is by having a human editing the system. In most cases, both are required. Even though a chatbot has artificial intelligence, a human still needs to audit the responses to make adjustments.

Is Microsoft bot framework free? ›

The Azure Bot Service has now been Generally Available, which means they're providing a paid service with incurring subscription cost. The free tier, as you mentioned above, is capped at 10000 msgs/month. The previous Preview (or Beta) did provide the channels for free although they don't guarantee the SLA.

Does Microsoft have a chatbot? ›

Azure Bot Service enables you to build intelligent, enterprise-grade bots with ownership and control of your data. Begin with a simple Q&A bot or build a sophisticated virtual assistant. Use comprehensive open source SDK and tools to easily connect your bot to popular channels and devices.

What is azure bot? ›

Azure Bot Service provides an integrated development environment for bot building. Its integration with Power Virtual Agents, a fully hosted low-code platform, enables developers of all technical abilities build conversational AI bots—no code needed.

What is chatbot API? ›

Chatbot API supports both advanced AI and machine learning to develop super-intelligent bots. It also allows you to integrate the bot with other messaging platforms. The Chatbot API helps to make stories related to bots that can answer various questions.

Is it easy to create a chatbot? ›

Because building a chatbot with code is immensely difficult for people with no development background and limited exposure to coding languages, it's good to research sample chatbot code from expert developers as a jumping-off point for those determined to learn how to build their own bot without help.

Is there any free chatbot IntelliTicks? ›

IntelliTicks is one of the Top Free AI-powered chatbots providers. It has a human fall-back option and allows the human agents to take over the conversation anytime. IntelliTicks Chatbots is an AI conversational tool with automated engagement in the form of personalized conversation.

Is Amazon Alexa a chatbot? ›

Alexa is officially a chatbot. Yesterday, Amazon began rolling out a new feature on iOS that enables users to type their requests to Alexa and see responses on the screen.

Is Amazon live chat a bot? ›

Using Amazon Lex, a service that allows you to create intelligent conversational chatbots, you can turn your call center contact flows into natural conversations that provide personalized experiences for your callers.

How do Chatbots work an overview of the architecture of Chatbots? ›

A chatbot interacts on a format similar to instant messaging. By artificially replicating the patterns of human interactions in machine learning allows computers to learn by themselves without programming natural language processing.

What is conversational architecture? ›

A conversation architect designs powerful, strategic conversations. They determine the questions to trigger the conversations and design the processes to convene and host them.

How are Chatbots built? ›

To create your own chatbot:

Design your bot conversation flow by using the right nodes. Test your chatbot and collect messages to get more insights. Use data and feedback from customers to train your bot. Check which conversation routes are the most popular and improve them for a better user experience.

What is chatbot and how it works? ›

A chatbot is a software or computer program that simulates human conversation or "chatter" through text or voice interactions. Users in both business-to-consumer (B2C) and business-to-business (B2B) environments increasingly use chatbot virtual assistants to handle simple tasks.

How chatbot works step by step? ›

Put simply, chatbots follow three simple steps: understand, act and respond. In the first step, the chatbot processes what the user sends. Then, it acts according to a series of algorithms that interpret what the user said. And finally, it picks from a series of appropriate responses.

How is NLP used in chatbots? ›

These AI-powered chatbots use a branch of AI called natural language processing (NLP) to provide a better user experience. Often referred to as virtual agents or intelligent virtual assistants, these NLP chatbots help human agents by taking over repetitive and time consuming communications.

What is chatbot Analytics? ›

Chatbot analytics is the real-time data that your chatbots is generating through its interaction with people. This data can be really useful and present a great value for your business.

What are conversational designs? ›

What is conversation design? Conversation design is a design language based on human conversation (similar to how material design is a design language based on pen and paper). The more an interface leverages human conversation, the less users have to be taught how to use it.

What is chatbot intent? ›

Intent is chatbot jargon for the motive of a given chatbot user. It's the intention behind each message that the chatbot receives. Intent is all about what the user wants to get out of the interaction. For example, a user says, 'I need new shoes.

What are the limitations of chatbots? ›

Each of them is strictly connected with a specific limitation of this technology or its bad implementation.
  • Unclear scope of the chatbot and/or too broad purposes of its utilization. ...
  • Setting unrealistic expectations is often the reason why chatbots fail. ...
  • Lack of customer perspective in building the chatbot.
12 Oct 2021

Is Siri a chatbot? ›

Yes! Technologies like Siri, Alexa and Google Assistant that are ubiquitous in every household today are excellent examples of conversational AI. These conversational AI bots are more advanced than regular chatbots that are programmed with answers to certain questions.

Is Alexa a chatbot? ›

Alexa is formally a chatbot. Recently, Amazon started revealing another component on iOS that empowers clients to type their solicitations to Alexa and see reactions on the screen.

What is the requirement of chatbot? ›

The most important requirements of chatbot software:

In addition to understanding and interacting within conversations, an outstanding chatbot software has NLP functions (Natural Language Processing) to analyze the context of a conversation.

How are chatbots trained? ›

Quite simply, you choose a common question, train the chatbot to recognize it, then create the answer. Once you set the answer live, the chatbot will reply to every customer who asks a matching question. The more answers you create, the more hours you save your team every week.

You might also like

Latest Posts

Article information

Author: Jerrold Considine

Last Updated: 09/24/2022

Views: 5952

Rating: 4.8 / 5 (58 voted)

Reviews: 81% of readers found this page helpful

Author information

Name: Jerrold Considine

Birthday: 1993-11-03

Address: Suite 447 3463 Marybelle Circles, New Marlin, AL 20765

Phone: +5816749283868

Job: Sales Executive

Hobby: Air sports, Sand art, Electronics, LARPing, Baseball, Book restoration, Puzzles

Introduction: My name is Jerrold Considine, I am a combative, cheerful, encouraging, happy, enthusiastic, funny, kind person who loves writing and wants to share my knowledge and understanding with you.