How to Develop a Chatbot From Scratch: Cost and Features

Chatbots are all around us. They help us get groceries delivered to our homes, search for the cheapest flights, grab a taxi, or even give medical advice based on the reported symptoms. Before we knew how far we’ve come, they had already settled down on our smartphones and become our personal pocket assistants.

However, it’s not only consumers who seem to benefit a lot from this artificial intelligence. Thousands of brands and businesses recognize chatbots as a new disruptive way to communicate with their target audiences. More so, instant messaging apps express a willingness to take over as the go-betweens and serve as hubs for chatbot building and integration.

In an interview with VentureBeat back in 2016, Seth Rosenberg, the product manager for Facebook Messenger, mentioned that Messenger could become a place where customers and businesses communicate, including with the help of chatbots.

“Our number one priority was to just give [the platform] to developers and help them to create great experiences and help them to promote the bots wherever it makes sense.”

Seth Rosenberg

In less than two years, his words became reality. In spring 2018, Facebook Messenger reached a milestone in 300,000 bots created on its basis, including those of Uber, Sephora, WSJ, Pizza Hut, and Marvel. Daily, Messenger handles 8 billion conversations taking place between businesses and customers, including those via bots, compared to 2 billion in 2017. And the game isn’t over yet.

But why? What is the secret behind this chatbot thing?

This is exactly what we’re talking about today. In the article, we are going to touch upon the reasons why businesses benefit from chatbots, give some guidelines on where to start if you want to build a chatbot from scratch, and share our own experience of chatbot creation for a mobile banking app.

What kind of a beast is a chatbot?

Chatbots are software programs that a user can communicate with via a chat interface using voice or text messages. The key feature of all chatbots is that they use the power of natural language processing (NLP) so that the interaction with them feels natural for people. Chatbots can live inside websites, apps, or messengers and are widely used for multiple purposes, from booking a hotel to chatting with Doctor Strange.

Types of chatbots

There are two main types of chatbots based on the way they are built: rule-based and AI-based.

• Rule-based or scripted chatbots. It implies that a chatbot is developed according to certain rules under which (and only under which) it’s able to operate. Such chatbots often rely on the keywords in a user inquiry and respond accordingly, or suggest a user choose from the predefined answering options. If a user’s message doesn’t contain any keywords, the chatbots “get lost” as they can’t understand the context of the conversation.
• AI-based chatbots. Chatbots powered by artificial intelligence and machine learning don’t need any keywords to count on. Unlike their rule-based counterparts, they are able to understand the context of the conversation. However, their conversational abilities are often limited to the topics they’ve been trained in — the times when chatbots will understand anything people tell them are still far away. The good news is that AI-based chatbots can be upgraded and trained to recognize new user intents, and thereby be further improved as talk partners.

How do chatbots work?

The main task of any chatbot is to perform online conversations. Both rule-based and AI-based chatbots perform that task well, but the way they do it will depend on how they were built.

Rule-based chatbots will follow the script precisely, and when something goes off-limits, they stop functioning. Let’s say, there is a bot that can get a pizza delivered to your doorway. The only things you have to tell it are the pizza name, quantity, and location. Therefore, if you type “I’m hungry” instead of “one Margarita, two Pepperoni”, the bot won’t understand a thing.

The reason for that is, first, there are no keywords in the message and, second, the bot won’t be able to “guess” that the intent behind the “I’m hungry” message might be “I want to order a pizza”.

The H&M bot is an example of a rule-based bot. It acts as a virtual sales assistant, suggesting outfits for users based on their preferences and style. The bot works within the predefined scenarios: It asks users certain questions and suggests the answering options, shows pictures of various outfits, and asks users to choose which one they like the most, or enables users to create outfits themselves by picking items from the suggested options. If a user types a random message, the bot replies in an “I’m sorry, I’m lost” manner.

AI-based chatbots are smart. They analyze what someone is saying and build their replies according to the input information.

Babylon, a UK-based medical chatbot, can recognize the symptoms a user reports to it and give medical advice based on them. To do this, it relies on its neural network and machine learning abilities to match the symptoms entered by the user with those it already “knows”. Moreover, it can learn from user data, such as the reported symptoms, to be able to recognize more conditions and give more accurate recommendations.

Chatbots are becoming widely used in various fields, from hospitality and eCommerce to government services and health care, taking over the most important and sensitive part of business operations — customer care. Gartner suggests that, by 2020, 25% of customer service and support operations will be handled by virtual customer assistants or chatbots.

Now, when we know what chatbots do and how they work, we can dive into some market trends that will help you make the decision.

Chatbot market trends and insights

Chatbots are now widely used in various fields, from hospitality and eCommerce to government services and health care. It’s definitely not the limit and the role of chatbots, especially in customer care, is only growing. A lot of businesses adapt this technology to their needs. Outgrow states that by this year, 80% of businesses would have a chatbot for process automation.

The market of chatbots grows accordingly. According to Business Insider, it’s expected to reach $9.4 billion by 2024. This industry has all the chances to become even more lucrative with time since more and more industries of any scale embrace automation.

The range of processes that chatbots can facilitate is extensive. 39% of businesses empower their websites with chatbots to make them more interactive and 36% use them to upgrade lead generating.

Sure, customer support is the primary field where chatbots showed their value. Most of the customers are fine with receiving messages from an AI companion. Chatbots collect the input information provided by clients and use it to provide the utter level of personalization during the dialog. Up to 80% of standard issues and questions can be answered by AI. It reduces customer support costs and builds more trustful relationships with the audience.

It would be an understatement to say that chatbots are becoming one of the leading tools for communicating with clients. But why? What are the reasons why businesses make chatbots and entrust them with customer interactions?

Why make a chatbot?

There are two major reasons why chatbots are taking the stage: the trend toward conversational commerce and the continuing rise of messaging platforms.

Messaging is the most popular way people communicate and get things done today. According to Statista, messaging platforms have been on an unprecedented rise for 5 years, with WhatsApp as the most popular messenger in the world with its 1.5 billion monthly active users (MAU) as of October 2018 compared to 465 back in 2014. Facebook Messenger is ranked second, having surged from 200 million MAU in 2014 to 1.3 billion in 2017. That’s mind-blowing growth, isn’t it?

The conversational commerce paradigm—getting in touch with brands and businesses simply via a chat—has also contributed a lot to the rise of messengers. Since it turned out that people find it natural to chat to businesses just as they chat to friends, messengers have transformed from the place where billions of people gab to the one where billions of potential customers hang out. Businesses got the point quickly and seized the opportunity to open themselves to larger audiences. And this is how chatbots found fertile soil to take root.

Pros and cons of developing a chatbot

Advantages of chatbots

Why do customers find chatbots useful? Drift , SurveyMonkey, and others conducted a chatbot survey among 1,000+ respondents from the US, aged 18-64. Among other questions, they were asked what it is about online services that get them frustrated the most. Let’s take a look at the answers.

It turned out that online services are a source of constant dissatisfaction, from poor designs to impersonality issues. And it seems (for a reason, however) that chatbots are able to cover most of the customer pains.

And here's how:

• Chatbots perform boring tasks for customers, such as browsing the stock or searching for the company’s contact details. These microtasks can be very time-consuming and frustrating, taking into account that users have to interact with lots of unknown websites and apps on a daily basis.
• Chatbots provide information instantly, and that’s the main reason why customers prefer chatbots versus other ways of reaching businesses, especially when they need a quick solution. And the figures prove this: according to Drift’s survey, 37% of respondents would use a chatbot in an emergency situation when a quick answer is required.
• Chatbots make customer experience more personal. That sounds funny because we are talking about software, but chatbots can really give businesses a personal voice. Chatbots use natural language to communicate with people, and the way they “talk” influences how consumers feel about the brand.
• Chatbots have no interface. Using a chatbot is simply using language. People can jump on chatting with a bot virtually on the go without having to learn how to use it.
• Chatbots don’t vie for smartphone storage. Apps do. And that’s the reason why people aren’t willing to install the app of every business they interact with. Moreover, chatbots are much cheaper to create and support than apps. Therefore, it’s not even reasonable for some brands, especially small ones, to embark on creating their own mobile app when a chatbot will suffice.

There are at least five reasons both consumers and businesses turn to chatbots to keep in touch. If you agree with just one of them, it’s a perfect argument for considering developing your own chatbot.

Disadvantages of chatbots

Despite all pros we mentioned above, there are still some things to consider:

• A chatbot is still not a human. It sounds quite obvious, but you need to bear it in mind. Even the most sophisticated AI-powered chatbot may not understand some queries and will still need a human intervention. Besides, some customers may consider chatbots unemotional and passionless.
• It needs time to implement. Any automation needs some time to smoothly integrate and chatbots are no exception. Besides a chatbot itself, you should organize a database for all the information it collects, plus train your employees to work with it.
• You need to keep an eye on a chatbot. Both rule-based and AI-based chatbots require constant updating and optimization. You need to add more information for a chatbot to process so it will learn how to handle most issues.

Now, when we highlighted all the positive and negative sides of chatbot creation, let’s talk about what functions this solution should have.

Must-have chatbot features

The basic functionality that will enable a chatbot to perform its tasks correctly includes:

• Integration with a CRM: A chatbot should be able to handle real-time conversations with customers, collect data about them, and organize it. This information can be used by your sales staff to connect with new clients and update the old ones.
• Natural language processing: If you empower your chatbot with AI, it should correctly understand everything customers say to it. Also, NLP will help your chatbot to become emotionally intelligent and detect what customers feel.
• Cross-platform usage: A chatbot should work flawlessly on each platform, mobile or desktop. There is how it can provide the best user experience.
• One main purpose: When developing a chatbot, bear in mind its main purpose. If your chatbot clearly understands what it has to do on your website or in your app, it can become more accurate and effective.
• Simple UX and UI: Since a chatbot isn’t the main element of a website, it shouldn’t be too fancy. A simple dialog window with several buttons won’t distract a user from the main purpose - conversation with a chatbot.

Chatbot development from scratch: Yellow’s experience

The chatbot we built for a mobile banking app is meant to save the customers some trouble when paying for the services online. The procedure of paying for the government services differs slightly depending on a user’s place of residence, and sometimes it’s really an ordeal to find the required transaction manually. The chatbot is able to do this in an instant.

Basically, it works like that: a user starts a conversation, typing something like “I want to pay a real estate tax in [city name]” ---------> the chatbot extracts the required info from the user intent (in this case, these are the user location and the transaction name) ----------> the chatbot finds the required transaction and sends the payment form to the user in response.

Therefore, the main task was to train the chatbot to derive the required data from the user intent. The process of extracting data is called named entity recognition. In our case, the named entities are the names of locations and transactions.

Below is a step-by-step description of how we handled the task.

Step 1. Parsing

To train our chatbot, we needed to compile a dictionary of the named entities—the names of all the populated areas and all the kinds of transactions that can be made via a mobile bank, as well as their variations (e.g. abbreviations or short names).

As there is a finite number of populated areas and transactions, we wrote a Python script to parse them all.

Step 2. Dividing entities into groups

We broke the obtained entities down into categories (or tags) so that they could further be recognized by the chatbot. To do it, we used Yandex Tomita, a parser that obtains structured data from unstructured texts based on certain rules. DeepPavlov is another great tool that can be used for this purpose.

Step 3. Creating intents

Based on the generated categories, we created a list of user intents and chatbot responses to them.

The chatbot responses are straightforward and aimed at obtaining the necessary information from the user — the location and transaction names. Once the user intent is covered (read: all the required info is obtained), the chatbot makes an inquiry containing the transaction name and the user location to a database, and it returns the transaction that matches the inquiry the best.

Here’s a simple chart showing how our chatbot works.

Sure, it’s not a “one-size-fits-all” list on how to write a chatbot. Each project is unique and requires teamwork to become successful. However, we hope that our experience will come in handy for the teams working on similar cases.

Development platforms and tools

If you’ve decided to create your own chatbot, you should first select a proper tool for chatbot development. Unfortunately, there’s no such thing as a universal ready-made solution that you can use for anything (bad news), but if you ever wondered how to build a chatbot using already available software, there still are some options. The choice primarily depends on whether you’re able to code, what level of customization you need, and what platform you want your chatbot to run on.

If you need your chatbot to live and operate inside a messenger, here are the bot starter kits of the world’s most popular messaging apps: Facebook Messenger, Slack, Kik, and Telegram. To build a chatbot for Facebook Messenger, you can also consider Chatfuel that features set are various bot templates that you can use, or Octane AI, a chatbot building platform for Shopify stores.

If you need integration with your website or app, look into Google’s DialogFlow, Botsify, or HubSpot's chatbot builder. Any of them will let you create intelligent, feature-rich chatbots without writing a single line of code.

If you need a totally custom solution, you can try to build a chatbot from scratch. In the next chapter we’ll tell you how we built a simple chatbot for a banking app using a conversational AI library.

The chatbot technology stack

The above-mentioned platforms can be used to create and host a chatbot, but there are plenty of other tools with which to create the individual parts of a chatbot.

If you plan to build an AI-based chatbot that will understand any input and learn from it, you’ll need NLP (natural language processing). The biggest, most famous platforms for that purpose are:

• Facebook Wit.AI
• IBM Watson
• Microsoft Bot Framework
• API.AI
• Nuance Mix
• RECAST.AI
• BotKit

These platforms can give your chatbot the ability to recognize not just certain words but also the meaning of an entire inquiry.

Additionally, the platforms can empower your chatbot with supervised learning. A team can work on the information the bot receives to train it and enrich its capability. Thus, a specialist can use one of two approaches: either be present all the time and qualify any unclear intent, or analyze all the conversations for a certain period of time and qualify the intent.

Third-party service integration is another aspect of chatbot construction. In order for a chatbot to work properly, there are several possible tools with which to merge it:

• Amazon Web Services
• Twilio
• MailChimp
• Zappier
• Salesforce

Before choosing one of those services, be sure your tech stack supports integration.

How much does it cost to develop a chatbot?

The total cost of chatbot creation depends on several factors, including the type of chatbot, the chosen tech stack and the range of desired functions.

A rule-based chatbot will cost less, but it will present less functionality. If you decide to create it from scratch and use third-party services to amplify it, the cost will be approximately $30,000.

An AI-based chatbot will require far greater resources. Tools will be more advanced, and consequently the team should have relevant experience in working with neural networks and artificial intelligence. A cutting-edge solution like this can cost $50,000 or more.

Conclusion

Chatbots are redefining the way businesses and customers communicate. They’ve taken over the tasks that humans are generally reluctant to bother with. As a result, site interactions are time-saving, which makes customers feel engaged and more willing to develop brand loyalty. They talk our language and handle thousands of inquiries simultaneously, making the customer experience quicker and more automated but also personalized and human-like. Chatbots have ushered in a new era where each brand has its own personal voice. With this powerful technology, customer-brand relations are closer and stronger than ever.

WWe believe these are the reasons to ask yourself a question: “Isn’t it the right time for me to learn how to develop a chatbot for my business?” If you agree, join the discussion on LinkedIn or drop us a line.