Hello Bots

This post is a summary containing my study notes of the MVA course Getting started with bots.

Bots are not necessarily artificial intelligence. Bots can be simple useful tasks like an Alarm Setter or a Password Resetter. Bots are not necessarily test interfaces. Bots can transmit and receive information without natural language. Bots are not necessarily Natural Language Processing only. Bots can be command line. Bots don’t necessarily require the user to type all out.

So what are bots? Bots are apps with a new interface. You build bots like apps, the same patterns and design apply. But unlike apps, bots have faster access, bots are available across more platforms, bots have less friction on deploy and last but not least bots require fewer steps to accomplish what you want to do.

You can build powerful bots with the cloud; take advantage of the cloud services to power your bot with machine learning and search.

Building a bot

You can build bots with 2 languages and platforms: C# and .Net or JavaScript and Node. So let’s start with Hello Bot in JavaScript.

Setting up the bot in Node using Visual Studio Code

Add package.json
Open the Integrated Terminal and type: npm install botbuilder –save

The botbuilder includes the bot that stores the logic, the connector used to talk to the bot via chat or console and other utilities.

Creating the bot

Create the connector: chat or console
Create the bot with the connector
Add logic with dialog
Communicte with the user via the session object

The UniversalBot with work with any connector. One bot to rule them all!

To run the bot, type: node filename.js. At this point the bot is waiting to initialize the communication.

Creating the bot with logic

You can ask the user for information by text and number via prompts which is stored in result.response. You can create a dialog with prompts using the waterfall which is just an array of functions: call first function, wait for user response, call second function, and so on. You can grab the message with session.message and send it to the Natural Language Processor.

Using the chat connector

The chat connector can be used to talk to the bot through IM channels, this can be accomplished through one connector. The logic of the bot does not change. You send and receive messages via REST API. You can host the bot with express or restify.

nmp install restify –save
Setup restify to listen

You can use the Bot Framework Channel Emulator for testing the bot and this acts as a client for the rest service.

Workflows for bots

Dialogs

Dialogs are for managing conversations. The dialog stack or callstack is an array of dialog objects. You can stear the conversation through the underlying stack.

session.beginDialog()
session.endDialog()
session.replaceDialog()
session.endDialogWithResults()

IntentDialog is for listening for keywords using regex: IntentDialog.matches().

Maintaining state

You can store state for conversations via the Bot Connector which includes a state management system that stores the state on decoupled service. Having multiple instances of the bot web service might lead to not having the same instance all the time, for example in a load balanced environment.

Data might have to be cleared explicitly.

session.privateConversationData
session.conversationData
session.dialogData – dialog lifespan
session.userData – bot lifespan

Interacting with the user

You can interact with the user via rich controls. You can ask the user questions via prompts like free form, a list of options or a number. You can help guide the user to minimize typing, you can direct the user and you can provide help with a list of commands. Keep in mind that best practices when building an app apply to bots as well, like data validation.

Free form: text
Data detection: number, time
Options: confirm, choice
Media: attachment

Using cards, you can move beyond text by displaying images, formatted text, adding clickability and interactivity. A card is a graphic containing a title, a subtitle, a text, an event handler for tap and a picture. A common limitation with bots is to keep the results relatively small (10). A carousel is multiple cards that the user can swipe through and tap on one card.

Useful: nodemon is a node service that monitors a file and restarts the service if the file has changed.

Natural Language Processing

Bots respond to commands but you can also talk to the bot in a more natural language, the same way that we talk to a friend. LUIS, Language Understanding Intelligence Service, is a tool for natural language processing. You have to train it with example messages that users might say, then identify the entities or nouns and intents or verbs of those messages. LUIS can be trained via GUI or programatically via API. LUIS includes built-in intent handling in the Bot Framework so you can map LUIS intent to execute code within the Both Framework.

Training a LUIS model

You can train a LUIS model using the website by adding utterances, intents and entities. It also includes pre-built entities, regex features and phrase list features.

You can enter utterance which is the user message. Add intent which is what a bot should do. You create a new intent with an example utterance, like search user or find user. Like this you train the model with one data point of data. For the intent and utterance you need to build entities, like user, so you can create a new utterance, like search for username. You train the LUIS model by adding different phrases and permutations. The more data you give it, the better it is going to be.

e.g. Intent = LoadProfile, Entity = Profile, Utterance = Load Profile profilename

Next you press the Train button and finally you have to publish it so it is available via the API. After you publish the web service, you can query the utterances which returns JSON.

Workflow: Bot => Model => Response = Entity + Intent => Handle in code

Using LUIS

Add LUIS model URL in .env, this URL contains a key so keep it safe and secure.
Create recognizer, built into the framework
Build IntentDialog object with recognizer
Create dialogs: a default and names of intents in LUIS model.
The functions have collection of arguments as parameter, this is the data that LUIS passed back, including the entities.

Previously we sent messages to wake up the bot and then we could send the actual message. With IntentDialog you get a better interface because you can get right to the point and ask for what you want.

Other cognitive services

You can use cognitive services to make your bots smarter, like Bing Speech API with Speech to Text and Text to Speech which weighs speech based on context, or Face API with face and age recognition, and Emotion API. You can interact with the service by sending content, e.g. you send an image to Face API and it returns a JSON response with data about the image.

Bots are like apps, you can use whatever services you like and call any API, you are not restricted to what you’re able to use or call within your bot.

There are many ways to make bots more intelligent with different cognitive services. You can also use services in Azure to make your bot more powerful and intelligent. Azure has Platform as Service offerings, these are packaged offerings that you can deploy and start to use.

Compute: Functions, Service Fabric, App Services
Data & Storage: DocumentDB, Blob, Table, Azure Search
Analytics: Azure ML, Stream Analytitcs, App Insights
Identity: Azure AD

Deploying bots

How do you introduce the bot to the real world? You can register a bot in 2 steps.

Hosting the bot

You have to host the bot. The bot is just like a web app, it doesn’t matter where you host it. But you need to know where it’s going to be so you can tell it to the Bot Framework.

Registering the bot

You have to register the bot with the Bot Framework so it appears in the channels. This is relatively easy, although depending on the channel additional steps are required.

You can register your bot by going to the Bot Framework. You have to login and fill out a form that will ask you for information about your bot. The messaging endpoint is where your code is hosted, although you don’t need to host it at this point yet, remember to add /api/messages at the end. You can easily create an Application ID and password by automatically pulling the information and generating the password, this is used for authentication to connect to the bot. You will also have to provide a page with the bot information, publisher, website, language, terms of use and privacy statement.

Finally you configure the bot to communicate by putting in the app id and password environmental variables in .env. This is for local use, you will still have to publish your bot for the real world.

Publishing the bot to Azure

Create App service of type Web app
Select your code source: Azure can read from your repo, like GitHub, by selecting the repo and the source project under Deployment Options, this will run npm install automatically.
Setup app id and password in App Settings

After registering the bot, you can add channels on the dashboard. The bot is already enabled for Skype. You can enable other channels by simply adding them on the dashboard, additional steps might be required depending on the channel. You can use the same bot on all channels without any changes in the code.

Sample bots

Project Murphy: you can ask Murphy on Skype theoretical questions and it sends back an image using Face API and Bing Images.

Sample code

I liked the demos from scratch in this MVA course because I could code along with the trainer. I uploaded the sample code to GitHub.