Next, let us select our machine learning scenario. Model Builder supports several scenarios:

In our case, we are going to predict sentiment based on the content (text) of customer reviews. So we select the Sentiment Analysis scenario, which is a binary classification ML task.

On the left hand side, we see the Menu items: Scenario, Data, Train, Evaluate, Code. These are the actual steps we perform when building a machine learning model. ML.NET has attempted to automate these steps and make it as simple as trying to configure few properties for each of the steps. Our scenario is decided and we move to the Data procurement stage.

We will use the WIKIPEDIA Detox project data. It can be downloaded here. It contains about 160 K rows of user comments that have been graded as 1 or 0 (0 is non-toxic, 1 is toxic)

The data looks as follows:

We include this file in our EchoBot project directory.

Add data

In Model Builder, you can add data from a local file or connect to a SQL Server database. Select File as the input data source in the drop-down, and in Select a file find and select wikiDetoxAnnotated160kRows.tsv.

Under Column to predict (Label), select “Label”. The Label is what you’re predicting, which in this case is the Label found in the first column of the dataset.

The columns that are used to help predict the Label are called Features. In this case, the review comment is the Feature, so leave “Comment” checked as the Input Column (Feature).

Now you’ll train your model with the wikiDeoxAnnoated160kRows.tsv dataset.

Model Builder evaluates many models with varying algorithms and settings to give you the best performing model.

In our case, we see that the time to Train has been auto populated as 600 seconds. For larger data sets, we will need longer times.

Select Start Training to start the training process.

Progress

You can keep track of the progress of model training in the Progress section.

• Status — This shows you the status of the model training process; this will tell you how much time is left in the training process and will also tell you when the training process has completed.
• Best accuracy — This shows you the accuracy of the best model that Model Builder has found so far. Higher accuracy means the model predicted more correctly on test data.
• Best algorithm — This shows you which algorithm performed the best so far during Model Builder’s exploration.
• Last algorithm — This shows you the last algorithm that was explored by Model Builder.

At the end of training, we see the following in the output window:

After model training finishes, go to the Evaluate step

After Model Builder trains and selects the best model, we move on to the Evaluate step, which shows you various output (like the best-performing algorithm, how many models were explored etc.

Try out your model

You can make predictions on sample input in the Try your model section. The textbox is pre-filled with the first line of data from your dataset, but you can change the input and hit Predict to try out different Sentiment predictions.

After evaluating and trying out your model, move on to the Code step.

In the Code step in Model Builder, select Add Projects.

Model Builder adds both the machine learning model and the projects for training and consuming the model to your solution. In the Solution Explorer, you should see the code files that were generated by Model Builder.

EchoBotML.ConsoleApp is a .NET console app that contains ModelBuilder.cs (used to build/train the model) and Program.cs (used to test run the model).

EchoBotML.Model is a .NET Standard class library that contains ModelInput.cs and ModelOutput.cs (input/output classes for model training and consumption), ConsumeModel.cs (class that contains method for model consumption), and MLModel.zip (trained serialized ML model).

For our case, we will use the ML Model.zip file to load the model and then make predictions. Those predictions will be output by the echo bot as a response to the chat we send to the bot.

To achieve this, we need to change the code in the EchoBot.cs file as follows:

protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken){MLContext mlContext = new MLContext();ITransformer mlModel = mlContext.Model.Load(ConsumeModel.ModelPath, out _);var predEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(mlModel);ModelInput sampleData = new ModelInput { Comment = turnContext.Activity.Text };ModelOutput predictionResult = predEngine.Predict(sampleData);var predictionBool = predictionResult.Prediction;await turnContext.SendActivityAsync(CreateActivityWithTextAndSpeak($"The sentence {turnContext.Activity.Text} is {(Convert.ToBoolean(predictionBool) ? "Toxic" : "Non Toxic")} sentiment"), cancellationToken);}
private IActivity CreateActivityWithTextAndSpeak(string message){var activity = MessageFactory.Text(message);string speak = @"<speak version='1.0' xmlns='https://www.w3.org/2001/10/synthesis' xml:lang='en-US'><voice name='Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'>" +$"{message}" + "</voice></speak>";activity.Speak = speak;return activity;}

The CreateActivityWithTextAndSpeak functionality will not work in the Bot Emulator (Don’t expect the bot emulator to start replying back to you in voice messages!!!). But we can test our sentiment prediction via the Bot emulator and what we see is as follows:

And voila! Here we are… we turned out rudimentary echo bot in to a sentiment prediction intelligent bot, within minutes. And pretty accurate too, at that.

The bot is able to reply back if the message sent to it was toxic or not, with high precision. Play with different sentences and see the output.

We have achieved 2 of our 3 steps with little or no hassle. Do not be overwhelmed by the steps above, those are performed in a matter of a few clicks. I have put out all the steps in details for convenience of the reader.

And now to the most interesting part! Infusing our sentiment analysis bot with voice.

We can use the power of the Speech service to easily voice-enable a chat bot.

We will achieve the task of our Sentiment analysis bot communicating back and forth in voice messages with the following workflow:

1. The sample client application is configured to connect to Direct Line Speech channel and the Echo Bot
2. Audio is recorded from the default microphone on button press
3. Using Speech SDK, the app connects to Direct Line Speech channel and streams audio
4. The audio is passed to the speech recognition service and translated to text
5. The recognized text is passed to the Echo-Bot as a Bot Framework Activity
6. The response text is turned into audio by the Text-to-Speech (TTS) service, and streamed back to the client application for playback.

As is the norm, while using Azure, we will need to create a subscription. Either create a 30 days free trial or use your personal/ enterprise subscription.

The details of the steps for configuring the speech to text and setting up the client etc. can be found here. As mentioned earlier, these Azure configuration steps are a bit tedious and in my personal experience, please operate using chrome browser. Some direct line speech options are not visible on IE Edge oft times.

Once the Direct line speech client is set up and running, you should see the settings screen like this:

From the azure portal, enter the subscription keys for the speech cognitive service. Clicking ‘Ok’ should route us to the next screen where you can click the ‘Reconnect’ button.

Then click on the ‘microphone’ icon to start recording your speech and wait for the Bot’s reply to predict the sentiment of your input text. Seeing the magic in action after a hard day’s work is really worth all the effort we have put it for so long 😀

Here is the final output:

You can tweak the code to build a fully functional Restaurant order bot or a technical FAQ bot that understand’s user input (either by voice or text)and the reply could be back in speech or plain text. Let your imagination run wild… happy coding!

All the resources and source code can be accessed through my GITHUB repository.

Source: https://chatbotslife.com/sentiment-analysis-voice-bot-12d8c34116f6?source=rss—-a49517e4c30b—4