The voice assistant market is also recently affecting the app market, people want smarter apps that do things for them.
Thanks to Dialogflow service we can create a bot in no time, defining training phrases, answers and parameters will be a breeze.
We can also connect webhooks to our backend to better manage requests and data.
By combining communication with the bot with speech to text and text to speach we obtain a real vocal assistant like Google Assistant or Alexa.
Ionic is a very powerful tool, if we want to create cross platform mobile applications without wasting time.
The first step is the creation of new bot (or agent) on Dialogflow.
We should go to Dialogflow console and log in with your google account.
Click on “Create Agent” and fill out the fields:
then, click on “CREATE”.
Now, you can edit other things, like avatar, description or additional languages but go on.
Now, it’s time to create the intents !
To create a new intent, we need to specify a list of training phrases, and a list of responses, and so doing we define a Presentation intent:
When user says “ Who speech?”, “ What are you?”, or “ Who are you?”, the bot responds with “Hi, i’m your personal assistant”, “I’m an artificial intelligence” or “I’m a bot at your service”.
Note: More training phrases make your bot better thanks to machine learning.
Moreover, by default there are 2 prebuilt intents: Default Fallback intent and Default Welcome intent that you can use, let’s not waste.
Now, you can already test!
To do it, you just need to go on the right side of console and ask a question at your bot:
Well, it was kind of easy, right?
Now, let’s go on to the ionic side, the first step is a creation of new project:
ionic start vocalAssistant blank
and install the (DialogFlow library) API-AI javascript library:
npm install api-ai-javascript@2.0.0-beta.21ionic cordova plugin add cordova-plugin-apiai@2.0.0 => to test
Note: We use a cordova plugin, then the app can be only deployed on real device.
Now, we need to retrieve an API key in settings page:
and save it in environments/environments.ts:
The next step is a genereation of “Chat Service”:
ionic generate service sevice/chat
First of all, we need to import environment, ApiAiClient, and BehaviorSubject, the latter allow to share values between componets.
Then, we define a Messageclass to handle users phrases and bot responses.
Following, we define the ChatService that allows to send requests to DialogFlow and manage responses.
Also, we put a profile_picproperty in the Message objects to differentiate user and bot.
Note: We have to remember to put 2 images in the asset folder, one for user and one for bot.
The following code shows what is described:
Now, it needs a page to display “conversation” flow, about that, we will create a SpeechPage.
ionic generate page speech
An example of UI could be the following:
In this example we have an Button to activate the speech recognition and a binding of messages, all with an activant UI thanks to this CSS:
Now, we need to add the following libraries for speech to text and text to speech:
ionic cordova plugin add cordova-plugin-speechrecognition
npm install @ionic-native/speech-recognitionionic cordova plugin add cordova-plugin-tts
npm install @ionic-native/text-to-speech
We have to remember to add the imports in app.module.ts and add it in providersarray under @NgModule
At this point, let’s move on Logic of the page:
in this file we have the core of our assistant, the startListeningfunction provides to activate microphone and sends detected sentence of user.
Instead, the ResponseHandler function takes care of responses of bot, adding the response in an array and activating the text to speech function, allowing to bring to life at out assistant.
In this example, I used the actions of DialogFlow intents to open a pages in Ionic, the doActionfunction takes care of this, but this aspect can be changed and used for different purpose.
Other functions need to handle permission, navigation etc.
The last step is a creation of assistant button that can be placed in all pages of our app:
ionic generate component assistant
Define the markup:
the style:
and the component class:
Ok, now we can use the Assistant component in every page, for example the home page:
Let’s add the selector in home HTML:
Let’s add import in home.module.tsand put in entryComponentsand declarationsArray:
The game is done !
Our assistant is complete, we just have to do the build and run.
You can run app on web browser but only to get UI preview, I suggest you to “mock” a conversation to edit your UI.
To run on web browser you can execute:
ionic serve
To running app you just need to execute:
ionic cordova run android
//OR
ionic cordova run ios
After clicking the assistant button, the app asks microphone permission for working, then we can ask something and the app will speech with bot response.
Note: We have to remember that question and answers dependes of intents on Dialogflow portal.
You can find the project on my github: https://github.com/alexminichino/vocalAssistant
I hope I’ve been of help, thank you for reading.
Enjoy! 😎
