How your Chatbot Can Learn to Understand Synonyms in Teneo
In language, there are many ways of saying the same thing. That creates a need to optimize the language conditions for your bot so that it can give the correct answer even when other words are used. Here is how you do it in Teneo Studio.
We have earlier seen examples of how to semi-automatically create language conditions based on positive example inputs. For example, we created a syntax trigger that can handle conversations like the following:
User: How much does it cost to join the loyalty program?
Bot: You can join the loyalty program for free!
However, as soon as you slightly change the question, you will get a wrong answer
User: How much does it cost to join the rewards program?
Bot: I am sorry, can you re-phrase your question?
In the following, we will shorten the language condition while at the same time broaden its coverage. After having investigated what went wrong in the second part of the example conversation above, we improve the language condition by following two strategies:
- Recombining language objects
- Exchanging language objects
If you have not yet created the ‘User wants to know if loyalty program is free’ flow, we recommend you to build it before you proceed.
Having a closer look at the language condition
This is the language condition that has automatically been created from our positive examples:
(%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL &^ %REWARD.NN.LEX &^ %PROGRAM.NN.LEX) / (%WHAT_IS_THE_PRICE_OF.PHR &^ %LOYALTY_PROGRAM.NN.SYN) / (%REWARD.NN.LEX &^ %PROGRAM.NN.LEX &^ %FREE.ADJ.LEX)
The reason why the question “How much does it cost to join the rewards program?” did not trigger this flow is that ‘rewards program’ is not among the synonyms listed in the LOYALTY_PROGRAM.NN.SYN language object. We will thus in the following recombine the language objects so that the different phrases for asking about the pricing are combined with the different ways of expressing “loyalty program” or “rewards program”.
Recombination of language objects
We start by updating each line of the language condition to make it match on both ‘loyalty program’ and ‘rewards program’. We do this by replacing each occurrence of ‘loyalty program’ or ‘rewards program’ with ‘(%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))’, as illustrated below:
(%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))) / (%WHAT_IS_THE_PRICE_OF.PHR &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))) / ((%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX)) &^ %FREE.ADJ.LEX)
You may have noticed that we replaced the &^ operator between REWARD.NN.LEX and PROGRAM.NN.LEX with the operator >>. This latter operator is called ‘directly followed by’ and does not allow for intervening words. For more information on the condition syntax language, go and have a look at the reference page.
Strictly speaking, our problem is now solved. However, we can go one step further. The first two lines of our condition look very similar, so let’s combine them:
(((%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL)/%WHAT_IS_THE_PRICE_OF.PHR) &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))) / ((%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX)) &^ %FREE.ADJ.LEX)
Much better, but why not combine these two lines as well?
((%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL) / %WHAT_IS_THE_PRICE_OF.PHR / %FREE.ADJ.LEX) &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))
The &^ operator matches regardless of the word order in the user input. Thus, we can move FREE.ADJ.LEX to the first part of the condition without losing coverage.
That’s it. What a nice and powerful language condition we now have created!
Exchange language objects
We now have a much more robust language condition than before. However, there are still some inputs like ‘Is the rewards program costless?’ that are not yet covered by this condition. So let’s broaden the coverage of the condition a bit.
As we can see, the condition currently uses the lexical language object FREE.ADJ.LEX. Lexical objects only cover inflections of a word. Perhaps there is another language object we can use that covers synonyms. To do that you can try this:
- Select %FREE.ADJ.LEX and replace it with the text
- Hit Ctrl + Space
- A list with language objects matching ‘free’ appears. One of them is FREE.ADJV.SYN and if you hover over it you will see it covers many synonyms of ‘free’ including ‘costless’.
- Now go ahead and replace ‘%FREE.ADJ.LEX’ with
%FREE.ADJV.SYNin your condition.
- Hit ‘Save’.
Your final condition should now look like this:
((%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL) / %WHAT_IS_THE_PRICE_OF.PHR / %FREE.ADJV.SYN) &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))
As you have noticed you can type words (and even parts of sentences) straight into the condition editor and ask Teneo for language objects matching those words. This auto-complete function is very handy if you have a bit of an idea which words you would like to cover and explore what is covered by the Teneo Language Resources. You can read more about it here: How to find language objects. Once you get a bit more experienced, you might know common language objects by heart and include them directly.