How your chatbot can learn to understand synonyms – in Teneo

In language, there are many ways of saying the same thing. That creates a need to optimize the language conditions for your bot so that it can give the correct answer even when other words are used. Here is how you do it in Teneo Studio.

We have earlier seen examples of how to semi-automatically create language conditions based on positive example inputs. For example, we created a syntax trigger that can handle conversations like the following:

User: How much does it cost to join the loyalty program?
Bot: You can join the loyalty program for free!

However, as soon as you slightly change the question, you will get a wrong answer

User: How much does it cost to join the rewards program?
Bot: I am sorry, can you re-phrase your question?

In the following, we will shorten the language condition while at the same time broaden its coverage. After having investigated what went wrong in the second part of the example conversation above, we improve the language condition by following two strategies:

  • Recombining language objects
  • Exchanging language objects

If you have not yet created the ‘User wants to know if loyalty program is free’ flow, we recommend you to build it before you proceed.

Having a closer look at the language condition

This is the language condition that has automatically been created from our positive examples:

(%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL &^ %REWARD.NN.LEX &^ %PROGRAM.NN.LEX)
    /
(%WHAT_IS_THE_PRICE_OF.PHR &^ %LOYALTY_PROGRAM.NN.SYN)
    /
(%REWARD.NN.LEX &^ %PROGRAM.NN.LEX &^ %FREE.ADJ.LEX)

The reason why the question “How much does it cost to join the rewards program?” did not trigger this flow is that ‘rewards program’ is not among the synonyms listed in the LOYALTY_PROGRAM.NN.SYN language object. We will thus in the following recombine the language objects so that the different phrases for asking about the pricing are combined with the different ways of expressing “loyalty program” or “rewards program”.

Recombination of language objects

We start by updating each line of the language condition to make it match on both ‘loyalty program’ and ‘rewards program’. We do this by replacing each occurrence of ‘loyalty program’ or ‘rewards program’ with ‘(%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))’, as illustrated below:

(%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX)))
    /
(%WHAT_IS_THE_PRICE_OF.PHR &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX)))
    /
((%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX)) &^ %FREE.ADJ.LEX)

You may have noticed that we replaced the &^ operator between REWARD.NN.LEX and PROGRAM.NN.LEX with the operator >>. This latter operator is called ‘directly followed by’ and does not allow for intervening words. For more information on the condition syntax language, go and have a look at the reference page.

Strictly speaking, our problem is now solved. However, we can go one step further. The first two lines of our condition look very similar, so let’s combine them:

(((%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL)/%WHAT_IS_THE_PRICE_OF.PHR) &^ (%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX)))
    /
((%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX)) &^ %FREE.ADJ.LEX)

Much better, but why not combine these two lines as well?

((%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL) / %WHAT_IS_THE_PRICE_OF.PHR / %FREE.ADJ.LEX) 
&^ 
(%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))

The &^ operator matches regardless of the word order in the user input. Thus, we can move FREE.ADJ.LEX to the first part of the condition without losing coverage.

That’s it. What a nice and powerful language condition we now have created!

Exchange language objects

We now have a much more robust language condition than before. However, there are still some inputs like ‘Is the rewards program costless?’ that are not yet covered by this condition. So let’s broaden the coverage of the condition a bit.

As we can see, the condition currently uses the lexical language object FREE.ADJ.LEX. Lexical objects only cover inflections of a word. Perhaps there is another language object we can use that covers synonyms. To do that you can try this:

  1. Select %FREE.ADJ.LEX and replace it with the text free
  2. Hit Ctrl + Space
  3. A list with language objects matching ‘free’ appears. One of them is FREE.ADJV.SYN and if you hover over it you will see it covers many synonyms of ‘free’ including ‘costless’.
  4. Now go ahead and replace ‘%FREE.ADJ.LEX’ with %FREE.ADJV.SYNin your condition.
  5. Hit ‘Save’.

Your final condition should now look like this:

((%SHOULD_I.PHR &^ %PAY_FOR.VB.MUL) / %WHAT_IS_THE_PRICE_OF.PHR / %FREE.ADJV.SYN) 
&^ 
(%LOYALTY_PROGRAM.NN.SYN / (%REWARD.NN.LEX>>%PROGRAM.NN.LEX))

As you have noticed you can type words (and even parts of sentences) straight into the condition editor and ask Teneo for language objects matching those words. This auto-complete function is very handy if you have a bit of an idea which words you would like to cover and explore what is covered by the Teneo Language Resources. You can read more about it here: How to find language objects. Once you get a bit more experienced, you might know common language objects by heart and include them directly.

Teneo Developers

Try Before You Buy.
Build. Deploy. Analyze.

Sign up to get your own developer sandbox of Teneo containing all the tools needed to build and manage advanced conversational solutions.

Get Started For Free

Author:
Gareth Walters is Vice President of Innovation and Community. Having moved into IT 20 years ago, he was responsible for Professional Services teams in several ERP, Supply Chain Management and Content Management software companies in a variety of sectors including Central Government, FMCG, Automotive and Pharmaceutical.

Leave a Reply

Your email address will not be published. Required fields are marked *

[Upcoming Webinar - April 7 at 11am CET]
[Upcoming Webinar - April 7 at 11am CET]