How a Machine Recognizes the Unknown

Conversational AI Machine Learning Linguistics

Discover the current use of voice assistants by consumers and the implications and likely changes in the near future.

As the saying goes, if it looks like a duck, swims like a duck and quacks like a duck. Then it’s probably a duck. But what does a machine do when it’s never seen one?

Think about how you might recognize the difference between a picture of a cow and a chicken. The first and most obvious characteristic might be that one has four legs, while the other has only two. This is the sort of feature a machine can use to tell one thing, from another. Using this information the machine learns how to recognize a cow from a chicken and creates a model to use next time.

So what happens when you show the machine a picture of a goat? Yes, you’ve guessed it. It thinks it is a cow. It does have four legs after all.

To fix this problem, you need another differentiator to train the machine. In this case, size might be an appropriate feature. So the machine learns that a chicken is small, a goat medium sized and a cow is large. With this information the machine creates a new model on which to base its calculations.

Then you show it a picture of an elephant…okay, I’ll stop. We could go on forever like this!

When a machine doesn’t recognize what’s on the picture is has to predict using the model it previously learnt. Teneo removes the complexity of building complicated models to recognize “cows and goats”.

One word can make a difference to conversational understanding

At Artificial Solutions we use natural language instead of pictures. Suppose your business was an airline. First you would need to feed the system with data. But instead of a few pictures of cows, goats and chickens, you would input examples about booking a flight, canceling a ticket, hand baggage weight etc. Teneo will learn from this information by looking for features that are different between the examples. Not just the words, but the sentence constructs such as booking a flight and reading a book on a flight.

To a human, the meaning is obvious. To a machine, not so much. It needs to learn that book is a verb in one sentence and a noun in the other, and that’s just for starters. Move on to more nuanced conversations such as I cancelled my flight and you cancelled my flight and most conversational tools will fail. There’s a fifty/fifty chance of it getting it right and a one hundred percent certainty of annoying your customer when you get it wrong—particularly when misunderstanding just one word is the difference between them getting a refund or not

This is why we use a variety of technologies, algorithms and our vast corpus of conversational data gathered over the last fifteen years so that Teneo knows what a cow is, can learn the difference between it and a goat, and predict correctly when it’s probably an elephant, with minimal developer intervention.

Teneo does more than recognizing “parts of speech”, the verbs, nouns etc. It delivers multilayered understanding that recognizes the morphology of the words, how they are used and their relationship to other words. So when someone says “I travelled to Birmingham”, Teneo understands that this was in the past. Other conversational development tools can’t implicitly understand this “out of the box”.

Read how Shell achieved a 40% reduction in call volume to live agents by answering 97% of questions correctly and resolving 74% of digital conversations.

Advanced language resources

We’ve already mapped out the structure of language itself and the relationship between words, phrases, sentences, synonyms, lexical entities, concepts. Known as the Teneo Language Resources (TLRs), they are crucial in enabling developers to rapidly build conversational AI applications.

Developed using machine learning techniques by some of the finest minds in computational linguists, the TLRs allow enterprises to teach new conversational applications all the possible language permutations in a matter of moments. The user simply enters a few representative queries, and the TLRs will enable the application to learn all the different ways a user might ask the same exact question.

Because Teneo is available in 35 languages, the TLRs enable the application to ‘think’ in your native tongue, while delivering the same linguistic sophistication across every other language required.

The advanced conversational ability of Teneo allows the interaction between user and applications to be more natural and humanlike, which in turn increases engagement. It also allows Teneo to predict more accurately what the user means when it come across something it hasn’t learned. It then lists these, along with the recommended dialogue flows it thinks they should be assigned to for a human developer to double check. In this way, Teneo helps to automate the optimization of the system, which in turn lowers maintenance costs.

A hybrid approach to conversational AI keeps on learning

But what happens if the machine can’t tell the difference between a cow and a goat, no matter how many pictures you show it? This is one of the constraints of a purely machine learning system. Adding more data may help if it just needs a little more information to clarify a point, but if for some reason it keeps on confusing the two, then you’re stuck.

Eventually machine learning will reach a point where no amount of extra data will make a difference to the accuracy.

Teneo on the other hand allows developers to easily intervene. Our technology uses a hybrid approach, combining linguistic and machine learning at a native level. We do this for several reasons such as to control the system so that it behaves in the manner an enterprises expects. Tay’s racist outburst might have brought a few wry grins to Microsoft’s detractors, but it didn’t really hurt the brand as it was just seen as an experiment. A customer service chatbot going on a rampage might not be viewed in the same light.

Where other purely machine learning systems stop improving, it’s Teneo’s linguistic abilities that allows developers to teach the system directly the correct response and then enable the machine learning to continue refining the performance.

It’s becoming increasingly apparent that customers are looking for a more humanlike, natural experience. They want to converse using their own words, phrases and terminology. Simple chatbots that demand formulaic, command style responses are no longer enough. But developing an application capable of understanding the complex way us humans speak is fraught with challenges.

It these challenges that Teneo addresses, and makes it easier for enterprises to build conversational AI applications that understand what the customer wants, and always delivers the right response.

Chatbots: The Definitive Guide (2020)

Everything you ever wanted
to know about Chatbots

Learn everything from chatbot fundamentals to AI chatbots, statistics and what we can expect in the future.

Read the Guide