What is Natural Language Interaction?
Natural Language Interaction (NLI) is the convergence of a diverse set of natural language principles that enables people to interact with any connected device or service in a humanlike way.
Increasingly known as conversational AI, NLI allows technology to understand complex sentences, containing multiple pieces of information and more than one request. It can then react accordingly, creating value and enhancing the user experience.
Natural language solutions in Teneo
Interacting with people in a humanlike way is a complex task for machines. The subtle way we understand each other, regardless of how we express it, is extremely complex to recreate artificially—but a fundamental building block of conversational AI.
At Artificial Solutions, we use NLI to take human interaction with machines to the next level. Our award-winning conversational development and analytics platform – Teneo – allows enterprises to deliver a wide range of conversational AI applications such as virtual assistants, chatbots, and speech-based conversational UIs for smart devices.
To deliver an intelligent, humanlike experience, Teneo uses a number of Natural Language Processing (NLP) and Natural Language Understanding (NLU) principles and technologies that are embedded into NLI. We do so in such a way as to allow developers to leverage the linguistic capabilities of NLI, without having to develop any natural language functionality themselves, or indeed even knowing how they work on the inside.
Natural Language Processing
Natural Language Processing is used by NLI to split the input text into sentences and words, and to normalize and pre-process it. For example, NLP might convert all the words to lowercase or correct spelling errors, before determining if the word is an adjective or verb etc. and tagging this for future reference.
This input processor layer in Teneo is modular and therefore customizable, for example allowing developers to add their own input processors if required. As standard, Teneo’s input processors can carry out many tasks including:
- Normalization: a process that converts a list of words to a more uniform sequence (e.g. converting all words to lowercase).
- Tokenization & Sentence Splitting: the task of splitting the input text into its constituent parts (such as words, sentences, etc.).
- Spelling Correction & Misspelling Tolerance: correcting spelling.
- Part of Speech (POS) tagging & Morphological Analysis: determining what type of word each of the pieces of the input is (for example a noun or a verb) and tagging this for future reference.
- Sentiment & Intensity Detection: looking for the presence of words that indicate sentiment and/or intensity inside the pieces of the input.
Natural Language Understanding
Natural Language Understanding (NLU) encompasses the building blocks to interpret human language. Teneo has lexical resources that are pre-built and cover the most frequent terms, expressions, vocabulary and phrases for a specific language and domain.
They are the base upon which both general and domain/client/project-specific Language Objects such as lexicon, synonyms and themes, NLU rules and dialogue flows can be constructed in the context of each NLI solution.
Installed in the Teneo Platform, Lexical Resources can be easily selected for use with a simple click. Once assigned to the solution, they can be reused in different contexts seamlessly, with no difference to the language objects defined by users.
Furthermore, thanks to a combination of the Lexical Resources and AI algorithms for automatic NLU rule generation, language conditions to understand the many different ways a person might ask the same question can be extrapolated easily in Teneo with just few simple example questions.
Natural Language Generation
Responding to a query using anything more that pre-scripted responses requires at a minimum, natural-language generation (NLG). This enables NLI to interrogate data, including integrated back-end systems and third-party databases, and to use that information in creating a response, combined with incorporating additional parameters which may be known, for instance user name, gender, location, time of day, appropriate tense, etc.
Teneo uses this functionality to give a different response depending on additional conditions/context/variables etc. creating an enhanced, humanlike experience.
Teneo Language Resources
In the Introduction to the Special Issue on Cross-Language Algorithms and Applications it is noted that Multilingual access and processing pose novel challenges to the core Artificial Intelligence discipline of speech and natural language processing.
Teneo overcomes these challenges by adding another layer to Natural language interaction that enables it to handle multiple languages with ease. Teneo Language Resources are a set of pre-built, natural language understanding blocks that cover every possible way to express language and crucially enable users to quickly build their own natural language applications. We build them using machine learning techniques combined with the brain power of a group of the world’s finest computational linguists.
Today, Teneo is available in 35 languages. This means that in each of these languages, the platform can identify the language’s alphabet and characters, recognize words and sentences, is able to normalize, pre-process text, tolerate misspellings and abbreviations, and perform morphological analysis. You can read more about our multilingual capabilities in We speak your language.
How Natural Language Interaction works
Most solutions in the current market stay at the input processing level but using NLI technology, Teneo goes far beyond. It’s the brains behind the conversational application and the source of our artificial intelligence and linguistic capabilities.
It listens to and interprets what customers are asking, and applies sophisticated linguistic algorithms and advanced reasoning rules to figure out the most appropriate response by using the following three step process:
Analyze: Teneo accepts input from the user via speech, text, touch and gesture and analyzes it using a robust linguistic understanding library to comprehend and derive the meaning. At this stage, it also eliminates ambiguity. For example, if a customer has been asking about an order and then queries ‘when will it be delivered’, Teneo already knows that the user is referring to the order.
Reason: Advanced linguistic interpretation and business rules are then used to simulate ‘intelligent thinking’. This allows Teneo to reason like a human and determine the most appropriate way to react, taking into account contextual factors such as the day of the week, user location, information from previous dialogs and data about the user retrieved from back-office systems such as CRM systems.
React: Finally, Teneo performs the necessary actions to achieve what it has determined as the most appropriate action. This might be giving a verbal or textual response, to ask for more information, open a webpage, play a video, open another app, automatically fill in a form or execute a transaction on backend or third-party systems.
These three steps all happen seamlessly in milliseconds, with Teneo able to handle thousands of interactions simultaneously.
Why syntax, spelling and semantics matter
Natural language interaction technology takes natural language processing (NLP) and natural language understanding (NLU) to the next level. It allows enterprises to create advanced dialogue systems that utilise memory, personal preferences and contextual understanding to deliver a proactive natural language interface.
These advanced natural language processing applications remove the need for the customer to constantly repeat information during a conversation. The NLI interface can ask clarifying questions if there is any ambiguity. It also enables for the conversation to be interrupted by the user. A common situation when talking to another human, but one even Intelligent Virtual Assistants can rarely handle. Not only that, natural language interaction can also bring the user back on track to the point of the original conversation.
In order to do this, NLI must understand exactly what the user means. This is a complex task. The way we talk in every day conversation is full of subtle nuances. “I had a nice vacation in nice” for example shows how a machine must understand grammar, syntax and spelling mistakes. But what about idioms, slang, SMS shorthand and even dialects?
To tackle semantics, that is to say the meaning of words, Teneo uses language objects to detect the meaning of a user input. These language objects test the input for its content by looking for words and word combinations. It can even exclude words by looking for special word forms, stems or particular spellings.
Teneo is also capable of detecting sentiment. It does so by analyzing the conversation in conjunction with other factors. These are then used for tagging and for defining specific actions in the solution. By keeping a running total of sentiment hits, it is possible to register and respond to trends. For example, if a user is especially negative, this could trigger an intelligent digital assistant to offer to hand over to a live agent.
How NLI reveals the hidden value in unstructured data
But NLI isn’t just great for creating humanlike conversations, it’s also great for analyzing those conversations too.
When people communicate in a natural, conversational way, they reveal more than just the words they’re saying. Their individual preferences, views, opinions, feelings, inclinations and more are all part of the conversation. This information is one of the reasons that makes the data collated during human-machine conversations so valuable.
However, enterprises frequently rely on their own, often prejudiced, interpretation of data, simply because they don’t have the necessary resources, or retrieving the relevant information takes too long.
Teneo Data uses Natural language interaction to mine the immense volumes of conversational data and unlock the knowledge held in sources such as virtual assistants, live chat, call transcripts and emails. Access to this data significantly increases insight into customer preferences and decision making, allowing companies to fully personalize the way they interact with each individual consumer.
But conversational data isn’t like typical data sets with neat rows and columns. It’s unstructured data and must be interpreted within its proper context before it can be turned into actionable information. According to Gartner by 2019, natural-language generation will be a standard feature of 90% of modern BI and analytics platforms.
However, most conversational analytic tools interpret the data simply as words; without context, without meaning, and without frequencies. This results in information overload. It’s a flat view of the data where nothing stands out as more important.
Teneo Data uses Natural language interaction to unlock the meaning in the conversations, understand the voice of the customer, and to improve the conversational AI application.
For example, an analysis of Carry Me Airlines conversational data, a fictitious name for an airline but based on real data, showed that questions about baggage are one of the more frequent topics, but when we drill down it’s possible to see that customers use “baggage” and “luggage” differently. Luggage is much more likely to refer to carry-on bags. This type of information is tremendously useful when building an NLI app that is sensitive to the expectations of customers.
In another example, the stemming feature within Teneo groups together all grammatical variants of the same word, in this case the word “book”. What is it that people book online? Intuitively, we’d think of booking flights, because that’s what airlines are all about and because there’s such a strong bond between those two words in everyday speech. And there are a lot of inputs about booking flights.
But this is where analysis on unstructured data using NLI comes into its own, because human intuitions about conversational data are often wrong. Teneo showed that “book” is actually most frequently used about seat reservations. Businesses need the facts that NLI provides to guide them, otherwise enterprises risk misunderstanding the voice of the customer.
Analysing the conversational data also allows businesses to look for what customers don’t like about a service or product by bringing together concepts such as can’t, don’t, not etc. In Carry Me’s case one of the main issues was around printing boarding passes. A simple revision to the website solved this issue.
Natural language interaction enables faster creation of conversational AI applications
From simple beginnings, natural language applications such as digital employees have grown to become the gateway to customer contact centers, the number one sales employee, the executive PA that updates corporate systems and the business analyst insights on markets and trends in real-time.
Natural language interaction removes the need for your customers to know and understand your terminology. It’s clever enough to figure out in over 35 languages what someone means when they use their everyday words and phrases, not yours.
While speech is a distinctive feature, it is how well conversational AI applications understand the complex sentences people use such as “Schedule a meeting with George for Thursday at 11, we’re meeting downtown” and more importantly, how accurately it responds that’s key to their effectiveness.
The deep understanding that Natural language interaction delivers gives enterprises the information they need to deliver a superior customer experience and have a positive impact on their bottom line. The fact that it automates the code that simulates the way a human thinks and makes creating a conversational AI application faster and easier than ever before is just an added bonus.