Try Out Your New Conversational AI Solution in Teneo Before Launch
Teneo offers you extensive possibilities to test and debug your solution before going live with changes and additions. The panels on the right in the main solution view work together to make this happen:
- Try out: this is the chat panel where you simulate user inputs and see the bot response.
- Inputs: this is the panel where you simulate input parameters to your bot.
- Response Info: this panel displays all information associated with each stage of the transaction.
- Engine: usually not needed: session details, including debug messages (printlns added to scripts).
You can open these panels via mouseclick as follows:
Before using Try Out be sure you:
- Save the relevant flow or items you were working on (press Save All in the Solution window to be on the safe side).
- If Try out shows a Restart now or Reload now notification, press the given button.
- If Try out shows model training started wait for it to show model training completed.
- Be sure you are testing with the right input.
Try out Panel
Try out is the most important panel when testing your solution. It is where you enter the inputs you want to test, and see the response that is returned. From here you can also navigate via mouseclick directly into the trigger, flow, output and any other compontents invovled with the answer that was given. You can mouseover the icons associated with the response until you find the right component to examine further. Take a closer look at Try out to acquaint yourself with the various panels:
Let’s zoom in on the icons shown under the bot response. Here we notice the following sequence of events: flow triggered -> flow raised -> output given -> 2nd output given -> flow dropped:
If you mousever one of the icons you see a preview of the element, and can also click your way directly into the flow at that location.
In this video we test an input, notice a typo in the output, then jump from Try out to the the flow to make the correction, and finally test what we’ve done:
Some Tryout tips:
- To open a flow in edit mode via tryout, use alt+mouseclick.
- To re-try a user input you already entered, click it with the mouse and it will appear automatically in the input field.
- If the Tryout panel becomes cluttered you can clear out old content via the option Clear All from the context menu (right mousekey).
Restart / Reload
As the name suggests, clicking the ‘Restart’ button will restart the session, i.e. active flows are ended and the flow stack is cleared. If you have made changes to a flow, these will become active by clicking ‘Restart’.
When clicking on the little black triangle next to ‘Restart’, you can open a drop down where the option ‘Reload Engine’ can be selected. In contrast to ‘Restart’ which only ends active flows , ‘Reload Engine’ will reload the entire solution. This includes ending all active flows, but also re-setting global variables and scripted contexts.
Sometimes Teneo detects that an engine reload is neccessary, for example, when you added or modified global variables, scripts, listeners or the trigger ordering. In that case the option will be made directly available to you:
If you happen to see the message Model training started above the restart button you should wait for Model training completed to appear before actually testing. This assures that the current training data (including all modifications you might have made) are taken into account when trying inputs. It will look like this:
Finding a Flagged Error
Script errors during processing, missing language object references or stuck flow warnings appear as visual notifications in the Tryout panel. The full information can then be found in the panels Response Info. To demonstrate this, we’ve taken a flow User wants bot suggest a coffee and built in some design flaws, as well as a faulty flow listener and bad script commands. This is what happens when we try to invoke the flow in Try out:
For every trigger or node (vertex) mentioned in the error message, you’ll see an associated ID you can click to directly open the flow in edit mode at that exact location.
Response Info Panel
This panel shows all information associated with a specific transaction, including a complete trace of global and flow variables as well as results of listeners, and other solution elements. It is also the place where you see warnings/error messages that occurred during the transaction. The information is divided into several sections which can be expanded or collapsed, as needed:
The section called Intent Classification shows the top intent along with its confidence level. It will optionally show up to four other intents with somewhat nearby confidence levels. This is useful in learning which class triggers in your solution are most likely to be relevant to a certain input.
In this example, we show an input that correctly runs into the safety net. In the Intent Classification section you see the two intents that were annotated with the highest confidence levels. If your input is supposed to go to one of those class triggers, you could click into the item directly and add additional training data to make it work better.
The Input Summary section shows the results of the Input Processor: which language was detected, which parts of speech, which named entities as well as the top intent.
It is also the place where you can discover that typos in the original input were corrected. In this particular example we can see that the input word ‘esspresso’ was corrected to ‘espresso.’ We can also see that the language English was detected, and that ‘espresso’ additionally was classed as a product entity.
For additional information about confidence levels and how typos were corrected you should check the detailed Input Processor result in the Processing Path.
In Answer details we can see the flows, triggers and outputs behind the response that was given. In our example here we see that the class trigger Can I order an espresso in the flow User wants to order a coffee was activated. The empty entry for ”(Prompt for coffee type) means that this prompt was skipped (a skip condition), as we already had that information. We also see that the output given was a prompt for the size.
Global Variables and Watched Variables
The two sections for variables work together. In Global Variables you see a complete list of global variables defined in the solution along with their values following completion of the transaction. Since this list can be rather long, Teneo offers a way to track specific variables, making it much easier for you to find them.
If you wish to follow a global variable click the eye icon next to the variable name in the Global Variables section. The variables you selected are then displayed separately in the Watched Variables part of the panel.
If no variables are currently followed, the Watched Variables section is not shown. Note that you can watch flow variables here as well.
In the Flow Variables section of Response Info you can see the values of any flow variables involved in the transaction. You can watch flow variables in the same way that you watch global variables.
In the example shown here we want to trace the type and size as they are set during the process of ordering a coffee.
The section for ‘Processing Path’ in ‘Response Info’ can be quite long, but it is also the most important in finding out exact details about the transaction. It shows you exactly the steps taken from the time the user hits enter to the point where the bot returns its response. This includes:
- Global or flow scripts that ran and the variables they changed.
- Input processor results, including topintent, and which words were spell-corrected.
- Global and flow listeners that triggered and the variables they changed.
- Which trigger responded to the input.
- Used words (if a syntax condition was involved)
- Outputs that were skipped.
- Outputs used to construct the answer.
- Flows that were raised, paused or dropped.
- Any errors or warnings received.
The information presented here can be extremely helpful when troubleshooting solution issues. For example, you can examine the input processor results to see exactly how suspected typos were corrected into words that the solution knows.
The Engine panel gives you details about the entire session. It repeats error and warning information seen during individual transactions but also offers you a helpful option for debugging your solution script. If you include println() calls at script locations, their output will be logged in the Engine panel.
When you start Try out, the first thing that happens is the Try out engine is started. You will see status Engine: Starting which should then be followed by the status Engine: Online. If the Engine cannot start, please check with your system administrator about a solution.
If the Engine panel becomes cluttered you can clear out old content via the option Clear All from the context menu behind the right mousekey.
The Inputs panel allows you to simulate sending parameters to your bot, which is useful if you want to simulate external context that you do not have available in Tryout.
All parameters are considered to be of type ‘string’. To add a parameter, fill out the name and the value. These will be sent to Teneo for that active transaction. If the values should change during the course of the conversation you can do that here too.
The page How to store input parameters shows an example working with input parameters.