Simple Sample of the Watson Dialog Service in Bluemix

Update November 2016: The dialog service has been deprecated and replaced with the Conversation service.

With the IBM Watson Dialog service on Bluemix developers can build applications using natural language to automatically respond to user questions, for example functionality to help users to reset their passwords. Below is a simple sample demonstrating how this service works.

Via the dialog users can enter a topic they are interested in to see recent tweets with a positive or negative sentiment. The tweets are received via the Insights for Twitter service.

dialogsimple

The dialog is defined via XML. There is a good tutorial describing how this works. In my sample I use two variables to store the input values from users. Via concepts you can also define synonyms and you can use wildcards to react to different types of user input.

<?xml version="1.0" encoding="UTF-8"?>
<dialog xsi:noNamespaceSchemaLocation="WatsonDialogDocument_1.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <flow>
        <folder label="Main">
            <output>
                <prompt selectionType="RANDOM">
                    <item>Hi, I'll show you the latest buzz around a topic of your choice. What topic are you interested in?</item>
                </prompt>
                <goto ref="getUserInput_2442994"/>
            </output>
            <output>
                <prompt selectionType="RANDOM">
                    <item>Bye</item>
                </prompt>
                <getUserInput id="getUserInput_2442994">
                    <search ref="folder_2442998"/>
                </getUserInput>
            </output>
        </folder>
        <folder label="Library">
            <folder label="Live Content" id="folder_2447777">
                <output>
                    <prompt selectionType="RANDOM">
                        <item>Alright. Open this URL to see the tweets: http://insights-search.mybluemix.net/api/1/messages/search?q={Topic}%20AND%20posted%3A2015-07-01%20AND%20sentiment%3A{Sentiment}</item>
                    </prompt>
                </output>
            </folder>
            <folder label="Live Content" id="folder_2442998">
                <input>
                    <grammar>
                        <item>*</item>
                    </grammar>
                    <action varName="Topic" operator="SET_TO_USER_INPUT"/>
                    <output>
                        <prompt selectionType="SEQUENTIAL">
                            <item>Are you interested in positive or negative tweets?</item>
                        </prompt>
                            <getUserInput>
                                <input>
                                    <grammar>
                                        <item>positive</item>
                                    </grammar>
                                    <action varName="Sentiment" operator="SET_TO">positive</action>
                                    <goto ref="folder_2447777"/>
                                </input>
                                <input>
                                    <grammar>
                                        <item>negative</item>
                                    </grammar>
                                    <action varName="Sentiment" operator="SET_TO">negative</action>
                                    <goto ref="folder_2447777"/>
                                </input>
                                <input>
                                    <grammar>
                                        <item>*</item>
                                    </grammar>
                                    <action varName="Sentiment" operator="SET_TO">nothing</action>
                                    <goto ref="folder_2442998"/>
                                </input>
                            </getUserInput>
                    </output>
                </input>
            </folder>
            <folder label="Storage"/>
        </folder>
        <folder label="Global"/>
        <folder label="Concepts">
            <concept>
                <grammar>
                    <item>positive</item>
                    <item>good</item>
                </grammar>
            </concept>
        </folder>
    </flow>
    <entities>
    </entities>
    <constants>
    </constants>
    <variables>
        <var_folder name="Home">
            <var name="Topic" type="TEXT"/>
            <var name="Sentiment" type="TEXT"/>
        </var_folder>
    </variables>
    <settings>
    </settings>
    <specialSettings>
    </specialSettings>
</dialog>

In order to test your dialog there is a dialog tool that can be run either locally or deployed to Bluemix. The tool helps testing the dialog and it also discovers syntax errors. To deploy your dialog you can use the Java or Node.js sample applications.

To find out more read the documentation, the API documentation or try the online pizza ordering sample.

In the simple sample I only display the URL that shows the tweets as JSON. There is a more advanced sample that finds certain movies based on user input. The movies are queried via a REST API provided by themoviedb.org. In order to do this the dialog service returns JSON with all filters to a Java application.

<prompt selectionType="RANDOM">
   <item>"{Search_Now:"{Search_Now}", Recency:"{Recency_Preference}", Rating:"{Certification_Preference}", Genre:"{Genre_Preference}", Index:"{Current_Index}", Page:"{Page}"}"</item>
</prompt>

When the dialog service returns this JSON to the Java application, the movies are queried and returned to the user.

Conversation conversation = dialogService.converse(converseParams);
wdsMessage = StringUtils.join(conversation.getResponse(), " ");
processedText = matchSearchNowPattern(wdsMessage);
...
SearchTheMovieDbProxyResource tmdb = new SearchTheMovieDbProxyResource();
conversationPayload = tmdb.discoverMovies(UtilityFunctions.getPropValue(paramsObj, "Genre"),
   UtilityFunctions.getPropValue(paramsObj, "Rating"),
   UtilityFunctions.getPropValue(paramsObj, "Recency"),
   currentIndex, pageNum, nextSearch || newSearch);
  • Sameh Yassin

    Great article. I have two questions:
    1) Is the dialog xml file executed sequentially as long as there is no ‘goto’?
    2) Will the dialog xml help the dialog service to learn new question answers or the service will always return values from the initial xml file?

    Thanks..

    • Dan O’Connor

      Hi Sameh.

      #1
      The dialog nodes will be executed sequentially so long as a go to is not encountered, and so long as any conditions relating to the next node/folder (i.e. the ‘if’ tag) are satisfied (evaluate to true).
      Obviously tags like ‘random’ etc will cause the order to change.

      #2
      The service always returns responses from the dialog xml file. There is no ‘corpus’ associated with the dialog service, so there is no place for the service to figure out ‘new’ answers/responses. However, you can always add new concepts over time which basically expand the system to handle more inputs.. For instance, in an insurance application the dialog might be trained how to respond to “How do I purchase car insurance”.. Through the use of concepts you can train the system to understand for instance that
      automobile=auto=car
      buy=purchase
      etc..
      This way further variations of the initial input are accepted by the system.. “How do I buy auto insurance”
      etc..
      Hope this helps!

      • Sameh Yassin

        Hi Dan, Thanks for your reply, it helped me. However, I’m a bit disappointed because I expected more than a large xml file. I thought that dialog service would truly learn from user interactions, and be smarter. I’ll try to use natural language classifier service with the dialog service. I hope this would make the dialog don’t look like an automated robotic reply :-)

  • Joseph Richardson

    How do we connect the Dialog service to a Watson Ecosystem instance or is this service making a big xml file with all the questions and answers in them?

    • Dan O’Connor

      Hi Joseph.. I am not familiar enough with your Watson environment to comment definitively, but I think you may be able to have dialog enabled within your Watson Ecosystem instance. You should talk to your Ecosystem representative to see if it is an option.
      Feel free to post back here if you need further help.

      • Joseph Richardson

        We found out that we will not be needing a Watson instance anymore. After going through the documentation for the Dialog service and finding that it accepts JSON answer units can be uploaded, we used the Document conversion service to change our Word documents to JSON answer units, when trying to upload the JSON answer units we consistantly get this error:

        { code: 400,

        error: ‘Failed to import file. Possibly due to corrupt or invalid file or system error. – com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized field “source_document_id” (class com.ibm.watson.wds.docformat.hierarchy.Chatflow), not marked as ignorable (13 known properties: “entities”, “constFolders”, “hitNodeRules”, “translations”, “settings”, “stopwords”, “profileValueRules”, “nodes”, “dictionaries”, “version”, “specialSettings”, “varFolders”, “reports”])n at [Source: java.io.InputStreamReader@da94a81f; line: 2, column: 26] (through reference chain: com.ibm.watson.wds.docformat.hierarchy.Chatflow[“source_document_id”])’ }

        Do you know how to create the necessary xml files from a word document for the dialog service?

  • Dan O’Connor

    Hi Balabhadra.

    The recommended approach here is to handle this at your app level. The generally recommended architecture in this case is one we refer to on the Watson team as the ‘proxy’ approach, whereby your web app acts as a proxy between the UI and the Watson service.
    Dialog has the concept of ‘profile variables’ (context variables). At any point in the conversation the dialog chat flow can be programed to set a context variable.. e.g. in a banking app..
    User> What is my checking account balance?
    At this point in the chat flow you could set two profile variables ‘check_balance=true’, ‘account=checking’..
    The application (proxy) layer in your app will receive a response from dialog which looks like:
    Watson> Your checking account balance is %s
    The idea here is that the proxy would check profile variables after each call, or alternatively you could include an indicator in the response that a profile lookup is required.. i.e. we have a couple customers who return meta data with each response
    {
    responses:[“Your checking account balance is %s”, “truechecking”]
    }
    In these cases the proxy is checking the response payload for each request, and once some indicator is identified (either by checking the profile variable, or the response payload) then the proxy knows that a server side action is required, e.g. look up db to get user’s account balance. The result of the lookup may require subsequent requests to dialog before the actual response is returned to the user.

    The dialog service is not able to make outbound requests as this poses a security risk, instead the approach is that the application should be aware of the state of the conversation, and if indicated by the dialog the app should act on Dialog’s behalf.

    I hope this answers your question?

    • Dan O’Connor

      Unfortunately the editor saved my json response with invalid content (extra “).. In any case I hope you get the idea.

    • Balabhadra

      thanks Dan..

  • DEBABRATA HALDER

    Hi Niklas,
    Can we share this url as a link ?

  • ruchi

    HI Niklas,

    Can we also display reports on chat box like pie charts ?