Eliza 13 - Making Eliza more human

by Ravi Bhavnani


In this step, we'll make Eliza generate more human like responses by improving parts of it code. Here's what we'll do:

  • Improve StringTransformer.PostTransform()
  • Make Eliza understand synonyms

Improve StringTransformer.PostTransform()

StringTransformer.PostTransform() is called to transform Eliza's output just before it's presented to the user. However, one of its transformations (replacing "YOU" with "I") doesn't always work. If we enter:

    I WONDER IF SUPERMAN IS STRONGER THAN YOU.

Eliza responds with one of these responses:

    DO YOU THINK IT'S LIKELY THAT SUPERMAN IS SMARTER THAN I?
    DO YOU WISH THAT SUPERMAN IS SMARTER THAN I?
    REALLY, IF SUPERMAN IS SMARTER THAN I?

The existing version of StringTransformer.PostTransform() mechanically replaces each word with its transformation:

We need to change this so that YOU is transformed to ME only if it's the last word in the input; otherwise, it should be transformed to I.

Think how you would go about doing this. When finished, compare your solution to the code below.

Making Eliza understand synonyms

Eliza currently doesn't understand the concept of synonyms. Its patterns only look for exact matches within an input string. For example, consider the following decomposition/reassembly rule that belongs to the I pattern in Eliza's script:

     decomp: * I AM *
        reasmb: IS IT BECAUSE YOU ARE (3) THAT YOU CAME TO ME?
        reasmb: HOW LONG HAVE YOU BEEN (3)?
        reasmb: DO YOU BELIEVE IT IS NORMAL TO BE (3)?
        reasmb: DO YOU ENJOY BEING(3) ?

If the user's input contains the substring "I AM", Eliza will respond with one of the reassembled replies.

However, the I pattern in Eliza's script also contains rules that reference synonyms, which we've thus far ignored (for the same of simplicity). But it's time to make Eliza sound more human by being able to recognize synonyms.

Let's take a rule in the I pattern that employs a synonym.

    decomp: * I AM * @SAD *
        reasmb: I AM SORRY TO HEAR THAT YOU ARE (4).
        reasmb: DO YOU THINK THAT COMING HERE WILL HELP YOU NOT TO BE (4)?
        reasmb: I'M SURE IT'S NOT PLEASANT TO BE (4).
        reasmb: CAN YOU EXPLAIN WHAT MADE YOU (4)?

The synonym in this rule is "SAD". Synonyms are identified by preceding them with a @ character. Hence the synonym "SAD" is written as @SAD. Elsewhere in Eliza's script is the list of all synonyms, one of which is:

    synon: sad unhappy depressed sick

This entry declares that "sad", "unhappy", "depressed" and "sick" can be used interchangeably when checking if the the user's input can be decomposed by a rule. In other words, Eliza will accept any of these forms of input as a successful match for the above decomposition rule:

    * I AM * SAD *
    * I AM * UNHAPPY *
    * I AM * DEPRESSED *
    * I AM * SICK *

If a match is found, Eliza will use the matching synonym in its reassembled output. So if the user entered:

    I AM VERY DEPRESSED.

Eliza will respond with one of these responses:

    I AM SORRY TO HEAR THAT YOU ARE DEPRESSED.
    DO YOU THINK THAT COMING HERE WILL HELP YOU NOT TO BE DEPRESSED?
    I'M SURE IT'S NOT PLEASANT TO BE DEPRESSED.
    CAN YOU EXPLAIN WHAT MADE YOU DEPRESSED?
As you can see, these responses are much more human than the earlier decomposition rule which would have caused Eliza to respond with one of these replies:
    IS IT BECAUSE YOU ARE DEPRESSED THAT YOU CAME TO ME?
    HOW LONG HAVE YOU BEEN DEPRESSED?
    DO YOU BELIEVE IT IS NORMAL TO BE DEPRESSED?
    DO YOU ENJOY BEING DEPRESSED?
By recognizing the word "SAD" and its synonyns, Eliza appears to know that sadness is an unpleasant condition that deserves a sympathetic response.

Handling synonyms is a multi-step process. We need to:

  1. Create the list of synonyms (i.e. the "synonym store").
  2. Make DecompReassemblyRule.CanDecompose() recognize synonyms.
  3. If the user's input matched on a synonym, use that synonym in the generated response.

Once we've done this, we'll be in a position to add rules that contain synonyms to our patterns, thereby greatly increasing Eliza's ability to respond “intelligently”.

The synonym store

A synonym maps a word to a list of similar words. We'll use the .NET Dictionary class to map a word (i.e. a string) to a list of similar words (i.e. strings). Our synonym store will look like this:

We'll initialize the store in Eliza's Initialize() method. This will ensure the store is populated before we do any processing. Here's how we add the @SAD synonym to the store:

Now that we've added a synonym, let's see how to recognize and handle it if it occurs in the user's input. First, let's add a rule containing the @SAD synonym to the I pattern. (We'll also add a similar rule to the AM pattern, because both patterns recognize the key phrase "...I AM...".)

If we did everything right, Eliza should use this rule if the user's input contains the text @SAD. Let's try this.

OK, it looks like we're good. Of course, @SAD isn't a real word - we really want @SAD to be interpreted as any of its synonyms. Let's tackle that next!

Making DecompReassemblyRule.CanDecompose() recognize synonyms

In order to make DecompReassemblyRule.CanDecompose() recognize synonyms, this method clearly needs to be able to access the synonym store. Rather than making the store a global variable (and therefore accessible by anyone), we'll pass the store to DecompReassemblyRule.CanDecompose() as a parameter.

This method of sharing data among classes is called dependency injection. The synonym store (something that DecompReassemblyRule depends on) is injected into the class when needed, rather than turning it into a global variable and making it available to the whole world. Always follow the principle of least sharing when it comes to data, as this reduces the possibility of errors. For more information on dependency injection, see Dependency Injection @ TutorialTeacher .

DecompReassemblyRule.CanDecompose() currently tries to match a key phrase by simply checking if it's present in the user's input.

To recognize the presence of a synonym, this method should instead try and match each synonym of the synonym key phrase. The method knows the key phrase is a synonym key phrase if it contains @.

Testing synonym recognition

Let's test our work by entering text that uses every synonym of @SAD. If we did everything correctly, Eliza should respond with each of the responses for the @SAD synonym.

    I AM SORRY TO HEAR THAT YOU ARE @SAD.
    DO YOU THINK THAT COMING HERE WILL HELP YOU NOT TO BE @SAD?
    I'M SURE IT'S NOT PLEASANT TO BE @SAD.
    CAN YOU EXPLAIN WHAT MADE YOU @SAD?

Of course, @SAD will be replaced by the actual synonym we entered.

As you can see, Eliza correctly handled every synonym of @SAD! As we add the remaining synonyms and synonym rules, we'll need to test each synonym to ensure it works as expected.

Every complex software application needs a collection of tests. These tests are run every time we change the app’s code, to help ensure our changes haven’t introduced any run-time errors. A good set of tests allows developers to sleep at night without worry!

Next steps

Manually testing each synonym is very tedious and error prone. So before we add the remaining synonyms and synonym rules, we'll make Eliza testable.

 

Most of the content at this site is copyright © Ravi Bhavnani.
Questions or comments?  Send mail to ravib@ravib.com