In this step, we'll make Eliza testable. Or to be precise, we'll make Eliza's patterns testable. We're going to do this because we want an automated way of ensuring that any new code we write (specifically the new synonyms we're going to add) doesn't break existing functionality. If Eliza was a small app, we could test it manually. But in its current form (and like all real complex apps), we need an automated way to ensure we don't introduce bugs as a result of adding new code.

We'll modify our Eliza Visual Studio solution so that instead of containing a single project, it will contain three projects:

1. A library containing the Eliza class we've built over the last 13 steps.
2. A console application (we'll call it ElizaMain) that allows a user to converse with Eliza (like it did in Step 13).
3. A tester console application (we'll call it ElizaTest) that gives a pattern specific input and checks whether it returns the expected response.

After performing steps 1 and 2, Eliza will operate exactly as it did in the previous step. The value of this exercise is really in step 3, as we'll soon see.

If you'd like to know more about Visual Studio solutions and projects, see this article:

• Solutions and projects in Visual Studio

Splitting the solution into multiple projects

Our original Visual Studio solution looked like this:

After separating the Eliza engine from the console main program, our solution looks like this:

The original Eliza project now contains only the Eliza engine. This project is of type "class library" instead of "console application". When you build this project, Visual Studio will create a dynamic link library (or .dll) file. A .dll cannot be run. Instead, it contains classes that can be used by other programs (and other .dll's).

The ElizaMain project is a small console application that makes use of Eliza. If you expand the ElizaMain's "References" node in the Solution Explorer, you'll notice we added a reference to the Eliza project. This is how we declare that the application ElizaMain depends on (or "references") classes in the Eliza project.

When you run ElizaMain, you'll notice it behaves exactly like our Eliza of Step 13. This should come as no suprise, because we haven't made any code changes to Eliza's internals. We simply separated the original app into a smaller app (ElizaMain.exe) and the Eliza engine library (Eliza.dll).

Adding the tester program

Now we'll add the tester console application project, ElizaTest, to our solution. Like ElizaMain, ElizaTest will contain a reference to the Eliza project so it can use the Eliza class.

However, instead of allowing a user to chat with Eliza, ElizaTest will give a pattern a pre-determined string and check whether it returns the expected response. If the expected response is received, the test will be deemed to have passed. Otherwise, the test will have failed. We'll use the .NET Debug.Assert() method to assert (check) if the result returned by GetResponse() is the expected one.

ElizaTest will display the name of each test as it runs. If a test fails, Debug.Assert() will display a message box that shows the stack trace and the point of failure. If all tests succeed, ElizaTest will display a summary at the end of its execution.

Let's add the console application project ElizaTest to our solution. Notice the reference to the Eliza project.

ElizaTest's default Main() method doesn't do anything useful.

So let's add our first test.

Writing our first test

Let's begin by testing the "AM" pattern. We know if we give this pattern the input:

    I am sad.

    I AM SORRY TO HEAR YOU ARE SAD.

So let's test this by doing the following:

1. Create an instance of AmPattern.
2. Call its GetResponse() method with the string "I am sad.".
3. Verify that the response is "I AM SORRY TO HEAR YOU ARE SAD.".

In order to allow ElizaTest to create an instance of AmPattern, there are a few things we need to change in Eliza:

1. Make the classes AmPattern, ComplexPattern, Pattern and DecompReassemblyRule public. This allows AmPattern to be used by code that resides in an assembly other than Eliza.
    
   
2. Make Eliza's collection of synonyms accessible by code that resides in an assembly other than Eliza. This allows the method AmPattern.GenerateResponse() (which requires the list of synonyms as a parameter) to be called by external code. We'll do this by turning Eliza's private member _synonyms into the public property Synonyms.

After making these changes, we can write our first test. Here's what it looks like:

Running ElizaTest produces the following output, indicating our test passed.

If the test failed, we'd see something like this:

The "Assertion Failed" window is displayed by Visual Studio whenever a call to Debug.Assert() fails. This also causes the program to pause execution. Clicking Abort ends the program, clicking Retry allows you to debug the program, and clicking Ignore continues execution of the program despite the assertion failure.

Getting ready to write more tests

You may think that the code we wrote for testing the response to "I am sad." seems like a lot of work, especially since we'd have to repeat this code in order to test every decomposition rule in every pattern. In reality, every test we write is virtually identical. The only difference between each test is the user input passed to GetResponse() and the response we're checking for.

To reduce the effort of writing tests, we'll create a helper class Test that will do the actual testing. ElizaTest will then simply use Test to specify what needs to be tested and will call its Test.Run() method to run the test. This will allow us to express our tests in a concise form, as shown below.

Improving the testing framework

Now that we have a Test class, we could simply instantiate and run it for every decomposition rule we want to test. But that would cause ElizaTest's Main() method to grow to an unwieldy size.

Instead, we'll create a new TestPattern class responsible for testing a specific pattern. TestPattern in turn will create and run instances of Test for every decomposition rule in the pattern. TestPattern will be an abstract class, which will require its pattern specific derived class to construct its Test instances. The first pattern we'll test will be the "AM" pattern, so AmPatternTest will be the first subclass of PatternTest.

Test and PatternTest constitute our testing framework. So we'll locate them in a "TestFramework" folder within the ElizaTest project. Pattern specific classes (like AmPatternTest) that derive from PatternTest will be located in the "PatternTests" folder.

Thanks to the utility provided by the testing framework, it's easy to write the AmPatternTest class. This class tests the rules defined for the "AM" pattern in Eliza's script.

Our first pattern test

Running the ElizaTest application produces this rather satisfying output indicating that the "AM" pattern behaves as expected.

Testable Eliza!

Eliza is finally testable! Here's what the ElizaTest project looks like after creating tests for every pattern. Notice that we moved complex and format free pattern tests to separate folders to keep things organized.

And here's what the full test run looks like. This is why we can sleep well at night.

Notice that our 24 patterns (that have a total of 260 individual responses) took all of 0.07 seconds to test. There's no way we could even come close to that time if we'd decided to test each pattern manually.

Next steps

We're now ready to code all the remaining patterns, including those that utilize synonyms. Our version of Eliza is getting close to becoming what Joseph Weizenbaum created in 1965.