Playing with a Testing Library

I’d like to be able to express my unit tests fairly naturally, using the conditional operators built into the language. So, for example, I’d want to write:

expect(factorial(5)) == 120
expect(factorial(10)) > 10000

I’d like the error messages to show both the code that caused the error and the values that caused the error. So, for example, I’d want the following (incorrect) test

expect(factorial(6)) == 600

to output something like

/Users/dave/tmp/tmc/blog_tests.rb:16
    the code was: expect(factorial(6)) == 600,
    but 720 != 600

and

expect(1) > 2

should say

/Users/dave/Play/tmc/blog_tests.rb:11
    the code was: expect(1) > 2,
    but 1 <= 2

(Note how the expression showing the actual values negates the comparison operator to make it easier to read.)

I annotate my code with comments, so I’d like to be able to annotate my tests the same way.

expect(factorial(6)) == 600  # Deliberate bad test

should produce something like

/Users/dave/tmp/tmc/blog_tests.rb:17
     **Deliberate bad test**
     the code was: expect(factorial(6)) == 600,
     but 720 != 600  

Sometimes I write longer comments.

# The factorial of 6 is a special case,
# because of the labor laws in Las Vegas
expect(factorial(6)) == 600
   

So the resulting errors are longer, too.

/Users/dave/Play/tmc/blog_tests.rb:21
    **The factorial of 6 is a special case, because of the labor laws in Las Vegas**
    the code was: expect(factorial(6)) == 600,
    but 720 != 600

I like to be able to group my tests.

testing("positive factorials") do
  expect(factorial(1)) == 1
  expect(factorial(2)) == 2
  expect(factorial(5)) == 120
end

testing("factorial of zero") do
  expect(factorial(0)) == 1
end

testing("negative factorials") do
  expect(factorial(-1)) == 1
  expect(factorial(-5)) == 1
end

I like the description of the group to appear along with any individual test annotation if a test fails.

testing("factorial of zero") do
  # this test is deliberately wrong
  expect(factorial(0)) == 0
end

will produce

/Users/dave/Play/tmc/blog_tests.rb:31--while testing factorial of zero
    this test is deliberately wrong
    the code was: expect(factorial(0)) == 0,
    but 1 != 0
    

I like to have the flexibility to set up the environment for a group of tests. I also like to have the idea of a global environment which doesn’t get messed up by the running of tests (so that subsequent tests can run in that environment. I don’t see why I should have to package things into methods with magic names to have that happen. Instead, why not just have transactional instance variables? That way, I can use regular methods to set up the state for a test.

@order = Order.new("Dave Thomas", "Ruby Book")

testing("normal case") do
  expect(@order.valid?) == true
end

testing("missing name in order") do
  @order.name = nil
  expect(@order.valid?) == false
  expect(@order.error)  == "missing name"
end

# Check that order is reset to valid state here
expect(@order.valid?) == true

So, in the preceding case, the second testing block changed the @order object. However, once the block terminated, the object was restored to its initial (valid) state.

So, while waiting for the last day of the Rails Studio to start, I hacked together a quick proof of concept. It’s less than 100 lines of code. All the output shown here was generated by it. Is this worth developing into something usable?