Meta-Testing your Assertions

As gems get more complex it is good practice to ship additional assertion methods with your library, so users can easily test code they wrote using your library.

Usually you do this by providing either a module or some derived TestCase, in a Test::Unit or MiniTest environment.

Tests without special assertions

The world-famous are-u-cool gem checks persons – whether they’re cool or not by querying some awesome web service.

Applications using this gem would usually have some test like this.

class PersonsTest < Unit::TestCase
  it "is the opposite of cool" do
    assert_not AreUCool.new.cool?("Helder")
  end
end

Domain-specific tests

Since this is a lot of work, the gem ships with its own test case to help you.

class AreUCool::TestCase < Unit::TestCase
  def setup
    @instance = AreUCool.new
  end
  
  # Asserts +name+ is cool.
  def assert_cool(name)
    assert @instance.cool?(name)
  end
end

Simplifying your application tests, an example usage would naturally look like the following.

class PersonsTest < AreUCool::TestCase
  it "is extremely cool" do
    assert_cool "Nick"
  end
end

See the difference?

The new test case makes it easy to test coolness.

Testing your test

Many gems do this. They provide modules or test classes to make your life easier.

However, mostly these test extensions aren’t tested at all, or are tested wrong. The gem authors usually “test” their test methods like this (as seen in Rails and in my own tests).

# in are-u-cool/test/test_case_test.rb
class TestCaseTest < AreUCool::TestCase
  it "should respond to assert_cool" do
    assert_cool "Nick"
  end
end

The problem here is: The tested test tests itself, which is almost like this unit test, for instance.

class PersonsTest < Test::Unit
  it "should respond to #name" do
    Person.new.name
  end
end

This test just tests if a method can be run in general. The same happens in the second last example. We use the new assertion, nothing more.

Don’t mix meta levels

We confused two different layers here! So, keep these rules in mind when testing your tests:

• Don’t derive your test case from that which you’re testing!
• Double-assert your new assertions!

Testing a TestCase is relatively simple. Look at how the new TestCaseTest looks.

class TestCaseTest < Unit::TestCase
  setup do
    @test = AreUCool::TestCase.new(:burp)
    @test.setup
  end
  
  it "should respond to assert_cool" do
    assert @test.assert_cool("Nick")
  end

Notice how I use two asserts in one line? This cleanly tests if the new assertion returns true on a 100% valid assertion.

Being true pessimists, we double-check a failing assertion, too.

  it "should complain about uncool persons" do
    exc = assert_raises(MiniTest::Assertion) do
      @test.assert_cool("Helder")
    end

    assert_equal "Expected true.", exc.message
  end
end

We simply catch the Assertion exception that is thrown if the assert_cool assertion fails.

Now, how would you do that in a self-contained test? You wouldn’t, right.

Following this, you should never use a test to test itself, but run tests against an instance of it – cleanly separating meta levels.

Advertisement