(Having trouble seeing the slides? Try here.)

Original article: RR for Test Doubles Presentation

The questions and their results

• What versions of Ruby have you ever tried out?
• What versions of Ruby do you currently use in production or for real apps?
• What testing frameworks are your active projects using?
• If you were starting a new Rails project right now, what testing frameworks would you use?
• What mocking/stubbing frameworks are your active projects using?
• If you were starting a new Rails project right now, what mocking/stubbing frameworks would you use?
• What do your active projects use to populate testing data?
• If you were starting a new Rails project right now, what would you use for populating testing data?

What versions of Ruby have you ever tried out?

Summary: a wide variety of Ruby versions used. What the heck is kiji, you might ask? This was a useful post on kiji.

What versions of Ruby do you currently use in production or for real apps?

Summary: Mostly 1.8.7 and 1.9.2 in production use right now. REE is production-ready. Tweet

What testing frameworks are your active projects using?

If you were starting a new Rails project right now, what testing frameworks would you use?

Conclusion: Expect to see MiniTest in more production apps in the future. Tweet

What mocking/stubbing frameworks are your active projects using?

If you were starting a new Rails project right now, what mocking/stubbing frameworks would you use?

Conclusion: RSpec mocks are here to stay. Tweet

What do your active projects use to populate testing data?

If you were starting a new Rails project right now, what would you use for populating testing data?

Conclusion: Factory Girl and Machinist are going to remain popular. Rails fixtures are hanging around. Tweet

The Data Source

Here is the spreadsheet of results (with contact information and bad rows stripped). I didn’t spend very much time making killer visualizations, and there are some great comments in there. Highly recommended to get a feel for what people are using. Someone else could probably create a correlation table of common tool sets. For example, showing that people who use RSpec commonly use a certain mocking framework. Also, there is geographic info there, so we can see which region is on the cutting edge…

Edit: A note on the charts. People could respond with multiple answers, and I just tallied the answers for each category. For example, if 50 people said they used 1.8.6 and 1.8.7 and 50 people said 1.8.7, 1.8.7 would get 2/3 of the chart (100 “votes”), and 1.8.6 would get 1/3 (50 “votes”). The graphs could have been clearer, feel free to create a better visualization with the data. Thanks to the Hacker News commenters for bringing this up.

Logistics

I took the data, and filtered it with this script:

#!/usr/bin/env ruby
 
types = []
File.open(ARGV[0]).each do |line|
  line.strip!
  types += line.split(", ") unless line.length == 0
end
 
counts = Hash.new(0)
types.each do |type|
  counts[type.downcase] += 1
end
 
counts.each do |key, value|
  puts "#{key}\t#{value}"
end

Then, I put it in an Open Office spreadsheet, cleaned up the data a bit and sorted by the number descending. I should have probably outsourced this task, as it took awhile to get things looking right. It was terrible getting the chart images out of OO though. Had to copy to Draw, then export selection to an image file…

What did I miss?

Is your favorite testing framework not represented here? I left out some frameworks that only one person said. Check the spreadsheet for all of the data and some great comments. If you want to fill out the survey after having read this, go ahead and do so here. I might update this blog post at some point in the future or create a second one with updates…

What else could I have done, and do you want to be notified when future surveys take place? I’d imagine something in my survey process could have been improved. Leave a comment or email me at panozzaj@gmail.com!

Original article: The State of Ruby and Testing

One thing that has piqued my interest of late is theories in JUnit 4.4+.

Overview

In JUnit, the still experimental theory functionality is derived from the open-source Popper project. The name comes from Karl Popper’s work on the philosophy of science. He was a major proponent of saying that something isn’t scientific unless it is falsifiable, and that theories cannot be proven correct, merely shown to have exceptions. If someone can provide an exception, it means that the theory does not match all of our observations. Theories are models of the world, so while a model may not be correct, it can still be useful.

So what are theories in software good for? When using test-driven development, the tests serve as a form of code documentation. However, there are certain properties of code that are known or assumed by the developers that are difficult to codify as simple tests. Theories provide a simple and expressive means for at least the following cases:

For example, you might have a theory that data that is serialized into XML and then unserialized should always result in the original object. This seems to make sense, but it might be time-consuming or error-prone to specify all of the different tests that you have to consider. A single theory can codify this assumption.

In JUnit, theories are a special case of parameterized test. So instead of using the standard test that has no parameters, you add parameters to make testing various combinations of values shorter and easier. You can then take these parameterized tests further by specifying data points to test for by default in an expressive manner.

Example

Alright enough theory. What does a JUnit theory actually look like? Given that you have a theory about two methods on a Currency object that should be inverses of each other, one nice example is:

    @Theory public void multiplyIsInverseOfDivide(
            @TestedOn(ints={0, 5, 10}) int amount,
            @TestedOn(ints={1, 2, 3})  int multiplier) {
        assertThat(new Currency(amount).times(multiplier).divideBy(multiplier).getAmount(), is(amount));
    }

On the first line, we declare that we are going to postulate a theory, and give it a unique name. This method would typically live in a test suite or test file of some sort. The next two lines are the parameters to the test method. The parameters are prefixed by annotations to put in sample test values. All permutations of these values will be attempted when running the theory, and the values will be reported for any failures so we know what happened when the theory failed. On the last line, we test that the functions are indeed inverses by invoking them and making sure that the original value is the same as the end value.

Going further

So big deal, we could easily just create a helper test method that does something similar and we could pass whatever values to the method that we wanted. Well, hold on a second. One thing I haven’t mentioned yet is that there are some nifty tools that support automatically inserting values in to test theories. The tools automate searching the theory space for members that fit the theory assumptions but cause the theory to fail. Something like this might be useful in detecting an off-by-one error or corner case that you hadn’t considered while implementing. Typically for development you would just run the test cases presented with the method, and then would search the theory parameter space with spare cycles. This enables you to have great power as well as having blazing unit tests, which is key for any test-driven development.

If we allow automated searching of the theory space, there is one small problem with the example theory listed above. Can you see it?

The assert attempts to multiply by the multiplier and then divide by the multiplier, so a zero value for the multiplier will cause a divide by zero error. When we use automated tools for searching the theory space, we want to say that this theory only holds when the multiplier is not zero. We can easily do this:

    @Theory public void multiplyIsInverseOfDivide(
            @TestedOn(ints={0, 5, 10}) int amount,
            @TestedOn(ints={1, 2, 3})  int multiplier) {
        assumeThat(multiplier, not(0));
        assertThat(new Currency(amount).times(multiplier).divideBy(multiplier).getAmount(), is(amount));
    }

Similar expressions exist for ensuring the value is positive, and so forth. You might create a different test or theory to show the expected behavior when the multiplier is zero.

I really liked the use of Java annotations to make this code considerably more readable. You would probably need a runner of some sort to specify the values if you didn’t do it right there in the method signature.

It seems like there is some overlap between theories and contracts, but I won’t go into that at this time. Theories exert positive design pressures like tests do, in that they tend to lead to leaner code that just does what the theory asks for.

Extensions

In Ruby on Rails, it would be nice to be able to easily test methods with multiple models that you have defined. For example, you might have a theory:

  fixtures :users

  def test_admin_user_can_access_this_page
    login(users(:admin_user_1))
    get :this_page
    assert_response :success
  end

  # the previous test, but better
  def theory_that_admin_users_can_access_this_page(User user)
    assume(user.is_a? AdminUser)
    login(user)
    get :this_page
    assert_response :success
  end

So, instead of using just one user or manually going through a set of users, you could just specify a whole bunch of fixtures and go to town. This might make fixtures less brittle by pointing out subtle interactions between parameters that would cause the theory to fail. I’m not sure how the syntax would look, and I’m not sure how you could use something better than annotations in Ruby.

Another thing that might be nice would be to say: I have a theory that any strings we put to the page from the database will always be escaped. The automated theory explorer could step through the pages that you define and try once with some basic data and then start generating HTML and JavaScript garbage and ensure that nothing gets through to the users. This might be a nice way of preventing cross-site scripting attacks (although I would imagine that there are tools that already exist for just this purpose.)

I looked, but could not find a Ruby plugin to handle theories. Hmmm…

Further reading

Some excellent resources are:
The Popper tutorial
A great paper with examples of use and benefits (see the pdf on that page)
General overview of Theories in JUnit [pdf]

Original article: Introduction to Theories

public Class Klass {
    ...
 
    public Klass(int foo, boolean flag, String s1, String s2, String s3, String s4) {
        initialize(foo, flag, s1, s2, s3, s4);
    }
 
    public Klass(int foo, boolean flag, String s1, String s2, String s3) {
        initialize(foo, flag, s1, s2, s3, null);
    }
 
    public Klass(int foo, boolean flag, String s1, String s2) {
        initialize(foo, flag, s1, s2, null, null);
    }
 
    public Klass(int foo, boolean flag, String s1) {
        initialize(foo, flag, s1, null, null, null);
    }
 
    public Klass(int foo, boolean flag) {
        initialize(foo, flag, null, null, null, null);
    }
 
    private initialize(int foo, boolean flag, String s1, String s2, String s3, String s4) {
        _foo = foo;
        _flag = flag;
        _s1 = s1;
        _s2 = s2;
        _s3 = s3;
        _s4 = s4;
    }
 
    ...
}

Immediately, my Don’t Repeat Yourself sensor went off. It felt like the previous coder was essentially trying to use some default values for a method. You can do something like this in other languages, even other statically typed ones. In Ruby, it would be pretty simple:

class klass
  ...
 
  def initialize(foo, flag, s1 = nil, s2 = nil, s3 = nil, s4 = nil)
    ...
  end
 
  ...
end

However, I looked online and saw several instances of people saying “you can’t do that in Java.” So how else can we reduce this duplication?

One way that was suggested was to make better use of the object itself:

public Class Klass {
    ...
 
    public Klass(int foo, boolean flag, String s1, String s2, String s3, String s4) {
        _foo = foo;
        _flag = flag;
        _s1 = s1;
        _s2 = s2;
        _s3 = s3;
        _s4 = s4;
    }
 
    public Klass(int foo, boolean flag, String s1, String s2, String s3) {
        this(foo, flag, s1, s2, s3, null);
    }
 
    public Klass(int foo, boolean flag, String s1, String s2) {
        this(foo, flag, s1, s2, null, null);
    }
 
    public Klass(int foo, boolean flag, String s1) {
        this(foo, flag, s1, null, null, null);
    }
 
    public Klass(int foo, boolean flag) {
        this(foo, flag, null, null, null, null);
    }
 
    ...
}

This at least takes away the initialization method. I suppose you could make it shorter by having each constructor call the one immediately prior. You would remove a few nulls here and there. However, there is a deeper problem here, which was demonstrated in the code I was actually working with. What if we wanted to add even more strings? Why not just store the strings in an array? But if we have one string, do we need to store it in an array?

Ruby has a solution for this problem–the splat (*) operator:

  def initialize(foo, flag, *rest)
    foo = foo
    flag = flag
    strings = *rest
  end

If the rest parameter is nil, then strings will be nil. If the rest parameter is just a single value, like “hello”, then strings will be ["hello"]. It follows that if multiple values are passed in, then strings will contain an array of those values.

My first instinct was that Java did not have a construct that would allow us to do the same thing. However, a search revealed that there was an operator in Java that would allow us to do something like Ruby’s args or splat. Enter the ellipsis (…) feature, first released in Java 1.5:

public Class Klass {
    ...
 
    public Klass(int foo, boolean flag, String... strings) {
        _foo = foo;
        _flag = flag;
        _strings = strings;
    }
 
    ...
}

Ahh, that makes me happy. We went from about twenty lines to five with a single trick. Now, if there is some reason that only four strings can be allowed, we can do some additional parameter checking, but it seems like this is considerably more robust already. Like in Ruby, the varargs feature can only be used at the end of the method signature.

There were other parts of the codebase that had something like:

public Class Klass {
    ...
    public Klass(String string1, Shape shape1, String string2, Shape shape2, String string3, Shape shape3) {
        ...
    }
    ...
}

Where each String refers somehow to the associated Shape. In this case, you could just use a HashMap with generics, as in HashMap<String, Shape>.

You probably already knew all of this, but I had never used the varargs feature before in Java. Hope this helped.

Original article: Java Anti-Pattern