Archive Page 2

Benchmarking Producer/Consumer in Akka

Further to this conversation on the Akka mailing list, I decided to benchmark various different approaches to implementing the producer/consumer pattern.

I wanted to choose a “real” problem, so I decided to count the words on the first 100,000 pages of Wikipedia. The producer parses the Wiki XML dump and the words are counted page-by-page by a pool of consumers.

I implemented three different approaches – producer pushes to a bounded queue, producer pushes to an unbounded queue together with a flow control protocol, and consumer pulls.

The source code for the different implementations is here, and the results are at the bottom of this message.

Some observations:

  1. I only timed to the nearest second as I see an approx. 3 second variation from run to run with identical parameters. I’m not sure why I see such a large variation—suggestions welcome.
  2. There’s basically no difference between the two “producer pushes” implementations. The “consumer pulls” implementation is much slower, however
  3. I tried both Dispatcher and BalancingDispatcher and RoundRobinRouter and SmallestMailboxRouter in the producer pushes implementations—the differences were too small to measure

I’m surprised that there is so much difference between producer pushes and consumer pulls. It’s quite possible that I’ve done something stupid in the implementation—I’d be very grateful for a pointer to what it is.

Here are the results (all on my i7 MacBook Pro—4 cores, 2 hyperthreads per core).

Bounded Unbounded Pull
Consumers Seconds Speedup Seconds Speedup Seconds Speedup
1 100 1.00 103 0.97 156 0.64
2 55 1.82 56 1.79 88 1.14
3 43 2.33 40 2.50 65 1.54
4 39 2.56 38 2.63 55 1.82
5 37 2.70 37 2.70 51 1.96
6 34 2.94 33 3.03 44 2.27
7 33 3.03 33 3.03 47 2.13
8 33 3.03 35 2.86 46 2.17
9 33 3.03 32 3.13 46 2.17

ScalaMock3 step-by-step

This post describes ScalaMock 3, which supports Scala 2.10 only. For ScalaMock 2, which supports earlier Scala versions, go here.

This post describes how to setup a project that uses ScalaMock in conjunction with ScalaTest and sbt. The sample code described in this article is available on GitHub.

The example assumes that we’re writing code to control a mechanical turtle, similar to that used by Logo programs. Mocking is useful in this kind of situation because we might want to create tests that function even if we don’t have the hardware to hand, which run more quickly than would be the case if we ran on real hardware, and where we can use mocks to simulate errors or other situations difficult to reproduce on demand.

  1. Create a root directory for your project:
    $ mkdir myproject[/soucecode]
    </li>
    	<li>Create <code>build.sbt</code> containing:
    organization := "com.example"
    
    version := "1.0"
    
    scalaVersion := "2.10.0"
    
    scalacOptions ++= Seq("-deprecation", "-unchecked")
    
    libraryDependencies +=
      "org.scalamock" %% "scalamock-scalatest-support" % "3.0" % "test"
  2. Now we’ve got a project, we need some code to test. Let’s start with a simple trait representing a turtle. Create src/main/scala/Turtle.scala containing:
    package com.example
    
    trait Turtle {
      def penDown()
      def penUp()
      def forward(distance: Double)
      def turn(angle: Double)
      def getPosition: (Double, Double)
      def getAngle: Double
    }
  3. The turtle API is not very convenient, we have no way to move to a specific position, instead we need to work out how to get from where we are now to where we want to get by calculating angles and distances. Here’s some code that draws a line from a specific point to another by doing exactly that.Create src/main/scala/Controller.scala containing:
    package com.example
    
    import scala.math.{atan2, sqrt}
    
    class Controller(turtle: Turtle) {
    
      def drawLine(start: (Double, Double), end: (Double, Double)) {
        moveTo(start)
    
        val initialAngle = turtle.getAngle
        val deltaPos = delta(start, end)
    
        turtle.turn(angle(deltaPos) - initialAngle)
        turtle.penDown
        turtle.forward(distance(deltaPos))
      }
    
      def delta(pos1: (Double, Double), pos2: (Double, Double)) =
        (pos2._1 - pos1._1, pos2._2 - pos1._2)
    
      def distance(delta: (Double, Double)) =
        sqrt(delta._1 * delta._1 + delta._2 * delta._2)
    
      def angle(delta: (Double, Double)) =
        atan2(delta._2, delta._1)
    
      def moveTo(pos: (Double, Double)) {
        val initialPos = turtle.getPosition
        val initialAngle = turtle.getAngle
    
        val deltaPos = delta(initialPos, pos)
    
        turtle.penUp
        turtle.turn(angle(deltaPos) - initialAngle)
        turtle.forward(distance(deltaPos))
      }
    }
  4. We can now write a test. We’ll create a mock turtle that pretends to start at the origin (0, 0) and verifies that if we draw a line from (1, 1) to (2, 1) it performs the correct sequence of turns and movements.

    Turtle diagram

    Create src/test/scala/ControllerTest.scala containing:

    package com.example
    
    import org.scalatest.FunSuite
    import org.scalamock.scalatest.MockFactory
    import scala.math.{Pi, sqrt}
    
    class ControllerTest extends FunSuite with MockFactory {
    
      test("draw line") {
        val mockTurtle = mock[Turtle]
        val controller = new Controller(mockTurtle)
    
        inSequence {
          inAnyOrder {
            (mockTurtle.penUp _).expects()
            (mockTurtle.getPosition _).expects().returning(0.0, 0.0)
            (mockTurtle.getAngle _).expects().returning(0.0)
          }
          (mockTurtle.turn _).expects(~(Pi / 4))
          (mockTurtle.forward _).expects(~sqrt(2.0))
          (mockTurtle.getAngle _).expects().returning(Pi / 4)
          (mockTurtle.turn _).expects(~(-Pi / 4))
          (mockTurtle.penDown _).expects()
          (mockTurtle.forward _).expects(1.0)
        }
    
        controller.drawLine((1.0, 1.0), (2.0, 1.0))
      }
    }

    This should (hopefully!) be self-explanatory, with one possible exception. The tilde (~) operator represents an epsilon match, useful for taking account of rounding errors when dealing with floating-point values.

  5. Run the tests with sbt test:
    $ sbt
    > test
    [info] ControllerTest:
    [info] - draw line
    [info] Passed: : Total 1, Failed 0, Errors 0, Passed 1, Skipped 0

ScalaMock 3.0-M4 for Scala 2.10.0-RC1

ScalaMock 3.0-M4 (scaladoc) for Scala 2.10.0-RC1 is now released. It supports:

  • Mock functions, traits and classes
  • Both expectation-first and record-then-verify (Mockito-style) mocking
  • ScalaTest and Specs2

To use with sbt and ScalaTest:

libraryDependencies +=
  "org.scalamock" % "scalamock-scalatest-support_2.10.0-RC1" % "3.0-M4"

or for Specs2:

libraryDependencies +=
  "org.scalamock" % "scalamock-specs2-support_2.10.0-RC1" % "3.0-M4"

For background information, see ScalaMock 3.0 Preview Release.

Known limitations (these should all be fixed when Scala adds support for mock types):

  • No support for mocking object creation (constructors)
  • No support for mocking singleton/companion objects
  • No support for mocking final classes or classes with private constructors
  • No support for mocking concrete vars
  • Limited support for overloaded methods
  • No support for mocking Java methods with repeated parameters

ScalaMock 3.0-M1

ScalaMock 3.0-M1 (scaladoc) for Scala 2.10.0-M4 is now released. It supports:

  • Mock functions, traits and classes
  • Both expectation-first and record-then-verify (Mockito-style) mocking
  • ScalaTest and Specs2

To use with sbt:

libraryDependencies +=
  "org.scalamock" %% "scalamock-scalatest-support" % "3.0-M1"

For background information, see ScalaMock 3.0 Preview Release.

It’s been tested against overloaded, curried and polymorphic methods, by-name parameters, path-dependent types and type projections. There may still be corner cases that aren’t handled correctly, please report a bug if you find one.

ScalaMock 3.0 Preview Release

Update: ScalaMock 3.0-M4 for Scala 2.10.0-RC1 is now released.

The current version of ScalaMock makes use of a compiler plugin to generate typesafe mock objects. This works, but has a number of problems, not least of which is the complicated nature of the build system required (meaning that it currently only works with sbt). The recent preview release of Scala 2.10 has opened up a new approach via its support for macros.

I’ve just released a preview of ScalaMock 3.0, a “from the ground up” rewrite using macros. It’s not yet as capable as ScalaMock 2.x (it can only mock traits – there’s no support for mocking classes, singleton/companion objects or object creation yet) but it’s certainly complete enough to be useful, I hope.

One significant (and frequently requested!) enhancement over ScalaMock 2.x is support for Mockito-style “record then verify” mocking as well as the more traditional “setup expectations” style.

The source is available on GitHub, and a snapshot release (Scala 2.10.0-M3 only) is available on Sonatype:

libraryDependencies +=
  "org.scalamock" %% "scalamock-scalatest-support" % "3.0-SNAPSHOT"

You can see examples of it in use in GitHub.

Here’s an example of a test written using the “setup expectations” style:

class ControllerTest extends FunSuite with MockFactory {

  test("draw line") {
    val mockTurtle = mock[Turtle]
    val controller = new Controller(mockTurtle)

    inSequence {
      inAnyOrder {
        (mockTurtle.penUp _) expects ()
        (mockTurtle.getPosition _) expects () returning (0.0, 0.0)
        (mockTurtle.getAngle _) expects () returning 0.0
      }
      (mockTurtle.turn _) expects ~(Pi / 4)
      (mockTurtle.forward _) expects ~sqrt(2.0)
      (mockTurtle.getAngle _) expects () returning Pi / 4
      (mockTurtle.turn _) expects ~(-Pi / 4)
      (mockTurtle.penDown _) expects ()
      (mockTurtle.forward _) expects 1.0
    }

    controller.drawLine((1.0, 1.0), (2.0, 1.0))
  }
}

and here’s the same example rewritten to use the Mockito-style record and then verify approach:

class ControllerTest extends FunSuite with MockFactory {
  import scala.language.postfixOps

  test("draw line") {
    val mockTurtle = stub[Turtle]
    val controller = new Controller(mockTurtle)

    inSequence {
      inAnyOrder {
        (mockTurtle.getPosition _) when () returns (0.0, 0.0)
        (mockTurtle.getAngle _) when () returns 0.0 once
      }
      (mockTurtle.getAngle _) when () returns Pi / 4
    }

    controller.drawLine((1.0, 1.0), (2.0, 1.0))

    inSequence {
      (mockTurtle.turn _) verify ~(Pi / 4)
      (mockTurtle.forward _) verify ~sqrt(2.0)
      (mockTurtle.turn _) verify ~(-Pi / 4)
      (mockTurtle.penDown _) verify ()
      (mockTurtle.forward _) verify 1.0
    }
  }
}

Right now, it supports:

  • Mock functions (including higher-order functions)
  • Polymorphic traits
  • Polymorphic methods
  • Curried methods
  • Overloaded methods

Here’s my todo list:

  • Sort out the documentation
  • Take advantage of macro types (type providers) when available in a subsequent Scala release. This will enable a slightly nicer syntax for method mocking:

    mockObject.expects.method(arguments)

    instead of:

    (mockObject.method _) expects (arguments)

  • Mocking classes (as well as traits)
  • Mocking singleton/companion objects
  • Mocking object creation

I’d be very grateful for any feedback, in particular bug reports.

An open letter to an IT recruitment consultant

Dear IT Recruitment Consultant,

For the purposes of this letter, I’m going to assume that you are one of the tiny minority of members of your profession who have some integrity and ability. Unfortunately for you, you have chosen a career with a deservedly dreadful reputation. Most IT recruitment consultants are no better than a combination of pimp and double glazing salesman.

But you’re not one of this majority. You can have a conversation with me where I don’t need to explain everything to you in words of one syllable. Repeatedly. And you have a unique source of amazing candidates unavailable to any other recruitment consultant and who would be just perfect for me.

So how do you get to talk to me?

Well this is where it gets difficult I’m afraid. You know that you’re one of this tiny minority, but I don’t. And unfortunately all of you claim that you’re different from all the rest. How do I tell the difference between someone who really does have a brain and the vast majority who just claim to?

Well, I’m sorry, but there is exactly one way to achieve this, and that’s to be good at your job. It’s a long, slow path, but eventually someone I trust will recommend that I talk to you. And if I need to, I will.

What you absolutely do not do is call me. Or e-mail me. Or try to link to me on LinkedIn (especially not by lying and claiming to be my friend or an ex-colleague in order to avoid paying LinkedIn). Or try to get in contact in any other way whatsoever.

My problem is not that I don’t know enough recruitment consultants. Nobody who has spent more than a few years in IT can possibly fail to know many (many!) recruitment consultants. I have found a small set that I trust, respect, and will work with again (you see – I do know that some of you are different).

I don’t want to be introduced to any more. There are too damn many of you, and most of you aren’t worth even the 60 seconds of my time it takes to tell you to go away. Not that it ever takes 60 seconds, because you always have to followup and tell me why I’m wrong about you and you’re different from all the rest. And ignoring you doesn’t work either, because you’ll just try again. And again. And again.

I will never do business with any recruitment consultant who cold calls. Ever.

So please stop? Please?

Mockito on Android step-by-step

One of my biggest frustrations with writing code for Android has been the fact that none of the current Java mocking frameworks work on Android’s Dalvik VM. I tried to address that myself by getting ScalaMock working on Android, but that’s been less successful than I hoped (primarily due to general problems with getting Scala working in the Android environment).

Happily there’s been a major breakthrough recently – Mockito, one of the established Java mocking frameworks now supports Android 🙂 In this article, I’ll go through getting an Android application up and running with tests written using Mockito.

This is an Android version of the example in Martin Fowler’s article Mocks Aren’t Stubs. The code is checked into GitHub here. You’ll need to have the Android SDK and ADT Plugin for Eclipse installed to run this code.

We’re going to build a (very) simple ordering system. Orders will succeed if there’s enough inventory in our warehouse and fail if not. Let’s start by creating a very simple little Android application for us to test:

  1. Create a new Android project in Eclipse called WarehouseManager. Select the “Create a Test Project” option.
  2. The core abstraction is a warehouse, represented by a Warehouse interface:

    package com.example;
    
    public interface Warehouse {
        boolean hasInventory(String product, int quantity);
        void remove(String product, int quantity);
    }
  3. And here’s a very simple concrete implementation of Warehouse:

    package com.example;
    
    import java.util.HashMap;
    
    public class RealWarehouse implements Warehouse {
    
        public RealWarehouse() {
            products = new HashMap();
            products.put("Talisker", 5);
            products.put("Lagavulin", 2);
        }
    
        public boolean hasInventory(String product, int quantity) {
            return inStock(product) &gt;= quantity;
        }
    
        public void remove(String product, int quantity) {
            products.put(product, inStock(product) - quantity);
        }
    
        private int inStock(String product) {
            Integer quantity = products.get(product);
            return quantity == null ? 0 : quantity;
        }
    
        private HashMap products;
    }
  4. We remove things from the warehouse by placing an Order:

    package com.example;
    
    public class Order {
    
        public Order(String product, int quantity) {
            this.product = product;
            this.quantity = quantity;
        }
    
        public void fill(Warehouse warehouse) {
            if (warehouse.hasInventory(product, quantity)) {
                warehouse.remove(product, quantity);
                filled = true;
            }
        }
    
        public boolean isFilled() {
            return filled;
        }
    
        private boolean filled = false;
        private String product;
        private int quantity;
    }
  5. We’ll need a UI to allow us to make orders, so modify main.xml to look like this:

    <?xml version="1.0" encoding="utf-8"?>
    <LinearLayout
        xmlns:android="http://schemas.android.com/apk/res/android"
        android:orientation="vertical"
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        >
      <TextView  
          android:layout_width="fill_parent" 
          android:layout_height="wrap_content" 
          android:text="Product:"
          />
      <EditText
          android:layout_width="fill_parent" 
          android:layout_height="wrap_content"
          android:id="@+id/product"
          />
      <TextView
          android:layout_width="fill_parent" 
          android:layout_height="wrap_content" 
          android:text="Quantity:"
          />
      <EditText
          android:layout_width="fill_parent" 
          android:layout_height="wrap_content"
          android:id="@+id/quantity"
          />
      <Button
          android:layout_height="wrap_content"
          android:layout_width="wrap_content"
          android:text="Place order"
          android:onClick="placeOrder" />
    </LinearLayout>
  6. And finally, here’s the implementation of WarehouseManagerActivity:

    package com.example;
    
    import android.app.Activity;
    import android.os.Bundle;
    import android.view.View;
    import android.widget.EditText;
    import android.widget.Toast;
    
    public class WarehouseManagerActivity extends Activity {
        /** Called when the activity is first created. */
        @Override
        public void onCreate(Bundle savedInstanceState) {
            super.onCreate(savedInstanceState);
            setContentView(R.layout.main);
    
            productEditText = (EditText)findViewById(R.id.product);
            quantityEditText = (EditText)findViewById(R.id.quantity);
        }
    
        public void placeOrder(View view) {
            String product = productEditText.getText().toString();
            int quantity = Integer.parseInt(quantityEditText.getText().toString());
            Order order = new Order(product, quantity);
            order.fill(warehouse);
    
            String message = order.isFilled() ? "Success" : "Failure";
            Toast toast = Toast.makeText(this, message, Toast.LENGTH_SHORT);
            toast.show();
        }
    
        private Warehouse warehouse = new RealWarehouse();
    
        private EditText productEditText;
        private EditText quantityEditText;
    }

You should now have a little Android application that can be compiled and installed with ant install. Here’s what it looks like:

Screenshot

So, now that we’ve got something to test, let’s test it:

  1. The version of Mockito that supports Android has not yet been released, so you’ll need to get the source and build it for yourself. After building, you should have mockito-all-1.9.1-SNAPSHOT.jar in the target directory.
  2. Create a libs directory underneath WarehouseManagerTest and copy mockito-all-1.9.1-SNAPSHOT.jar into it. Download the latest dexmaker into the same directory.
  3. Finally, we can write our tests, which create mock instances of the Warehouse interface:

    package com.example.test;
    
    import android.test.InstrumentationTestCase;
    import com.example.*;
    
    import static org.mockito.Mockito.*;
    
    public class OrderTest extends InstrumentationTestCase {
    
        public void testInStock() {
            Warehouse mockWarehouse = mock(Warehouse.class);
    
            when(mockWarehouse.hasInventory("Talisker", 50)).thenReturn(true);
    
            Order order = new Order("Talisker", 50);
            order.fill(mockWarehouse);
    
            assertTrue(order.isFilled());
            verify(mockWarehouse).remove("Talisker", 50);
        }
    
        public void testOutOfStock() {
            Warehouse mockWarehouse = mock(Warehouse.class);
    
            when(mockWarehouse.hasInventory("Talisker", 50)).thenReturn(false);
    
            Order order = new Order("Talisker", 50);
            order.fill(mockWarehouse);
    
            assertFalse(order.isFilled());
        }
    }
  4. Run your tests 🙂