Developing an Akka edge special code
Chapter 9: Testing
In this chapter, we focus our attention on testing actor systems. We start with understanding the testing strategy and toolkit that Akka has to provide and we will use examples to illustrates these test strategies. In this chapter, we will be using ScalaTest as our test framework which works well with SBT. The nice thing about sbt is that the popular Jenkins and Hudson Continuous Integration environments gel together pretty nice and allows the developer to execute test runs on a automated basis, mostly happening right after code checkins. Writing tests for actors is a little trickier that writing actor code for the reason that when you are writing actor code, you tend to express the code in a sequential manner that reflects the work process requirement (see illustration below) but before you launch it into production, you definitely want to gain a certain level of confidence that the work process is correct for all users, in a concurrent / asynchronous environment.
Akka provides a tool kit akka-testkit which provides you the capability to write tests that validate your actor's processing logic (in isolation, without threads) and its inter-actor processing (with threads, communicating with other actors) and it's quite natural. Taking a step back about what you've learnt so far and reflecting on crafting actors, you would notice that the Akka model focuses your attention into crafting your application's logic into the message handler i.e. def receive = { ... }
class YourActor extends Actor {
def receive = {
// business logic
// step 1, do this
// step 2, do that
...
// step N, complete
}
}
and Akka's actor model detaches you from thinking about how data/messages are transported between actors; in fact Akka provides the developer a configuration based approach to configure how the actor system might be customized based on your requirements on scaling by increasing/decreasing parallelism, message routing / dispatching etc as illustrated in previous chapters.
Setting Up
Setting up the dependency is easy, you just add the following line to your build.sbt
libraryDependencies ++= Seq( ...
"com.typesafe.akka" %% "akka-testkit" % "2.2.1")
Akka is agnostic about which test framework you would like to use though Akka's tests was written using ScalaTest and in this case we'll follow suit too and we add the following dependency to our build.sbt too
libraryDependencies ++= Seq(...
"com.typesafe.akka" %% "akka-testkit" % "2.2.1",
"org.scalatest" % "scalatest_2.10" % "2.0.M6" % "test")
The next few sections will examine how you would go about using Akka's testing toolkit to write tests and you'll notice some parallels in the JVM world of unit-testing.
Isolation Testing
This style of writing tests is almost the first form you will begin with, reason being that you would naturally want to validate your application's logic before subjecting that logic to a concurrency test (which is the subject of the next chapter). Let's use an example assuming an crazy problem of an Actor hold a reference to a stack of invoices, modelled by Order, and let's assume that this actor can only add invoices upon receiving a message, Checkin, and finally it'll tally the total monies on that stack upon receiving the message, Checkout. Let's take a look at our code
import akka.actor._
import collection.mutable.SynchronizedStack
sealed trait Message
case class Checkin(amount: Int) extends Message
case class Checkout extends Message
class InvoiceActor extends Actor with ActorLogging {
private[this] var orders = Order()
def receive = {
case Checkin(amount) ⇒ orders.add(amount)
case Checkout ⇒ sender ! orders.tally
}
def topOfOrder = orders.top
}
trait Order {
// Our restaurant is special: we only accept dollars, no coins!
// we need to synchronized access to our shared structure
private[this] var orders : SynchronizedStack[Int] = new SynchronizedStack[Int]()
def add(amount: Int) : Unit = orders = orders push amount
def apply(x: Int) = orders.applyOrElse(x, (_:Int) match { case _ ⇒ 0 })
def tally : Int = orders.foldLeft(0)(_ + _)
def numOfOrders : Int = orders.size
def top : Int = orders.top
}
object Order {
def apply() = new Order {}
}
There are a few ways you can go about testing this code and for starters, we can identify two things we need to test in isolation and subsequently under concurrency. Admittedly, we cheated a little here by using the synchronized stack in our choice because otherwise you might start with a stack and run into concurrency problems before switching out for a synchronized stack.
- Test the validity of Order
- Test the validity of the business logic within the actor, Invoices
It's probably a good idea to test Order in isolation so that we can iron the bugs before we begin writing tests for our business logic in our actor. Here is how that test might look like using a test framework like ScalaTest:
import org.scalatest.FlatSpec
class OrderSpec extends FlatSpec {
behavior of "An default Order i.e. no orders have been added"
private[this] val order = Order()
it should "have zero orders" in {
assert(order.numOfOrders === 0)
}
it should "return 0 when tally is invoked" in {
assert(order.tally === 0)
}
behavior of "An order when > 1 order(s) have been added"
it should "have exactly 2 orders when 2 order are added" in {
order.add(1)
assert(order.top === 1)
order.add(2)
assert(order.numOfOrders === 2)
}
it should "return 13 when tally is invoked for orders of values {1,2,5,5}" in {
order.add(5)
order.add(5)
assert(order.tally === 13)
}
}
At this point in time, we are satisfied that Order has been tested and we can move on to begin writing tests for our actor. In our actor, Invoices, we would like to actually make sure the behavior is not only consistent in a single-threaded execution environment but also in a multi-threaded environment. To achieve the former, we can hook our actor into the akka-testkit module by extending from akka.testkit.TestKit and providing an container in which to host our actors i.e. ActorSystem
The following code shows one way of crafting this test using the BDD i.e. Behavior Driven Developmentstyle in ScalaTest.
import akka.testkit._
import akka.actor.{ActorRef, ActorSystem, Props}
import org.scalatest.{FlatSpecLike, BeforeAndAfterAll}
class InvoiceActorSpec extends TestKit(ActorSystem()) with ImplicitSender
with FlatSpecLike
with MustMatchers with BeforeAndAfterAll {
behavior of "`Invoice` actor taking 2 orders"
it should "have two values {400, 500} when the same two orders are issued" in {
val actor = TestActorRef[InvoiceActor]
val ref = actor.underlyingActor
actor ! Checkin(400)
ref.topOfOrder must be (400)
actor ! Checkin(500)
ref.topOfOrder must be (500)
}
it should "return a tally of 900 after two orders of value 400 & 500" in {
val actor = TestActorRef[InvoiceActor]
actor ! Checkin(400)
actor ! Checkin(500)
actor ! Checkout
expectMsg(900)
}
override def afterAll() { system.shutdown }
}
Let's go through what we did to arrive at this test. The first thing we did was to extend from TestKit and provided TestKit with a container in which all actors are hosted (for this test only); next, we mixed in a trait called FlatSpecLike which is the cousin of FlatSpec and we did this because FlatSpec is a class and we cannot extend from two classes, even in Scala. Moving on, we mixed in another trait called ImplicitSender and this trait is necessary for the test to succeed because it allows the test code to receive the response messsages.
The last statement we made about ImplicitSender deserves more attention. This particular trait is needed in your tests because it provides a special environment in which your tests will execute on and that environment is actually an actor that's single-threaded so that your tests are executed in written order.
Inside InvoiceActorSpec, we have two tests and once testing completes it is necessary to shutdown the ActorSystem or else you would have unintentionally leaked the ActorSystem (that's bad since we are essentially wasting the resources on the computer). We need to find a way to inject the clean up code and we would like it to occur when all of our test completes; but then again you may argue that you would like to create and destroy an ActorSystem before and after every test and you can definitely do that but if you recall one of the core principles of unit testing is that it should complete rather quickly and this cycling of ActorSystem's do take time which leaves us the former option. We are fortunate as ScalaTest's BeforeAndAfterAll trait provides a method, afterAll, which allows us to place our clean up code.
Since we've understood the basics of our actor testing lifecycle, we can dive into the two tests we have. You will notice that they look similar and differ in ways. Common to both tests is the fact that we need to create a reference to our actor under test and Akka named it TestActorRef and this reference basically wraps around our actor, InvoiceActor, along with a special dispatcher called CallingThreadDispatcher (it doesn't create a new thread, but uses the current one i.e. piggy back). This dispatcher is ideal for our cause because we wanted to make sure our test succeeds in isolation (which means our actor's logic works!) before subjecting it to concurrency.
The next thing you almost want to do when writing tests for actors is to make sure:
- Messages are received and acted upon
- Message handler logic is correct
In the former, we send messages to our actor under test via the familiar tell syntax i.e. ! and we know that its acted upon because our actor's logic says to send back the results to the actor and by using the function, expectMsg, we know that it did. There is a entire plethora of functions that allows the developer to test the validity of responses that's embedded in akka.testkit.TestKit and we encourage you to explore the Scala docs on that.
In the latter, our actor's code indicates that upon receiving a Checkin message, we'll deposit that amount into the Order held by the actor and we wanted to make sure that the said amount is the received amount and let's assume we are not allowed to add another message, LastCheckin, to add this feature (though we see no real harm, really) like this:
case class LastCheckin
class InvoiceActor extends Actor with ActorLogging {
def receive = {
case Checkin(amount) ⇒ // as above
case Checkout ⇒ // as above
case LastCheckin ⇒ sender ! topOfOrder
}
// other code omitted
private def topOfOrder = orders.top
}
In honesty, this would work but there's another way! Akka provided us another way of achieving this without much fanfare by allowing us access to our actor, InvoiceActor, by invoking a method, underlyingActor, and this allows us to add a function to check the state of the Order after each Checkin by the expression
ref.topOfOrder must be (400) // now, topOfOrder is made 'public' & no synchronization
There are a few ways we can see the tests we have written in action. Since we are using SBT, we could try the following
>test
and that would trigger a one-time only trigger of all tests in our project. You would see the following output on the console window:
[info] OrderSpec:
[info] An default Order i.e. no orders have been added
[info] - should have zero orders
[info] - should return 0 when tally is invoked
[info] An order when > 1 order(s) have been added
[info] - should have exactly 2 orders when 2 order are added
[info] - should return 13 when tally is invoked for orders of values {1,2,5,5}
[info] InvoiceActorSpec:
[info] `Invoice` actor taking 2 orders
[info] - should have two values {400, 500} when the same two orders are issued
[info] - should return a tally of 900 after two orders of value 400 & 500
[info] Passed: : Total 6, Failed 0, Errors 0, Passed 6, Skipped 0
[success] Total time: 1 s, completed Sep 15, 2013 11:51:54 AM
At this point, you should see that Akka does not interfere with how we write tests and this beneficial to developers as they can continue the testing and mocking frameworks that worked well for them and only worry about asynchronicity when they have to.
Parallel Testing
This section is really about running tests in parallel and you normally would want to do that is so that your test cycle can complete quicker than when running your test suites in sequence. There are two ways you can accomplish this and one way is to not involve Akka and the unit testing framework will handle running parallel tests; the second way is to involve Akka and this manner allows not only parallelism in your unit tests but also you may launch multiple Scala modules, dependent on your requirement.
In the scenario where you want your unit test to take care of launching tests in parallel, in the context of ScalaTest, we need to create tests suites (its really a Scala class which you use to categorize your tests, in a meaningful way) with the following mixins
- FunSuite [with BeforeAndAfter] with ParallelTestExecution
- FunSuiteLike [with BeforeAndAfter] with ParallelTestExecution
where the trait BeforeAndAfter should be mixed in before ParallelTestExecution so ensure the As far as we know in the the ScalaTest 2.0.M7 release, ParallelTestExecution must be the last trait to be mixed in a method, runTest, has been marked final.
The following code demonstrates what the test code looks like after the appropriate traits have been mixed in to allow parallel execution.
import akka.testkit._
import akka.actor.{ActorRef, ActorSystem, Props}
import org.scalatest.{FlatSpecLike, BeforeAndAfterAll, FunSuiteLike, ParallelTestExecution}
import org.scalatest.matchers.MustMatchers
class InvoiceActorFunParSpec extends TestKit(ActorSystem()) with ImplicitSender
with FunSuiteLike
with BeforeAndAfterAll
with MustMatchers
with ParallelTestExecution {
test("have three values {400, 500, 600} when the same three orders are issued") {
val actor = TestActorRef[InvoiceActor]
val ref = actor.underlyingActor
actor ! Checkin(400)
ref.topOfOrder must be (400)
actor ! Checkin(500)
ref.topOfOrder must be (500)
actor ! Checkin(600)
ref.topOfOrder must be (600)
}
test("return a tally of 1,500 after three orders of value 400, 500 & 600") {
val actor = TestActorRef[InvoiceActor]
actor ! Checkin(400)
actor ! Checkin(500)
actor ! Checkin(600)
actor ! Checkout
expectMsg(1500)
}
override def afterAll() = { system.shutdown}
}
class InvoiceActorFunParSpec2 extends TestKit(ActorSystem()) with ImplicitSender
with FunSuiteLike
with BeforeAndAfterAll
with MustMatchers
with ParallelTestExecution {
test("have one value {400} when one order is issued") {
val actor = TestActorRef[InvoiceActor]
val ref = actor.underlyingActor
actor ! Checkin(400)
ref.topOfOrder must be (400)
}
test("return a tally of 500 after one order of value 500") {
val actor = TestActorRef[InvoiceActor]
actor ! Checkin(500)
actor ! Checkout
expectMsg(500)
}
override def afterAll() = { system.shutdown}
}
To run these new tests, we go back to SBT and trigger them again. Depending on how many times you trigger these tests, the order of execution, for InvoiceActorFunParSpec and InvoiceActorFunParSpec2, would be different. A sample output from my machine (with the test suite names in bold) is as follows:
[info] OrderSpec:
[info] An default Order i.e. no orders have been added
[info] - should have zero orders
[info] - should return 0 when tally is invoked
[info] An order when > 1 order(s) have been added
[info] - should have exactly 2 orders when 2 order are added
[info] - should return 13 when tally is invoked for orders of values {1,2,5,5}
[info] InvoiceActorFunParSpec2:
[info] - have one value {400} when one order is issued
[info] - return a tally of 500 after one order of value 500
[info] InvoiceActorFunParSpec:
[info] - have three values {400, 500, 600} when the same three orders are issued
[info] - return a tally of 1,500 after three orders of value 400, 500 & 600
[info] InvoiceActorSpec:
[info] `Invoice` actor taking 2 orders
[info] - should have two values {400, 500} when the same two orders are issued
[info] - should return a tally of 900 after two orders of value 400 & 500
[info] Passed: : Total 10, Failed 0, Errors 0, Passed 10, Skipped 0
[success] Total time: 2 s, completed Sep 16, 2013 10:07:31 AM
The way ScalaTest handles parallel test execution is that it would run the test suites in parallel while each test within the test suite would execute in sequence i.e. written text order. We'll take a look at another form of parallel testing, in the following section. In the second scenario of involving Akka to take care of launching your tests in parallel, then you have the choice of running tests in parallel on a single host, running tests in a coordinated manner across distributed hosts etc.
Concurrent Testing
In the previous section, you've seen how the test framework we've chose allows the developer the capability of writing test logic, validating your actor and also the capability to execute those tests in parallel. Well, Akka has another tool in the toolbox which allows us, to launch tests in parallel across multiple JVM instances. The module we will be discussing is the sbt-multi-jvm and it's different from akka-testkit in that it just cares about one thing: detects the applications it needs to work with and launch each of them into separate JVMs, possibly on different machines.
The sbt-multi-jvm module is actually part of a larger module, akka-multi-node-testkit which allows the developer to craft distributed tests that reflects more of a reality than not i.e. coordinated distributed test. We'll take a look at how to accomplish that a little while later.
In this section, we will take a look at hooking our current tests i.e. written with ScalaTest into sbt-multi-jvm, followed by an illustration on how to launch Scala modules within a test (i.e. controlled) environment. Finally, we'll demonstrate how to use the multi-node testkit.
Setting Up
Akka, the cool dude that he is, has only two requirements in order to get this to work. First, we need to incorporate the plugin into your build file for your project/plugins.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-multi-jvm" % "0.3.8")
You can either restart sbt or issue the update command to the sbt console to update this new setting. The second thing is to include this new addition to the build file, project/Build.scala and for starters, it should look like this:
import sbt._
import Keys._
import com.typesafe.sbt.SbtMultiJvm
import com.typesafe.sbt.SbtMultiJvm.MultiJvmKeys.{ MultiJvm }
object Chapter9Build extends Build {
lazy val root = Project(id = "chapter9-testing-actors",
base = file("."),
settings = Project.defaultSettings ++ multiJvmSettings) configs(MultiJvm)
lazy val multiJvmSettings = SbtMultiJvm.multiJvmSettings ++ Seq(
// make sure that MultiJvm test are compiled by the default test compilation
compile in MultiJvm < val results = testResults ++ multiJvmResults
(Tests.overall(results.values), results)})}
The final thing you need to do is to create a directory src/multi-jvm/scala and start depositing the tests you will write in distributed mode over there. The way in which plugin knows what test to pick out is based on the naming convention that candidates for such execution runs are identified when this string, MultiJvm, is detected as part of the test name and this applies to the name of the source file. As an example of how this particular test would look like now with this modification contained in a file named InvoiceActorMultiJvmNode1.scala (also InvoiceActorMultiJvmNode2.scala which you'll see later is grouped and executed as part of a group named InvoiceActor)
class InvoiceActorMultiJvmNode1 extends TestKit(ActorSystem()) with ImplicitSender
with FlatSpecLike
with BeforeAndAfterAll
with MustMatchers { // as before }
The portion of the above code marked in bold shows the difference; you may wish to contrast this approach against the previous approach we took with ScalaTest and you might actually like it. The mechanism for discovering which tests to execute is based on naming convention. We briefly mentioned earlier that it's looking the files and tests with the same string as MultiJvm and more specifically we are talking about the value abstracted by multiJvmMarker (See the tip below on how to change that value). The test kit starts pouring over the files which respect the naming convention and groups them according to {Test Name}MultiJvm{Node Name} and the following schematics on our directory structure illustrates the schematics where the tests found in those two files would be grouped with the name InvoiceActor and the test output would reflect that they ran on two JVMs.
multi-jvm/
└── scala
├── InvoiceActorMultiJvmNode1.scala
└── InvoiceActorMultiJvmNode2.scala
The following figure illustrates how the tests are grouped, as we described previously.
Tip: If you, or your organization, isn't too keen on the name MultiJvm, you can change it by placing an expression similar to this in your Build.scala file:
multiJvmMarker in MultiJvm := "GroupTest"
and the placement of that expression can be like this:
object Chapter9Build extends Build {
lazy val root = Project(id = "chapter9-testing-actors",
base = file("."),
settings = Project.defaultSettings ++
multiJvmSettings ++
Seq(multiJvmMarker in MultiJvm := "GroupTest")) configs(MultiJvm) // as before
When you next trigger the multiple jvm test to run, it'll look for files which has GroupTest embedded within and start inspecting those files and executing those tests with a name that has GroupTest.
To illustrate how this would work, you would again employ sbt and enter the following expression into the console
> multi-jvm:test
And in our scenario, we have two tests running concurrently i.e. InvoiceActorMultiJvmNode1 and InvoiceActorMultiJvmNode2 and when you trigger that test, you should see an output on the sbt console like the following:
[info] * InvoiceActor
[JVM-1] Run starting. Expected test count is: 2
[JVM-1] InvoiceActorMultiJvmNode1:
[JVM-1] Node1 : `Invoice` actor taking 2 orders
[JVM-2] Run starting. Expected test count is: 2
[JVM-2] InvoiceActorMultiJvmNode2:
[JVM-2] Node2 : `Invoice` actor taking 2 orders
[JVM-1] - should have two values {400, 500} when the same two orders are issued
[JVM-1] - should return a tally of 900 after two orders of value 400 & 500
[JVM-1] Run completed in 636 milliseconds.
[JVM-1] Total number of tests run: 2
[JVM-1] Suites: completed 1, aborted 0
[JVM-1] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[JVM-1] All tests passed.
[JVM-2] - should have two values {400, 500} when the same two orders are issued
[JVM-2] - should return a tally of 900 after two orders of value 400 & 500
[JVM-2] Run completed in 666 milliseconds.
[JVM-2] Total number of tests run: 2
[JVM-2] Suites: completed 1, aborted 0
[JVM-2] Tests: succeeded 2, failed 0, canceled 0, ignored 0, pending 0
[JVM-2] All tests passed.
[success] Total time: 1 s, completed Sep 19, 2013 4:48:01 PM
When you constrast the approach we took here with the one we took with ScalaTest, you'll quickly realize that it doesn't take too much effort to convert those ScalaTests (or whatever test framework you're familiar with) to the ones here. We believe they are not meant to replace those tests you've already prepared with your favourite test framework but its always good to know you have alternatives.
Next, let's take a look at another use case for this module. Let's assume that you have crafted several applications i.e. Scala modules and they start up depending on certain attributes or characteristics that you've represented in a configuration file or even passed them through the command line using the familiar -Dkey=value pairs that we are accustomed in the Java world. The following pseudocode represents a typical (skeletal) Scala module, that starts up based on some assumptions:
object MyApplication {
val property1 = System.getProperty("some_property1")
val property2 = System.getProperty("some_property2")
// ...
def main(args: Array[String]) = {
// Asserting properties
// Start up your application depending on the characteristics of those properties
}
}
At this point, we can enlist the help of sbt-multi-jvm to help us out! Assuming that we have factored out the nasty looking System.getProperty("XXX") methods out to a trait Configuration and mixed in that into our two fictitious Scala modules, the code now looks cleaner and prettier! Now, our Scala modules / applications can mixin the traits and the work of discovering system specific properties is abstracted away.
trait Configuration {
val default = "Nothing"
// System.getProperty is weird...but that's in the Java world...
def getPropertyOrElse(prop: String, default: ⇒ String = default) =
System.getProperty(prop) match { case null ⇒ default; case x ⇒ x }
def getCustomMessageForNode = getPropertyOrElse("custom_message")
def getThreadPoolName = getPropertyOrElse("custom_pool_name")
def getThreadPoolType = Class.forName(getThreadPoolName)
}
object Configuration {
def apply = new Configuration {}
}
object SimpleModuleMultiJvmNode1 extends Configuration with FlatSpecLike with MustMatchers {
def main(args: Array[String]) : Unit = {
val p = getCustomMessageForNode
val pn = getThreadPoolName
pn must include ("ForkJoinPool") // assertion
}
}
object SimpleModuleMultiJvmNode2 extends Configuration with FlatSpecLike with MustMatchers {
def main(args: Array[String]) : Unit = {
val p = getCustomMessageForNode
val pn = getThreadPoolName
pn must include ("ThreadPoolExecutor") // assertion
}
}
In our example, we again followed the naming conventions as described earlier to group those use cases that we would like to categorized them by. When we accomplished this (as shown), we have two modules that can be started up in separate JVMs that tests for the validity of our properties. To trigger this, we input the expression multi-jvm:run SimpleModule into the sbt console and a test failure would result in error messages spewed onto the console and below is a sample run on our sbt console:
> multi-jvm:run SimpleModule [info] Compiling 5 Scala sources to /Users/raymondtay/akka-edge/ch9-testing-actors/target/scala-2.10/multi-jvm-classes... [warn] there were 2 deprecation warning(s); re-run with -deprecation for details [warn] one warning found [info] * SimpleModule [success] Total time: 5 s, completed Sep 22, 2013 9:51:26 PM
Before concluding this section, let us demonstrate to you how we can bootstrap our InvoiceActor into a Scala module and have it run a test; this covers the use case where you have actors and you like them to be launched from a Scala module that resembles a standalone application. Here's what happened to that piece of code, now, as compared to what we did earlier
object SimpleModuleMultiJvmNode1 extends TestKit(ActorSystem("SimpleModuleAS")) with Configuration with ImplicitSender
with FlatSpecLike
with MustMatchers
with BeforeAndAfterAll {
def doAction(f: ⇒ Unit) = f
def main(args: Array[String]) : Unit = {
try doAction{
val p = getCustomMessageForNode
val pn = getThreadPoolName
pn must include ("ForkJoinPool")
val invoice = TestActorRef[InvoiceActor]
invoice ! Checkin(500)
invoice ! Checkout
expectMsg(500)
} finally afterAll
}
override def afterAll = { system.shutdown }
}
What happened here is that we used the previous technique of creating a TestActorRef to wrap around our InvoiceActor (which is the real actor we want to protect) and have it execute the simple test of checking in and out and making sure that the amount he/she pays is exactly what he/she ordered. Once the actor has been bootstrapped into the module, you could probably recognize that the test pretty much stays the same and we can still conduct an array of tests on the use case but from the standpoint of a single JVM (as compared to within a test suite for example).
Multi-Node Testing
Previously, we demonstrated that you can improve your testing cycle by running your tests concurrently but that doesn't really tell the whole story because a big part of your actor application would be communicating with other machines in the network. As you continue to write actor code, you would quickly realize that testing on a single host would out lived its usefulness and we need to find some manner to put the application to a distributed test. The following simplified diagram illustrates the concurrent testing we have encountered so far where actor-1 delivers the payload to actor-3 through actor-2 (which we'll assume is performing some middle-man processing) and the right-hand side of the diagram illustrates how each actor code is tested by launching parallel JVMs.
When we want to test our application in a distributed way, ideally we would need a framework that allows our actor code to be hosted within a test framework like ScalaTest and the tests would be packaged along with our dependencies and shipped off to the destination nodes and begins the distributed testing. Fortunately for us, the framework is already present and is known as akka-multi-node-testkit and we are going to take a look at it.
How this testkit works is that there is a TestConductor that coordinates and runs the tests across the nodes that we have deployed our application to. The next thing is for us to hook our tests to a specification which allows our tests to be hooked to the multi-node test framework's lifecycle and its called MultiNodeSpecCallbacks (a Scala trait) and the two abstract methods in it allows the developer to define lifecycle methods that controls what happens before and after all the tests are run by providing your own version of the trait's methods multiNodeSpecAfterAll and multiNodeSpecBeforeall. Next, we need to provide a configuration object that allows this testkit to know how many nodes are undergoing the test and placing barriers at appropriate places to correctly simulate the test; these barriers are meant to simulate the various startup/shutdown sequences of your machines or application code. Let's begin by setting this up.
Setting up
Edit the build.sbt file and include the following dependency to it:
"com.typesafe.akka" %% "akka-multi-node-testkit" % "2.2.1"
and update the project by triggering the update command to the sbt console and sbt should shortly download the dependency to your local file system if it hasn't already done so. This testkit allows your test to be delivered and executed on various machines, so its probably a good idea to create password-less SSH access to those distributed nodes designated for your tests.
Example
Let's take a further look by building the classic example of messaging forwarding involving three actors. The first actor will begin by sending the message to the middle-actor and that would be forwarded to the last actor, by which the last actor will send a reply back to the original actor. Next, our test environment would consists of a single host that's attached to two virtual machines with the localhost acting as the master / TestConductor for this example. We have configured password-less access to those virtual machines via SSH using the conventional techniques (e.g. creating a passwordless ssh access within a trusted data centre environment). The diagram below illustrates our scenario
The following code demonstrates this (we'll explain in more detail in a short while)
import org.scalatest.{BeforeAndAfterAll, WordSpecLike}
import org.scalatest.matchers.MustMatchers
import akka.actor._
import akka.testkit.ImplicitSender
import akka.remote.testkit._
// Your specification should mixin what you need for your test
trait ForwardMultiNodeSpec extends MultiNodeSpecCallbacks
with WordSpecLike
with MustMatchers
with BeforeAndAfterAll {
override def beforeAll() = multiNodeSpecBeforeAll()
override def afterAll() = multiNodeSpecAfterAll()
}
//Create as many `roles` as you have `nodes`
object MConfig extends MultiNodeConfig {
val node1 = role("Actor-1")
val node2 = role("Actor-2")
val node3 = role("Actor-3")
}
// the sbt-multi-jvm testkit would launch 1 `class` per `node
class ForwardMNMultiJvmNode1 extends ForwardMultipleNodeDemo
class ForwardMNMultiJvmNode2 extends ForwardMultipleNodeDemo
class ForwardMNMultiJvmNode3 extends ForwardMultipleNodeDemo
class ForwardMultipleNodeDemo extends MultiNodeSpec(MConfig)
with ForwardMultiNodeSpec
with ImplicitSender {
import MConfig._
def initialParticipants = roles.size
// Override these two methods if you like to conduct some
// setup/shutdown for `all` tests
override def atStartup() = println("STARTING UP!")
override def afterTermination() = println("TERMINATION!")
"A multi-node illustrating `message-forwarding`" must {
"wait for all nodes to enter barrier" in {
enterBarrier("startingup")
}
"send to and receive from a remote node via `forwarding`" in {
runOn(node1) {
enterBarrier("deployed")
var msgReturned = false
val next = system.actorFor(node(node2) / "user" / "MiddleMan")
val me = system.actorOf(Props(new Actor with ActorLogging {
def receive = {
case "hello" ⇒ next forward (self, "hello")
case ("hello",Stop) ⇒ condition = true
}}), "Initiator")
me ! "hello"
awaitCond(msgReturned == true)
}
runOn(node2) {
val next = system.actorFor(node(node3) / "user" / "Destinator")
system.actorOf(Props(new Actor with ActorLogging {
def receive = {
case (to, msg) ⇒ next forward (to, msg)
}}), "MiddleMan")
enterBarrier("deployed")
}
runOn(node3) {
system.actorOf(Props(new Actor with ActorLogging {
def receive = {
case (to:ActorRef, msg) ⇒ to ! (msg, Stop)
}}), "Destinator")
enterBarrier("deployed")
}
enterBarrier("finished")
}
}
}
Let's go through in more detail what we've done over here. What we did was to craft a test that consists of three actors with names Initiator, MiddleMan and Destinator; each of these actors are to be executed on the nodes node1, node2, node3 respectively through the function, runOn. If you examine the actors code, you would notice that its just regular akka actor code but there's a caveat and that is we need to make sure the order of startup is correct for our small simulation.
The Initiator would need to be started up after MiddleMan and Destinator otherwise Initiator would not be able to locate the appropriate actor to forward the message to, and this testkit gave us the answer by allowing us to place barriers i.e. barriers dictate that no one crosses the barrier untill everyone has arrived at the barrier.
Using the barrier, we can control the order of the startup and in our example MiddleMan and Destinator would startup before Initiator (which is what we want). Notice that we place barriers upon startup and shutdown in our test lifecycle and that is probably a good practice to abide by since you won't run into situations where the tests runs out of control because of these synchronization issues. When writing tests, there's often a placement in the framework which prepares and subsequently releases resources for the tests, akin to test fixtures. This testkit has it too and if you want to modify the default behavior beyond the default, then you need to provide your own implementations of:
def atStartup() : Unit def afterTermination() : Unit
which alters the default behavior, which is to do nothing, when your tests are starting up or shutting down. In our example, we simply gave it a printout and during the test run, you would notice this output. To give this example a run, enter the expression into the sbt console:
multi-node-test-only ForwardMN
and the testkit would package our application and its dependencies together producing an artifact jar ch9-testing-actors_2.10.2-1.0-multi-jvm-assembly.jar and transports this over to the hosts. Here's a sample output:
[info] Including protobuf-java-2.4.1.jar [info] Including akka-actor_2.10-2.2.1.jar [info] Including akka-multi-node-testkit_2.10-2.2.1.jar ... some lines omitted [info] Including netty-3.6.6.Final.jar [info] Including scalatest_2.10-2.0.M7.jar [info] Including uncommons-maths-1.2.2a.jar [info] Including scala-library.jar [info] SHA-1: WrappedArray(-22, 53, -18, 49, -65, 38, -30, -5, 89, -62, 123, -65, -86, 27, -85, 90, -77, 104, -55, 19) [info] Packaging /Users/tayboonl/akka-edge/ch9-testing-actors/target/ch9-testing-actors_2.10.2-1.0-multi-jvm-assembly.jar ... [info] Done packaging. [info] * ForwardMN ... more lines omitted reporting test results
and you can correlate this action by observing that several SSH processes have been started and would live as long as the tests lasts, and the test would report that it completely successfully !
Timed Tests
Thus far, we have made no assumptions about how quickly should our transactions should complete and to be specific we made no assertions about the latency we are willing to compromise for when placing orders or checking out. The akka-testkit provides us a mini-DSL in which we can express our latency concerns by placing the logic in which we want accomplished within a time range using this
within([min, ] max) {
// time sensitive business logic
}
What this means is the business logic must be completed between the timed window between (min, max) and this mini-DSL can be embedded. Let's take a look at how our tests can be crafted to account for this. Let's assume that our business owner lays the requirement that any single order should take no more than 0.5 seconds to register while 2 or more orders should take no more than 0.8 seconds to register then our system can be validated for this requirement with the following changes made to our code
class InvoiceActorTimeSpec extends TestKit(ActorSystem()) with ImplicitSender
with FlatSpecLike
with BeforeAndAfterAll
with MustMatchers
with ParallelTestExecution {
behavior of "`Invoice` actor taking 2 orders must complete within 800 millis"
import scala.concurrent.duration._
it should "have two values {400, 500} when the same two orders are issued" in {
val actor = TestActorRef[InvoiceActor]
val ref = actor.underlyingActor
within (800 millis) {
within(500 millis) {
actor ! Checkin(400)
ref.topOfOrder must be (400)
}
within(500 millis) {
actor ! Checkin(500)
ref.topOfOrder must be (500)
}
}
}
On the subject of time comes the question: "Would the tests still give the same results when executed on a faster machine compared to a slower machine? " many of the testing techniques with Akka carries a timeout value in which the test must complete or it would report test failures. Most of timeout values are implicit but they can be made explicit and in situations where you need to deliberately slow down the test(s), you can include the expression in your tests:
1.second.dilated
That's with the assumption of that the expression is given within a ActorSystem with the necessary imports. Be careful when the criteria for the deemed success or failure of your tests depends on time since a heavily loaded system would mean that your application has to compete with other stuff for CPU-/IO-time; tests should be, afterall, deterministic.