tysonjh

making mistakes everyday

Spray Custom 404

| Comments

Spray is a great web framework for slapping an app together quickly once you know your way around. Also it uses Akka and that’s cool. Having a custom 404 page is standard on the interwebs these days, but it took me a bit of work to figure out how to implement it with Spray.

Goals

A simple app that has routes and serves static assets. If an asset can not be loaded, a custom 404 page should be returned.

Solution Highlights

Spray uses directives to express complex routing patterns elegantly. There are many predefined directives available which should satisfy most application needs. Directives are composable and extensible. They can be nested and/or chained together using the ~ operator, combined with | and & operators – the possibilities are quite endless.

To implement the custom 404 page, we will make use of the following predefined directives:

  • path: to match the complete unmatched path
  • respondWithStatus: to override the inner route status code
  • getFromFile: to get the static asset from disk (or recover with the 404 template from disk)
  • handleRejections: to handle the rejection when getFromFile can not find the static asset
Defining a RejectionHandler
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
//import spray.routing.{ Directives, Route }

val notFoundTemplate = new File("public/404.html")

val assetNotFoundHandler = RejectionHandler {
  case Nil /* secret code for path not found */ 
    respondWithStatus(404) {
      getFromFile(notFoundTemplate)
    }
}

def staticAssets: Route = path(Rest) { pathRest: String 
  handleRejections(assetNotFoundHandler) {
    getFromFile(new File(Conf.publicAssetDir + pathRest))
  }
}

The notFoundTemplate value is a plain old HTML file. The staticAssets method can be combined with other directives.

Using the Static Assets Route
1
2
3
4
5
6
val routes: Route = {
  path("helloworld") {
    complete("Hello World!")
  } ~
  staticAssets
}

The staticAssets route will happily load all of the .js, .css, .html, .txt, etc. files. In fact, I’m using this exact approach with the angular and bootstrap frameworks in an app right now.

Notice that the static asset directive is the last in the chain. This makes it the final attempt to resolve the URL before returning the 404. Quite a common approach in various sample projects and production solutions alike.

Scala DelayedInit Trait Explored

| Comments

The App trait is often used for making easily executable code. It is a display of how elegant the language can be. A hook for those new to the Scala language. Put a “Hello World!” app from Scala beside a Java app and you’ll notice among other things the lack of a main method from the former. Cool – look how succinct Scala is!

The darker side of this nifty feature is rather subtle. Especially until the open major bug since 2.9.1 is resolved. I came across it while using the Smoke HTTP service library and mixing it into a trait. Smoke extends the DelayedInit trait for initializing the server before executing the body of the main App object. It’s a nice use case.

The Scaladoc for DelayedInit explicitly states that traits don’t automagically benefit from mixing in DelayedInit like a class or object would. The doc has an example of how to define the delayedInit method within a trait that will be executed as expected. In search of the SI-4680 bug I performed a series of investigative scenarios, which are presented here.

The gist is here.

Test Traits

A simple trait that extended the DelayedInit trait, along with a couple others for mixins.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
trait Stage1 extends DelayedInit {
  println("Stage1 constructor")

  override def delayedInit(x: => Unit) {
    println("Stage1 delayedInit before x")
    x
    println("Stage1 delayedInit after x")
  }
}

trait Stage2 {
  println("Stage2 constructor")
}

trait Stage3 {
  println("Stage3 constructor")
}

Traits can be mixed in using several different techniques. Here are some with regular mixins and some with dependency injection.

1
2
3
4
5
6
7
8
9
10
11
12
13
trait Stage123 extends Stage1 with Stage2 with Stage3 {
  println("Stage123 constructor")
}

trait Stage1Di23 extends Stage1 {
  this: Stage2 with Stage3 =>
  println("Stage1Di23 constructor")
}

trait Stage12Di3 extends Stage1 with Stage2 {
  this: Stage3 =>
  println("Stage12Di3 constructor")
}

SI-4680 Bug Example

A trait with The empty initialization code in the instantiated trait results in the delayedInit method not being invoked.

1
2
3
4
5
6
7
8
9
10
11
12
/*
Stage1 constructor
Stage2 constructor
Stage3 constructor
Stage123 constructor
Main application constructor end
 */
object Main123 extends App {
  println("Main application constructor start")
  new Stage123 {}
  println("Main application constructor end")
}

The result of running this object is shown in the comments. All of the initializations of the various traits were executed, but the delayedInit method was never called! Note that it is not possible to instantiate a trait, there is no constructor. By providing {} we are providing a constructor and an anonymous class that subclasses the trait Stage123. This is the empty constructor that results in bug SI-4680.

SI-4680 Bug Workaround

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
/*
Main application constructor start
Stage1 constructor
Stage2 constructor
Stage3 constructor
Stage123 constructor
Stage1 delayedInit before x
Stage123 NonEmpty constructor
Stage1 delayedInit after x
Main application constructor end
 */
object Main123NonEmpty extends App {
  println("Main application constructor start")
  new Stage123 {
    println("Stage123 NonEmpty constructor")
  }
  println("Main application constructor end")
}

In this example the constructor is populated with a println for illustrating the example but the constructor could be { { } } or {Unit}, comments don’t count.

Dependency Injection

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
trait Stage1Di23 extends Stage1 {
  this: Stage2 with Stage3 =>
  println("Stage1Di23 constructor")
}

/*
Main application constructor start
Stage1 constructor
Stage1Di23 constructor
Stage2 constructor
Stage3 constructor
Stage1 delayedInit before x
Stage1Di23 NonEmpty constructor
Stage1 delayedInit after x
Main application constructor end
 */
object Main1Di23NonEmpty extends App {
  println("Main application constructor start")
  new Stage1Di23 with Stage2 with Stage3 {
    println("Stage1Di23 NonEmpty constructor")
  }
  println("Main application constructor end")
}

There are lots of great articles on DI in Scala, don’t use this as one. What is interesting here is the order of execution. The body of our instantiated anonymous class of type Stage1Di23 with Stage2 with Stage3 is executed within the delayedInit method now that it is not empty. I was surprised by the output, what I expected to see was Stage1, Stage1Di23, Stage3, Stage2, following the idea where traits are generally layered on from right to left. To better understand what is happening with the template evaluation, the Stage2 trait has been mixed in as a base class instead of an injected dependency,

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
trait Stage12Di3 extends Stage1 with Stage2 {
  this: Stage3 =>
  println("Stage12Di3 constructor")
}

/*
Main application constructor start
Stage1 constructor
Stage2 constructor
Stage12Di3 constructor
Stage3 constructor
Stage1 delayedInit before x
Stage12Di3 NonEmpty constructor
Stage1 delayedInit after x
Main application constructor end
 */
object Main12Di3NonEmpty extends App {
  println("Main application constructor start")
  new Stage12Di3 with Stage3 {
    println("Stage12Di3 NonEmpty constructor")
  }
  println("Main application constructor end")
}

This clears things up a bit more. This is similar to the Example 5.1.1 from the Scala Specification,

Example 5.1.1 Consider the following class definitions:
class Base extends Object {}
trait Mixin extends Base {}
object O extends Mixin {}
In this case, the definition of O is expanded to:
object O extends Base with Mixin {}

http://www.scala-lang.org/docu/files/ScalaReference.pdf Scala Specification

It would be nice to see the actual output from the compiler. The scalac -print option can do this and removes all Scala specific features and print to standard output. Now it’s clear that first the superclass constructor of the anonymous class is evaluated. Followed by constructors of all base classes that are mixed in, in reverse order of the occurrence in linearization. Then the delayedInit method is called for each trait that extends the DelayedInit trait and implements the delayedInit method, first App, then Stage1.

1
2
3
4
5
6
7
8
9
10
11
12
final class Test$$anon$1 extends Object with Stage12Di3 with Stage3 {
  override def delayedInit(x: Function0): Unit = Stage1$class.delayedInit(Test$$anon$1.this, x);
  def <init>(): anonymous class anon$1 = {
    Test$$anon$1.super.<init>();
    Stage1$class./*Stage1$class*/$init$(Test$$anon$1.this);
    Stage2$class./*Stage2$class*/$init$(Test$$anon$1.this);
    Stage12Di3$class./*Stage12Di3$class*/$init$(Test$$anon$1.this);
    Stage3$class./*Stage3$class*/$init$(Test$$anon$1.this);
    Test$$anon$1.this.delayedInit(new anonymous class anon$1$delayedInit$body(Test$$anon$1.this));
    ()
  }
}

Scala Macros: Accessing Case Class Parameters

| Comments

Gaining access to a case class’ constructor parameters within a macro implementation was not obvious at first. One approach is to make use of the WeakTypeTag. Consider the example macro implementation with the following signature,

1
def macroImpl[A: c.WeakTypeTag](c: Context)

To gain access to the Type of A,

1
val tpe: Type = weakTypeOf[A]

The trait Type has a method declarations that returns a MemberScope which is essentially an Iterable[Symbol] collection. The Symbol is of interest, for it contains useful information about anything in Scala that can be assigned a name.

1
val decl: MemberScope  = tpe.declarations

A simple collect and the right partial function and presto,

1
2
3
val params: Iterable[Name] = decl.collect {
  case param if param.isMethod && param.asMethod.isCaseAccessor => field.name
}

params contains the iterable collection of the case class parameter names.

Feature Toggling Using Scala Quasiquote Macros

| Comments

It’s tough to dream up reasons to use macros in Scala. When our team began considering feature toggling to avoid experiencing another merging nightmare, it seemed like an excellent use case for a macro. A feature toggle is simply a construct that wraps a block of code, executing it if the feature is enabled. It is also referred to as kill switching, feature flipping, etc. If you are interested on learning more, check out this article by Martin Fowler. According to Fowler’s article the toggles we will be supporting are business toggles. Here is the basic idea,

Toggle Flip Pseudocode
1
2
3
4
5
if(featureEnabled) {
  doFeatureStuff()
} else {
  doNotDoFeatureStuff()
}

Scala macros are great for generating code, so it should be a simple matter to generate the conditional wrapper around a feature block.

Goals

  1. Macro is simple to use
  2. Macro wraps feature code with a conditional
  3. Macro boolean condition implementation is extensible

The third point is to facilitate different mechanisms for querying a feature’s toggle state. The first use case for this feature will be writing hard wired scenarios of the boolean condition for testing purposes. Already I am working on a Redis backed boolean conditional that polls periodically for changes in feature flags.

Macros

Jumping right into the coding turned out to be the wrong approach. Macro programming in Scala is tough. It requires an intimate knowledge of how the AST is built and the symbols used. The documentation scratches the surface but leaves many questions unanswered. Finding examples online helped but I still struggled to massage them into a workable solution. Furthermore the examples I found used a lot of deprecated methods.

I’m not going to bother writing about directly manipulating the AST to create macros in Scala. At the end of this post exists a link section, in it you will find some great articles and examples of manipulating the AST. We are going to take a look at using quasiquotes.

Quasiquotes

Shipping with Scala 2.11 but supported in 2.10 with the macro paradise plugin is quasiquotes. The basic idea is that they allow macros to be written similar to normal Scala code. Quasiquotes use the familiar concept of string interpolation. If you are new to writing macros start with quasiquotes!

There is an excellent sbt-example-paradise repository on github that will help get started.

Define the macro

Flipper Macro DefinitionSource File
1
2
3
4
trait Dolphin {
  def flipper[T](featureName: String, config: QueryableConfig)
                (fn: => T): Unit = macro DolphinMacro.flipperImpl[T]
 }

Nothing too magical happening here, the macro keyword is followed by the name of the static macro implementation method.

Implement the macro

Flipper Macro ImplementationSource File
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
object DolphinMacro {
  def flipperImpl[T](c: Context)
                    (featureName: c.Expr[String],
                     config: c.Expr[QueryableConfig])
                    (fn: c.Expr[T]): c.Expr[Unit] = {
    import c.universe._
    val result = {
      q"""
        if($config.isFeatureOn($featureName)) {
          $fn
        } else {}
      """
    }
    c.Expr[Unit](result)
  }
}

This represents the implementation of the macro using quasiquotes. The import of c.universe._ is common in most macro definitions. It provides many useful methods, for us it provides the q string interpolator on line 8. This is the start of the quasiquote. Appreciate how much it looks like plain old scala, compare it to any AST constructions on the macro overview page on the Scala macro documentation to understand how awesome this is.

Conceptually we are printing out the symbols that construct the desired outcome and letting the macro paradise plugin handle the rest of the heavy lifting. Much easier to read isn’t it? Compare it to the showRaw output of the AST we would need to build without quasiquotes in res0:

The desired AST
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
scala> import scala.reflect.runtime.{universe => u}
import scala.reflect.runtime.{universe=>u}

scala> def flipper(featureEnabled: Boolean)(doFeatureStuff: => Unit)(doNotDoFeatureStuff: => Unit) = {
     | if(featureEnabled){ doFeatureStuff }
     | else doNotDoFeatureStuff
     | }
flipper: (featureEnabled: Boolean)(doFeatureStuff: => Unit)(doNotDoFeatureStuff: => Unit)Unit

scala> val expr = u reify (flipper _)
expr: reflect.runtime.universe.Expr[Boolean => ((=> Unit) => ((=> Unit) => Unit))] =
Expr[Boolean => ((=> Unit) => ((=> Unit) => Unit))] ({((featureEnabled) => ((doFeatureStuff) => ((doNotDoFeatureStuff) => $read.flipper(featureEnabled)(doFeatureStuff)(doNotDoFeatureStuff))))})

scala> u showRaw expr.tree
res0: String = Block(List(), Function(List(ValDef(Modifiers(PARAM | SYNTHETIC), newTermName("featureEnabled"), TypeTree(), EmptyTree)), Function(List(ValDef(Modifiers(PARAM | SYNTHETIC), newTermName("doFeatureStuff"), TypeTree(), EmptyTree)), Function(List(ValDef(Modifiers(PARAM | SYNTHETIC), newTermName("doNotDoFeatureStuff"), TypeTree(), EmptyTree)), Apply(Apply(Apply(Select(Select(Select(Ident($line7.$read), newTermName("$iw")), newTermName("$iw")), newTermName("flipper")), List(Ident(newTermName("featureEnabled")))), List(Ident(newTermName("doFeatureStuff")))), List(Ident(newTermName("doNotDoFeatureStuff"))))))))

A more accurate way according to Imran Rashid’s great post on Learning Scala Macros is to let the scalac compiler do it for you (below). Though I still prefer quasiquotes to this!

1
scalac -Xplugin macro-paradise_2.10.2-2.0.0-SNAPSHOT.jar -deprecation -Xprint:parser -Ystop-after:parser -Yshow-trees-compact *.scala

Test the macro

Flipper Macro TestsSource File
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
class DolphinSpec extends FunSpec with Dolphin {
  def featureAlwaysOnConfig = new QueryableConfig {
    def isFeatureOn(name: String): Boolean = true
  }

  def featureAlwaysOffConfig = new QueryableConfig {
    def isFeatureOn(name: String): Boolean = false
  }

  describe("Flipper") {
    it("should execute function if feature is on"){
      var cnt = 0
      flipper("a", featureAlwaysOnConfig){cnt = cnt + 1}
      assert(cnt == 1)
    }

    it("should not execute function if feature is off"){
      var cnt = 0
      flipper("a", featureAlwaysOffConfig){cnt = cnt + 1}
      assert(cnt == 0)
    }
  }
}      

This is how the macro can be tested, again nothing special here. An interesting detail to note is that in the macro definition we have a call by name parameter fn: => T, but in this implementation of the macro we have the parameter fn: c.Expr[T] which appears to be call by value. The tests clearly show that the call by name parameter is only evaluated by name. Attempting to change the implementation to fn: c.Expr[=> T] or fn: => c.Expr[T] causes a compilation error.

The limitation is that one may only inline fn and not invoke it, at least until Scala 2.11 where fn: c.Tree may be used as the implementation parameter (see here).

Summary

Quasiquotes are a welcomed addition to the macro scene. They make writing macros more accessible by eliminating the necessity for deep AST knowledge before one gets started. The code in this post is part of the Dolphin library for feature toggling in Scala.

Links

Hello Octopress

| Comments

This is a POC for migrating my blog to Jekyll. I’m going to be focussing on technical content, more specifically software development.