While I find it fun to write SBT plugins, I’ve always found them tricky to test. Yes, there is the scripted plugin but each time I read the documentation, it put me off using it. It seemed complex with too many steps involved.

More recently I decided to give scripted another go. It was a little frustrating at first because the documentation is not specific enough about which directories need which files. There are a bunch of similarly named files and directories which need to go into specific locations. Confused? I was. But then I had a look at some of the examples provided and pieced together the required steps which I have outlined below.

Use a SNAPSHOT version

If you’re not already doing so, update your plugin version in the build.sbt file to use a -SNAPSHOT suffix. When the scripted plugin runs, it will locally install your plugin. By using a snapshot version, you prevent version clashes when you do finally publish your plugin; such as where your version X locally is different to your published version X.

Within Plugin Root Directory

  1. Create a file named project/scripted.sbt with the following content:
libraryDependencies += { "org.scala-sbt" % "scripted-plugin" % sbtVersion.value }

This just adds the scripted plugin as a dependency to your project.

  1. Create a file named scripted.sbt with the following content:
ScriptedPlugin.scriptedSettings

scriptedLaunchOpts := { scriptedLaunchOpts.value ++
  Seq("-Xmx1024M", "-XX:MaxPermSize=256M", "-Dplugin.version=" + version.value)
}

scriptedBufferLog := false

The above defines the options supplied to the scripted plugin when it is run. It also supplies the plugin version which will be taken from the version value defined in your build.sbt file.

Within src/sbt-test Directory

One of the steps is to create a new test source directory for your scripted tests which resides in: src/sbt-test. This will be further subdivided by [testGroup] and [testName] to be of the form: src/sbt-test/testGroup/testName.

For example to write a simple test for your XYZ plugin you’d create the following directory structure:

src/sbt-test/XYZ/simple

The above test directory (simple) contains a full sbt project within it which will be used to test your plugin. This sbt project will be copied to a temporary directory when the scripted plugin is run.

Within your test directory create the following files:

  1. src/sbt-test/XYZ/simple/build.sbt

This is the build file for the project that will test your plugin. A basic example is given below:

lazy val root = (project in file("."))
  .settings(
    version := "0.1",
    scalaVersion := "2.10.6"
    //any other config you need here
  )
  1. src/sbt-test/XYZ/simple/project/plugins.sbt

This is the plugin file for the project that will test your plugin. It will need to include your plugin as a dependency:

sys.props.get("plugin.version") match {
  case Some(x) => addSbtPlugin("your_org" % "your_plugin_name" % x)
  case _ => sys.error("""|The system property 'plugin.version' is not defined.
                         |Specify this property using the scriptedLaunchOpts -D.""".stripMargin)
}

Now you have a basic sbt project that uses your plugin as a dependency.

  1. src/sbt-test/testGroup/testName/test

This is the test script that will be called from scripted. Write it using the script syntax specified.

For example to verify that your plugin works and creates a file called your_output_file in the target directory use:

> yourPluginTask
$ exists target/your_output_file

Make sure the the test script is executable:

chmod +x test

Run the Script

Run the scripted plugin through SBT to run your test script with:

> scripted

As far as I can tell scripted does the following:

  1. Installs your plugin SNAPSHOT to your local ivy cache.
  2. For each src/sbt-test/testGroup/testName directory, copies the content to a temporary directory.
  3. Runs your test script in the temporary directory.

Here is the structure of the sbt-test directory in my sbt-scuggest plugin:

src
└── sbt-test
    └── sbt-scuggest
        ├── emptyProject
        │   ├── build.sbt
        │   ├── project
        │   │   └── plugins.sbt
        │   └── test
        ├── existingProject
        │   ├── build.sbt
        │   ├── existingProject.sublime-project
        │   ├── project
        │   │   └── plugins.sbt
        │   └── test
        └── simulate
            ├── build.sbt
            ├── project
            │   └── plugins.sbt
            └── test

Notice how each testName directory contains a full SBT project.

Here is the sample output for my emptyProject test:

> scripted
[info] Packaging /Volumes/Work/projects/code/scala/toy/sbt-scuggest/target/scala-2.10/sbt-0.13/sbt-scuggest-0.0.6.0-SNAPSHOT-sources.jar ...
[info] Done packaging.
[info] Updating {file:/Volumes/Work/projects/code/scala/toy/sbt-scuggest/}sbt-scuggest...
[info] Resolving org.scala-sbt#sbt-launch;0.13.8 ...
[info] Done updating.
[info] :: delivering :: net.ssanj#sbt-scuggest;0.0.6.0-SNAPSHOT :: 0.0.6.0-SNAPSHOT :: integration :: Thu Sep 07 00:06:33 AEST 2017
[info]  delivering ivy file to /Volumes/Work/projects/code/scala/toy/sbt-scuggest/target/scala-2.10/sbt-0.13/ivy-0.0.6.0-SNAPSHOT.xml
[info] Main Scala API documentation to /Volumes/Work/projects/code/scala/toy/sbt-scuggest/target/scala-2.10/sbt-0.13/api...
[info] Compiling 2 Scala sources to /Volumes/Work/projects/code/scala/toy/sbt-scuggest/target/scala-2.10/sbt-0.13/classes...
model contains 17 documentable templates
[info] Main Scala API documentation successful.
[info] Packaging /Volumes/Work/projects/code/scala/toy/sbt-scuggest/target/scala-2.10/sbt-0.13/sbt-scuggest-0.0.6.0-SNAPSHOT-javadoc.jar ...
[info] Done packaging.
[info] Packaging /Volumes/Work/projects/code/scala/toy/sbt-scuggest/target/scala-2.10/sbt-0.13/sbt-scuggest-0.0.6.0-SNAPSHOT.jar ...
[info] Done packaging.
[info]  published sbt-scuggest to /Users/sanj/.ivy2/local/net.ssanj/sbt-scuggest/scala_2.10/sbt_0.13/0.0.6.0-SNAPSHOT/jars/sbt-scuggest.jar
[info]  published sbt-scuggest to /Users/sanj/.ivy2/local/net.ssanj/sbt-scuggest/scala_2.10/sbt_0.13/0.0.6.0-SNAPSHOT/srcs/sbt-scuggest-sources.jar
[info]  published sbt-scuggest to /Users/sanj/.ivy2/local/net.ssanj/sbt-scuggest/scala_2.10/sbt_0.13/0.0.6.0-SNAPSHOT/docs/sbt-scuggest-javadoc.jar
[info]  published ivy to /Users/sanj/.ivy2/local/net.ssanj/sbt-scuggest/scala_2.10/sbt_0.13/0.0.6.0-SNAPSHOT/ivys/ivy.xml
Running sbt-scuggest / emptyProject
[error] Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256M; support was removed in 8.0
[info] Getting org.scala-sbt sbt 0.13.8 ...
[info] :: retrieving :: org.scala-sbt#boot-app
[info]  confs: [default]
[info]  52 artifacts copied, 0 already retrieved (17674kB/116ms)
[info] Getting Scala 2.10.4 (for sbt)...
[info] :: retrieving :: org.scala-sbt#boot-scala
[info]  confs: [default]
[info]  5 artifacts copied, 0 already retrieved (24459kB/57ms)
[info] [info] Loading project definition from /private/var/folders/zr/63yqmtjx6yn4k35f4kvvtnqw0000gn/T/sbt_3224922a/emptyProject/project
[info] [info] Updating {file:/private/var/folders/zr/63yqmtjx6yn4k35f4kvvtnqw0000gn/T/sbt_3224922a/emptyProject/project/}emptyproject-build...
[info] [info] Resolving net.ssanj#sbt-scuggest;0.0.6.0-SNAPSHOT ...
       [info] Resolving net.ssanj#sbt-scuggest;0.0.6.0-SNAPSHOT ...
       [info] Resolving com.typesafe.play#play-json_2.10;2.4.8 ...
       [info] Resolving org.scala-lang#scala-library;2.10.4 ...
       [info] Resolving com.typesafe.play#play-iteratees_2.10;2.4.8 ...
       [info] Resolving org.scala-stm#scala-stm_2.10;0.7 ...
       [info] Resolving com.typesafe#config;1.3.0 ...
       [info] Resolving com.typesafe.play#play-functional_2.10;2.4.8 ...
       [info] Resolving com.typesafe.play#play-datacommons_2.10;2.4.8 ...
       [info] Resolving joda-time#joda-time;2.8.1 ...
       [info] Resolving org.joda#joda-convert;1.7 ...
       [info] Resolving org.scala-lang#scala-reflect;2.10.4 ...
       [info] Resolving com.fasterxml.jackson.core#jackson-core;2.5.4 ...
       [info] Resolving com.fasterxml.jackson.core#jackson-annotations;2.5.4 ...
       [info] Resolving com.fasterxml.jackson.core#jackson-databind;2.5.4 ...
       [info] Resolving com.fasterxml.jackson.datatype#jackson-datatype-jdk8;2.5.4 ...
       [info] Resolving com.fasterxml.jackson.datatype#jackson-datatype-jsr310;2.5.4 ...
       [info] Resolving org.scala-sbt#sbt;0.13.8 ...
       [info] Resolving org.scala-sbt#main;0.13.8 ...
       [info] Resolving org.scala-sbt#actions;0.13.8 ...
       [info] Resolving org.scala-sbt#classpath;0.13.8 ...
       [info] Resolving org.scala-lang#scala-compiler;2.10.4 ...
       [info] Resolving org.scala-sbt#launcher-interface;0.13.8 ...
       [info] Resolving org.scala-sbt#interface;0.13.8 ...
       [info] Resolving org.scala-sbt#io;0.13.8 ...
       [info] Resolving org.scala-sbt#control;0.13.8 ...
       [info] Resolving org.scala-sbt#completion;0.13.8 ...
       [info] Resolving org.scala-sbt#collections;0.13.8 ...
       [info] Resolving jline#jline;2.11 ...
       [info] Resolving org.scala-sbt#api;0.13.8 ...
       [info] Resolving org.scala-sbt#compiler-integration;0.13.8 ...
       [info] Resolving org.scala-sbt#incremental-compiler;0.13.8 ...
       [info] Resolving org.scala-sbt#logging;0.13.8 ...
       [info] Resolving org.scala-sbt#process;0.13.8 ...
       [info] Resolving org.scala-sbt#relation;0.13.8 ...
       [info] Resolving org.scala-sbt#compile;0.13.8 ...
       [info] Resolving org.scala-sbt#classfile;0.13.8 ...
       [info] Resolving org.scala-sbt#persist;0.13.8 ...
       [info] Resolving org.scala-tools.sbinary#sbinary_2.10;0.4.2 ...
       [info] Resolving org.scala-sbt#compiler-ivy-integration;0.13.8 ...
       [info] Resolving org.scala-sbt#ivy;0.13.8 ...
       [info] Resolving org.scala-sbt#cross;0.13.8 ...
       [info] Resolving org.scala-sbt.ivy#ivy;2.3.0-sbt-fccfbd44c9f64523b61398a0155784dcbaeae28f ...
       [info] Resolving com.jcraft#jsch;0.1.46 ...
       [info] Resolving org.scala-sbt#serialization_2.10;0.1.1 ...
       [info] Resolving org.scala-lang.modules#scala-pickling_2.10;0.10.0 ...
       [info] Resolving org.scalamacros#quasiquotes_2.10;2.0.1 ...
       [info] Resolving org.json4s#json4s-core_2.10;3.2.10 ...
       [info] Resolving org.json4s#json4s-ast_2.10;3.2.10 ...
       [info] Resolving com.thoughtworks.paranamer#paranamer;2.6 ...
       [info] Resolving org.spire-math#jawn-parser_2.10;0.6.0 ...
       [info] Resolving org.spire-math#json4s-support_2.10;0.6.0 ...
       [info] Resolving org.scala-sbt#run;0.13.8 ...
       [info] Resolving org.scala-sbt#task-system;0.13.8 ...
       [info] Resolving org.scala-sbt#tasks;0.13.8 ...
       [info] Resolving org.scala-sbt#tracking;0.13.8 ...
       [info] Resolving org.scala-sbt#cache;0.13.8 ...
       [info] Resolving org.scala-sbt#testing;0.13.8 ...
       [info] Resolving org.scala-sbt#test-agent;0.13.8 ...
       [info] Resolving org.scala-sbt#test-interface;1.0 ...
       [info] Resolving org.scala-sbt#main-settings;0.13.8 ...
       [info] Resolving org.scala-sbt#apply-macro;0.13.8 ...
       [info] Resolving org.scala-sbt#command;0.13.8 ...
       [info] Resolving org.scala-sbt#logic;0.13.8 ...
       [info] Resolving org.scala-sbt#compiler-interface;0.13.8 ...
       [info] Resolving org.scala-sbt#precompiled-2_8_2;0.13.8 ...
       [info] Resolving org.scala-sbt#precompiled-2_9_2;0.13.8 ...
       [info] Resolving org.scala-sbt#precompiled-2_9_3;0.13.8 ...
       [info] Resolving org.scala-lang#jline;2.10.4 ...
       [info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] [info] Done updating.
[info] [info] Set current project to emptyProject (in build file:/private/var/folders/zr/63yqmtjx6yn4k35f4kvvtnqw0000gn/T/sbt_3224922a/emptyProject/)
[info] [info] Defining *:scuggestSimulate
[info] [info] The new value will be used by *:scuggestGen
[info] [info] Reapplying settings...
[info] [info] Set current project to emptyProject (in build file:/private/var/folders/zr/63yqmtjx6yn4k35f4kvvtnqw0000gn/T/sbt_3224922a/emptyProject/)
[info] [info] Updating {file:/private/var/folders/zr/63yqmtjx6yn4k35f4kvvtnqw0000gn/T/sbt_3224922a/emptyProject/}root...
[info] [info] Resolving org.scala-lang#scala-library;2.10.6 ...
       [info] Resolving org.scala-lang#scala-compiler;2.10.6 ...
       [info] Resolving org.scala-lang#scala-reflect;2.10.6 ...
       [info] Resolving org.scala-lang#jline;2.10.6 ...
       [info] Resolving org.fusesource.jansi#jansi;1.4 ...
[info] [info] Done updating.
[info] [info] successfully updated emptyProject.sublime-project
[info] [success] Total time: 0 s, completed 07/09/2017 12:06:47 AM
[info] [success] Total time: 0 s, completed 07/09/2017 12:06:47 AM
[info] + sbt-scuggest / emptyProject

Custom Assertions

Custom assertions can be created as given in the docs or examples.

Debugging

There are some additional niceties like pausing your test with:

//your test script steps
$ pause

which allows you to poke around the temporary project directory if you need to before continuing with assertions.

And that should be about all you need to know to get up and running with scripted. There is some additional doco here.