Testing

← Previous

Motivation

Kiji comes with a testing framework that makes it very easy to ensure that code you have written works as expected.

Setup

To use the Kiji testing framework, you must depend on the Kiji testing artifact: org.kiji.schema:kiji-schema:<kiji-version>:test-jar.

<dependency>
  <groupId>org.kiji.schema</groupId>
  <artifactId>kiji-schema</artifactId>
  <version>${kiji-schema.version}</version>
  <type>test-jar</type>
  <scope>test</scope>
</dependency>

Classes Overview

Kiji provides two base classes for unit and integration tests that depend on Kiji instances. For unit tests, the Kiji framework provides org.kiji.schema.KijiClientTest. For integration tests, the Kiji framework provides org.kiji.schema.testutil.AbstractKijiIntegrationTest.

Finally, Kiji comes with a helper builder to setup testing environments: org.kiji.schema.util.InstanceBuilder allows one to easily create test Kiji instances and populate them with tables and cells with minimal effort.

Note: in the future, as part of SCHEMA-166, KijiClientTest and AbstractKijiIntegrationTest are likely to merge into a single base class KijiTest.

Using the API

The two base classes KijiClientTest and AbstractKijiIntegrationTest provide very similar sets of functionality. The first one targets unit tests that may use a fake Kiji instance, ie. Kiji instance entirely backed by an in-memory HBase implementation; whereas the second one is designed for integration tests that must run against real Kiji instances.

Kiji unit tests inherit from the base class KijiClientTest which provides an environment suitable for tests of Kiji schema logic and MapReduce logic. It provides a testing Hadoop configuration accessible through KijiClientTest.getConf(). This configuration includes a default Hadoop FileSystem URI that is living in the local file system and an in-process MapReduce job tracker. The KijiClientTest base class keeps track of Kiji instances created for testing and cleans them up after each test. In particular, it provides a default Kiji instance accessible with KijiClientTest.getKiji(). Other Kiji instances may be created on demand with KijiClientTest.createKiji().

Should you need to create files for the purpose of your test, you may use the temporary directory KijiClientTest.getLocalTempDir(). This directory is unique for each test and automatically deleted after each test.

Kiji instance and table builder

The Kiji framework provides the InstanceBuilder helper class to define and populate testing Kiji environments. InstanceBuilder may create new Kiji instances with new InstanceBuilder().withX(...).build() or populate an existing Kiji instance with new InstanceBuilder(kiji).withX(...).build(): * InstanceBuilder creates a table using the .withTable(tableName, tableLayout) statement; * Within a table declaration, new rows are created using the .withRow(rowKey) statement; * Within a row declaration, new families are added using .withFamily(family); * Within a family declaration, new columns are specified using .withQualifier(qualifier); * Finally, within a column definition, cell versions are specified using .withValue(value) or .withValue(timestamp, value);

Example

Below is an example of a unit test that uses a fake in-memory Kiji instance, populates it with one table and some cells using the InstanceBuilder, and runs a gatherer with a reducer on it.

public class SomeTests extends KijiClientTest {
  @Test
  public void testSomeFeature() throws Exception {
    // Use the default testing Kiji instance managed by KijiClientTest.
    // Do not release this instance, KijiClientTest will clean it up automatically:
    final Kiji kiji = getKiji();

    // Create or load a table layout:
    final KijiTableLayout tableLayout = ...;
    final String tableName = tableLayout.getName();

    // Populate the existing Kiji instance 'kiji':
    new InstanceBuilder(kiji)
        // Declare a table
        .withTable(tableName, tableLayout)
            // Declare a row for the entity "Marsellus Wallace":
            .withRow("Marsellus Wallace")
                 .withFamily("info")
                     .withQualifier("first_name").withValue("Marsellus")
                     .withQualifier("last_name").withValue("Wallace")
            // Declare another row for the entity "Vincent Vega":
            .withRow("Vincent Vega")
                 .withFamily("info")
                     .withQualifier("first_name").withValue("Vincent")
                     .withQualifier("last_name").withValue("Vega")
        .build();

    // Run a gatherer/reducer on the test table:

    final KijiURI inputTableURI =
        KijiURI.newBuilder(kiji.getURI).withTable(tableName).build();

    final File outputDir = File.createTempFile("gatherer-output", ".dir", getLocalTempDir());
    Preconditions.checkState(outputDir.delete());

    // In-process MapReduce are currently limited to one reducer:
    final int numSplits = 1;

    // Run gatherer:
    final KijiMapReduceJob job = KijiGatherJobBuilder.create()
        .withConf(getConf())
        .withGatherer(GathererClassToTest.class)
        .withReducer(ReducerClassToTest.class)
        .withInputTable(inputTableURI)
        .withOutput(new TextMapReduceJobOutput(new Path(outputDir.toString()), numSplits))
        .build();
    assertTrue(job.run());
  }
}

This test assumes a fictitious input table whose rows describe people with columns such as info:first_name and info:last_name; it also assumes the usage of a gatherer named GathererClassToTest and a reducer named ReducerClassToTest.

The gatherer and the reducer will run in-process and write output files to the temporary directory outputDir.

Note: the Hadoop MapReduce local job runner that runs MapReduce jobs in-process does currently not allow running jobs with more than one reducer. See MAPREDUCE-434 for more details.

You may find additional concrete examples of unit tests and integration tests in the Kiji code base.

← Previous