Kiji Community

Accessing Data

The KijiTableReader class provides a get(...) method to read typed data from a Kiji table row. The row is addressed by its EntityId (which can be retrieved from the KijiTable instance using the getEntityId() method). Specify the desired cells from the rows with a KijiDataRequest. See the KijiDataRequest documentation for details.

In general, Kiji and KijiTable instances should only be opened once over the life of an application. (EntityIdFactorys should also be reused). KijiTablePool can be used to maintain a pool of opened KijiTable objects for reuse. To initially open a KijiTable:

Configuration conf = HBaseConfiguration.create();
KijiConfiguration kijiConf = new KijiConfiguration(conf, "your-kiji-instance-name");
Kiji kiji = Kiji.open(kijiConf);
KijiTable table = kiji.openTable("the-table-name");

To read from an existing KijiTable instance, create a KijiDataRequest specifying the columns of data to return. Then, query for the desired EntityId, using a KijiTableReader. You can get a KijiTableReader for a KijiTable using the openTableReader() method. For example:

KijiTableReader reader = table.openTableReader();

// Select which columns you want to read.
KijiDataRequest dataRequest = new KijiDataRequest()
    .addColumn(new KijiDataRequest.Column("your-family", "your-qualifier"));

// Try to reuse EntityIds when possible.
// If a need for this entityId comes up again, reuse
// this same entityId object.
EntityId entityId = table.getEntityId("your-row");
KijiRowData rowData = reader.get(entityId, dataRequest);

// Make sure to close the reader once you're finished.
reader.close();

The KijiTableReader also implements a bulkGet(...) method for retrieving data for a list of EntityIds. This is more efficient than a series of calls to get(...) because it uses a single RPC instead of one for each get.

Modifying Data

The KijiTableWriter class provides a put(...) method to write/update cells to a Kiji table. The cell is addressed by its entity ID, column family, column qualifier, and timestamp. You can get a KijiTableWriter for a KijiTable using the openTableWriter() method.

KijiTableWriter writer = table.openTableWriter();

long timestamp = System.currentTimeMillis();
writer.put(table.getEntityId("your-row"), "your-family", "your-qualifier", timestamp,
    "your-string-value");
writer.flush();
writer.close();

Counters

Incrementing a counter value stored in a Kiji cell would normally require a “read-modify-write” transaction using a client-side row lock. Since row locks can cause contention, Kiji exposes a feature of HBase to do this more efficiently by pushing the work to the server side. To increment a counter value in a Kiji cell, the column must be declared with a schema of type “counter”. See Managing Data for details on how to declare a counter in your table layout.

The KijiTableWriter class provides methods for incrementing counter values. Non-counter columns can not be incremented, and counter columns support only the increment operation. In other words, attempting to increment a column value that is not declared to be a counter will throw an exception. Likewise, attempting to put(...) a value into a column that is declared to be a counter will also throw an exception. The setCounter(...) method should be used when setting a counter value.

KijiTableWriter writer = table.openTableWriter();

// Incrementing a counter type column by 1.
// The column represented by counter-type-qualifier must
// of type counter otherwise an exception will be thrown.
writer.increment(table.getEntityId("your-row"), "your-family", "counter-type-qualifier", 1);

writer.flush();
writer.close();

MapReduce

The KijiTableInputFormat provides the necessary functionality to read from a Kiji table in a MapReduce job. To configure a job to read from a Kiji table, use KijiTableInputFormat’s static setOptions method. For example:

Configuration conf = HBaseConfiguration.create();
Job job = new Job(conf);

// * Setup jars to ship to the hadoop cluster.
job.setJarByClass(YourClassHere.class);
GenericTableMapReduceUtil.addAllDependencyJars(job);
DistributedCacheJars.addJarsToDistributedCache(job,
    new File(System.getenv("KIJI_HOME"), "lib"));
job.setUserClassesTakesPrecedence(true);
// *

KijiDataRequest request = new KijiDataRequest()
    .addColumn(new KijiDataRequest.Column("your-family", "your-qualifier"));

// Setup the InputFormat.
KijiTableInputFormat.setOptions(job, "your-kiji-instance-name", "the-table-name", request);
job.setInputFormatClass(KijiTableInputFormat.class);

The code contained within “// *” is responsible for shipping Kiji resources to the DistributedCache. This is so that all nodes within your hadoop cluster will have access to Kiji dependencies.

KijiTableInputFormat outputs keys of type EntityId and values of type KijiRowData. This data can be accessed from within a mapper:

@Override
public void map(EntityId entityId, KijiRowData row, Context context) {
  // ...
}

To write to a Kiji table from a MapReduce job, you should use KijiTableWriter as before. You should also set your OutputFormat class to NullOutputFormat, so MapReduce doesn’t expect to create a directory full of text files on your behalf.

To configure a job to write to a Kiji table, refer to the following example:

Configuration conf = HBaseConfiguration.create();
Job job = new Job(conf);

// Setup jars to ship to the hadoop cluster.
job.setJarByClass(YourClassHere.class);
GenericTableMapReduceUtil.addAllDependencyJars(job);
DistributedCacheJars.addJarsToDistributedCache(job,
    new File(System.getenv("KIJI_HOME"), "lib"));
job.setUserClassesTakesPrecedence(true);

// Setup the OutputFormat.
job.setOutputKeyClass(NullWritable.class);
job.setOutputValueClass(NullWritable.class);
job.setOutputFormatClass(NullOutputFormat.class);

And then, from within a Mapper:

public class MyMapper extends Mapper<LongWritable, Text, NullWritable, KijiOutput> {
  private KijiTableWriter writer;
  private Kiji kiji;
  private KijiTable table;

  @Override
  public void setup(Context context) {
    // Open a KijiTable for generating EntityIds.
    kiji = Kiji.open("your-kiji-instance-name");
    table = kiji.openTable("the-table-name");

    // Create a KijiTableWriter that writes to a MapReduce context.
    writer = table.openTableWriter();
  }

  @Override
  public void map(LongWritable key, Text value, Context context) {
    // ...

    writer.put(table.getEntityId("your-row"), "your-family, "your-qualifier", value.toString());
  }

  @Override
  public void cleanup(Context context) {
    writer.close();
    kiji.close();
    table.close();
  }
}

KijiSchema User Guide

Need Help?