JSON Item Readers And Writers

Spring Batch provides support for reading and Writing JSON resources in the following format:

[
  {
    "isin": "123",
    "quantity": 1,
    "price": 1.2,
    "customer": "foo"
  },
  {
    "isin": "456",
    "quantity": 2,
    "price": 1.4,
    "customer": "bar"
  }
]

It is assumed that the JSON resource is an array of JSON objects corresponding to individual items. Spring Batch is not tied to any particular JSON library.

JsonItemReader

The JsonItemReader delegates JSON parsing and binding to implementations of the org.springframework.batch.infrastructure.item.json.JsonObjectReader interface. This interface is intended to be implemented by using a streaming API to read JSON objects in chunks. Two implementations are currently provided:

  • Jackson through the org.springframework.batch.infrastructure.item.json.JacksonJsonObjectReader

  • Gson through the org.springframework.batch.infrastructure.item.json.GsonJsonObjectReader

To be able to process JSON records, the following is needed:

  • Resource: A Spring Resource that represents the JSON file to read.

  • JsonObjectReader: A JSON object reader to parse and bind JSON objects to items

The following example shows how to define a JsonItemReader that works with the previous JSON resource org/springframework/batch/infrastructure/item/json/trades.json and a JsonObjectReader based on Jackson:

@Bean
public JsonItemReader<Trade> jsonItemReader() {
   return new JsonItemReaderBuilder<Trade>()
                 .jsonObjectReader(new JacksonJsonObjectReader<>(Trade.class))
                 .resource(new ClassPathResource("trades.json"))
                 .name("tradeJsonItemReader")
                 .build();
}

JsonFileItemWriter

The JsonFileItemWriter delegates the marshalling of items to the org.springframework.batch.infrastructure.item.json.JsonObjectMarshaller interface. The contract of this interface is to take an object and marshall it to a JSON String. Two implementations are currently provided:

  • Jackson through the org.springframework.batch.infrastructure.item.json.JacksonJsonObjectMarshaller

  • Gson through the org.springframework.batch.infrastructure.item.json.GsonJsonObjectMarshaller

To be able to write JSON records, the following is needed:

  • Resource: A Spring Resource that represents the JSON file to write

  • JsonObjectMarshaller: A JSON object marshaller to marshall objects to JSON format

The following example shows how to define a JsonFileItemWriter:

@Bean
public JsonFileItemWriter<Trade> jsonFileItemWriter() {
   return new JsonFileItemWriterBuilder<Trade>()
                 .jsonObjectMarshaller(new JacksonJsonObjectMarshaller<>())
                 .resource(new ClassPathResource("trades.json"))
                 .name("tradeJsonFileItemWriter")
                 .build();
}

JSON Lines

Spring Batch also supports the JSON Lines format, where each line of a file is a valid JSON object corresponding to an item:

{"isin": "123", "quantity": 1, "price": 1.2, "customer": "foo"}
{"isin": "456", "quantity": 2, "price": 1.4, "customer": "bar"}

Since each line is an item, there is no need for a dedicated item reader or writer: the FlatFileItemReader and FlatFileItemWriter are used, with a JSON-aware LineMapper and LineAggregator. Like the JSON item readers and writers, they delegate the JSON processing to a collaborator, so they are not tied to any particular JSON library.

JsonLineMapper

The JsonLineMapper is a LineMapper that delegates the conversion of a line to an item to the org.springframework.batch.infrastructure.item.json.JsonObjectUnmarshaller interface. The contract of this interface is to take a JSON String and unmarshall it to an object. Two implementations are currently provided:

  • Jackson through the org.springframework.batch.infrastructure.item.json.JacksonJsonObjectUnmarshaller

  • Gson through the org.springframework.batch.infrastructure.item.json.GsonJsonObjectUnmarshaller

The following example shows how to define a FlatFileItemReader that reads the previous JSON Lines file, using an unmarshaller based on Jackson:

@Bean
public FlatFileItemReader<Trade> jsonLinesItemReader() {
   return new FlatFileItemReaderBuilder<Trade>()
                 .name("tradeJsonLinesItemReader")
                 .resource(new ClassPathResource("trades.jsonl"))
                 .lineMapper(new JsonLineMapper<>(new JacksonJsonObjectUnmarshaller<>(Trade.class)))
                 .build();
}

A custom JsonMapper or Gson can be provided to the unmarshaller, for example new JacksonJsonObjectUnmarshaller<>(jsonMapper, Trade.class). For parameterized target types, such as Map<String, Object>, which cannot be expressed with a Class, the unmarshallers accept a Jackson TypeReference or a Gson TypeToken:

JsonLineMapper<Map<String, Object>> lineMapper = new JsonLineMapper<>(
		new JacksonJsonObjectUnmarshaller<>(new TypeReference<Map<String, Object>>() {}));
A line that cannot be parsed causes a ParseException that wraps the exception of the JSON library. The FlatFileItemReader reports it as a FlatFileParseException that also contains the line and its number.

JsonLineAggregator

The JsonLineAggregator is a LineAggregator that delegates the conversion of an item to a line to the org.springframework.batch.infrastructure.item.json.JsonObjectMarshaller interface, which is the same one used by the JsonFileItemWriter (see JsonFileItemWriter).

The following example shows how to define a FlatFileItemWriter that writes a JSON Lines file, using a marshaller based on Jackson:

@Bean
public FlatFileItemWriter<Trade> jsonLinesItemWriter() {
   return new FlatFileItemWriterBuilder<Trade>()
                 .name("tradeJsonLinesItemWriter")
                 .resource(new FileSystemResource("trades.jsonl"))
                 .lineAggregator(new JsonLineAggregator<>(new JacksonJsonObjectMarshaller<>()))
                 .build();
}
Each item is written as a single line, so the marshaller must produce a JSON object that fits on one line. This is the case of the default Jackson and Gson marshallers, but not of a marshaller that is configured to pretty-print its output.