This version is still in development and is not considered stable yet. For the latest stable version, please use Spring Batch Documentation 6.0.5!

Configuring Skip Logic

There are many scenarios where errors encountered while processing should not result in Step failure but should be skipped instead. This is usually a decision that must be made by someone who understands the data itself and what meaning it has. Financial data, for example, may not be skippable because it results in money being transferred, which needs to be completely accurate. Loading a list of vendors, on the other hand, might allow for skips. If a vendor is not loaded because it was formatted incorrectly or was missing necessary information, there probably are not issues. Usually, these bad records are logged as well, which is covered later when discussing listeners.

  • Java

  • XML

The following Java example shows an example of using a skip limit:

Java Configuration
@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
    int skipLimit = 10;
    var skippableExceptions = Set.of(FlatFileParseException.class);
    SkipPolicy skipPolicy = new LimitCheckingExceptionHierarchySkipPolicy(skippableExceptions, skipLimit);

	return new StepBuilder("step1", jobRepository)
				.<String, String>chunk(10).transactionManager(transactionManager)
				.reader(flatFileItemReader())
				.writer(itemWriter())
				.faultTolerant()
				.skipPolicy(skipPolicy)
				.build();
}

Note: The skipLimit can be explicitly set using the skipLimit() method.

The following XML example shows an example of using a skip limit:

XML Configuration
<step id="step1">
   <tasklet>
      <chunk reader="flatFileItemReader" writer="itemWriter"
             commit-interval="10" skip-limit="10">
         <skippable-exception-classes>
            <include class="org.springframework.batch.infrastructure.item.file.FlatFileParseException"/>
         </skippable-exception-classes>
      </chunk>
   </tasklet>
</step>

In the preceding example, a FlatFileItemReader is used. If, at any point, a FlatFileParseException is thrown, the item is skipped and counted against the total skip limit of 10. Exceptions (and their subclasses) that are declared might be thrown during any phase of the chunk processing (read, process, or write). Separate counts are made of skips on read, process, and write inside the step execution, but the limit applies across all skips. Once the skip limit is reached, the next exception found causes the step to fail. In other words, the eleventh skip triggers the exception, not the tenth.

The skip limit applies to all skips (read, process and write).

Skipping Write Failures and Chunk Scanning

Read and process failures affect a single item, so they can be skipped without discarding the rest of the chunk. A write failure is different: the ItemWriter receives the whole chunk at once, so the framework cannot tell which item caused the failure. When a write fails with a skippable exception, the chunk transaction is rolled back and the step enters scan mode: the items of the failed chunk are re-attempted one item per transaction, so that only the offending item is skipped and the others are still written.

Giving every scanned item its own transaction matters for writers whose failures poison the underlying resource. A JpaItemWriter whose flush() fails marks the persistence context as rollback-only, which would make every subsequent item in the same transaction fail too.

By default, the ItemProcessor is transactional, which means it is re-invoked for every item that is re-attempted during a chunk scan. The output produced in the rolled back transaction is discarded. This is the safe default, because the writer may have mutated the output item before failing: a JPA entity, for example, keeps the generated id assigned by the failed flush, and writing that same instance again fails with an unrelated exception that hides the original failure.

If your processor is expensive and its output is safe to write again after a failed, rolled back write, you can cache the processor output and re-use it during chunk scanning with processorNonTransactional():

@Bean
public Step step1(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
	return new StepBuilder("step1", jobRepository)
				.<String, String>chunk(10).transactionManager(transactionManager)
				.reader(itemReader())
				.processor(expensiveItemProcessor())
				.writer(itemWriter())
				.faultTolerant()
				.processorNonTransactional()
				.skip(MyBusinessException.class)
				.skipLimit(10)
				.build();
}

Do not set processorNonTransactional() if the item writer mutates the items it is given. Doing so makes the second write attempt operate on a stale item, which typically fails with an exception unrelated to the original failure. If that exception is not skippable, the step fails instead of skipping the offending item.