.NET 1.1+

Object Pipelines

by Richard Carr, published at http://www.blackwasp.co.uk/Pipeline.aspx

A pipeline is a chain of connected steps that process information. In object pipelines, each step receives an object and performs an action using it before passing an object to the next step. This repeats until every step is complete.

What is a Pipeline?

Often problems can be solved by applying a series of steps, with the output of one step being passed as the input to the next. One way in which the steps can be chained together is using a pipeline. A pipeline connects processing units, which may be classes, methods or larger modules, each of which performs a small portion of the overall task. When combined in the correct sequence, a pipeline containing simple processing units can perform complex tasks. For example, pipelines are often used in audio processing. The individual processing units may perform tasks such as normalising volume, applying filters to remove noise or add special effects, and converting between file formats. These units can be combined in pipelines, to which music files can be passed to apply multiple filters in series. Different results can be achieved by creating new pipelines that connect the same units in different combinations.

In this article we'll look at how you can create object pipelines. An object pipeline follows the same pattern of a chain of steps executed in sequence, using objects as the input and output of each step. Such pipelines are often used to process data in collections or datasets. Individual processing units may perform filtering or sorting of the data or transform the items in the collection.

When creating object pipelines we can use a design that is similar to the Chain of Responsibility design pattern. The difference is that every step of a pipeline is visited, whereas a chain of responsibility stops when the first step that can process the input object is reached.

Implementing a Pipeline

The UML diagram above describes an implementation of an object pipeline. The classes in the diagram are described below:

• Client. The client is the class that creates an object to be processed and passes it to the first step in the pipeline.
• PipelineStepBase. This class is the base class for all of the steps that can be added to a pipeline. It defines an abstract method, "HandleRequest", that must be implemented each step. This method receives an input object, performs some processing using it and passes an object to the next step. The class also defines a protected member that holds the next step in the pipeline and a method that allows that step to be set.
• PipelineStepA/B. The concrete pipeline step classes inherit from PipelineStepBase. They add the functionality to handle the input object and produce the desired output to pass to the next step.

NB: This is just one way to implement a pipeline. An alternative is to create a controller class that holds a list of steps to execute in series. One advantage of the approach described in the UML diagram is the ability to create branching pipelines, where processing conditionally continues with one of several possible steps. It is also possible to create looping pipelines, though you must be careful to avoid infinite loops.

The code below implements the types in the UML diagram using C#:

public abstract class PipelineStepBase
{
    protected PipelineStepBase _nextStep;

    public void SetNextStep(PipelineStepBase nextStep)
    {
        _nextStep = nextStep;
    }

    public abstract string HandleRequest(string input);
}


public class ConcretePipelineStepA : PipelineStepBase
{
    public override string HandleRequest(string input)
    {
        string output = input + " -> Step A";
        return _nextStep == null ? output : _nextStep.HandleRequest(output);
    }
}


public class ConcretePipelineStepB : PipelineStepBase
{
    public override string HandleRequest(string input)
    {
        string output = input + " -> Step B";
        return _nextStep == null ? output : _nextStep.HandleRequest(output);
    }
}