Below is an excerpt from my book "RESTful Web API Patterns and Practices Cookbook" from O'Reilly Media
When talking about coordinating services, it is common to encounter the notions of choreography and orchestration. Each represents architectural points of view on how to design and implement service coordination. In my experience, the web offers a third, more direct (and, I think, more powerful) point of view on coordination: that of describing workflows as declarative documents that, like HTTP, provide a consistent interface that services can use to interact with each other. I’ll get to this hypermedia workflow shortly. First, it is worthwhile to review just what choreography and orchestration offer and how they differ from a hypermedia approach.
Centralized Orchestration
The service orchestration model is a centralized approach to defining workflows in a single context and then submitting that document to a workflow “engine.” This has the effect of creating an integration programming model (e.g., BPEL) that solution designers can use to connect services in a logical execution flow. The orchestra metaphor is a common way to talk about this approach since there is a single “conductor” (the engine) that leads a group of people all working from essentially the same agreed-upon “music” (the workflow document).
ORCHESTRATION HISTORY
Service orchestration has a long and varied history. Probably the best known “origin story” for service orchestration comes from XLang, Web Services Flow Language (WSFL), and its standardized offshoot Business Process Execution Language or WS-BPEL. These are designed to support describing a series of workflow steps and decisions and then controlling the execution of that workflow by submitting the document to a workflow “engine,” such as Microsoft’s BizTalk, IBM’s WebSphere, Apache ODE, and others.
Stateless Choreography
Typically, choreography is defined as a kind of “dance” of services. Each service knows its own “moves,” and each acts as an independent entity that interacts with others. With choreography, the workflow is a by-product of the interactions between existing services. Workflow is an “emergent” behavior.
LET’S DANCE
The dance metaphor appears often in descriptions of the choreography approach to workflow. In fact, this dance theme is so strong with the choreography-style workflow crowd that an entire programming language designed for integration work was released in 2017 called Ballerina.
Both centralized orchestration and point-to-point choreography have their advantages and drawbacks. In general, workflows without many steps and few branching options are well-served with an orchestration approach. Define the work in a single place (a document) and use that document as the execution script for the solution. Often, this can be written up in the native source code.
However, if the workflows are quite involved, it can be more beneficial to use the choreography approach. As the number of steps goes up, the sequential approach of most orchestration engines becomes a liability. Also, extensive branching points in workflows can make it hard to reason about outcomes and difficult to “roll back” problematic workflows. Choreographed solutions are sometimes expressed as source code but can also be expressed in a domain-specific language (DSL) tuned for the task at hand.
Hypermedia Workflow
There is, however, another approach that I have found quite reliable; one that depends on the use of hypermedia controls (forms and links) that describe the details of service interactions at runtime. I have always called this hypermedia workflow. This approach works well both for cases with just a few steps and ones with several involved processes. It takes advantage of the independent, parallel processing of choreography as well as the centralized approach of orchestration.
JAZZ AS A WORKFLOW ANALOGY
Just as it has become popular to use orchestra and dance analogies to talk about service coordination, I have been using jazz music as a way to talk about hypermedia workflow. With orchestration, there is “someone in charge.” With choreography, it is important that everyone knows their part and works closely with each other to reach the final outcome. With jazz music, everyone has a basic idea of what the song is and they all get to contribute in their own way to create a final performance.
The key to designing and implementing hypermedia workflows is to make sure each service that is enlisted in the flow has a composable service interface that supports the following actions:
Execute: Do the work (e.g., computeTax, createCustomerResource, etc.).
Repeat : Do the same work again in case something didn’t work the first time.
Revert : Undo the work in case there was an error somewhere else.
Cancel : Stop processing the task and undo any previous work.
When each service (called a task) supports these actions, it is possible to collect them together in a job. Workflow jobs run all tasks in parallel and offer two other important actions:
Continue : Pick up the work where it last left off.
Restart : Start from the very beginning again.
Cancel : Stop processing the job and tell all tasks to undo any previous work.
There are some other details for this workflow language regarding shared state and time-outs that can be covered by additional internal implementation details. The important thing to keep in mind is that the workflow interface described here is very simple and usually straightforward to implement for an individual service. There are definitely challenges. For example, supporting the Revert action has implications if the service calls other services under the covers. But that set of details can be sorted out within source code and does not need to be exposed as part of the service interface.
Another advantage of relying on a hypermedia workflow language is that all the details of a workflow can be expressed as one or more documents instead of source code or script. Most programming languages are imperative. They describe how something is done. Hypermedia documents are, however, declarative. Declarative languages describe what needs to be done without explicitly specifying a step-by-step process.
This ability to describe actions as tasks, and flows as jobs, makes it possible to write composable services that can be successfully enlisted into a workflow without worrying about how all the parts will interact. Just like HTTP, hypermedia workflow standardizes the external interface and allows services to use any internal programming languages and paradigms they wish.