What is Durable Functions?
Durable Functions is an extension of Azure Functions and Azure WebJobs that lets you write stateful functions in a serverless environment. The extension manages state, checkpoints, and restarts for you. Basically, it allows you to write Functions orchestrations, in code. The extension allow us to define stateful workflows in a new type of function called an orchestrator function which in short achieves the following:
- Define workflows in code. No JSON schemas or designers are needed.
- Ability to call other functions synchronously and asynchronously. Output from called functions can be saved to local variables.
- Automatically checkpoint their progress whenever the function awaits. Local state is never lost if the process recycles or the VM reboots.
However, for the time being, the orchestrator function can only be written in C# while the other functions can be written in any language (supported by Azure Functions).
What are the current challenges?
One of the challenges that happens when you’re dealing with Azure Functions in general which is unique to Azure is – What happens if I have a process that is complex or long-running and I need to manage that entire process?
Let’s take for example the scenario I used in the previous article, where the process requires a certain job to get done before a next function can work its task. That is so because I wanted to take the output of one function and use it as the input into another function, hence, I had to co-ordinate that somehow by using Queues and Blobs.
Of course, in a distributed architecture, the former approach works great as it provides a stateless, resilient and scalable architecture. However today, what I wanted to showcase is, how we could take that scenario and translate it into Durable Functions.
To solve the above scenario or problem, there is a new piece of functionality for Azure Functions called Durable Functions. What Durable Functions do is, it’s essentially an orchestration piece that is within your Functions that can run as long as you want while managing the state.
Pattern: Function Chaining
Let’s walk through the previous example using Azure Functions to execute OCR on our processed images. We’ll use the pattern called Function Chaining. Function chaining refers to the pattern of executing a sequence of functions in a particular order. Often the output of one function needs to be applied to the input of another function. The beauty of durable functions is we can implement this pattern concisely in code.
I’m going to skip the setup process. To develop durable functions, you would need these prerequisites and installing durable functions here. Instead, I’ll dive straight into the codes.
Start with an empty project
We’ll be using Visual Studio Code to achieve the following.
As mentioned earlier, the orchestrator function is the piece that orchestrates your functions. It orchestrates what happens between the functions and it’s responsible for calling and receiving the output from the “Activity Functions” (equivalent to our normal functions which we are familiar with).
This function is an orchestration trigger function that is triggered from the orchestrator function which sends its output to
SayHello activity function which receives it as an input to be processed. It uses the
This code is the activity function. It uses the
activityTrigger binding. When an activity function returns, its return value becomes part of the local state of the orchestrator function so that we are able to work with it as we would as any other variable.
To this point, if you had followed the documentation on Azure Docs - Durable Functions, and if you had ran your function, you would have realised that it would not run. It’s not because that the code is wrong or there’s a bug. It’s because an important piece is left out. This is the most important part of the entire example, the orchestration function.
This is the orchestrator function that orchestrates your functions. It is using a
httpTrigger which defines the orchestration client.
Run the sample
To execute the
E1_HelloSequence orchestration, send the following HTTP POST request.
Let’s go back to the example scenario we built using Azure Functions. What I had just shown using Function Chaining works, but wait, could we do better? I would consider building a combination of “Function Chaining” with “Fan-in / Fan-out” architecture that not only takes one function’s output as an input to another function; I would also have one function that take an event then parallelize it’s operations and when all of those tasks have been completed, I’ll aggregate the results.
Remember we used Logic Apps in the previous example to sort of orchestrate the OCR capabilities from Azure Functions. Now, we can do that with Durable Functions.
In this short article, I did not dive into writing Durable Functions with the scenario I shared in the previous article. Instead, I shared some of the concepts and experiences I had while building a Durable Function.
I recommend taking a look at the examples here.