Manual JWT Validation in .NET Core

Recently, I have been working with Jeff Fritz over at as part of his effort to build a TikTok like site for Twitch, uniquely called KlipTok ( Mainly my efforts have been on shoring up the backend code in the BackOffice using Azure Functions.

This was one of my first major exposures with the Twitch API. Its fine overall but, it oddly does not use JWT tokens to communication states back and forth, rather an issues string is required for authenticated requests. I wanted to try a different approach to handling token auth and refresh so, I devised the following POC:

One of the aspects of the Twitch API is that tokens can expire and calls should be ready to refresh an access token which enters this state. The trouble is, these are two tokens and I didnt want the clients required to send both tokens, nor did I want the client to have to resubmit a request. I decided, I would create my own token and store within it, as claims, the access token and refresh token.

Taking this approach would allow the POC to, in effect, make it seem like Twitch is issues JWT tokens while still allowing the backend to perform the refresh. I decided, for additional security, I would encrypt the token claims in my JWT using Azure Key Vault Keys.

Part 1: Creating the Token

This approach hinges on what I refer to as token interception. As part of any OAuth/OIDC flow, there is a callback after the third party site (Twitch in this case) has completed the login. Tokens are sent to this callback for the sole purpose of allowing the caller to store them.

In order to achieve this, I created a method which a client would call at the very start. This contacts Twitch and reissues the active tokens, if they exist, or requests the user to login in again:

public IActionResult Get()
var redirectUri = WebUtility.UrlEncode("https://localhost:5001/home/callback");
var urlString = @$"{_configuration["TwitchClientId"]}"
+ $"&redirect_uri={redirectUri}"
+ "&response_type=code"
+ "&scope=openid";
return Redirect(urlString);
view raw login.cs hosted with ❤ by GitHub

The key here is the redirectUri which redirects the provided response code back to the application. Here we can create the token and send it to the client. You can find this method in the provided GitHub repository, HomeController.

You can find MANY examples of creating a JWT Token on the internet, I will use this one for reference:

Here is my code which creates the token string with the access token and refresh token as claims:

public async Task<string> CreateJwtTokenString(string accessToken, string refreshToken)
var jwtSigningKey = await _keyVaultService.GetJwtSigningKey();
var securityKey = new SymmetricSecurityKey(Encoding.UTF8.GetBytes(jwtSigningKey));
var signingCredentials = new SigningCredentials(securityKey, SecurityAlgorithms.HmacSha256Signature);
var secToken = new JwtSecurityToken(
issuer: _configuration["Issuer"],
audience: _configuration["Audience"],
claims: new List<Claim>
new Claim("accessToken", await _cryptoService.Encrypt(accessToken)),
new Claim("refreshToken", await _cryptoService.Encrypt(refreshToken))
notBefore: null,
expires: DateTime.Now.AddDays(1),
return new JwtSecurityTokenHandler().WriteToken(secToken);
view raw jwt-create.cs hosted with ❤ by GitHub

The actual signing key is stored as a secret in Azure Key Vault with access controlled using ClientSecretCredentials, those values are stored in environment variables and not located in source code. You can find more information on this approach here: The one critical point I will make is ClientSecretCredential is only appropriate for local development – when deploying into Azure be sure code is using a Managed Identity driven approach.

I defined a simple method which grabs the Encryption key from Azure Key Vault and encrypts (or decrypts the data).

// getting the key
private KeyClient KeyClient => new KeyClient(
vaultUri: new Uri(_configuration["KeyVaultUri"]),
credential: _getCredentialService.GetKeyVaultCredentials());
public async Task<KeyVaultKey> GetEncryptionKey()
var keyResponse = await KeyClient.GetKeyAsync("encryption-key");
return keyResponse.Value;
// usage
public async Task<string> Encrypt(string rawValue)
var encryptionKey = await _keyVaultService.GetEncryptionKey();
var cryptoClient = new CryptographyClient(encryptionKey.Id, _getCredentialService.GetKeyVaultCredentials());
var byteData = Encoding.Unicode.GetBytes(rawValue);
var encryptResult = await cryptoClient.EncryptAsync(EncryptionAlgorithm.RsaOaep, byteData);
return Convert.ToBase64String(encryptResult.Ciphertext);
view raw gistfile1.txt hosted with ❤ by GitHub

The beauty of using Azure Key Vault is NO ONE but Azure is aware of the key. Using this, even if our JWT token is somehow leaked, the data within is not easy to decipher.

Once generated, this token can be passed back to the client either as data or in some header, allowing the client to store it. We can then use the built-in validation to require the token with each call.

Part 2: Validating the Token

Traditionally, tokens are signed by an authority and the underlying system will contact that authority to validate the token. However, in our case, we have no such authority so, we will want to MANUALLY validate the token, mainly its signature.

It turns out this is rather tricky to perform in ASP .NET Core due to the way the validation middleware is implemented. The best way I found to get it work and be clean is to adjust the way you register certain dependencies in ConfigureServices, as such:

var keyVaultService = new KeyVaultService(new GetCredentialService(Configuration), Configuration);
var tokenSecurityValidator = new JwtSecurityTokenValidator(Configuration, keyVaultService);
.AddSingleton(p => keyVaultService)
.AddTransient(p => tokenSecurityValidator)
// add auth middleware
.AddJwtBearer(options =>
options.RequireHttpsMetadata = false;
view raw statup.cs hosted with ❤ by GitHub

You can see the keyVaultService and tokenSecurityValidator are defined as concrete dependencies and we use the provider override syntax for AddSingleton to pass the instance directly. This is done so we can pass the direct instance of tokenSecurityValidator to our the options for validating our Bearer token.

This class calls on its dependencies and validates the signature of the token and ensures it matches with our expectations:

The result of adding this (and the appropriate Use methods in the Configure method) is we can fully leverage [Authorize] on our actions and controllers. Users who pass no token or a token that we cannot validate will receive a 401 Unauthorized.

Part 3: Performing the Refresh

First step with any call is the ability to GET the token for the request so it can be used. There are MANY ways to do this. As I wanted to keep this simple I elected to use the IHttpContextAccessor. This is a special dependency you can have ASP .NET Core inject that lets you access the HttpContext anywhere in the call chain. I wrapped this in a service:

This class very simply yanks the token from the incoming request and return the specific claim that represents the token. It also calls the decryption method so the fetched token is ready for immediate use.

This is by no means a perfect approach, in fact were I to see this in Production code I would comment that its a violation of the separation of concerns since a web concerns is being accessed in the services layer. More ideally, you would want to use middleware or similar to hydrate a scoped dependency which can be injected into your layers.

The TwitchApiService ( houses the logic to perform the request for user data from Twitch that I chose to showcase the refresh functionality.

This code is crucial for the functionality:

client.DefaultRequestHeaders.Add("Client-Id", _configuration["TwitchClientId"]);
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", await _getTokensFromHttpRequestService.GetAccessToken());
var result = new ApiResult<TwitchUser>();
var response = await client.GetAsync($"helix/users?login={loginName}");
if (response.StatusCode == HttpStatusCode.Unauthorized)
// refresh tokens
var (accessToken, refreshToken) = await _authService.RefreshTokens(await _getTokensFromHttpRequestService.GetRefreshToken());
result.TokensChanged = true;
result.NewAccessToken = accessToken;
result.NewRefreshToken = refreshToken;
// re-execute the request with the new access token
client.DefaultRequestHeaders.Authorization =
new AuthenticationHeaderValue("Bearer", accessToken);
response = await client.GetAsync($"helix/users?login={loginName}");
if (response.IsSuccessStatusCode == false)
throw new Exception($"GetUser request failed with status code {response.StatusCode} and reason: '{response.ReasonPhrase}'");
var responseContent = await response.Content.ReadAsStringAsync();
view raw call.cs hosted with ❤ by GitHub

I wrote this in a very heavy fashion, it simple makes the call, check if it failed with a 401 Unauthorized and, if so, refreshes the token using the TwitchAuthService () and then makes the same call again.

The result is a return to the caller with the appropriate data (or an error if the request still failed).

Part 4: Notify of new Token

Something you may have noticed in the previous code, the use of a generic ApiResult<T>. This is necessary because JWT tokens are designed to be immutable. This means they cannot be changed once created, its this aspect which makes them secure. However, in this case, we are creating a token with data that will change (on a refresh) and thus necessitate a regeneration of the token.

The purpose of this ApiResult<T> class it to hold NOT JUST the result but to tell us if the token needs to change. If it does change, that new version must be passed to the client so it can be saved. This may seem like a drawback to the approach but, in actuality this is a typical part of any application interacting with an OAuth flow where token refresh is being used.

However, what we DO NOT want to do is require logic in every action to check the result, rebuild the token, and pass it to the caller. Instead, we want to intercept the return result and, in a central spot, strip away the extra data and ensure our new token, if appropriate, is in the response headers.

To that end I created the following ActionFilter:

public class ProcessApiResultFilter : IActionFilter
private readonly JwtTokenService _jwtTokenService;
public ProcessApiResultFilter(JwtTokenService jwtTokenService)
_jwtTokenService = jwtTokenService;
public void OnActionExecuting(ActionExecutingContext context)
// no action
public void OnActionExecuted(ActionExecutedContext context)
if ((context.Result as OkObjectResult)?.Value is ApiResult result)
if (result.TokensChanged)
var newTokenString = _jwtTokenService.CreateJwtTokenString(
result.NewAccessToken, result.NewRefreshToken).Result;
context.HttpContext.Response.Headers.Add("X-NewToken", newTokenString);
context.Result = new ObjectResult(result.Result);
view raw filter.cs hosted with ❤ by GitHub

Our ApiResult<T> inherits from ApiResult which gives it the non-generic read only Result property, which is used in the code sample above. The ApiResult<T> includes a setter whose accepted type is T. This allows the application to interact with it in a type-safe way.

Above you can see the Result being sent to the user is altered so its the inner result. Meanwhile, if the token changes we regenerate that token using our JwtTokenService and its stored in the X-NewToken header in the response. Client can now check for this header when receiving the response and update their stores as needed.

One final thing, I am using Dependency Injection in the filter. To achieve this you must wrap its usage in the ServiceFilterAttribute. Example here:

And that is it. Let’s walk through the example again.

Understanding what happens

A given client will make its initial page the response to /Login which will return the Twitch Login screen OR, if a token is already present, the callback will be called instantly. This callback will generate a token and send it down to the caller (right now its printed to the screen), generally this would be a page in your client app that will store the token and show the initial page.

When the client makes a request, they MUST pass the custom JWT Token given to them, the application will be checking for it as an Authorization Bearer token – failure to pass it will result in a 401 Unauthorized being sent back.

The application, after validating the token, will proceed with its usual call to the Twitch API. Part of this will access whatever the Access Token was passed. If Twitch responds with a 401 Unauthorized, the code will extract the refresh token from the JWT Token and refresh the access token. Upon successfully doing this, the call to Twitch will be executed again.

The result is sent back to the caller in a wrapper, ApiResult<T> which, along with carrying the call result, also contains information on whether the token changed. The caller will simply return this result as it would any normal Action call.

We use a special ActionFilter to intercept the response, and rewrite it so the caller returns the expected result in the response body. If the token did change, the new token is written into the response behind the X-NewToken header.

Throughout the process, we never reveal the tokens and all of the values involved in signing, encryption, and decryption are stored in Azure Key Vault outside of our application. For local dev, we are using an App Registration to govern access to the Key Vault, if we were deployed in Azure we would want to associate our Azure service to a managed identity.


Hopefully, this example has been instructive and helpful. I know I learned quite a bit going through this process. So, if it helps you, drop me a comment and let me know. If something does not make sense feel free to also drop me a comment. Cheers.

Getting Started with KEDA and Queues

One of the limitations inside Kubernetes was the metrics that were supported to allow for scaling within the cluster for a deployment. The HorizontalPodAutoScaler or HPA for short, could only monitor CPU Utilization to determine if more Pods needed to be added to support a given workload. As you can imagine, in a queue based or event system, CPU usage wont tell, accurately, whether or not more pods are needed.

Note: The Kubernetes team realizing has added support for custom-metrics into the platform:

Noticing this, Microsoft engineers began work on a project to address this, called KEDA (Kubernetes Event-Driven Autoscaling) comprised of custom resources which were capable of triggering scaling events based on external cluster criteria: queue tail length, message availability, etc. Now in 2.1 the team has added support for MANY popular external products which would dictate scaling needs in unique ways.

Here is the complete list:

For this post, I wanted to walk through how to set up a configuration whereby I could use KEDA to create jobs in Kubernetes based on the tail length of an Azure Storage Queue. As is expected with a newer project, KEDA’s documentation still needs work and certain things are not entirely clear. So I view this as an opportunity to supplement the teams work. That being said, this is still very much an alpha product and, as such, I expect future iterations to not work with the steps I lay out here. But as of right now, Feb 2020, they work.

Full source code:

First step, Create a cluster an Azure Queue Storage

Head out to the portal and be sure to create an AKS cluster (or a Kubernetes cluster in general, doesnt matter who the provider is) and an Azure Storage account (this one you will need in Azure). Once the storage account is created, create a Queue (shown below) and saved the connection string off somewhere you can copy from later.

As indicated, you could use GKE (Google Kubernetes Engine) or something else if you wanted. KEDA also supports other storage and events outside of Azure but, I am using Azure Queue Storage for this demo hence why I will assume the Queue Storage is in Azure.

Now, let’s install KEDA

As with anything involving custom resources in Kubernetes, KEDA must be installed for those resources to exist. KEDA has a variety of ways it can be installed, laid out here:

A quick note on this, BE CAREFUL of the version!! I am using v2.1 for this and that is important since the specification for ScaledJob changes between 2.0 and 2.1. If you read through the third approach to deployment, where you run kubectl apply against a remote file, be sure to replace the version of the file to v2.1.0. I noted with Helm at least I did NOT get v2.1 from the given charts repo.

If you run the third approach, creation of the keda namespace will happen for you, this is where the internal of KEDA will be installed and run from, your code does NOT need to go in here and I wont be doing that just to put you at ease.

Once the installation completes I recommend running the following command to make sure everything is up and running:

kubectl get all -n keda

Note that I used the shorthand -n because I have had it happen where the –namespace doesnt copy correctly and you end up with command syntax errors. If you see something like this, KEDA is up and running:

Let’s setup the KEDA Scaler

For starters, we need a secret to hold that connection string for our Queue Storage from earlier. Here is a simple secret definition to create a secret that KEDA can use to monitor the queue tail length. REMEMBER when you provide the value to the secret it MUST be base64 encoded. I wont show my value as I do not wish to dox myself.

Linux users you can use the built-in base64 command to generate the value for the secret file. Everyone else, you can quickly Google a Base64 encoder and convert your string.

echo “your connection string” | base64

Use kubectl apply -f to create the secret. Since the namespace is provided in file, it will be placed in that namespace for you.

Next, we are going to get into KEDA specific components TriggerAuthentication and ScaledJob. These two resources will be critical to supporting our intended functionality.

First, there is the specification for TriggerAuthentication:

As you can see, there are a number of ways to provide authentication, we will be using secretTargetRef. The purpose is to give our trigger a way to authentication to our Queue Storage such that it can determine the various property values it needs to find out if a scaling action needs to be taken (up or down).

Building on what we did with the creation of our Secret we add the following definition and apply it via kubectl apply -f

Comparing the Secret with this file you can see where things start to match up. We are simply telling the trigger it can find the connection string at the appropriate key in a certain secret. Many of the examples on the KEDA website will use podIdentity which as I have come to understand refers back to MSI. This is a better approach, albeit more complicated, than what I am showing here. We should always avoid storing sensitive information in our cluster (like connection strings) due to the less than stellar security around Secrets in general – base64 is not in anyway secure.

The final piece is the creation of the ScaledJob. KEDA mostly focuses around scaling deployments, which makes a lot of sense but, it can also serve to scale up Kubernetes Jobs as needed to fulfill deferred processing. Effectively, KEDA creates a psuedo deployment around the job and scales the number up as needed based on the scaling strategy specified.

This looks like quite a bit but, when you break it down it has a very straightforward purpose and a structure that is consistent with other Kubernetes objects. Let’s break it down in four parts:

The first part is identification, what we are naming the ScaledJob and where it is going to be stored within the cluster. Notice the apiVersion value this is a clear indication of the Spec being in ALPHA meaning, I fully expect this to change.

The second part is the details for the actual ScaledJob, that is things which are specific to this instance of the resource. Here we tell the resource to check the length of our queue every 5 seconds and that it should trigger based on an azure-queue with authentication stored in our trigger auth that we defined previously.

The third and fourth part are actually all relating to the same thing which is the configuration of the created Kubernetes Job instances that will perform the work – I broke this apart based on my own personal style when constructing YAML files for Kubernetes. To keep things simple we are not going to have the job leverage parallelism, so we leave this at 1, which is also the default.

The last section lays out the template for the Pods that will carry out the work. You notice the custom image xximjasonxx/printmessage which will grab the message from the queue and print out its contents. We are also reusing the Secret here to provide the container with the connection string of the Queue so it can take items off.

All of this is available for reference in the GitHub repo I linked above.

Let’s test it

In the provided source code, I included a command line program that can send messages to our queue in the form of random numbers – SendMessage. To run this, open a Command Line window up to the directory holding the .csproj file and run the following command:

dotnet run “<connection string>” 150

The above command will send 100 messages to the queue – I should note that the queue name in the container is HARD CODED as test-queue. Feel free to download the code and make the appropriate change for your own queue name if need be – you will need to do it for both Print and Send message programs.

After running the above command you can run the following kubectl command to see the results of your experiment. Should look something like this:

This shows that it is working and, in fact, we can do a kubectl logs on one of the pods and we can see the output message sent to the queue. Or so it appears, let’s take a closer look.

Execute the following command to COUNT how many pods were actually created:

kubectl get po | wc -l

Remember to subtract one as the wc program will also count the header line. If you get similar to what I got it will be around 300. But that does not make any sense, we only sent 150 items to our queue. The answer is, the way printmessage:v3 is written, it contains logic to print that no data was found as the queue becomes empty. While valid, with the 10 completion rule being enforced this will spin up unnecessary pods. Let’s change the image used for the job to a special image: printmessage:v3-error. This image will throw an uncaught exception when the queue is empty. The updated definition for ScaledJob is below:

Before running things again I recommend executing these two commands, they assume the ONLY thing in the current space are jobs and pods related to KEDA. If you are sharing the namespace with other resources you will have to modify these commands.

kubectl delete po –all

kubectl delete job –all

Make sure to run kubectl apply to get the updated ScaledJob definition into your cluster. Run the SendMessage program again. This is what I got:

Notice how, even though we specified the job needs to complete 10 times, none of these did. Your results are likely going to vary depending on when items were pulled from the queue. But as the queue gets shorter more jobs will start to fail as the Pods attempt to grab data that does not exist.

The other thing to notice is that the Pods, if they fail, will self terminate. So, if I run my wc -l check again on the Pods I get a number that makes more sense:

kubectl get po | wc -l

Result should be 151 which, subtracting the header row gives us the 150 items we sent to the queue

Why is this happening?

The key value for controlling this behavior is the backoffLimit specified as part of the job spec. It tells a job how many times it should try to restart failing pods under its control. I have set it to 1 which effectively means it will not retry and only accept one failure.

The reason this is so important is control over resources that are scaling to match processing workloads is crucial from the standpoint of maintaining healthy resource consumption. We do not want our pods to go crazy and overwhelm the system and starve other processes.

Storage Class with Microsoft Azure

One of the things I have been focusing on lately is Kubernetes, its always been an interest to me but, I recently decided to pursue the Certified Kubernetes Application Developer (CKAD) and so diving into topics that I was not totally familiar with has been a great deal of fun.

One topic that is of particular interest is storage. In Kubernetes, though really in Containerized applications, state storage is an important topic since the entire design of these systems is aimed at being transient in nature. With this in mind, it is paramount that storage happen in a centralized and highly available way.

A common approach to this is to simply leverage the raw cloud APIs for things like Azure Storage, S3, etc as the providers will do a better job ensuring the data is stored securely and in a way that makes it hard for data loss to occur. However, Kubernetes enables the mounting of the cloud systems directly into Pods through Persistent Volumes and Storage Classes. In this post, I want to show how to use Storage Class with Azure so I wont be going to detail about the ins and out of Storage Classes or their use cases over Persistent Volumes, frankly I dont understand that super well myself, yet.

Creating the Storage Class

The advantage to Storage Class (SC) over something like Persistent Volume (PV) is the former can automatically create the later. That is, a Storage Class can received Claims for volume and will, under the hood, create PVs. This is why SC’s have become very popular with developers, less maintenance.

Here is a sample Storage Class I created for this demo:

kind: StorageClass
name: file-storage
skuName: Standard_LRS
view raw sc.yaml hosted with ❤ by GitHub

This step is actually optional – I only did it for practice. AKS will automatically create 4 default storage classes (they are useless without a Persistent Volume Claim (PVC)). You can see them by running the follow command:

kubectl get storageclass

Use kubectl create -f to create the storage class based on the above, or use one of the built in ones. Remember, by itself, the storage class wont do anything. We need to create a Volume Claim for the magic to actually start.

Create the Persistent Volume Claim

A persistent volume claim (PVC) is used to “claim” a storage mechanism. The PVC can be, depending on its access mode, attached to multiple nodes where its pods reside. Here is a sample PVC claim that I made to go with the SC above:

apiVersion: v1
kind: PersistentVolumeClaim
name: fileupload-pvc
storageClassName: file-storage
storage: 5Gi
view raw pvc.yaml hosted with ❤ by GitHub

The way PVCs work (simplistically) is they seek out a Persistent Volume (PV) that can support the claim request (see access mode and resource requests). If nothing is found, the claim is not fulfulled. However, when used with a Storage Class its fulfillment is based on the specifications of the Storage Class provisioner field.

One of the barriers I ran into, for example, was that my original provisioner (azure-disk) does NOT support multi-node (that is it does not support ReadWriteMany used above). This means, the storage medium is ONLY ever attached to a single node which limits where pods using the PVC can be scheduled.

To alleviate this, I opted to use, as you can see, the azure-file provisioner, which allows multi node mounting. A good resource for reading more about this is here: Concepts – Storage in Azure Kubernetes Services (AKS) – Azure Kubernetes Service | Microsoft Docs

Run a kubectl create -f to create this PVC in your cluster. Then run kubectl get pvc – if all things are working your new PVC should have a state of Bound.

Let’s dig a bit deeper into this – run a kubectl describe pvc <pvc name>. If you look at the details there is a value with the name Volume. This is actually the name of the PV that the Storage Class carved out based on the PVC request.

Run kubectl describe pv <pv name>. This gives you some juicy details and you can find the share in Azure now under a common Storage Account that Kubernetes has created for you (look under Source).

This is important to understand, the claim creates the actual storage and Pods just use the claim. Speaking of Pods, let’s now deploy an application to use this volume to store data.

Using a Volume with a Deployment

Right now, AKS has created a storage account for us based on the request from the given PVC that we created. To use this, we have to tell each Pod about this volume.

I have created the following application as Docker image xximjasonxx/fileupload:2.1. Its a basic C# Web API with a single endpoint to support a file upload. Here is the deployment that is associated with this:

apiVersion: apps/v1
kind: Deployment
name: fileupload-deployment
replicas: 1
app: fileupload
name: fileupload-app
app: fileupload
name: fileupload
image: xximjasonxx/fileupload:2.1
containerPort: 80
value: "/app/output"
mountPath: /app/output
name: save-path
name: save-path
claimName: fileupload-pvc
view raw deploy.yaml hosted with ❤ by GitHub

The key piece of this the ENV and Volume Mounting specification. The web app looks to a hard coded path for storage if not overridden by the Environment Variable SAVE_PATH. In this spec, we specify a custom path within the container via this environment variable and then mount that directory externally using the Volume created by our PVC.

Run a kubectl create -f on this deployment spec and you will have the web app running in your cluster. To enable external access, create a Load Balancer Service (or Ingress), here is an example:

apiVersion: v1
kind: Service
name: fileupload-service-lb
app: fileupload
– protocol: TCP
port: 80
targetPort: 80
type: LoadBalancer
view raw gistfile1.txt hosted with ❤ by GitHub

Run kubectl create -f on this spec file and then run kubectl get svc until you see an External IP for this service indicating it can be addressed from outside the cluster.

I ran the following via Postman to test the endpoint:

If all goes well, the response should be a Guid which indicates the name of the image as stored in our volume.

To see it, simply navigate to the Storage Account from before and select the newly created share under the Files service. If you see the file, congrats, you just used a PVC through a Storage Class to create a place to store data.

What about Blob Storage?

Unfortunately, near as I can tell so far, there is no support for saving these items to object storage, only file storage. To use the former, at least with Azure, you would still need to use the REST APIs.

This also means you wont get notifications when new files are created in the file share as you would with blob storage. Still, its useful and a good way to ensure that data provided and stored is securely and properly preserved as needed.

Using Scoped Dependencies

I was recently asked by a client how I would go about injecting user information into a service that could be accessed anywhere in the call chain. They did not want have to capture the value at the web layer and pass it to what could be a rather lengthy call stack.

The solution to this is to leverage scoped dependencies in ASP .NET Core which will hold an object for the duration of the request (default). In doing this, we can gather information related to the request and expose it. I also wanted to add an additional twist. I wanted to have two interfaces for the same object, one that enable writing and the other that would enable reading, like so:

The reason for doing this is aimed at being deterministic. What I dont want to support is the ability for common code to accidentally “change” values, for whatever reason. When the injection is made, I want the value to be read only. But, to get the value in there I need to be able to write it, so I segregate the operations into different interfaces.

This may be overkill for your solution but, I want the code to be as obvious in its intent and capabilities – this helps instruct users of this code how it should be used.

Configuring Injection

Our ContextService, as described above, contains only a single property: Username. For this exercise, we will pull the value for this out of the incoming query string (over simplistic I grant you, but it works well enough to show how I am using this).

I am going to define two interfaces which this class implements: IContextReaderService and IConextWriterService, code below:

public class ContextService : IContextReaderService, IContextWriterService
public string Username { get; set; }
public interface IContextReaderService
string Username { get; }
public interface IContextWriterService
string Username { set; }
view raw service.cs hosted with ❤ by GitHub

The tricky part now is, we want the instance of ContextService created with and scoped to the incoming request to be shared between IContextReaderService and IContextWriterService, that is I want the same instance to comeback when I inject a dependency marked with either of these interfaces.

In Startup.cs I need to do the following to achieve this:

services.AddScoped<IContextReaderService>(p => p.GetService<ContextServiceFactory>().GetCurrentContext());
services.AddScoped<IContextWriterService>(p => p.GetService<ContextServiceFactory>().GetCurrentContext());
view raw startup.cs hosted with ❤ by GitHub

The secret here is the request scoped ContextServiceFactory which is given as the parameter to AddScoped that allows us to tell .NET Core how to resolve the dependency. This factory is defined very simply as such:

public class ContextServiceFactory
private ContextService _currentContext;
public ContextService GetCurrentContext()
if (_currentContext == null)
_currentContext = new ContextService();
return _currentContext;
view raw factory.cs hosted with ❤ by GitHub

Remember, by default, something added as a scoped dependency is shared throughout the lifetime of the request. So here, we maintain state within the factory to know if it has created an instance of ContextService or not, and if it has, we will return that one. This factory object will get destroyed when the request completed and recreated when a new request is processed.

Hydrating the Context

Now that we have our context split off, we need to hydrate the values, thus we need to inject our IContextWriterService dependency into a section of code that will get hit on each request. You might be tempted to use a global filter, which will work but, the better approach here is custom middleware. Here is what I used:

// middleware.cs file
public class HydrateContextMiddleware
private RequestDelegate _next;
public HydrateContextMiddleware(RequestDelegate next)
_next = next;
public async Task Invoke(HttpContext context, IContextWriterService contextService)
contextService.Username = context.User.Identity.Name;
await _next(context);
// startup.cs Configure method
view raw middleware.cs hosted with ❤ by GitHub

Because of the way they are used, you can only use constructor injection for singleton scoped dependencies, if you attempt to use a Scoped or Transient scoped dependency in the middleware constructor, it will fail to run.

Fear not, we can use method injection here to inject our dependency as a parameter to the Invoke method which is what ASP .NET Core will look for and execute with each request. Here you can see we have defined a parameter of type IContextWriterService.

Within Invoke perform the steps you wish to take (here we are extracting the username from the name parameter in the Query String, for this example). Once you complete your steps be sure to call the next bit of middleware in sequence (or return a Completed Task to stop the chain).

Using the Reader dependency

Now that we have configured the dependency and hydrated it using middleware we can no reference the IContextReaderService to read the value out. This works in the standard way as you would expect:

public class UserController : ControllerBase
private readonly IContextReaderService _contextService;
public UserController(IContextReaderService contextService)
_contextService = contextService;
public IActionResult GetUser()
return Ok(_contextService.Username);
view raw controller.cs hosted with ❤ by GitHub

We can inject this dependency wherever we need (though more specifically, wherever we can access the IContextReaderService).

Mutability vs Immutability

The main goal I was trying to illustrate here is to leverage immutability to prevent side effects in code. Because of the interface segregation, a user would be unable to change the given value of the context. This is desirable since it lends to better code.

In general, we want to achieve immutability with objects in our code, this is a core learning from functional programming. By doing this, operations become deterministic and less prone to sporadic and unexplainable failures. While the example presented above is simplistic in nature, in a more complex systems, having assurances that users can only read or write depending on which interface is used allows for better segregation and can yield cleaner and more discernable code.

Hope you enjoyed. More Testing posts to come, I promise.

Test Series: Part 2 Unit Testing

Part 1 is here – where I intro Testing Strategies.

Unit testing is the single most important test suite within ANY application. It is the first line of defense guarding against defects and is paramount to instilling confidence in developers that the application of changes does not break any existing logic. This being the case, they are (should be) the most numerous type of test authored for a system. High performing teams will run them often as a verification step and ensure their runs are as fast as possible to save time. By doing so and building confidence they are able to achiever ever higher levels of efficiency and quality.

What do we Unit Test?

This is perhaps the single most important and common question you will get from teams or you will discuss within your own teams. Making the right decision here is critical for the long term success of the project and preventing quality and performance issues from negatively impacting your teams.

As a fundamental rule, we do not unit test external dependencies, that is database calls, network calls, or any logic that might involve any sort of external dependencies. Our unit test runs need to be idempotent such that we can run them as much as we like without having to worry about disk space, data pollution, or other external factors.

Second, the focus must be on a unit of code. In this regard, our tests do not test multi-step processes. They test a single path through a unit of code; the need for a unit test to be complex is often an indicator of a code smell: either the logic is overly complicated and needs refactoring or, the test itself is wrong and should either be broken down or tested with a different form of testing such as integration tests.

Finally, we should test known conditions for external dependencies through the use of mocking. By using a mocking library we can ensure that code remains resilient and that our known error cases are handled. Further, using a mocking library often forces us to use design by contract which can improve the readability of our code.

Making the wrong choice – a story from the past

I worked with a team in a past life that made the wrong choice when it came to their testing. As part of an effort to improve quality the client (astutely) asked the team to ensure testing was being done against database and networking calls. Leaders on the team, due to poor knowledge around testing or poor decision making, opted to work these tests into the unit test library. Over the course of the project, this caused the test run time to increase to greater than 40m.

One of the critical elements to high functioning teams is the notion of fast feedback. We want to ensure developers are given immediate feedback when something breaks. Unit tests are a core part of achieving this and their speed is paramount to the teams effectiveness. What happens when you allow tests times to balloon as mentioned? Disaster.

When the turnaround time is that long, developers will seek ways to avoid incurring that time cost (there is still the pressure to get work done). Generally this involves not writing tests (we dont want to increase the time cost), running them minimally (get the work done and test at the end), or turning them off. None of these options improve efficiency and, in fact, make an already bad problem that much worse.

In this case, the team adopted a branching model that called for entire features to be developed in a “feature” branch before merging. With any development environment we always want to minimize “drift”, that is differences between master and any branches. The less drift the fewer merge conflicts and the quicker problems are discovered.

By not understanding this principle, the team unknowingly, compounded their problem. In some cases these “features” would be in flight for 10+ days, creating enormous amounts of drift. And, as the team was looking to avoid running the tests too often, the changes were not being checked regularly by the tests. As you can imagine, issues were found persistently near the end of sprints, as code was merged. And due to the size of the incoming changes debugging became a massive task.

This created more problems for the beleaguered teams as they were forced to spend time after hours routinely debugging and trying to finish features before the end of the sprint. Burnout was rampant and the team members became jaded with one another and the company itself – they endured this for 10+ months. While the project ultimately did complete, the client relationship was ruined and several good developers left the company.

To be clear, the bad choices around testing alone were not the single cause of this failure, there were numerous other problems. However, I have found that that even a difficult client can be assuaged if code quality is maintained and the team delivers. I can recall a team that I led where we had unit testing and continuous delivery processes in place such that, even though we had delays and bugs, these processes enabled us to respond quickly – the client remained delighted and worked with us.

The lesson here is, no matter what, we MUST ensure the development team has the tools needed to support automation processes. These processes form the core of the ability to deliver and lend themselves to building healthy and sustainable client relationships.

How do I write a Unit Test?

So, now you have an understanding of what can be unit tested, let’s talk about how you write them. First, I wish to introduce you to the AAA pattern: Arrange, Act, Assert. This pattern is crucial as you write your tests to check yourself against the warning signs for bad unit tests.

  • Arrange: In this step we “arrange” the unit, that is we do all of the things to prepare for executing our unit. Be wary at this level if the steps to arrange feel too cumbersome, it likely indicates that your design need refactoring
  • Act: In this step we “invoke” the unit. This executes our the code we are specifically testing. Be wary at this level if more than two executions are necessary. This means you are NOT testing a unit and your design needs to be re-evaluated. Remember, we do not test multi-part flows with unit tests.
  • Assert: In this step we check the outcome of our unit. Important here is to only assert on the minimum amount of information needed to verify the unit. I have seen teams assert on 20+ properties for an object, this is excessive. Think carefully about what indicates a failure. My rule of thumb is never more than three asserts. If you need more, create another test.

Here is an example of a simple math problem under unit test:

public void assert_adding_two_numbers_gives_their_sum()
// arrange
var numberOne = 10;
var numberTwo = 20;
// act
var result = numberOne + numberTwo;
// assert
Assert.Equal(30, result);

As you can see, in this example we define our two variables (numberOne and numberTwo) in the arrange section, we then invoke our add operation in the act and finally we assert that the value meets with our expectations.

The [Fact] is a part of the xUnit testing library. xUnit is a popular open source testing framework commonly used with .NET Core. There are other libraries available. The use of a library for unit testing makes great sense and will greatly aid in your productivity. Below are a few of the common ones in the .NET ecosystem:

  • nUnit ( – the grand-daddy of them all. Base dont JUnit from Java and one of the first unit testing frameworks devised for .NET
  • MSTest – Microsoft’s testing framework. It offers the same functionality as nUnit and is built into .NET Framework
  • xUnit – as mentioned above, similar to nUnit in functionality and aimed at supporting testing in an OS agnostic programming world. This is my default

The next common problem is organization. When you start talking about an application that has thousands, if not tens of thousands (or more) tests, it becomes very apparent that a clear and consistent strategy must be adopted. Over the course of my career I have seen many different approaches but, the one that I favor is the given and assert naming convention. Mainly because it plays very well with most test reporters. Here is an example.

Imagine we have defined the following Web API Controller:

public class CalculationController : Controller
public IActionResult Add([FromBody]TwoNumberViewModel viewModel)
return Ok(viewModel.FirstNumber + viewModel.SecondNumber);
view raw controller.cs hosted with ❤ by GitHub

In this case we might define our test fixture (that is the class that contains our test) as such:

public class given_an_instance_of_calculation_controller
view raw testfixture.cs hosted with ❤ by GitHub

Notice the name of the class here, while it does violate traditional C# naming convention, when you run the test runner, it will precede your method name. Therefore, if we expand this to include a test like so:

public class given_an_instance_of_calculation_controller
public void assert_that_given_two_numbers_the_result_returned_is_the_correct_sum()
// arrange
var controller = new CalculationController();
var viewModel = new TwoNumberViewModel
FirstNumber = 10,
SecondNumber = 20
// act
var result = controller.Add(viewModel) as OkObjectResult;
// assert
Assert.Equal("30", result.Value.ToString());

The above example is a product of over simplification and ONLY for demonstration purposes. When unit testing controllers, the emphasis needs to be on result types returned NOT values. Testing the outcome of operations should be done with unit tests against services. The above represents code that violates the separation of concerns principle.

With this in place, if we run a test runner and view the results in the reporter we will see the following:


As you can see, the advantage to this strategy is it lines up nicely and produces a readable English sentence detailing what the test is doing. There are other strategies but, as I said, this is my go to in most cases due to the readability and scalable nature of this naming method.

Further, it bakes into it a necessary check to ensure unit tests are not checking too much. As a rule, the assert portion should never contain the word and as that it implies more than one thing is being checked which violates the unit principle.

How do I test external dependencies?

The short answer is, you dont, you generally write integration tests (next part in this series) to cover those interactions. However, given the speed and criticality of the logic checked by unit tests we want to maximize their ability as best we can.

A classic example of this case is Entity Framework. If you have worked with Entity Framework you will be familiar with the DbContext base class that denotes the context which handles querying our underlying database. As you might expect, our unit tests should NEVER invoke this context directly, not even the InMemory version but, we do need to ensure our logic built on the context works properly. How can we achieve this?

The short answer is: we can define an interface which exposes the necessary methods and properties on our context and have our classes take a dependency on this interface rather than the concreate context class itself. In doing so, we can use mocking libraries to mock the context allowing testing against these lower level classes.

The long answer is, honestly, an entire blog post (Learning Tree has a good write up that uses NSubstitute here) that I will try to add on later.

But this strategy of using interfaces also allows us to take dependencies on static components as well. In older versions of ASP .NET it was common for applications to utilize the HttpContext.Current property to reference the incoming ISAPI results. But, because this property was static it could not be unit tested directly (it would always be null unless running in the web context).

Using the interface approach, we commonly saw things like this:

public class ContextAccessor : IContextAccessor
public IDictionary<string, string> QueryString
// assume .AsDictionary() is an extension method that takes the QueryString struct and converts it to a Dictionary
get { return HttpContext.Current.Request.QueryString.AsDictionary(); }
public interface IContextAccessor
IDictionary<string, string> QueryString { get; }
public class TestController : ControllerBase
private readonly IContextAccessor _contextAccessor { get; set; }
public TestController(IContextAccessor contextAccessor)
_contextAccessor = contextAccessor;
public IActionResult Get()
return Ok(_contextAccessor.QueryString["name"]);

Using this approach the controller, which will have unit tests, is dependent on the injected IContextAccessor interface instead of HttpContext. This fact is crucial as it allows us to write code like such:

public class given_an_instance_of_test_controller
public void assert_that_the_name_query_string_parameter_is_returned_in_the_result()
// arrange
var contextMock = new Mock<IContextAccessor>();
contextMock.Setup(x => x.QueryString).Returns(new Dictionary<string, string> { { "name", "TestUser" } });
var controller = new TestController(contextMock.Object);
// act
var result = controller.Get() as OkObjectResult;
// assert
Assert.Equal("TestUser", result.Value.ToString());
view raw mocking.cs hosted with ❤ by GitHub

This is the result. This code validates that our logic is correct but, it does NOT validate that HttpContext gets built properly at runtime, this is not our responsibility, it is the author of the framework (Microsoft in this case) whose responsibility that is.

This brings a very clear and important point when writing tests: some tests are NOT yours to right. It is not on your team to validate that, for example, Entity Framework works properly, or that a request through HttpClient works – these components are already (hopefully) being tested by their authors. Attempting to go down this road will not lead you anywhere where the test drive value.

A final point

The final use case with testing I would like to make, and this is especially true with .NET is, tests should ALWAYS be synchronous and deterministic. Parallel code needs to be broken down into its discrete pieces and those pieces need to be tested. Trying to unit test parallel code is fraught with the risk of introducing “flakiness” into tests – these are tests that pass sometimes and other times not.

.NET developers commonly use the async/await syntax in their code. Its very useful and helpful however, when running unit tests it needs to be forced down a synchronous path.

We do not test external dependencies so, the use of async/await should not be needed for ANY test. Our dependencies should be mocked and thus will return instantaneously.

To do this, it is quite easy, we can call GetAwaiter and GetResult methods which will force the resolution of the Task return variable. Here is an example:

public interface IDataService
Task<List<DataModel>> GetData();
public class TestController : ControllerBase
private readonly IDataSerivce _dataService;
public TestController(IDataService dataService)
_dataService = dataService;
public async Task<IActionResult> Get()
return Ok(await _dataService.GetData());
public class given_an_instance_of_test_controller
public void assert_that_data_is_returned_from_get_call()
// arrange
var dataServiceMock = new Mock<IDataService>();
dataServiceMock.Setup(x => x.GetData()).ReturnsAsync(new List<DataModel> { new DataModel() } });
var controller = new TestController();
// act
var result = controller.Get().GetAwaiter().GetResult() as OkObjectResult;
// assert
var resultValue = result.Value as List<DataModel>();
view raw async_await.cs hosted with ❤ by GitHub

By calling GetAwaiter() and GetResult() we force the call to be synchronous. This is important since, in some cases, the Asserts may run BEFORE the actual call completes, resulting in increased test flakiness.

The most important thing is not just to test but also to be fast

Hopefully this post has shown you some of the ways you can test things like databases, async calls, and other complex scenarios with unit tests. This is important. Due to their speed, it makes sense to use them to validate wherever possible.

One of the uses that I did not show here is “call spying“, this is where the mocking framework can “track” how many times a method is called which can serve as another way to assert.

But the most important thing I hope I can impress is the need to not only ensure unit tests are built with the application but, also that you continually are watching to ensure they remain fast enough to be effective for your developers to perform validation on a consistent ongoing basis.

The next topic which I intend to cover will focus on Integration Tests, primarily via API testing through Postman.

Test Series: Part 1 – Understanding Testing Strategies

One of the challenges with incorporating DevOps culture for teams is understanding that greater speed yields better quality. This is often foreign to teams, because conventional logic dictates that “the slow and steady win the race”. Yet, in every State of DevOps report ( since it began Puppet ( has consistently found that teams which move faster see higher quality that those that move slower – and this margin is not close and the gap continues to accelerate. Why is this? I shall explain.

The First Way: Enable Fast and Increasing Flow

DevOps principles (and Agile) were born out of Lean Management which is based on the Toyota Production System ( Through these experience we identify The Three Ways, and the first of these specifically aims for teams to operating on increasingly smaller workloads. With this focus, we can enable more rapid QA and faster rollback as it far easier to diagnose a problem in one thing than in 10 things. Randy Shoup of Google observed:

“There is a non-linear relationship between the size of the change and the potential risk of integrating that change—when you go from a ten-line code change to a one-hundred-line code change, the risk of something going wrong is more than 10x higher, and so forth”

What this means is, the more changes we make the more difficult it is to diagnose and identify problems. And this relationship is non-linear meaning, this difficulty goes up exponentially as the size of our changes increase.

In more practical terms, it argues against concepts such as “release windows” and aims for a more continuous deployment model whereby smaller changes are constantly deployed and evaluated. The value here is, by operating on these smaller pieces we can more easily diagnose a problem and rollbacks become less of an “event”. Put more overtly, the aim is to make deployments “normal” instead of large events.

This notion is very hard for many organizations to accept and it often runs counter to how many IT departments operate. Many of these departments have long had problems with software quality and have devised release and operations plans to, they believe, minimize the risk of these quality issues. However, from the State of DevOps reports, this thinking is not backed up by evidence and tends to create larger problems. Successful high functioning teams are deploying constantly and moving fast. Speed is the key.

The secret to this speed with quality is the confidence created through a safety net. Creating a thorough safety net can even create enough confidence to let newest person on the team deploy to Production on Day 1 (this is the case at Etsy).

Creating the Safety Net

In the simplest terms, the safety net is the amalgamation of ALL your tests/scans running automatically with each commit. The trust and faith in these tests to catch problems before they reach production allows developers to move faster with confidence. It also being automated means it does not rely on a sole person (or group) and can scale with the team.

Ensuring the testing suite is effective is a product of having a solid understanding of the breakdown of testing types and adopting of the “Shift Left” mindset. For an illustration of testing breakdown, we can reference the tried and true “Testing Pyramind”:

As illustrated, unit tests comprise the vast majority of tests in the system. The speed of these tests is something that should be closely monitored as they are run the most often. Tips for ensuring speed:

  • Do NOT invoke external dependencies (database, network calls, disk, etc)
  • Focus on a UNIT, use Mocking libraries to fulfill dependencies
  • Adhere to the AAA model (Arrange, Act, Assert) and carefully examine tests for high complexity

Unit tests must be run frequently to be effective. In general, a minimum of three runs should occur with any change: Local run, run as part of PR validation, and a run when the code is merged to master. The speed is crucial to reduce, as much as possible, the amount of time developers have to wait for these tests.

At the next level we start considering “Integration tests”. These are tests which require a running instance of the application and thus need to follow a deploy action. Their inclusion of external dependencies makes them take longer to run, hence we decrease the frequency. There are two principle strategies I commonly see when executing these tests:

  1. Use of an Ephemeral “Integration” environment – in this strategy, we use Infrastructure as code to create a wholly new environment to run our Integration tests in – this has several advantages and disadvantages
    • Benefit – avoids “data pollution”. Data pollution occurs when data created as part of these tests can interfere with future test runs. A new environments guarantees a fresh starting point each time
    • Benefit – tests your IaC scripts more frequently. Part of the benefit in modern development is the ability to fully represent environments using technologies like Terraform, ARM, and others. These scripts, like the code itself, need exercising to ensure they continue to meet our needs.
    • Negative – creating ephemeral environments can elongate the cycle time for our release process. This may give us clues when one “thing” is more complex than it should be
  2. Execute against an existing environment. Most commonly, I recommend this to be the Development environment as it allows the testing to serve as a “gate” to enable further testing (QA and beyond)
    • Benefit – ensures that integration testing completes before QA examines the application
    • Negative – requires logic to avoid problems with data pollution.

What about Load Testing?

Load Testing is a form of integration testing with some nuance. We want to run them frequently but, their running must be in a context where our results are valid. Running them in, let us say, a QA environment is often not helpful since a QA server likely does not have the same specs as Production. Thus problems in QA with load may not be an issue in higher environments.

If you opt for the “ephemeral approach” you can conduct load testing as part of these integration tests – provided your ephemeral environment is configured to have horsepower similar to production.

If the second strategy is used, I often see Load Testing done for staging, which I disagree with – it is too far to the right. Instead, this should be done in QA ahead of (or as part of) the manual testing effort.

As you can see above in the pyramid, ideally these integration tests comprise about 20% of your tests. Typically though, this section is where the percentage will vary the most depending on the type of application you are building.

Finally we do our Manual Testing with UI testing

UI tests and/or acceptance testing comprises the smallest percentage (10%), mainly because the tests are so high level that they become brittle and excessive amounts will generate an increased need for maintenance. Further, testing here tends to be more subjective and strategic in nature, thus exploratory testing tends to yield more results and inform the introduction of more tactical tests at other levels.

QA is a strategic resource, not a tactical one

A core problem that is often seen within teams and organizations prior is how QA is seen and used. Very often, QA is a member of the team or some other department that code is “thrown over the wall to” as a last step in the process. This often leads to bottlenecks and can even create an adversarial relationship between Engineering and QA.

The truth is, the way QA is treated is not fair and nor is it sensible. I always ask teams “how often has QA been given 4 features to test at 445pm the day before the Sprint Demo?”. And each time, this is not an exception, it is consistent. And of course, QA finds issues and results in the whole team staying late or “living with bugs”. The major mistake that is made

The truth is, this creates a bottleneck with QA, a rather unfair one at that. How often has QA been asked to work long hours the day before the sprint ends after being given 5 features that “just finished and need testing”? This is not acceptable and underlines the misunderstanding organizations have for QA.

QA is not responsible for testing, per se, they are responsible for guiding testing and to ensure it is happening. Testing, ultimately, falls to developers as they are the closest to the code and have the best understanding of it. This is why automated testing (unit in particular) is so vital to the “safety net” concept. Getting developers to understand that testing and writing tests is their responsible is vital to adopting DevOps culture.

This is not to say QA does NO testing, they do. But it is more strategic in nature; aimed at exploratory testing and/or ensuring the completness of the testing approach. They also lead in the identification of issues and their subsequent triaging. Key to high function teams is, whenever an issue is found, the team should remediate it but also create a test which can prevent it from appearing in the future. As the old saying goes “any problem is allowed to happen once”.

Moving away from this relience on the QA department/individual can feel rash to teams that have become overly dependant on this idiom. But rest assured, the best way forward is to focus on automation to create and maintain a suitable safety net for teams.

Safety Nets Take Time

Even introducing 1000 unit tests tomorrow is not going to immediately give your developers the confidence to move faster. Showing that you can deploy 6x a day is not going to immediately see teams deploying 6x a day. Confidence is earned and, there is a saying, “you only notice something when it breaks”. DevOps is not a magic bullet or even a tool – it is a cultural shift, one that, when properly done, touchest every corner of the organization and every level, from the most junior developer to the CEO.

The culture implores participants to constantly challenge themselves and application to ensure that safety measures in place work correctly and complete. High functioning teams want to break their systems, notable Netflix will often break things in Production intentionally to ensure failsafes are working properly.

More canonically, if a test never breaks, how do we know it works at all? This is the reason behind the Red-Green-Refactor development methodology ( I see a lot of teams simply write tests with the assumption that they work, without actually creating a false condition to test if they break.

But the effort is worth it to move faster and see higher quality. In addition, adopting this aspect of DevOps culture means teams can have higher confidence in their releases (even if they are not deploying all the time). This makes for decreased burn out and better morale/productivity. Plus, you get a full regression suite for free.

I plan to continue this series by diving more deeply into many of the concepts I covered here with unit testing likely being the next candidate.

Durable Functions: Part 4 – Analyze and Download

All code for this series can be found here:

We are here now at the final part of our example (Part 1, Part 2, Part 3) that will focus on what happens after we approve our upload as shown in Part 3. That is, we will leverage Cognitive Services to gather data about image and store it in the Azure Table Storage we have been using. As to why this part is coming so much later, I moved into a house so I was rather busy 🙂

In the previous blog posts we built up a Durable Function Orchestrator which is initiated by a blob trigger from the file upload. To this point, we have uploaded the file and allowed a separate HTTP Trigger function to “approve” this upload, thereby demonstrating how Durable Functions enable support of workflows that can advance in a variety of different ways. Our next step will use an ActivityTrigger, which is a function that is ONLY ever executed in the context of an orchestrator by an orchestrator.

Building the Activity Trigger

ActivityTriggers are identified by their trigger parameter as shown in the code sample below (only the function declaration):

public static async Task<bool> ProcessFile(
[ActivityTrigger] string fileId,
[Blob("files/{fileId}", FileAccess.Read, Connection = "StorageAccountConnectionString")] Stream fileBlob,
[Table("ocrdata", Connection = "TableConnectionString")] CloudTable ocrDataTable,
ILogger log)
view raw trigger1.cs hosted with ❤ by GitHub

In this declaration we are indicating this function is called as an activity within an orchestration flow. Further, as we have with other functions we are referencing the related Blob and, new here, the ocrData cloud table which will hold the data outputted from the OCR process (Optical Character Recognition, Computer Vision essentially).

To “call” this method we expand our workflow and add the CallActivityAsync call:

public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
[Table("metadata", Connection = "TableConnectionString")] CloudTable metadataTable,
ILogger log)
var input = context.GetInput<ApprovalWorkflowData>();
var uploadApprovedEvent = context.WaitForExternalEvent<bool>("UploadApproved");
await Task.WhenAny(uploadApprovedEvent);
// run through OCR tools
var ocrProcessTask = context.CallActivityAsync<bool>(nameof(ProcessFileFunction.ProcessFile), input.TargetId);
await Task.WhenAny(ocrProcessTask);
view raw workflow3.cs hosted with ❤ by GitHub

This approach enables us to fire “parallel” tasks and further leverage our pool of Azure Functions handlers (200 at a time). This can be more effective than trying to leverage the parallel processing within the Azure Function instance itself, but always consider how best to approach a problem needing parallelism to solve.

I am not certain if the Function attribute is necessary on the function since, as you can see, we are referring to by its canonical name in C#. We also pass in the Target Id for the Azure Table record, this so a FK relationship can exist for this data. This is purely stylistic – in many cases it may make more sense for all data to live together, this is one of the strengths of document databases like DocumentDb and Mongo.

Finally, we have our Function “wait” for activity to complete. This activity, as I indicated, can spawn other activities and use its separate function space as needed.

Using Cognitive Services

A discussion on how to setup Cognitive Services within Azure is outside the scope of this article instead, I would invite you to follow Microsoft’s documentation here:

Once you have cognitive services setup, you can update your settings so that your keys and URL match your service, install the necessary Nuget package:

  • Microsoft.Azure.CognitiveServices.Vision.ComputerVision (link)

As a first step, we need to make sure the OcrData table is created and indicate what bits of the computer vision data we want. To do this efficient I created the follow extension method:

public static List<OcrResult> AsResultList(this ImageAnalysis analysisResult, string fileId)
var returnList = new List<OcrResult>();
returnList.AddRange(analysisResult.Adult.AsOcrPairs(fileId, OcrType.ComputerVision));
returnList.AddRange(analysisResult.Color.AsOcrPairs(fileId, OcrType.ComputerVision));
returnList.AddRange(analysisResult.ImageType.AsOcrPairs(fileId, OcrType.ComputerVision));
returnList.AddRange(analysisResult.Description.Captions.FirstOrDefault()?.AsOcrPairs(fileId, OcrType.ComputerVision));
//returnList.AddRange(analysisResult.Brands.AsOcrPairs(fileId, OcrType.ComputerVision));
//returnList.AddRange(analysisResult.Faces.AsOcrPairs(fileId, OcrType.ComputerVision));
return returnList;
static IEnumerable<OcrResult> AsOcrPairs(this object obj, string fileId, OcrType ocrType)
foreach (var propertyInfo in obj.GetType().GetProperties())
if (typeof(IEnumerable).IsAssignableFrom(propertyInfo.PropertyType) == false || propertyInfo.PropertyType == typeof(string))
yield return new OcrResult(fileId)
KeyName = propertyInfo.Name,
OcrValue = propertyInfo.GetValue(obj).ToString(),
OcrType = ocrType
view raw extension.cs hosted with ❤ by GitHub

All this does is allow me to specify parent object points in the return structure for Ocr results and create a name value pair that I can return and more easily insert into the Table Storage schema I am aiming to achieve. Once I have all of these OcrPairs, I use a batch insert operation to update the OcrData table.

var computerVisionResults = await ProcessWithComputerVision(fileBlob, fileId);
// save the batch data
var batchOperation = new TableBatchOperation();
computerVisionResults.ForEach(result => batchOperation.Insert(result));
batchOperation.Insert(new OcrResult(fileId) { KeyName = FileLengthKeyName, OcrValue = fileBlob.Length.ToString(), OcrType = OcrType.None });
var executeReslt = await ocrDataTable.ExecuteBatchAsync(batchOperation);
view raw insert.cs hosted with ❤ by GitHub

Approve and allow the file to be downloaded

Now that the Ocr data has been generated our Task.WhenAny will allow the orchestrator to proceed. The next step is to wait for an external user to indicate their approval for the data to be downloaded – this is nearly a carbon copy of the step which approved the uploaded file for processing.

Once the approval is given, our user can call the DownloadFile function to download the data and get a tokenized URL to use for raw download (our blob storage is private and we want to control access to blobs). Here is our code for the download action:

public static IActionResult Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = "download/{fileId}")] HttpRequest req,
string fileId,
[Table("metadata", "{fileId}", "{fileId}", Connection = "TableConnectionString")] FileMetadata fileMetadata,
[Table("ocrdata", Connection = "TableConnectionString")] CloudTable fileOcrDataTable,
[Blob("files/{fileId}", FileAccess.Read, Connection = "StorageAccountConnectionString")] CloudBlockBlob fileBlob,
ILogger log)
if (!fileMetadata.ApprovedForDownload)
return new StatusCodeResult(403);
var readQuery = new TableQuery<OcrResult>();
TableQuery.GenerateFilterCondition(nameof(OcrResult.PartitionKey), QueryComparisons.Equal, fileId);
var ocrResults = fileOcrDataTable.ExecuteQuery(readQuery).ToList();
return new OkObjectResult(new DownloadResponse
Metadata = ocrResults,
FileId = fileId,
DownloadUrl = GenerateSasUrlForFileDownload(fileBlob, fileId)
static string GenerateSasUrlForFileDownload(CloudBlockBlob blob, string fileId)
var policy = new SharedAccessBlobPolicy()
SharedAccessExpiryTime = DateTime.Now.AddHours(1),
Permissions = SharedAccessBlobPermissions.Read
return blob.Uri + blob.GetSharedAccessSignature(policy);
view raw download.cs hosted with ❤ by GitHub

That is quite a bit of code but, in essence, we are simply gathering all data associated with the data entry being requested for download and generating a special URL for download out of our blob storage that will be good for only one hour – a lot more restrictions can be placed on this so its an ideal way to allow external users to have temporary and tightly controlled access to blobs.

And that is it, you can call this function through Postman and it will give you all data collected for this file and a link to download the raw file. There is also a check to ensure the file has been approved for download.


When I started to explore Durable Functions this was the antithesis of what I was after: event based workflow execution with a minimal amount of code needing to be written and managed.

As I said in Part 1 – for me event driven programming is the way to go in 95% of cloud based backends; the entire platform is quite literally begging us to leverage the internal events and APIs to reduce the amount of code we need to write while still allowing us to deliver on value propositions. True, going to event approach does create new challenges but, I feel that trade-off in most cases is well worth it.

In one of my training classes I explore how we can write “codeless” applications using API Management by effectively using APIM to “proxy” Azure APIs (Key Vault and Storage notably). Sure, there are cases where we need to support additional business logic but, there are also many cases where we write a service to store data to blob storage when we dont need to – when we can just store it there and use events to filter and process things.

In the end, the cloud gives you a tremendous amount of options for what you can do and how to solve problems. And that really is the most important thing: having options and realizing the different ways you can solve problems and deliver value.

Durable Functions: Part 3 – Approve the Upload

All code for this series can be found here:

In Part 1 of this series, we explained what we were doing and why including the aspects of Event Driven Design we are hoping to leverage using Durable Functions (and indeed Azure Functions) for this task.

In Part 2, we build our file uploader that sent our file to blob storage and recorded a dummy entry in Azure Table Storage that will later hold our metadata. We also explained why we choose Azure Table Storage over Document DB (Cosmos default offering)

Here in Part 3, we will start to work with Durable Functions directly by triggering it based on the afore mentioned upload operation (event) and allowing its progression to be driven by a human rather than pure backend code. To that end, we will create an endpoint that enables a human to approve a file by its identifier which, advances the file through the workflow represented by the Durable Function.

Defining Our Workflow

Durable Function workflows are divided into two parts: The Orchestrator Client and the Orchestrator itself.

  • The Orchestrator Client is exactly what it sounds like, the client which launches the orchestrator. Its main responsibility is initializing the long running Orchestrator function and generating an instanceId which can be thought of as a workflow Id
  • The Orchestrator, as you might expect, represents our workflow in code with the stopping points and/or fan outs that will happens as a result of operations. Within this context you can start subworkflows if desired or (as we will show) wait for a custom event to allow advancement

To that end, I have below the code for the OrchestratorClient that I am using as part of this example.

public static async Task HttpStart(
[BlobTrigger("files/{id}", Connection = "StorageAccountConnectionString")] Stream fileBlob,
string id,
[Table("metadata", "{id}", "{id}", Connection = "TableConnectionString")] FileMetadata metadata,
[Table("metadata", Connection = "TableConnectionString")] CloudTable metadataTable,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
// Function input comes from the request content.
string instanceId = await starter.StartNewAsync("ProcessFileFlow", new ApprovalWorkflowData { TargetId = id });
metadata.WorkflowId = instanceId;
var replaceOperation = TableOperation.Replace(metadata);
var result = await metadataTable.ExecuteAsync(replaceOperation);
log.LogInformation($"Started orchestration with ID = '{instanceId}'.");
log.LogInformation("Flow started");
Approve Start Function (Github Gist)

First, I want to call attention to the definition block for this function. You can see a number of parameter, most of which have an attribute decoration. The one to focus on is the BlobTrigger as it does two things:

  • It ensures this functions is called whenever a new object is written to our files container in the storage account defined by our Connection. The use of these types of triggers is essential for achieving the event driven style we are after and which yield substantial benefit when used with Azure Functions
  • It defines the parameter id via its binding notation {id} through this, we can use this value in other parameters which feature binding (such as if we wanted to output a value to a Queue or something similar)

The Table attributes each perform a separate action:

  • The first parameter (FileMetadata) extracts from Azure Table Storage the row with the provided RowKey/PartitionKey combination (refer to Part 2 for how we stored this data). Notice the use of {id} here – this value is defined via the same notation used in the BlobTrigger parameter
  • The second parameter (CloudTable) brings forth a CloudTable reference to our Azure Storage Table. Table does not support an output operation, or at least not a straightforward one. So, I am using this approach to save the entity from the first parameter back to the table, once I update some values

What is most important for this sort of function is the DurableClient reference (Need this Nuget package). This is what we will use to start the action workflow.

Reference Line 11 of our code sample and the call to StartNewAsync. This literally starts an orchestrator to represent the workflow. It returns an InstanceId which we save back to our Azure Table Storage Entity. Why? We could technically have the user pass the InstanceId received from IDurableOrchestrationClient but, for this application, that would run contrary to the id they were given after file upload so, instead we choose to have them send us the file id, perform a look up so we can access the appropriate workflow instance, your mileage may vary.

Finally, since this method is pure backend there is no reason to return anything though you certainly could. In the documentation here Microsoft lays out a number of architectural patterns that make heavy use of the parallelism offered through Durable Functions.

Managing the Workflow

Noting the above code on Line 11 we actually name the function that we want to start, this function is expected to have one argument of type IDurableOrchestrationContext (Note Client vs Context) that is decorated with the OrchestratioinTrigger attribute. This denotes the method is triggered by a DurableClient starting a workflow with this given name (the name here is ProcessFileFlow).

The code for this workflow (at least the initial code) is shown below:

public static async Task RunOrchestrator(
[OrchestrationTrigger] IDurableOrchestrationContext context,
ILogger log)
var uploadApprovedEvent = context.WaitForExternalEvent<bool>("UploadApproved");
await Task.WhenAny(uploadApprovedEvent);
log.LogInformation("File Ready");
view raw workflow1.cs hosted with ❤ by GitHub
Workflow Function – Part 1

I feel it is necessary to keep this function very simple and only contain code that represents steps in the flow or any necessary logic for branching. Any updates to the related info elements is kept in the functions themselves.

For this portion of our code base, I am indicating to the Orchestration Context that advancement to the next step can only occur when an external event called UploadApproved is received. This is, of course, an area that we could provide a split of even a time out concept (this so we dont have n number of workflows sitting waiting for an event that may never be coming).

To raise this event, we need to build a separate function (I will use an HttpTrigger) that can raise this event. Here is the code I choose to use:

public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "approve/{fileId}")] HttpRequest req,
[Table("metadata", "{fileId}", "{fileId}", Connection = "TableConnectionString")] FileMetadata fileMetadata,
[Table("metadata", Connection = "TableConnectionString")] CloudTable metadataTable,
[DurableClient] IDurableOrchestrationClient client,
ILogger log)
var instanceId = fileMetadata.WorkflowId;
fileMetadata.ApprovedForAnalysis = true;
var replaceOperation = TableOperation.Replace(fileMetadata);
await metadataTable.ExecuteAsync(replaceOperation);
await client.RaiseEventAsync(instanceId, "UploadApproved", fileMetadata.ApprovedForAnalysis);
return new AcceptedResult(string.Empty, fileMetadata.RowKey);
Upload Approve Http Function

Do observe that, as this an example, we are omitting a lot of functionality that would pertain to authentication and authorization to allow the UploadApprove action – as such this code should not be taken literally and used only to understand the concept we are driving towards.

Once again, we leverage bindings to simplify our code, mainly based on the fileId provided by the caller we can bring in the FileMetadata reference represented in our Azure Table Storage (we also bring in CloudTable so the afore mentioned entry can be updated to denote the file upload has been approved).

Using the IDurableOrchestrationClient injected into this function we can use the RaiseEventAsync method with the InstanceId extracted from the Azure Table Storage record to raise the UploadApproved event. Once this event is raised, our workflow advances.

Next Steps

Already we see the potential use cases for this approach, as the ability to combine workflow advancement with code based approaches makes our workflows even more dynamic and flexible.

In Part 4, we will close out the entire sample as we add two more approval steps to the workflow (one code driven and the other user driven) and then add a method to download the file.

I hope this was informative and has given you an idea of the potential durable functions hold. Once again, here is the complete code for reference:

Durable Functions: Part 2 – The Upload

I covered the opening to this series in Part 1 (here). Our overall goal is to create a File Approval flow using Azure Durable Functions and showcase how complex workflows can be me managed within this Azure offering. This topic exists in a much larger topic that is Event Driven design. Under EDD we aim to build “specialized” components which respond to events. By doing taking this approach we write only what code should do an alleviate ourselves of boilerplate and unrelated code. This creates a greater degree of decoupling which can help us when change inevitably comes. It also allows us to solve specific problems without generating wasteful logic which can hide bugs and creates other problems.

In this segment of the series, we will talk through building the upload functionality to allow file ingestion. Our key focus will be the Azure Function binding model that allows boilerplate code to be neatly extracted away from our function. Bindings also underpin the event driven ways we can working with functions, specifically allowing them to be triggered by an Azure Event.

Let’s get started. As I move forward I will be assuming that you are using Visual Studio Code with the Azure Function tools to create your functions. This is highly recommended and is covered in detail in Part 1.

Provision Azure Resources

The first thing we will want to do is setup our Azure resources for usage, this includes:

  • Resource Group
  • Azure Storage Account (create a container)
  • Azure Table Storage (this is the Table option within Cosmos)
  • Cognitive Services (we will use this in Part 3)

As a DevOps professional my recommended approach to deploying infrastructure to any cloud environment is to use a scripted approach, ideally Terraform or Pulumi. For this example, we will not go into that since it is not strictly my aim to extol good DevOps practices as part of this series (we wont be deploying CI/CD either).

For this simple demo, I will leave these resources publically available thus, we can update local.settings.json with the relevant connection information as we develop locally. local.settings.json is a special config file that, by default, the template for an Azure Function project created by the VSCode Azure Functions will excludes from source control. Always be diligent and refrain from checking credentials into source control, especially for environments above Development.

Getting started, you will want to having the following values listed in local.settings.json:

  • AzureWebJobsStorage – used by Azure Functions runtime, the value here should be the connection string to the storage account you created
  • FUNCTIONS_WORKER_RUNTIMEdotnet – just leave this alone
  • StorageAccountConnectionString – this is the account where our uploads are saved too, again it can share the same Storage Account that you previously created
  • TableConnectionString– this is the connection string to the Azure Table Storage instance
  • CognitiveServicesKey – the value of the key given when you create an instance of the Azure Cognitive Services resource
  • CognitiveServicesEndpoint – the value of the endpoint to access your instance of the Azure Cognitive services

Here is the complete code for the Azure Function which handles this upload:

public static class UploadFile
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "file/add")] HttpRequest uploadFile,
[Blob("files", FileAccess.Write, Connection = "StorageAccountConnectionString")] CloudBlobContainer blobContainer,
[Table("metadata", Connection = "TableConnectionString")] CloudTable metadataTable,
ILogger log)
var fileName = Guid.NewGuid().ToString();
await blobContainer.CreateIfNotExistsAsync();
var cloudBlockBlob = blobContainer.GetBlockBlobReference(fileName);
await cloudBlockBlob.UploadFromStreamAsync(uploadFile.Body);
await metadataTable.CreateIfNotExistsAsync();
var addOperation = TableOperation.Insert(new FileMetadata
RowKey = fileName,
PartitionKey = fileName,
await metadataTable.ExecuteAsync(addOperation);
return new CreatedResult(string.Empty, fileName);
view raw upload.cs hosted with ❤ by GitHub
Figure 1 – File Upload for Azure Function

The code looks complex but, it is actually relatively simple given to the heavy use of Azure Function bindings. There are three in use:

  • HttpTrigger – most developers will be familiar with this trigger. Through it, Azure will listen for Http requests to a specific endpoint and route and execute this function when such a request is detected
  • Blob – You will need this Nuget package. This creates a CloudBlobContainer initialized with the given values. It makes it incredibly easy to write data to the container.
  • Table – Stored in the same Nuget as the Blob bindings. This, like the blob, opens up a table connection to make it easy to add data, even under high volume scenarios

Bindings are incredibly useful when developing Azure Functions. Most developers are only familiar with HttpTrigger which is used to respond to Http requests but there is a huge assortment and support for events from many popular Azure resources. Using these removes the need to write boilerplate code which can clutter our functions and obscure their purpose.

Blob and Table can be made to represent an item in their respective collections or a collection of items. The documentation (here) indicates what types method arguments using these attributes can be. Depending on how you use the attribute it can be a reference to the table itself, a segment of data from that table (using the partition key), or an item itself. The Blob attribute has similar properties (here).

One thing to keep in mind is a “bound parameter” must be declared as part of a trigger binding attribute to be used by other non-trigger bindings. Essentially it is important to understand that bindings are bound BEFORE the function is run, not after. Understanding this is essential to creating concise workflows using bindings.

Understanding Binding through an example

Taking our code sample from above (repasted here)

public static class UploadFile
public static async Task<IActionResult> Run(
[HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = "file/add")] HttpRequest uploadFile,
[Blob("files", FileAccess.Write, Connection = "StorageAccountConnectionString")] CloudBlobContainer blobContainer,
[Table("metadata", Connection = "TableConnectionString")] CloudTable metadataTable,
ILogger log)
var fileName = Guid.NewGuid().ToString();
await blobContainer.CreateIfNotExistsAsync();
var cloudBlockBlob = blobContainer.GetBlockBlobReference(fileName);
await cloudBlockBlob.UploadFromStreamAsync(uploadFile.Body);
await metadataTable.CreateIfNotExistsAsync();
var addOperation = TableOperation.Insert(new FileMetadata
RowKey = fileName,
PartitionKey = fileName,
await metadataTable.ExecuteAsync(addOperation);
return new CreatedResult(string.Empty, fileName);
view raw upload.cs hosted with ❤ by GitHub

So here I am creating the unique Id (called fileName) in code. If I wanted to, I could specify {id} in the HttpTrigger as part of the path. This would give me access to the value of {id} in other bindings or as a parameter to the function called id. In this case, it would amount to relying on the user to give me a unique value, which would not work.

I hope that explains this concept, I find understanding it makes things easier and more straightforward with how you may choose to write your code. If not, there will be other examples of this in later sections and I am happy to explain more in the comments.

The Upload Process

Now that we have covered the binding the code should make a lot more sense if it did not before.

Simply put, we:

  1. Call Guid.NewGuid().ToString() and get a string representation or a new Guid. This is our unique Id for this file upload
  2. The binary stream accepted through the Http Post request is saved to a block reference into our Azure Storage Account
  3. Next, the initial record for our entry is created in the Azure Table Storage (Approval flags are both set to false)
  4. We return a 201 Created response as is the standard for Post operations which add new state to systems

Straightforward and easy to understand and thanks to the bindings all heavy lifting was done outside of the scope our function allowing it to clearly express its intent.

Why Azure Table Storage?

Azure Table Storage is an offering that has existed for a long time in Microsoft Azure, it only recently came to be under the Cosmos umbrella along with other NoSQL providers. The use of Table Storage here is intentional due to its cost effectiveness and speed. But, it does come with some trade-offs:

  • Cosmos Core (DocumentDb) offering is designed as a massive scalable NoSQL system. For larger systems with more volume, I would opt for this over Table Storage – though you get what you pay for, its not cheap
  • DocumentDb is a document-database meaning the schema is never set in stone and can always be changed as new records are added. This is not the case with Table Storage which will set its schema based on the first record written.

When making this decision it is important to consider not just current requirements but also near-term requirements as well. I tend to like Table Storage when the schema is not going to have a lot of variance and/or I want a NoSQL solution that is cheap and still effective. Cosmos Core is the other extreme where I am willing to pay more high redundancy and greater performance as well as using a document database where my schema can be different insert to insert.

Triggering the Workflow

Reading the upload code you may have wondered where the workflow is triggered or how it is triggered. By now the answer should not surprise you: binding. Specifically, a BlobTrigger which can listen for new blobs being added (or removed) and trigger a function when that case is detected. Here is declaration of the Azure Durable Function which represents the bootstrapping of our workflow.

public static async Task HttpStart(
[BlobTrigger("files/{id}", Connection = "StorageAccountConnectionString")] Stream fileBlob,
string id,
[Table("metadata", "{id}", "{id}", Connection = "TableConnectionString")] FileMetadata metadata,
[Table("metadata", Connection = "TableConnectionString")] CloudTable metadataTable,
[DurableClient] IDurableOrchestrationClient starter,
ILogger log)
Figure 2 – Workflow start declaration

As you can see here, we are starting to get a bit nuts with our triggers. Here is a brief summary:

  • We use a BlobTrigger to initiate this function and {id} to grab the name of newly created blob (this will be the Guid which is generated during upload)
  • The Table attribute is used twice: once to reference the actual Table Storage record represented by the newly created blob and another as a reference to the metadata table where the referenced row exists (we need this to write the row back once its updated)
  • Finally DurableClient (from this Nuget package) which provides the client that allows us to start the orchestrator that will manage the workflow

I will go into much more depth on this in Part 3 but the one point I do want to call out is Table attribute is NOT two way. This means, even if you reference the single item (as we did in our example), changes to that single item are NOT saved back to the table – you must do this manually. This is important as it drives the reason we see some rather creative uses of this attribute.


We explored some code in this portion of the series, though it was not immediately tied to Durable Functions it was, tied to event driven programming. Using these bindings we can create code that alleviates itself from mundane and boilerplate operations and allow other systems to manage this on our behalf.

Doing this gets us close to the event driven model discussed in Part 1 and allows each function to specialize in what it must do. By cutting our excess and unnecessary code we can remove bugs and complexities that can make it more difficult to manage our code base.

In Part 3, we are going to dive deeper and really start to explore Durable Functions and showing how they can be initiated and referenced in subsequent calls, including those that can advance the workflow, circa a human operation.

The complete code for this entire series is here:

Durable Functions: Part 1 – The Intro

No Code in this post. Here we establish the starting point

Event Driven Programming is a popular way to approach complex systems with a heavy emphasis on breaking applications apart and into smaller, more fundamental pieces. Done correctly, taking an event driven approach can make coding more fun and concise and allow for “specialization” over “generalization”. In doing so, we get closer to the purity of code that does only what it needs to do and nothing more, which should always be our aim as software developers.

In realizing this for cloud applications I have become convinced that, with few exceptions, serverless technologies must be employed as the glue for complex systems. The more they mature, the greater the flexibility they offer for the Architect. In truth, not using serverless can, and should be, viewed in most cases as an anti-pattern. I will note that I am referring explicitly to tooling such as AWS Lamba, Google Cloud Functions, and Azure Functions, I am not speaking to “codeless” solutions such as Azure Logic Apps or similar tools in other platforms – the purpose of such tools is mainly to allow less technical persons to build out solutions. Serverless technologies, such as those mentioned, remain in the domain of the Engineer/Developer.

Very often I find that engineers view serverless functions as more of a “one off” technology, good for that basic endpoint that can run in Consumption. As I have shown before, Azure Functions in particular are very mature and through the use of “bindings” can enable highly sophisticated scenarios without need for writing excessive amounts of boilerplate code. Further, offerings such as Durable Functions in Azure (Step Functions in AWS) enable serverless to go a step further and actually maintain a semblance of state between calls – thus enabling sophisticated multi-part workflows that feature a wide variety of inputters for workflow progression. I wanted to demonstrate this in this series.

Planning Phase

As with any application, planning is crucial and our File Approver application shall be no different. In fact, with event driven applications planning is especially crucial because while Event Driven systems offer a host of advantages they also require certain questions to be answered. Some common questions:

  • How can I ensure events get delivered to the components of my system?
  • How do I handle a failure in one component but success in another?
  • How can I be alerted if events start failing?
  • How can I ensure events that are sent during downtime are processed? And in the correct order?

Understandable, I hope, these questions are too big to answer as part of this post but, are questions I hope you, as an architect, are asking your team when you embark on this style of architecture.

For our application, we will adopt a focus on the “golden path”. That is, the path which assumes everything goes correctly. The following diagram shows our workflow:

Our flow is quite simple and straightforward

  • Our user uploads a file to an Azure Function that operates off an HttpTrigger
  • After receiving this file, the binary data is written to Azure Blob Storage and a related entry is made in Azure Table Storage
  • The creation of the blob triggers Durable Function Orchestration which will manage a workflow that aims to gather data about the file contents and ultimately allow users to download it
  • Our Durable workflow contains three steps, two of which will pause our workflow waiting for human actions (done via Http API calls). The other is a “pure function” that is only called as part of this workflow
  • Once all steps are complete the file is marked available for download. When requested the Download File function will return the gathered metadata for the file AND the generated SAS Token allowing persons to download the file for a period of 1hr

Of course, we could accomplish this same goal with a traditional approach but, that would leave us to write a far more sophisticated solution than I ended up with. For reference, here is the complete source code:

Azure Function Bindings

Bindings are a crucial components of efficient Azure Function design, at present I am not aware of a similar concept in AWS but, I do not discount its existence. Using bindings we can write FAR LESS code and make our functions easier to understand with more focus on the actual task instead of logic for connecting and reading from various data source. In addition, the triggers tie very nicely into the whole Event Driven paradigm. You can find a complete list of ALL triggers here: this is a direct link to the Blob storage triggers, see the left hand side for a complete list.

Throughout my code sample you will see references to bindings for Durable Functions, Blobs, Azure Table Storage, and Http. Understanding these bindings is, as I said, crucial to your sanity when developing Azure Functions.

Visual Studio Code with Azure Function Tools

I recommend Visual Studio Code when developing any modern application since its lighter and the extensions give you a tremendous amount of flexibility. This is not to say you cannot use Visual Studio, the same tools and support exist, I just find Visual Studio Code (with the right extensions) to be the superior product, YMMV.

Once you have Visual Studio Code you will want to install two separate things:

  • Azure Functions Extension VSCode
  • Azure Function Core Tools (here)

I really cannot say enough good things about Azure Function Core Tools. It has come a long way from version 1.x and the recent versions are superb. In fact, I was able to complete my ENTIRE example without ever deploying to Visual Studio, using breakpoints all along the way.

The extension for Visual Studio Code is also very helpful for both creating and deploying Azure Functions. Unlike traditional .NET Core applications, I do not recommend using the command line to create the project. Instead, open Visual Studio Code and access your Azure Tools. If you have the Functions extension installed, you will see a dedicated blade – expand it.

The first icon (looks like a folder) enables you to create a Function project through Code. I recommend this approach since it gets you started very easily. I have not ruled out the existence of templates that could be downloaded and use through dotnet new but this works well enough.

Keep in mind that a Function project is 1:1 with a Function app so, you will want to target an existing directory if you play to have more than one in your solution. Note that this is likely completely different in Visual Studio, I do not have any advice for that approach.

When you go through the creation process you will be asked to create a function. For now, you can create whatever you like, I will be diving into our first function in Part 2, as you create subsequent functions use the lightning icon next to the folder. Doing this is not required, it is perfectly acceptable to build your functions up but, using this gets the VSCode settings correct to enable debugging with the Core Tools so, I highly recommend it.

The arrow (third icon) is for deploying. Of course, we should never use this outside of testing since we would like a CI/CD process to test and deploy code efficiently – we wont be covering CI/CD for Azure Functions in this series but, we will certainly in a future series.


Ok so, now we understand a little about what Durable Functions are and how they play a role in Event Driven Programming. I also walked through the tools that are used when developing Azure Functions and how to use them.

Moving forward into Part 2, we will construct our File Upload portion of the pipeline and show how it starts our Durable Function workflow.

Once again the code is available here: