Recently while working with a client, we were asked to implement an Authorization model that could crosscut our various services with the identity tying back to B2C and the OAuth pattern in use. A lot of ideas were presented, and it was decided to explore (and ultimately use) the preview feature Token Enrichment: https://learn.microsoft.com/en-us/azure/active-directory-b2c/add-api-connector-token-enrichment?pivots=b2c-user-flow

How does OAuth handle Authorization?

Prior to the rise of OAuth, or delegated authentication really, authorization systems usually involved database calls based on a user Id which was passed as part of a request. Teams might even leverage caching to speed things up but, in the world of monoliths this was heavily used, realistically for most teams there was no alternative.

Fast forward to the rise in distributed programming and infrastructure patterns like Microservices or general approaches using SOA (Service Oriented Architecture) and this pattern falls on its face, and hard. Today, the very idea of supporting a network call for EVERY SINGLE request like this is outlandish and desperately avoided.

Instead, teams leverage an identity server concept (B2C here, or Okta, Auth0 [owned by Okta] or Ping] whereby a central authority issues the token and embeds the role information into the token’s contents; the names of roles should never constitute sensitive information. Here is a visual:

The critical element to understand here is that tokens are hashed and signed. Any mutation of the token will render it invalid and unable to pass any authoritative check. Thus, we just need to ensure no sensitive data is exposed in the token, as they can be easily decoded by sites like https://jwt.ms and https://jwt.io

Once the Access Token is received by the service and it is verified, the service can then strip claims off the token and use it for its own processing. I wont be showing you in this article BUT dotnet (and many other web frameworks) natively support constructs to make this parsing easy and enable easy implementation of Auth systems driven by claims.

How do I do this with B2C?

B2C supports API Connectors per the article above. These connectors allow B2C to reach out at various stages and contact an API to perform additional work; including enrichment.

The first step in this process is the creation of a custom attribute to be sent with the Access Token to hold the custom information, I called mine Roles.

Create the Custom Attribute for ALL Users

1. From your Azure B2C Tenant select User Attributes
2. Create a new Attribute called extension_Roles of type string
3. Click Save

The naming of the attribute here is crucial. It MUST be preceded be extension_ for B2C to return the value.

This attribute is created ONLY to hold the value coming from token enrichment via the API, it is not stored in B2C, only returned as part of the token.

Configure your sign-in flow to send back our custom attribute

1. Select User Flows from the main menu in Azure B2C
2. Select your sign-in flow
3. Select Application claims
4. Find the custom claim extension_Roles in the list
5. Click Save

This is a verification step. We want to ensure our new attribute is the Application Claims for the flow and NOT, the user attributes. If it is in the user attributes, it will appear in the sign-up pages.

Deploy your API to support the API Connector

The link at the top shows what the payload to the API connector looks like as well as the response. I created a very simple response in an Azure Function, shown below:

	public class HttpReturnUserRolesFunction
	{
	[FunctionName("HttpReturnUserRoles")]
	public IActionResult HttpReturnUserRoles(
	[HttpTrigger(AuthorizationLevel.Anonymous, "get", "post", Route = null)] HttpRequest req,
	ILogger log)
	{
	return new OkObjectResult(new {
	version = "1.0.0",
	action = "Continue",
	postalCode = "12349",
	extension_Roles = "Admin,User"
	});
	}
	}

view raw function.cs hosted with ❤ by GitHub

We can deploy this the same way we deploy anything in Azure; in my testing I used right-click publishing to make this work.

Setting the API Connector

We need to configure B2C to call this new endpoint to enrich the provided token.

1. Select API Connectors from the B2C menu
2. Click New API Connector
3. Choose any Display name (I used Enrich Token)
4. For the Endpoint URL input the web page to your API
5. Enter whatever you want for the Username and Password
6. Click Save

The username and password can provide an additional layer of security by sending a base64 encoded string to the API endpoint which the endpoint can decode and validate the caller is legitimate. In the code above, I choose not to do this, though I would recommend it for a real scenario.

Configure the Signup Flow to call the Enrich Token API endpoint

The last step of the setup is to tell the User Flow for Signup/Sign-in to call our Enrich Token endpoint.

1. Select User Flows
2. Select the User Flow that represents the Signup/Signin operation
3. Select API Connectors
4. For the Before including application claims in token (preview) select the Enrich Token API Connector (or whatever name you used)
5. Click Save

This completes the configuration for calling the API Connector as part of the user flow.

Testing the Flow

Now let’s test our flow. We can do this using the built-in flow tester in B2C. Before that though, we need to create an Application Registration and set the a reply URL so the flow has some place to dump the user when validation is successful.

For this testing, I recommend using either jwt.ms or jwt.io which will receive the token from B2C and show its contents. For more information on creating an Application Registration see this URL: https://learn.microsoft.com/en-us/azure/active-directory-b2c/tutorial-register-applications?tabs=app-reg-ga

Once you have created the registration return to the B2C page and select User Flows. Select your flow, and then click Run User Flow. B2C will ask under what identity do you want to run the flow as. Make sure you select the identity you created and validate that the Reply URL is what you expect.

Click Run user flow and login (or create a user) and you should get dumped to the reply URL and see your custom claims. Here is a sample of what I get (using the code above).

	{
	"alg": "RS256",
	"kid": "X5eXk4xyojNFum1kl2Ytv8dlNP4-c57dO6QGTVBwaNk",
	"typ": "JWT"
	}.{
	"ver": "1.0",
	"iss": "b2clogin_url/v2.0/",
	"sub": "d0d196a4-96b3-4c46-b550-842ab59cd4d8",
	"aud": "3a61cc01-104a-44c8-a3ff-d895a860d70e",
	"exp": 1695000577,
	"nonce": "defaultNonce",
	"iat": 1694996977,
	"auth_time": 1694996977,
	"extension_Roles": "Admin,User",
	"tfp": "B2C_1_Signup_Signin",
	"nbf": 1694996977
	}.[Signature]

view raw token.json hosted with ❤ by GitHub

Above you can see extension_Roles with the value Admin,User. This token can then be parsed in your services to check what roles are given to the user represented by this token.

Recently I finished “Team Topologies: Organizing Business and Technology Teams for Fast Flow” written by Matthrew Skelton and Manuel Pais – Amazon: https://a.co/d/1U8Gz56

I really enjoyed this book because it took a different tactic to talking about DevOps, one that is very often overlooked by organizations: Team Structure and Team Communication. A lot of organizations that I have worked with misunderstand DevOps as simply being automation or the use of a product like GitHub Actions, CircleCI, Azure DevOps, etc. But the truth is, DevOps is about so much more than this and the book really goes deep into this explore team topologies and emphasizing the need to organize communication.

In particular the book calls out four core team types:

• Stream aligned – in the simplest sense these are feature teams but, really they are so much more. If you read The Phoenix Project by Gene Kim you start to understand that IT and engineering are not really its own “thing” but rather, a feature of a specific department, entity, or collab. Thus, what stream-aligned actually means is a set of individuals, working together to handle changes for that part of the organization

• Enabling – this one I was aware of though, I had never given it a formal name. This team is designed to assist steam aligned teams enable something. A lot of orgs make the mistake of creating a DevOps team, which is a known anti-pattern. DevOps (or automation as it usually is) is something you enable teams with, with things like self-service and self-management. The goal of the enabling team is to improve the autonomy of the team.

• Platform – platform teams can be stream-aligned teams but, their purpose is less about directly handling the changes for a part of the org than it is support other stream aligned teams. Where enabling teams may introduce new functionality, platform teams support various constructs to enable more streamlined operation. Examples might include a wiki with documentation for certain techniques to even a custom solution enabling the deployment of infrastructure to the cloud.

• Complicated Sub-system – the author views this a specialized team that is aligned to a single, highly complex, part of a system or the organization (can even be a team managing regulatory compliance). The author uses the example of a trading platform, where individuals on the team manage a highly complex system performing market trades, where speed and accuracy must be perfect.

The essence of this grouping is to align teams to purpose and enable fast flow, what Gene Kim (in The DevOps Handbook) calls The First Way. Speed is crucial for an organization using DevOps, as speed to deploy also means speed to recover. And to enable that speed teams need focus (and to reduce change sizes). Too often organizations get into sticky situations and respond with still more process. While the general thought is it makes things better, really it is security theater (link) – in fact I observed this often leads to what I term TPS (Traumatic Process Syndrome) where processes become so bad, that teams do every thing they can to avoid the trauma of going through with them.

Team Topologies goes even deeper than just talking about these four team types, going even into office layouts and referencing the Spotify idea of squads. But, in the end, as the author indicates, this is all really a snapshot in time and it is important to constantly be evaluating your topology and make the appropriate shifts as priorities or realities shift – nothing should remain static.

To further make this point, the book introduces the three core communication types:

• Collaboration – this is a short lived effort so two teams can perform discovery of new features, capabilities, and techniques in an effort to be better. The author stresses this MUST be short lived, since collaborating inherently brings about inefficiencies and blurs boundaries of responsibilities, and increased cognitive load for both teams.

• X-as-a-Sevice – this is the natural evolution from collaboration, where one team provides functionality “as a service” to one or more teams. This is not necessarily a platform model but, instead, enforces the idea of separation of responsibilities. Contrasting with collaboration, cognitive load is minimal here as each knows their responsibilities

• Facilitating – this is where one team is guiding another. Similar, in my view, to collaboration, it is likewise short-lived and designed to enable new capabilities. Therefore, this is the typical type of communication a stream-aligned and enabling team will experience.

One core essence of this is avoid anti-patterns like Architectural Review Boards, or another other ivory-tower planning committee. Trying to do this sort of planning up front is at best, asking for continuous stream of proposals as architectures morph, and at worst a blocking process that delays projects and diminishes trust and autonomy.

It made me recall an interaction I had with a client many years ago. I had asked “how do you ensure quality in your software?” to which they replied “we require a senior developer approve all PRs”. I looked at the person and then asked “about how many meetings per day is that person involved in?” I asked. They conferred for a moment and came back and said “8”. I then looked at them and said, “how much attention would you say he is actually exercising against the code?” It began to dawn on them. It came to light much later that that Senior Developer had not been actively in the code in months and was just approving what he was asked to approve. It was the Junior developers approving and validating their work with each other – further showing that “developers will do what it takes to get things done, even in the face of a bad process”.

And this brings me to the final point I was to discuss from this book, cognitive load. Being in the industry for 20yrs now I have come to understand that we must constantly monitor how much cognitive load an action takes, people have limits. For example, even if its straightforwad, opening a class file with 1000 lines will immediately overload cognitive load for most people. Taking a more complex approach, or trying to be fancy when it is not needed also affects cognitive load. And this makes it progressively harder for the team to operate efficiently.

In fact, Team Topologies talks about monitoring cognitive load as a way to determine when a system might need to be broken apart. And yes, that means giving time for the reduction of technical debt, even in the face of delaying features. If LinkedIn can do it (https://www.bloomberg.com/news/articles/2013-04-10/inside-operation-inversion-the-code-freeze-that-saved-linkedin#xj4y7vzkg) your organization can do it, and in doing so shift the culture to “team-first” and improve its overall health.

I highly recommend this book for all levels and roles, technologists will benefit as much as managers. Organizing teams is the key to actually getting value from DevOps. Anyone can write pipelines and automate things but, if such a shift is done without actually addressing organizational inefficiencies in operations and culture, you may do more harm than good.

Team Topologies on Amazon – https://a.co/d/1U8Gz56

Deploying to the Cloud makes a lot of sense as the large number of services in Azure (and other providers) can help accelerate teams and decrease time to market. However, while many services are, with their defaults, a great option for hosting applications on the public Internet, it can be a bit of a mystery for scenarios where applications should be private. Here I wanted to walk through the steps of privatizing a Function App and opening it to the Internet via an Application Gateway.

Before we start, a word on Private Endpoint

This post will heavily feature Private Endpoint as a means to make private connections. Private Endpoints and the associated service, Private Link, enable to very highest levels of control over the flow of network traffic by restricting it ONLY within the attached Virtual Network.

This, however, comes at a cost as it will typically require the usage of Premium plans for services to support the feature. What is important to understand is that service-service (even cross region) communication in Azure is ALL handled on the Microsoft backbone, it never touches the public internet. Therefore, your traffic is, by default, traveling in a controlled and secure environment.

I say this because I have a lot of clients whose security teams set Private Endpoint as the default. For the vast majority of use cases, this is overkill as the default Microsoft networking is going to be adequate for majority of data cases. The exceptions are the obvious ones: HIPPAA, CIJIS, IRS, and Financial (most specifically PCI), and perhaps others. But, in my view, using it for general data transfer, is overkill and leads to bloated cost.

Now, on with the show.

Create a Virtual Network

Assuming you already have a Resource Group (or set of Resource Groups) you will first want to deploy a Virtual Network with an address space, for this example I am taking the default of 10.0.0.0/16. Include the following subnets:

• functionApp – CIDR: 10.0.0.0/24 – will host the private endpoint that is the Function App on the Virtual Network
• privateEndpoints – CIDR: 10.01.0/24 – will host our private endpoints for related services, Storage Account in this case
• appGw – CIDR: 10.0.2.0/24 – will host the Application Gateway which enables access to the Function App for external users
• functionAppOutbound – CIDR: 10.0.3.0/24 – This will be the integration point where the function app will send outbound requests

The Region selected here is critical, as many network resources can either not cross a regional boundary OR can only cross into their paired region. I am sing East US 2 for my example.

Create the Storage Account

Function Apps rely on a storage account to support the runtime. So we will want to create one to support our function app. One thing to keep in mind, Private Endpoints are NOT supported on v1 of Storage Account, only v2. If you attempt to create the Storage Account through the Portal via the Function App process, it will create a v1 account and NOT support Private Endpoint.

When you create this Storage Account, be sure the network settings are wide-open; we will adjust it after the Function App is successfully setup.

Create the Function App

Now with the Function App we want to keep a few things in mind.

• Use the same region that the Virtual Network is deployed into
• You MUST use either a Premium or App Service plan type, Consumption does not support privatization
• For hosting, select the storage account you created in the previous section.
• For the time being do NOT disable public access – we will disable it later

For added benefit I recommend picking Windows for the Operating system as it will enable in-portal editing. This will let you quickly setup the Ping endpoint I am going to describe later. Note this post does NOT go into deploying – without public access additional configuration may be required to support automated deployments.

Allow the process to complete to create the Function App.

Enable VNet Integration

VNet integration is only available on Premium SKUs and above for both Function Apps and App Services. It enables a service to sit effectively on the boundary of the VNet and communicate with private IPs in the attached VNet as well as peered VNets.

For this step, access the Networking blade in your Function App and look for the VNet integration link on the right side of the screen.

Next, click “Add VNet” and select the Virtual Network and Subnet (functionAppOutbound) which receive the outbound traffic from the Function App.

Once complete, leave ROUTE ALL enabled. Note that for many production scenarios leaving this on can create issues, as I explain next. But for this simple example, having it enabled will be fine.

What is ROUTE ALL?

I like to view an App Service, or Function App, as having two sides, inbound and outbound. VNet integration allows the traffic coming out of the service to enter a Virtual Network. Two different modes are supported: ROUTE ALL and default. With ROUTE ALL enabled ALL traffic enters the VNet, including traffic perhaps bound for an external host (https://www.google.com for example). Thus, to support this YOU must add the various control to support egress. With ROUTE ALL disabled, routing will simply follow rules within RFC1918 (link) and send 10.x and a few others into the Virtual Network and the rest will follow Azure Routing rules.

Microsoft documentation explains it more clearly: Integrate your app with a Virtual Network

Setup Private Connection for Storage Account

Function Apps utilize two sub-services within Storage Account for operation: blob and file. We need to create private endpoints for these two sub-services so that, using the VNet Integration we just enabled, the connection to the runtime is handled via private connection.

Access the Storage Account and select the Networking blade. Immediately select Disabled for Public network access. This will force the use of Private Endpoint as the sole means to access the storage account. Hit Save before continuing to the next step.

Select the Private endpoint connections tab from the top. Here is a screen shot of the two Private Endpoints I created to support my case:

We will create a Private Endpoint for file share and blob services, as these are being used by the Function App to support the runtime. By doing this through the portal, other networking elements, such as setup of the Private DNS Zone can be handled for us. Note, in an effort to stay on point, I wont be discussing how Private Endpoint/Link routing actually works.

Click the + Private endpoint button and follow the steps for both file and blob subresource types. Pay special attention to the values the defaults select, if you have other networking in the subscription, it can select these components and cause communication issues.

Each private endpoint should link into the privateEndpoints subnet that was created with the Virtual Network.

Remember, it is imperative that the Private Endpoint MUST be deployed in the same region and same subscription as the Virtual Network to which it is being attached to.

More information on Private Endpoint and the reason for the Private DNS Zone here

Update the Function App

Your Function App needs to be updated to ensure it understands that it must get its content over a VNet. Specifically this involves updating Configuration values.

Details on what values should be updated: Configure your function app settings

The one to key on is the WEBSITE_CONTENTOVERVNET setting and ensuring it is set to 1. Note the documentation deploys a Service Bus, we are not doing so here so you can skip related fields.

Be sure to check that each values matches expectation. I skipped over this the first time and ran into problems because of it.

Click Save to apply and before moving on.

Go into General Settings and disable HTTPS Only. We are doing this to avoid dealing with certificate in the soon to be created Application Gateway. In a Production setting you would not want this turned off.

Click Save again to apply the changes.

Next, create a new HttpTrigger Function called HttpPing. Use the source code below:

	#r "Newtonsoft.Json"
	using System.Net;
	using Microsoft.AspNetCore.Mvc;
	using Microsoft.Extensions.Primitives;
	using Newtonsoft.Json;
	public static IActionResult Run(HttpRequest req, ILogger log)
	{
	return new OkObjectResult("ping");
	}

view raw ping.cs hosted with ❤ by GitHub

Again, I am assuming you used Windows for your Plan OS otherwise, you will need to figure out how to get custom code to this function app so you can validate functionality, beyond seeing the loading page.

Once you complete this, break out Postman or whatever and hit the endpoint to make sure it’s working.

Coincidentally, this will also validate that the Storage Account connection is working. Check for common errors like DLL not found or Runtime unreachable or the darn thing just not loading.

Create Private Endpoint for Function App

With the networking features in place to secure the outbound communications from the Function App we need to lock down the incoming traffic. To do this we need disable private access and use Private Endpoint to get a routable private IP for the Function App.

Return to the Networking blade and this time, select Private Endpoints from the screen (shown below):

Using the Express option, create a private endpoint attached to the functionApp subnet in our Virtual Network – choose Yes for Integrate with private DNS zone (this will create the Private DNS zone and allow routing to work). Once complete, attempt to hit your Function App again, it should still work.

Now, we need to disable Public Access to the function app. Do this by returning to the Networking blade of the Function App, this time we will select Access restriction.

Declick the Allow public access checkbox at the top of the page. And click Save.

If you attempt to query the Function App now, you will be met with an error page indicating a 403 Forbidden.

Remember, for most PaaS services, unless an App Service Environment is used, it can never be fully private. Users who attempt to access this function app now will receive a 403 – as the only route left to the service is through our Virtual Network. Let’s add an Application Gateway and finish the job.

Create an Application Gateway

Application Gateway are popular networking routing controls that operate at Layer-7, the HTTP layer. This means they can route based on pathing, protocol, hostname, verb, really any feature of the HTTP payload. In this case, we are going to assign the Application Gateway a Public IP and then call that Public IP and see our Function App respond.

Start by selecting Application Gateway for the list of available services:

On the first page set the following values:

• Region should be the SAME as the Virtual Network
• Disable auto-scaling (not recommended for Production scenarios)
• Virtual Network should be the Virtual Network created previously
• Select the appGw subnet (App Gateway MUST have a dedicated subnet)

On the second page:

• Create a Public IP Address so as to make the Application Gateway addressable on the public internet, that is it will allow external clients to call the Function App.

On the third page:

• Add a backend pool
• Select App Service and pick you Function App from the list
• Click Add

On the fourth page:

• Add a Routing Rule
• For Priority make it 100 (can really be whatever number you like)
• Take the default for all fields, but make sure the Listener Type is Basic Site and the Fronend IP Protocol is HTTP (remember we disabled HTTPS Only on the Function App)
• Select Backend Targets tab
• For Backend Target select the pool you defined previously
• Click Add new for Backend Settings field
• Backend protocol should be HTTP, with port 80
• Indicate you wish to Override with new host name. Then choose to Pick host name from backend target – since we will let the Function App decide the hostname
• Click Add a couple times

Finish up and create the Application Gateway.

Let’s test it out

When we deployed the Application Gateway we attached it to a public IP. Get the address of that Public IP and replace the hostname in your query – REMEMBER we must use HTTP!!

If everything is setup properly you should get back a response. Congratulations, you have created a private Azure Function App routable only through your Virtual Network.

Options for SSL

To be clear, I would not advocate the use of HTTP for any scenario, even in Development. I abstained from that path to make this walkthrough easier. Apart from create an HTTPS listener in the Application Gateway, Azure API Management operating in External mode with Developer or Premium SKU (only they support VNet Integration) would be the easiest way of support TLS throughout this flow.

Perhaps another blog post in the future – just APIM takes an hour to deploy so, it is a wait 🙂

Closing Remarks

Private endpoint is designed as a way to secure the flow of network data between services in Azure, specifically it is for high security scenarios where data needs to meet certain regulatory requirement for isolation. Using Private Endpoint for this case, as I have shown, is a good way to approach security without taking on the expense and overhead of an App Service Environment which creates an isolated block within the data center for your networking.

That said, using them for all data in your environment is not recommended. Data, by default, goes over the Azure backbone and stays securely on Microsoft networks so long as the communication is between Azure resources. This is advised for most data scenarios and can free your organization from the cost and overhead of maintaining Private Endpoints and Premium SKUs for apps that make no sense to have such capability.

So over the last week I have been battling an issue in Terraform that truly drove me nuts and I think understanding can help someone else who is struggling with the same issue.

What is a data source?

In Terraform there are two principle elements when building scripts: resources and data sources. Resource is something that will be created by and controlled by the script. A data source is something which Terraform expects to exist. Keep this in mind, as it will be important in a moment.

What is a module?

Modern Infrastructure as Code approaches focus on modules as a way to encapsulate logic and standards which can be reused. It is this approach which underpins the problem I found. That is, when a module is built that needs to look up existing resources to hydrate fields on encapsulated resources.

What is the problem?

The problems a sequence of events like this:

	terraform {
	required_providers {
	azurerm = {
	source = "hashicorp/azurerm"
	version = "=3.30.0"
	}
	}
	}
	provider "azurerm" {
	features {}
	}
	resource azurerm_resource_group this {
	name = "rg-terraform"
	location = "East US 2"
	}
	module "storage-account" {
	source = "./storage-account"
	resource_group_name = azurerm_resource_group.this.name
	}
	module "image-container" {
	source = "./storage-account-container"
	storage_account_resource_group_name = azurerm_resource_group.this.name
	storage_account_name = module.storage-account.storage_account_name
	container_name = "images"
	}

view raw main.tf hosted with ❤ by GitHub

Where the problem rears its head is in the source for the image-container

	terraform {
	}
	variable storage_account_resource_group_name {
	type = string
	}
	variable storage_account_name {
	type = string
	}
	variable container_name {
	type = string
	}
	data azurerm_storage_account sa {
	name = var.storage_account_name
	resource_group_name = var.storage_account_resource_group_name
	}
	resource azurerm_storage_container container {
	name = var.container_name
	storage_account_name = data.azurerm_storage_account.sa.name
	}

view raw main.tf hosted with ❤ by GitHub

The key here is the data source reference to the azurerm_storage_account. Here, DESPITE the reference to the storage-account module used in the root, Terraform will attempt to resolve the data source within the image-container before ANYTHING, which results in this error:

As you can see, Terraform will NOT wait for the storage-account module to complete before trying to resolve the data source within the container module.

What is the solution?

Frankly, I am not sure I would classify this as a bug so much as “by design” but it still annoying. The way to get around it is to not have ANY data source in your modules that reference components created as part of your root scripts. So for example, our test script would look like this:

	terraform {
	}
	variable storage_account_resource_group_name {
	type = string
	}
	variable storage_account_name {
	type = string
	}
	variable container_name {
	type = string
	}
	resource azurerm_storage_container container {
	name = var.container_name
	storage_account_name = var.storage_account_name
	}

view raw main.tf hosted with ❤ by GitHub

Notice how I have removed the data source and simply used the variable for the storage account name directly? This is all you have to do. This is, I admit, a simple script but, it is something you will want to be thinking about. Where I ran into this problem was defining custom domains and mTLS certificates for API Management,

So that is it, that is what I found. Maybe it is not new and it was something obvious, though I venture otherwise due to the lack of this mention in the Terraform documentation. It might be something HashiCorp considers “works as designed” but I still found it annoying. So if this helps you, let me know in the comments.

Cheers.

One of the things I have been looking at it recent weeks is Event Streaming, or basically democratizing data within a system so that it can be freely accessed by any number of microservices. It is a wonderful pattern and Azure Functions are an ideal platform for implementation. However, over the course of this process I came to realize that while Azure Functions has a bevy of bindings available to it, one that is very clearly missing is for Redis. So I set about building one.

I am pleased to say I have released version 1.1.0 on nuget.org – https://www.nuget.org/packages/Farrellsoft.Azure.Functions.Extensions.Redis/

Under the hood there are two supporting libraries:

• NewtonSoft Json
• Stack Exchange Redis

Reading Values

	public static async Task<IActionResult> Run(
	[Redis(key: "value", Connection = "RedisConnectionString")] string value)
	{
	}

view raw simple-read.cs hosted with ❤ by GitHub

Here, the binding is reading a string value from Redis (connected using the RedisConnectionString app setting value) with the key of value. The binding is, right now, limited to only reading data from either a Redis string or a Redis list. However, it can support reading C# classes – which are stored as JSON strings and deserializable using Newtonsoft’s Json.NET. For example:

	public static async Task<IActionResult> Run(
	[Redis(key: "object", Connection = "RedisConnectionString")] Person person)
	{
	}

view raw object-read.cs hosted with ❤ by GitHub

When we get into reading lists, the internal type can either be string or a normal C# class. For example:

	public static async Task<IActionResult> Run(
	[Redis(key: "objects", Connection = "RedisConnectionString")] List<Person> people)
	{
	}

view raw objects-read.cs hosted with ❤ by GitHub

As when reading a single object our of the cache, the underlying value is stored as a JSON string and will be deserialized using Newtonsoft Json.NET.

Writing Values

This is currently the newest use case I have added to the binding, and it, like reading, only supports the string and basic C# types saved using either ICollector<T> or IAsyncCollector<T>, currently out parameter is NOT supported, I plan to add it in the future.

	public async Task<IActionResult> Run(
	[Redis(key: "value", valueType: RedisValueType.Single, Connection = "RedisConnectionString")] ICollector<string> values)
	{
	values.Add("testvalue")
	}

view raw string-write.cs hosted with ❤ by GitHub

When doing an output (write) case, you must specify the underlying type the value is stored as, right now either Single or Collection. In the above example, the use of Single will invoke JSON serialization logic and store the value given using StringSet. If the given key already has a value, the new value sent through the collector will overwrite the existing value.

When using Collection the underlying code will use List functions against the Redis class, with two potential execution paths. For example, the following code will append JSON strings for objects given:

	public async Task<IActionResult> Run(
	[Redis(key: "values", valueType: RedisValueType.Collection, Connection = "RedisConnectionString")] ICollector<Person> values)
	{
	values.Add(new Person { Name = "Name 1" });
	values.Add(new Person { Name = "Name 2" });
	}

view raw objects-write.cs hosted with ❤ by GitHub

Its important to understand that, the above code will keep adding values to the end of the Redis list. If you want to update values in a list, you need to have your C# object implement the IRedisListItem interface, which will force an Id property. Note this approach is NOT available for strings.

	public class Person : IRedisListItem
	{
	public string Name { get; set; }
	public string Id { get; set; }
	}

view raw person.cs hosted with ❤ by GitHub

The binding will key off this Id value when it receives a value for the specific Redis key. The one drawback in the current implementation is the entire list has to be pulled, so if you are adding a lot of values for a C# class you will notice performance degradation. Here is an example of this approach being used:

	public async Task<IActionResult> Run(
	[Redis(key: "values", valueType: RedisValueType.Collection, Connection = "RedisConnectionString")] ICollector<Person> values)
	{
	values.Add(new Person { Id = "1", Name = "Name 1" });
	values.Add(new Person { Id = "1", Name = "Name 2" });
	}

view raw objects-update.cs hosted with ❤ by GitHub

In the example above, because the Id value is found in the list, the final value for Name will be Name 2.

The source code for the binding is located at https://github.com/farrellsoft/azure-functions-redis-binding and I am open to taking feedback and I look forward to people having a positive usage experience.

Governance is a very important component of a strong security and operational posture. Using Azure Policy, organizations can define rules around how the various resources in Azure may be used. These cover a wide range of granularity from the general “resources may only be deployed to these regions” to “VMs must be of certain SKUs and certain resource tiers are prohibited” to finally, “network security groups must contain a rule allow access over port 22”. Policies are essential to giving operators and security professionals peace of mind that rules and standards are being enforced.

As organizations have moved to adopt Cloud Native technologies like Kubernetes, the governance question continues to come up as a point of concern. Kubernetes resources are, after all, outside the true scope of the Cloud Provider, Azure included. Thus, in many cases, teams rely on each other to avoid any pitfalls, in lieu of proper governance.

Open Policy Agent (https://openpolicyagent.org) is a Graduated CNCF (https://cncf.io) project aimed at providing an agnostic platform for enforcing policy rules across a variety platform, one of which is Kubernetes (a fully intro on this topic is not covered here). Open Policy Agent hooks into the Kubernetes Admission Controller and works to prevent the creation of resources that violate defined rules. To aid in applying this at scale, Microsoft created the Azure Policy for Kubernetes (https://docs.microsoft.com/en-us/azure/governance/policy/concepts/policy-for-kubernetes) feature to allow the provisioning of OPA policies (written in Rego) to be created in an assigned AKS or Arc connected cluster.

The feature is enabled via an add-on to the target AKS cluster. Azure Policy authors have already written a large amount of these policies that you can use for free with an Azure subscription. Support for custom OPA policies is, at this time, still in Preview mode but, it is stable enough that walking through it should prove beneficial.

Enable the Add-on

Support for OPA type policies in AKS is done through an add-on, which must be enabled to support proliferation of the policies. Instructions for enabling this add-on can be done using the following command:

az aks enable-addons --addons azure-policy --resource-group $RG_NAME --name $CLUSTER_NAME

As with ANY addon, you should run a list first to see if its already installed. Be prepared for the enablement process to take a non-trivial amount of time.

Create your Rego Policy

Whether you use OPA in the traditional sense or with Azure Policy you start with creating a ConstraintTemplate. This template is what enables the creation of the custom kind (the name of a specific resource type in Kubernetes) that will enable low level assignment of your policy. Below is a simple ConstraintTemplate which restricts a certain for a target namespace:

	apiVersion: templates.gatekeeper.sh/v1beta1
	kind: ConstraintTemplate
	metadata:
	name: k8srestricttype
	spec:
	crd:
	spec:
	names:
	kind: K8sRestrictType
	validation:
	openAPIV3Schema:
	properties:
	namespace:
	type: string
	targets:
	– target: admission.k8s.gatekeeper.sh
	rego: |
	package k8srestrictype
	violation[{ "msg": msg }] {
	object := input.review.object
	object.metadata.namespace == input.parameters.namespace
	msg := sprintf("%v is not allowed for creation in namespace %v", [input.review.kind.kind, input.parameters.namespace])
	}

view raw template.yaml hosted with ❤ by GitHub

This code is not something I would use in a production setting, but it gets the point across without being overly complicated. The policy registers a violation if within a namespace as indicated by an input parameter namespace, if creation of a certain type of resource is attempted.

VERY IMPORTANT!! Note the use of the openAPIV3Schema block to indicate the supported parameters and their type. Including this is vital, otherwise the addon will not generate the relevant constraint with the parameter value provided.

Once you have created the ConstraintTemplate you should store it a place which is accessible, I recommend, for ease of use, Azure Blob Storage with a public container.

Create the Azure Policy

The Azure Policy team has a wonderful extension for VSCode that can aid in generating a strong starting point for a new Azure Policy: https://docs.microsoft.com/en-us/azure/governance/policy/how-to/extension-for-vscode. The below is the Policy JSON I used for the above policy:

	{
	"properties": {
	"policyType": "Custom",
	"mode": "Microsoft.Kubernetes.Data",
	"policyRule": {
	"if": {
	"field": "type",
	"in": [
	"Microsoft.ContainerService/managedClusters"
	]
	},
	"then": {
	"effect": "[parameters('effect')]",
	"details": {
	"templateInfo": {
	"sourceType": "PublicURL",
	"url": "https://stopapolicyjx01.blob.core.windows.net/templates/template.yaml"
	},
	"apiGroups": [
	""
	],
	"kinds": [
	"[parameters('kind')]"
	],
	"namespaces": "[parameters('namespaces')]",
	"excludedNamespaces": "[parameters('excludedNamespaces')]",
	"labelSelector": "[parameters('labelSelector')]",
	"values": {
	"namespace": "[parameters('namespace')]"
	}
	}
	}
	},
	"parameters": {
	"kind": {
	"type": "String",
	"metadata": {
	"displayName": "The Kind of Resource the policy is restricting",
	"description": "This is a Restriction"
	},
	"allowedValues": [
	"Pod",
	"Deployment",
	"Service"
	]
	},
	"namespace": {
	"type": "String",
	"metadata": {
	"displayName": "The namespace the restriction will be applied to",
	"description": ""
	}
	},
	"effect": {
	"type": "String",
	"metadata": {
	"displayName": "Effect",
	"description": "'audit' allows a non-compliant resource to be created or updated, but flags it as non-compliant. 'deny' blocks the non-compliant resource creation or update. 'disabled' turns off the policy."
	},
	"allowedValues": [
	"audit",
	"deny",
	"disabled"
	],
	"defaultValue": "audit"
	},
	"excludedNamespaces": {
	"type": "Array",
	"metadata": {
	"displayName": "Namespace exclusions",
	"description": "List of Kubernetes namespaces to exclude from policy evaluation."
	},
	"defaultValue": [
	"kube-system",
	"gatekeeper-system",
	"azure-arc"
	]
	},
	"namespaces": {
	"type": "Array",
	"metadata": {
	"displayName": "Namespace inclusions",
	"description": "List of Kubernetes namespaces to only include in policy evaluation. An empty list means the policy is applied to all resources in all namespaces."
	},
	"defaultValue": []
	},
	"labelSelector": {
	"type": "Object",
	"metadata": {
	"displayName": "Kubernetes label selector",
	"description": "Label query to select Kubernetes resources for policy evaluation. An empty label selector matches all Kubernetes resources."
	},
	"defaultValue": {},
	"schema": {
	"description": "A label selector is a label query over a set of resources. The result of matchLabels and matchExpressions are ANDed. An empty label selector matches all resources.",
	"type": "object",
	"properties": {
	"matchLabels": {
	"description": "matchLabels is a map of {key,value} pairs.",
	"type": "object",
	"additionalProperties": {
	"type": "string"
	},
	"minProperties": 1
	},
	"matchExpressions": {
	"description": "matchExpressions is a list of values, a key, and an operator.",
	"type": "array",
	"items": {
	"type": "object",
	"properties": {
	"key": {
	"description": "key is the label key that the selector applies to.",
	"type": "string"
	},
	"operator": {
	"description": "operator represents a key's relationship to a set of values.",
	"type": "string",
	"enum": [
	"In",
	"NotIn",
	"Exists",
	"DoesNotExist"
	]
	},
	"values": {
	"description": "values is an array of string values. If the operator is In or NotIn, the values array must be non-empty. If the operator is Exists or DoesNotExist, the values array must be empty.",
	"type": "array",
	"items": {
	"type": "string"
	}
	}
	},
	"required": [
	"key",
	"operator"
	],
	"additionalProperties": false
	},
	"minItems": 1
	}
	},
	"additionalProperties": false
	}
	}
	}
	}
	}

view raw policy.json hosted with ❤ by GitHub

Now, that is a rather long definition so, lets hit on the key points, focusing on the then block:

• OPA supports many standard parameters, and these are listed in the policy JSON under parameters. Important to understand is that OPA ALWAYS expects these, so we do not need to do anything extra. You may also omit them, and the system does not seem to care.

• Custom parameters should be placed under the values key. Parameters specified here MUST have a corresponding openAPIV3Spec definition.

• kind and apiGroups relate directly to concepts in Kubernetes and help declare for what resources and actions against those resources the rule applies.

• Note the templateInfo key value under then. This indicates where Azure Policy can find the OPA constraint template – that which we created earlier. There are a few different sourceType values available. For this example, I am referencing a URL in the storage account where the template was uploaded

• Note the mode value (Microsoft.Kubernetes.Data). This value must be used so that the Azure Policy runners know this policy contains an OPA

In the case of our example, we are piggybacking off whatever kind the policy applies to and then indicating a specific namespace to make this restriction. We could achieve the same with the namespaces standard parameter but, in this case, I am using a singular namespace property to demonstrate passing properties.

Once this is created, create the Policy definition in Azure Policy using either the Portal or the Azure CLI, instructions here: https://docs.microsoft.com/en-us/azure/governance/policy/tutorials/create-custom-policy-definition#create-a-resource-in-the-portal

I also recommend giving the new definition a very clear name indicating it is custom, I tend to recommend this regardless of whether or not it’s for Kubernetes or uses OPA, just helps to more easily find them in the portal or when searching with the CLI.

Make the Assignment

As with any policy in Azure Policy, you must assign it to a specific scope for it to take effect. Policy assignments in Azure are hierarchical in nature thus, making the assignment at a higher level affects the lower levels. In MCS, we typically recommend assigning policies to Management Groups in order to create easier maintenance – but you may assign all the way down to Resource Groups.

More information on making assignments: https://learn.microsoft.com/en-us/azure/governance/policy/assign-policy-portal

Once the assignment is made you will have to wait for the AKS addon to pick it up and create the new type and constraint definition on your behalf – my experience has had it take around 15 minutes. At the time of this writing, there is no way to speed it up – even using a trigger-scan command from the AZ CLI does not work.

One critical detail while doing the assignment, ensure that the value provided to the effect parameter is in lower case and either audit (which will map to dryrun in OPA) or deny. In some of the built-in templates Audit and Deny are offered as options. In my experience, the addon gets confused by this.

Validate the assignment

After a period of time run a kubectl get <your CRD type name> and it will eventually return a generated resource of that type representing the policy assignment. Once you see the result there, you can attempt to create an invalid resource in your cluster to validate enforcement.

Something to keep in mind, is the CRD Constraints will NOT report on deny actions, only dryrun (at this time warn is not supported, nor do I see much sense in the team supporting it) enforcements get reported on – this makes sense since with deny in place, invalid resources cannot enter the system.

I also recommend starting with dryrun if your governance process is new, this will give teams time to make change per the policies. Starting with deny can cause work disruptions and less then chances of success.

Debugging the Assignment

One thing I found helpful is to use -o yaml to see what the generated CRD and Constraint look like in the cluster. I used this when I was working with the Engineering team to determine why parameters were not being mapped.

I still recommend building your Rego using the Rego Playground (https://play.openpolicyagent.org/)

Closing

I am deeply intrigued by the use OPA in Kubernetes to enforce governance, as I believe strongly in the promise of governance and its criticality in the DevSecOps movement. I also like what support in Azure Policy means for larger organizations looking to adopt OPA at a wider scale. Combine it with what Azure Arc brings to the table and suddenly any organization in any cloud has the ability to create a single pane of glass to monitor the governance status of their cluster regardless of platform or provider.

Please let me know if you have any questions in the comments.