One of the most interesting scenarios the Cloud gives rise to is better support for building event driven backends. Such backends are necessary as the complexity and scale of applications grow and developers want to decouple service references. Additionally, cloud providers are raising events from their own services and allow developers to hook into these events, this is most noticeable on AWS.

While this tend is not new on Azure (EventHubs have existed for a while) support for it compared to AWS has lagged. But by introducing EventGrid, Microsoft seems to be positioning themselves to support the same sort of event driven programming as allowed with in Amazon with the added benefit of additional flexibility supported, mainly, by EventGrid.

What is Event Grid?

In the most simplest sense, EventGrid is a router. It allows events to be posted to a topic (similar to a Pub/Sub) and can support multiple subscriptions, each with differing configurations, including what eventType to support and what matching subjects will be sent to subscribers.

In effect, EventGrid allow you to define how you want the various services you are using to communicate. In that way, it can potentially provide a greater level of flexibility, I feel, than AWS, whose eventing generally lacks a filtering mechanism.

Creating an Event Topic

So, the first thing to understand is that the concept of topic will be appear twice. The first is the name of the EventGrid itself, which is a topic itself.

Choose to Add a Resource and search for event grid. Select Event Grid Topic from Microsoft from the returned results.

By default, the name of the topic will match the given name of the Event Grid so, pick wisely if this is for something where the name will need to have meaning.

Once this completes we will need to create a subscription.

Before you create a subscription

When you create a subscription you will be asked to give an endpoint type. This is where matched events to the topics will be sent. If you use a standard Azure service (ie Event Hubs) you can select and go, however, if you use an Azure Function you will need to do something special to ensure the subscription can be created.

If you fail to follow this, you will receive (at the time of this writing) an undefined error in the Portal. The only way to get more information is to use the Azure CLI to create the subscription.

The error is in reference to a validation event that is sent to the perscribed endpoint to make sure it is valid. There is a good guide on doing this here. Essentially, we have to look for a certain event type and return the provided validation code as proof that we own the event endpoint.

I do not believe you have to do this with other Azure service specific endpoints, but you certainly have to do it if your endpoint is going to be of type WebHook.

The destination for this Event Grid subscription is the Azure Function Url.

Regardless of your endpoint type…

You will need to setup the various filtering mechanisms that indicate which events, when received, are passed through your subscription to the endpoint.

Most of these are fairly self explanatory except for the Event Schema field which refers to the structure of the data coming through the topic. It is necessary for the subscription to understand the arrangement so it can be analyzed.

The default is the pre-defined EventGridEvent which looks like this:

[
    {
        "id": "string",
        "eventType": "string",
        "subject": "string",
        "eventTime": "string",
        "data": "object",
        "dataVersion": "string"
    }
]

The key here is the data object which will contain your payload.

I am not sure of the other schemas as I have not explored them, mainly because the .NET EventGrid libraries have good native support for the EventGridEvent schema.

Let’s Send an Event

There are quite a few ways to send events to EventGrids. For any attempt you are going to need your Topic Endpoint which you can find on the Overview section for your Event Grid. This is the Url you will submit new events to for propagation within Azure, it is also the value you will need to get from any Azure service that you want to submit events to the Event Grid.

Disclaimer: At the time of this writing, there was some weirdness in the portal where you needed to use the CLI to route events from Azure services (like Storage) to an EventGrid.

Receiving Events from an Azure Resource
https://docs.microsoft.com/en-us/azure/storage/blobs/storage-blob-event-quickstart?toc=%2fazure%2fevent-grid%2ftoc.json

Within this tutorial you are shown how to use the Azure CLI (via the command shell or through the portal) to create the appropriate subscriptions. Do note that, once created, you will not see this listed in the portal

Send a Custom Event
This is probably the case you came here to read about. Principally, sending custom events involves posting to the Event Grid Topic Url with a schema for the body that matches a subscription.

Doing this is easiest if you use the Microsoft.Azure.EventGrid NuGet library which will expose the the PublishEventAsync method, here is an example:

You can also send to this endpoint manually using the HttpClient library, but I recommend using the NuGet package. To be a better idea of what is in there, here is the source page: https://github.com/Azure/azure-sdk-for-net/tree/psSdkJson6/src/SDKs/EventGrid/DataPlane

Keep in mind that this will send an event using the EventGridEvent schema, which is what I recommend.

How do I test that things are working…

Once you have things in place, I recommend selecting the Azure Function and clicking the Run button. This will enter into a mode where you can see a streaming log at the bottom. This is the stream for the function and is not relegated to just what is passed via testing.

Once you have this up, send an event to your Event Grid Topic and wait for a log message to appear which will tell you things are connected correctly.

What are my next steps?

This gets into a bigger overall point but, in general, I do NOT recommend using the Event Grid Topic Url for Production because it couples you to a Url that cannot be versioned and breaks the identity of your API.

Instead, as should be the case when you use Azure Functions you need to set things up behind an Azure API Management service. This lets you customize what the Url looks like, version, and maintain better control over its usage.

Additionally, in an event driven system you should view events as a way to maximize throughput so users should be posting to this endpoint and the event should fan out.

To put our Event Grid Topic endpoint behind an Azure API Management Route, do this:

1. Create an API Management service instance (this will take time) and wait for it to activate
2. Create a Blank API
   • Tip: When you create the API you will be asked to select a Product. Without getting too much into what this is, select Unlimited to make things easy on yourself
3. Access your API definition via the APIs link navigation bar
4. Add a POST route –  you can provide whatever values you want to the required fields
5. Copy the aeg-sas-key from the Access Keys section under your Event Grid Topic and make this a required header for your POST route
6. Once the route is created selected the Edit icon for the Backend
7. Select Http(s) endpoint and paste the Event Grid Topic Url, make sure to NOT include the /events at the end of this URL
   
8. Hit Save
9. Click on your API Operation (POST route) in the selection method to the left
   
10. Select Inbound Processing
11. Change the Backend line such that it reads with the POST method and the resource is /events
12. Hit Save and you can click Test to check this. Building the payload is a bit tedious. Use the source here to understand how the JSON should look for the EventGridEvent that I assume you are sending.

Ok so let’s recap.

The first bit of this is to create an Azure API Management service instance and then create a POST path. Azure API Management is a huge topic that I have covered part of in the past. It is an essential tool when building a MicroService architecture in Azure as it allows for unification of different service types (serverless, containerized, traditional, etc) into what appears to be a single unified API.

All requests to Azure Event Grid require the aeg-sas-key so, we make sure that Azure API Management does not allow any requests to the proxy to come through unless that header is at least present. Event Grid will determine its validity.

Once the general route is created we need to tell it where to forward the request it receives. This is simply the Event Grid Topic endpoint that we can get from the Event Grid Topic Overview page in the Azure Portal. However, we do NOT want to take the entire Url as we will need to use Url Rewriting instead of Url Forwarding. So, I recommend taking everything but the trailing /events which we will use in the next section. Be sure to also check Override to allow for text entry into the Url field.

Ok, now lets complete the circle. You need to click Inbound Processing and change the default processing from URL Forwarding to URL Rewrite. There is a bit of a quirk here where you cannot leave the text field for your backend blank. This is where you will want to drop the /events that we omitted from our backend URL.

You will want to use the Test feature to test this. I provided the source code for the Azure Event Grid EventGridEvent class so you can see how the JSON needs to be in test.

Conclusion

Event Driven Programming is a very important and vital concept in enabling high throughput endpoints. The design is inherently serverless and enables developers to tap into more of Azure’s potential and easily declare n number of things that happen when an endpoint is called.

API Management is a vital tool in creating a unified and consistent look interface for external developers to use. There are many tools it comes with including Header enforcement and event event body validation and transformation and it can integrated with an authentication mechanism. If you are going to create a Microservice type API or if you intend to break up a monolith, API Management is a create way to get free versioning and abstraction.

The biggest thing to remember is that Event Grid is geared towards enabling Event Driven scenarios, it is not designed for processing a high number of events, that is the realm of Event Hubs. If you were going to do real time processing, you would feed your data pipeline into Event Hubs and route those events to Event Grid (potentially).

More information here