Serverless Proxy Pattern: Part 4

See Part 3 here

To this point, we have defined what I would say are the “backing” services for our application. We could write frontend apps to send images to our bucket which would trigger the overall process but, in keeping with the theme, I want to provide a “codeless” way (or serverless) to interact with our services.  Enter API Gateway.

API Gateway: The Heart of the Serverless Proxy Pattern

API Gateway (and API Management on Azure) can act as proxies forwarding received calls to backing services to perform operations. This means we can dispense with writing any code to work with these services and instead rely on the API Gateway to handle the underlying calls for us.

To create an API Gateway in AWS we must first create a RestApi type resource. This serves as the container for related resources of type Resource and Method. Here is the YAML:

 

 

 

 

 

AWSTemplateFormatVersion: 2010-09-09
Description: "Creates infrastructure for Thumnbail Creator"
Parameters:
ApiGatewayName:
Type: String
Default: tcapigateway
Description: Enter the name of the API Gateway
Resources:
ApiGatewayRest:
Type: AWS::ApiGateway::RestApi
Properties:
Name: !Ref ApiGatewayName
Description: Api Gateway to enable reading and writing of image data
BinaryMediaTypes:
image/png
EndpointConfiguration:
Types:
REGIONAL
DependsOn:
ImageTable

view raw
gateway.yaml
hosted with ❤ by GitHub

Next, we need to define our resources. If you are familiar with the nomenclature from REST standards you will recognize this term. It is the “thing” that defines what is being acted against. It can often map to database models but, does not necessarily have to.

In our case we will define our resources as such:

 

 

 

 

AWSTemplateFormatVersion: 2010-09-09
Description: "Creates infrastructure for Thumnbail Creator"
Resources:
BucketApiResource:
Type: AWS::ApiGateway::Resource
Properties:
PathPart: !Ref RawBucketName
RestApiId: !Ref ApiGatewayRest
ParentId: !GetAtt ApiGatewayRest.RootResourceId
DependsOn:
ApiGatewayRest
ImagesApiResource:
Type: AWS::ApiGateway::Resource
Properties:
PathPart: "all-images"
RestApiId: !Ref ApiGatewayRest
ParentId: !GetAtt ApiGatewayRest.RootResourceId
DependsOn:
ApiGatewayRest
BucketItemApiResource:
Type: AWS::ApiGateway::Resource
Properties:
PathPart: "{item}"
RestApiId: !Ref ApiGatewayRest
ParentId: !Ref BucketApiResource

view raw
resource.yaml
hosted with ❤ by GitHub

Each level of the API is defined as a resource such that we end up supporting the two following path structures:

  • /all-images
  • /<bucket_name>/<key>

Of particular note is the usage of the bucket name for the root resource for key – this appears to be required or, at least, I cannot find a way around it. Now that our resource structure in place we define the “actions” that will take place when the path is matched:

PUT an Object

First, we will configure out /<bucket_name>/<key> route to allow a POST verb to invoke it and forward that onto S3 using PUT to create (or update) the object in blob storage.

 

 

AWSTemplateFormatVersion: 2010-09-09
Description: "Creates infrastructure for Thumnbail Creator"
Resources:
BucketItemApiMethod:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApiGatewayRest
ResourceId: !Ref BucketItemApiResource
HttpMethod: POST
AuthorizationType: AWS_IAM
OperationName: SaveImage
MethodResponses:
StatusCode: 201
RequestParameters:
method.request.header.Content-Disposition: false
method.request.header.Content-Type: true
method.request.path.item: true
Integration:
Type: AWS
Credentials: !GetAtt AppRole.Arn
IntegrationHttpMethod: PUT
PassthroughBehavior: WHEN_NO_MATCH
RequestParameters:
integration.request.header.Content-Disposition: method.request.header.Content-Disposition
integration.request.header.Content-Type: method.request.header.Content-Type
integration.request.path.key: method.request.path.item
Uri:
Fn::Join:
"/"
– "arn:aws:apigateway:us-east-1:s3:path"
!Ref RawBucketName
"{key}"
IntegrationResponses:
StatusCode: 201

view raw
putobject.yaml
hosted with ❤ by GitHub

There is quite a bit here so lets hit the crucial areas. We define HttpMethod to indicate that the method can only be invoked using POST and that the Authorization will be gleaned from IAM credentials associated with the underlying integration calls (recall our role we defined in Part 1).

Next we define RequestParameters which in this case lets us enforce that item must be provided. Where the real magic happens is the IntegrationHttpMethod and Integration sections:

  • IntegrationHttpMethod: This is the verb that API Gateway will use when invoking whatever service it will call when the resource path matches.
  • Integration: This is the meat of this operation, here we define various mappings for values to pass to the underlying service call, what kind of integration we will (AWS Service in this case) and the Uri we will call (in this case it is the S3 bucket path)

What this ends up facilitating is a PUT using the bucket-name and key against the S3 bucket object path to pass the contents of the request to S3 API which will create the appropriate object – all without writing a line of code.

The final bit of this is the IntegrationResponses where we define that the integration will respond with a 201 (Created) – this lines up with MethodResponses intentionally.

Get All Image Data

The second resource will return the contents of our DynamoDB table ImageDataTable that we created in Part 3. For this one, the integration portion is a bit trickier, have a look:

 

AWSTemplateFormatVersion: 2010-09-09
Description: "Creates infrastructure for Thumnbail Creator"
Resources:
ImagesApiMethod:
Type: AWS::ApiGateway::Method
Properties:
RestApiId: !Ref ApiGatewayRest
ResourceId: !Ref ImagesApiResource
HttpMethod: GET
AuthorizationType: AWS_IAM
MethodResponses:
StatusCode: 200
Integration:
Type: AWS
Credentials: !GetAtt AppRole.Arn
IntegrationHttpMethod: POST
PassthroughBehavior: WHEN_NO_MATCH
Uri: !Sub "arn:aws:apigateway:${AWS::Region}:dynamodb:action/Scan"
RequestTemplates:
application/json: >-
{ "TableName": "ImageDataTable2" }
IntegrationResponses:
StatusCode: 200

view raw
getimages.yaml
hosted with ❤ by GitHub

The key thing to understand here is we indicating we want to call an action against the DynamoDB API – the action in this being Scan (wont go into the differences here between Query and Scan).

Scan expects a JSON block to define various parameters that it will use to perform filtering and selection. In this example, we are dropping the filtering aspect and simply specifying our DynamoDB table name so that all contents of that table are returned.

Also of note here is the IntegrationHttpMethod. Despite the fact that our API Gateway resource is called using GET, we must use POST to call the DynamoDB Action API.

Once this is in place you can get a dump of the table contents from DynamoDB. The one outstanding issue I have with this is the JSON comes back as it exists in DyanmoDB which will not look like most JSON blocks you have likely seen – but it is easily parsable by DyanamoDB libraries. Still, a standing goal of mine is to get this into a more normalized JSON block for easier consumption.

Deploying the API

The final bit here is making the API Gateway accessible via the web. This is done by deploying an API Gateway Deployment and Stage resources. In practice, these are designed to serve as a means enable environmentalization of the API Gateway so cahnges can be promoted, similar to code changes.  Here is what I defined for these resources:

 

 

 

AWSTemplateFormatVersion: 2010-09-09
Description: "Creates infrastructure for Thumnbail Creator"
Resources:
ApiGatewayDeployment:
Type: AWS::ApiGateway::Deployment
Properties:
RestApiId: !Ref ApiGatewayRest
StageName: DefaultDeployment
DependsOn:
ApiGatewayRest
BucketItemApiMethod
DefaultApiGatewayStage:
Type: AWS::ApiGateway::Stage
Properties:
RestApiId: !Ref ApiGatewayRest
TracingEnabled: true
DeploymentId: !Ref ApiGatewayDeployment

view raw
api-deployment.yaml
hosted with ❤ by GitHub

Ending this Part

In this part I went through the complete setup of the API Gateway that we are using to proxy Amazon services: S3 and DynamoDB. This approach gives a lot of the basic functionality we find in applications but without needing to write any code and all definable via Cloud Formation. Now that our application is stood up, our next parts will focus on tweaking things to make them more secure and more efficient.

As always you can reference the complete source here: https://github.com/xximjasonxx/ThumbnailCreator/tree/release/version1.1

Part 5 is here

3 thoughts on “Serverless Proxy Pattern: Part 4

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s