Part 4 is here
Source code here
Full Disclaimer
It would appear the script referenced in previous parts was not re-entrant, that is, it contained good information but, over the course of building it, things fell into place rather than being in place. Thus, I noted it failed to setup when I ran it clean. Version 1.1 addresses this and will be referenced here
Now that we our app up and running its time to be critical of things, mainly permissions. As I stated in Part 1 AWS recommends the use of roles to fulfill service level tasks so that credentials are never stored in source control or within applications, thus lessening the blast radius in the event of compromise. However, in saying that we recognize that have a setup where all of the roles we use are open to everything is not good either. Thus first we will focus on splitting and refining AppRole.
Get the Order Right
Cloud Formation makes every attempt to determine the order in which to create resources, when it needs help you can often use the DependsOn attribute. I have personally found this wanting so, the first thing I did was establish the order of my resources so things could be efficiently and correctly created. Here is that order:
- ThumbnailBucket (S3 Bucket)
- ThumbnailBucketPolicy (S3 Bucket Policy)
- ImageTable (DynamoDB Table)
- CreateThumbnailFunctionRole (IAM Role)
- CreateThumbnailFunction (Lambda)
- AnalyzeImageFunctionRole (IAM Role)
- AnalyzeImageFunction (Lambda)
- ImageUploadTopic (SNS Topic)
- ImageUploadTopicPolicy (SNS Topic Policy)
- RawBucket (S3 Bucket)
- RawBucketPolicy (S3 Bucket Policy)
- CreateThumbnailFunctionSNSInvokePermission (Lambda Permission)
- AnalyzeThumbnailFunctionSNSInvokePermission (Lambda Permission)
- ApiGatewayRest (API Gateway)
- BucketApiResource (API Gateway Resource)
- ImagesApiResource (API Gateway Resource)
- BucketItemApiResource (API Gateway Resource)
- BucketItemApiMethodRole (IAM Role)
- BucketItemApiMethod (API Gateway Method)
- ImagesApiMethodRole (IAM Role)
- ImageApiMethod (API Gateway Method)
- ApiGatewayDeployment (API Gateway Deployment)
- DefaultApiGatewayStage (API Gateway Stage)
The items highlighted in BOLD are new moving into this part and mostly represent the result of splitting AppRole. As an example here is CreateThumbnailFunctionRole:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| CreateThumbnailFunctionRole: | |
| Type: AWS::IAM::Role | |
| Properties: | |
| AssumeRolePolicyDocument: | |
| Version: "2012-10-17" | |
| Statement: | |
| – Effect: Allow | |
| Principal: | |
| AWS: "*" | |
| Service: lambda.amazonaws.com | |
| Action: "sts:AssumeRole" | |
| Path: "/" | |
| Policies: | |
| – PolicyName: "tc-createthumbnailfunctionrole" | |
| PolicyDocument: | |
| Version: "2012-10-17" | |
| Statement: | |
| – Effect: "Allow" | |
| Action: "s3:Put*" | |
| Resource: !GetAtt ThumbnailBucket.Arn | |
| – Effect: "Allow" | |
| Action: "s3:Get*" | |
| Resource: !Sub "arn:aws:s3:::${RawBucketName}" | |
| – Effect: "Allow" | |
| Action: "logs:*" | |
| Resource: "arn:aws:logs:*:*:*" | |
| – Effect: "Allow" | |
| Action: "cloudwatch:*" | |
| Resource: "*" | |
| – Effect: "Allow" | |
| Action: "xray:*" | |
| Resource: "*" |
You can see that I kept the open resource for CloudWatch, XRay, and Logs but I restricted the rights to the ThumbnailBucket to Put and Get type actions only. This disallows the role from reading information out of the Thumbnail bucket.
Further supporting this is the AssumeRolePolicy which enforces that ONLY Lambda can assume this role and no other service. This is key given the very tight use case the function and its coupling to this specific bucket.
The other interesting thing here is the user of !Sub to specify the ARN of the S3 Bucket holding raw images. Cloud Formation will always do its best to mitigate circular dependencies but, especially wit this sort of role structure, has a hard time. Thankfully AWS has standard format for resource names. Using that we can specify it here to prevent Cloud Formation from looking at its template for the ARN – this mitigates the Circular Dependency that would happen here otherwise.
Create Bucket Policies
One thing I did learn while working through this was, curiously, if I specified Resource: “*” for within my Roles for the S3 bucket things worked fine but, the moment I went resource specific I got Access Denied.
After much Googling (seriously why do all of the AWS docs never call out resource specific roles) I guessed and was correct that in addition to the Role I needed a Bucket Policy to allow the action as well. Here is the policy for RawBucket which allows API Gateway to Proxy the PUT call to create objects:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| awBucketPolicy: | |
| Type: AWS::S3::BucketPolicy | |
| Properties: | |
| Bucket: !Ref RawBucket | |
| PolicyDocument: | |
| Statement: | |
| – Action: "s3:Put*" | |
| Effect: Allow | |
| Resource: | |
| Fn::Join: | |
| – "" | |
| – | |
| – !GetAtt RawBucket.Arn | |
| – "/*" | |
| Principal: | |
| AWS: "*" | |
| Service: apigateway.amazonaws.com | |
| – Action: "s3:Get*" | |
| Effect: Allow | |
| Resource: | |
| Fn::Join: | |
| – "" | |
| – | |
| – !GetAtt RawBucket.Arn | |
| – "/*" | |
| Principal: | |
| AWS: "*" | |
| Service: lambda.amazonaws.com |
Here we specify that, for RawBucket, we allow Lambda services to read from it (necessary to support the Thumbnail creation process) and we allow API Gateway to write it (necessary to support the PUT operations our API Gateway will proxy. Of particular note here is the /* you see – this is necessary to indicate our policy is on the bucket contents and not the bucket itself.
SNS Endpoint Verification Check
One of the other problems I came across with testing was that SNS, when it creates, will send a pulse to the Lambda functions to ensure they are there. I didnt catch this the first time because of the order in which I created things the first time. Handling this is rather easy as SNS simply sends a s3:TestEvent to the function. Both CreateThumbnail and AnalyzeImage have this code to handle the check:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| var jsonEvent = JObject.Parse(evnt.Records?[0].Sns.Message); | |
| if (jsonEvent.Value<string>("Event") == "s3:TestEvent") | |
| return; |
Conclusion
I hope this series has been informative and worthwhile. It took a lot of care and thought to make and a lot of trial and error. Cloud Formation, and similar tools, take a lot of effort to learn and understand but, ultimately, they can save you a ton of grief. One of the value props for Infrastructure as Code (IaC) is that it enables both quick creation of environments and access to more efficient DevOps processes.
This is best manifested, right now, in JenkinsX approach to spin up entire environments in Kubernetes for testing per PR. This way, we have a way to run a complete battery of tests to ensure the code entering the trunk (and possibly being deployed automatically to production) is valid and tested.
Again here is the completed code:
https://github.com/xximjasonxx/ThumbnailCreator/tree/release/version1.1
Random Blurbs and Things of this Nature 
One thought on “Serverless Proxy Pattern: Part 5”