peter siemen, dipl.-inf.

Attention: This is a meta-blog post about this blog's development and deployment lifecycle.

2 motivating definitions that explain my motivation to write this blog and host it on AWS.

Cloud Native Development: Super agile software development through writing Lambda Functions that feel like UNIX commands, single purpose containers & infrastructure-as-code cloud services and not loosing track in the presumable consequence of deployment hell by having a scalable, maintainable, cost-optimized and secure CI/CD pipeline in place. ¹

JAM Stack: The promise of joy in software development (again) through sticking to code JavaScript, API's & Markup. ²

?!

Because there is still no definite answer to the question if the process of developing complex but yet scalable infrastructure inevitably has to slow down when projects, teams, problems have grown to a certain size.

The definition of Cloud Native Development above is just a frame. What it really means to me is that with all the power and reach of the public cloud the min-max problem of software development can be taken to new extremes.

• Minimize the lines of code you write
• Maximize your lever in business impact, scalability, maintainability & time-to-market

Sounds great. But how can I as a business owner empower my Software Engineering team to embrace this new power of the public cloud and rest assured that the business will not come to a halt because nobody knows how to deploy all of this fragmented (but yet so very powerful ;)) infrastructure?

CI/CD

When a large enterprise stack is deconstructed (or build up from the ground) as single-purpose, UNIX-commands-like functions it is easy to see that there will be a lot of them. So whatever your CI/CD solution looks like it needs to be able to handle possibly hundreds of build simultaneously, support deployments of applications of any color and it needs to securely fit into your high-secret enterprise cloud.

Alright, the CI/CD infrastructure needs to be highly available, scaling horizontally, and for sure you do not want to pay money for it when nobody is piping her code through it.

So who should manage this crucial piece of infrastructure in the world of cloud native development? Hopefully nobody. Because if so, you would probably need to hire an whole DevOps company solely for keeping the pipeline running.

So get over it, leave Jenkins behind, and start using a managed service for your CI/CD pipeline. If you are running your workloads on AWS there is really no reason not to use AWS CodePipeline for CI/CD!

This blog???

I have to admit that this blog is really, really far away from a high-secret, super complex enterprise stack but I think it can used to test-drive a CI/CD pipeline with a bit of imagination.

So what are we looking at?

• gatsby for the M in JAM
• API Gateway with AWS Lambda functions attached for the contact form - which makes an A for JAM
• The AWS Lambda functions are written in JavaScript. Gatsby == React. The J for JAM
• AWS Multi-Account Structure
  • Master Account (master account for AWS Organizations, used as an Identity Account to log into the console and switch into the other accounts - a dedicated Identity Account might be added at some point)
  • Shared Services (AWS CodeCommit, AWS CodePipeline, Route53, S3 Buckets for build artifacts)
  • Development Account (S3 Static Website Hosting, API Gateway, AWS Lambda, AWS SES)
  • Production Account (S3 Static Website Hosting, API Gateway, AWS Lambda, AWS SES)

Check out my ready to use terraform/terragrunt codebase which I continuously update as I add more features to this blog for all the remaining details...

CI/CD (cont.)

The most interesting part of this blog (at least for me - for now) is the Multi-Account/Multi-Region CodePipeline.

The CodePipeline which lives in the Shared Services account deploys upon every push to the develop branch of the gatsby homepage project or the Lambda functions's develop branches into the Development account. Pushes into the master branches trigger builds and deployments into the Production account. The cross account deployments are done by the CodePipeline assuming appropriate roles in the Development and the Production account during the deploy stage of the pipeline.

The cross-region functionality is another absolute must for a "real world" CI/CD pipeline on AWS. AWS Services are mostly always launched in a region by region fashion. So if you want to be ahead of the pack and use AWS services that are freshly released you will have get used to multi-region deployment pipelines. In my case AWS SES for receiving e-mails is not available in eu-central-1 yet so I have to use it in eu-west-1.

Noteworthy in regards to terraforming a multi-region CodePipeline is that build/deploy artifacts need to exist in the same region as the pipeline stage you are running.

So let's say you have Lambda Function that should be deployed to region eu-west-1 but the CodePipeline is managed centrally with all other CodePipelines of your organization in eu-central-1. In order to be able to configure a CodePipeline stage into another region you need to provision artifact stores (encrypted S3 buckets) in every region and make them read and/or writable from every region.

In order to share an encrypted artifact store, you need to create a Customer managed key (CMK) in AWS KMS for every regional bucket and grant access to the IAM role that runs the Pipeline - for the cross-account deployment you need to grant access to the IAM role from the sub account that is assumed in the deploy stage access as well. In the CodePipeline config you can then explicitly select an AWS_REGION for the different stages (as long as a resp. regional bucket exists). CodePipeline will take care of replicating the artifacts across your regional buckets under the hood.

When your interested in more details, check out my terraform/terragrunt codebase and use it to deploy your own projects.

And if you need professional help with AWS, get in touch...