In my blog post about DevOps I argued that a good Software Engineer knows not only about his code, but about how and where it is going to be ran. In the past I have been guilty of having an idea which service I might use to run a particular application (usually Microsoft Azure as I get big discounts as a former MSP) but not having had a full idea of the exact environment said application will execute in.
Recently I have been treating infrastructure as a first class citizen of any project I work on using the process of code-defined infrastructure. Instead of manually provisioning servers, databases, message queues and load balancers — as you might do when using the AWS Console or cPanel on a shared webhost — I create a deployment script and some configuration files. I have actually banned myself from manually creating new instances of anything in the cloud, in order to avoid unwittingly allowing any single instance to become a Pet.
Defining the infrastructure required by any given application in code has a few advantages:
- If someone tears down an environment by mistake we don’t have to try and remember all of the configuration to relaunch it. We have it in code.
- We can easily spin up new environments for any given feature branch rather than being stuck with just “staging” and “production” environments.
- Infrastructure changes show up in version control. You can see when and why you started shelling out for a more expensive EC2 instance.
- At a glance you can see everything an application requires and the associated names. This makes it easier to be sure that when you’re deleting the unhelpfully named “app” environment it won’t affect production.
- Scripting anything you can imagine is a lot easier, such as culling servers with old versions of an application programatically.
Introducing code defined infrastructure on one of our greenfield projects at PepperHQ has already paid dividends and the team has a desire to back-port the infrastructure management scripts and config files I developed to other services.
I developed the system with the idea of a git commit being the true “version” of a given application and that all infrastructure associated with a given version should be totally immutable. In order to deploy a new version of our new greenfield system you don’t upgrade the application on the server, you create a whole new environment with the new version on and then delete the old one — or keep it around for a hot swapping back in the event of an issue.
The infra-scripts also allow you to see what is live identified in a way useful to developers — by commit id and message. Other features include turning any environment into “production” or “staging” by updating their CNAMEs through AWS Route 53. When using the yarn terminate to kill a particular environment checks are ran to ensure you’re not deleting “production” or “staging”. The scripts are developed in TypeScript using the aws-sdk npm package.
Whilst this approach does require some more up-front investment than just manually creating environments as and when you need them, I recommend any developer writing systems people will really be relying on at least investigates this approach as there is a very quick return on investment. Some more out of the box solutions you could look into are Chef and AWS CloudFormation, though I ruled out using these for Pepper based on our internal requirements.
Danny
dannybrown.blog 