Introduction to Amazon Code Pipeline with Java part 1: basics of CI/CD

Introduction

Amazon has a relatively new service out called Code Pipeline (CP). It is a Continuous Delivery tool that enables users to run builds, tests and deploys automatically. Its purpose is similar to other CI tools such as TeamCity or Jenkins but there are some fundamental differences in the architecture and customisation options.

The company I work for had the honour to team up with Amazon and be among the first to integrate a custom job processor in CP before it was made public in June 2015. I was very fortunate to take part in this project as the developer who was responsible for writing the job processor. Our tool is a selectable option among the job processor tools of type “Test”:

In this series we’ll take a closer look at Code Pipeline and also how a new job processor can be integrated with it using Java. It’s a large topic so the series will also consist of many posts. At this point I’m not sure yet how many there will be but 15-18 is my initial estimate since I’d like to be as detailed as possible. The AWS CP home page provides a lot of details both about the general architecture and setup. The developer pages provide the API for the CP related classes and functions.

You’ll need at least a test AWS account if you want to try the tool and have a go at building a custom job processor. However, even if you don’t have an account it can be interesting for you to learn about this new technology.

Continuous Delivery

Let’s step back a little and first explain what Continuous Delivery means. There are actually various ideas that are often used to mean the same thing:

• Continuous Delivery
• Continuous Integration
• Continuous Deployment

This thread on StackOverflow has an excellent explanation on what these terms mean. Here’s a summary:

• Continuous Integration (CI) is a strategy for how a developer can integrate code to the mainline continuously. If you work at a company that has a central repository for its sourcecode, like MS TFS or GitHub then you have the potential for continuous integration. The idea is that a team works together on the same project and each team member continuously pulls the changes made by others from the repository and then pushes their own updates to the same repository. New code is thereby continuously integrated in the solution. Wikipedia has also a very similar definition of CI: Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies to a shared mainline several times a day
• Continuous Deployment refers to the practice where the continuously integrated code can be deployed frequently, possibly several times a day. It describes the ability to move the result of the CI development process to the production environment.
• Continuous Delivery has a larger scope and usually means a software production process where CI and CD are 2 substeps. There can be other substeps however, such as automated unit tests, integration tests, builds etc. From Wikipedia: Continuous delivery (CD) is a software engineering approach in which teams produce software in short cycles, ensuring that the software can be reliably released at any time.

Note that the above definitions are not set in stone. 10 developers may give 11 different definitions of these concepts. In practice, however, they are often used interchangeably to mean approximately the following steps:

1. The developer checks in the changes to the code repository, e.g. GitHub
2. A CI/CD tool, such as TeamCity can “see” that new code was pushed to GitHub which in turn triggers one or more actions in TeamCity
3. The actions in TeamCity can be whatever was set up using the available plugins: build the project, run the unit tests, run a custom script, execute a load test etc.
4. TeamCity can even deploy the project to a deployment server
5. TeamCity can also perform additional tests on the deployed application such as GUI tests with Selenium

All of the above can be automated nowadays. There are free and paid tools out there that make it possible, like the ones mentioned above: TeamCity, Jenkins and various others. Amazon have entered this scene with Code Pipeline.

We’ll continue with a quick visual tour of AWS Code Pipeline.

View all posts related to Amazon Web Services and Big Data here.