Introduction to Amazon Code Pipeline with Java part 12: the job agent entry point in code

May 27, 2016 Leave a comment

Introduction

In the previous post we started looking into the Code Pipeline job agent application. The job agent can be any application that is capable of executing long-running threads that periodically communicates with a Code Pipeline endpoint through the AWS SDK. The model application we’re going to look at is a Java Maven web app. It can be deployed directly on a Tomcat server or in AWS Elastic Beanstalk – which is actually a wrapper around an EC2 server with Tomcat installed.

In this post we’ll start looking into some actual Java code within the job agent. Note that I won’t provide every single detail in this series as most of the code is related to our business and is therefore irrelevant to the current discussion. You should also have at least some experience with Maven and things like the web.xml file. I will also omit infrastructure related code such as reading the properties or logging with log4j as much as possible. I’ll only show that type of code where it would otherwise be difficult to follow where a certain variable is coming from. We’ll actually see some of that in this post. This is in order to minimise the “noise”. You are welcome to ask specific questions in the comments section.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 11: starting with the job agent

May 22, 2016 Leave a comment

Introduction

In the previous post we looked a bit at the web pages that must be up and running for the third party action to work with code pipeline. There are 3 types of URL that must be set up. The external configuration URL defines a web page where the CP user can provide the necessary inputs for the job execution. The execution URL is where the CP user can follow the job execution, i.e. where you can show tables, graphs etc. that show the job progress. Lastly we have the entity URL which is a web page to show some resources or job summary page to the user.

In this post we’ll start looking into the job agent itself. It’s probably the largest component you’ll need to build for the CP integration.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 10: the client web pages

May 21, 2016 1 Comment

Introduction

In the previous post we looked at an important component that comes up during a job execution in CodePipeline, namely the continuation token. The continuation token can be supplied in the response to CP when a new job is processed. The token will then appear the next time the job agent gets a signal about a new job. That way the job agent will know that it’s a job that has already been initiated. The job is in effect divided up into multiple small partial jobs coming from CP. The partial jobs will keep coming until the job agent has indicated a 100% completion rate.

We’re almost at the point where we can start looking at the structure of the job agent itself. Before we do that I’d like to present you what type of web pages you’ll need to build for a third party action. We’ve already seen some pictures but I’d like to provide some further details.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 9: the job agent continuation token

May 15, 2016 Leave a comment

Introduction

In the previous post we went through the details behind the communication between CodePipeline and the third part action. The job agent is responsible for for this communication on the side of the third party action. It must monitor a CodePipeline endpoint for new jobs and then acknowledge the job in some way, e.g. by sending a success or failure message. We also mentioned the role of AWS S3 which will store the artifact that enters the pipeline.

We’re at the point where the job agent will act upon the job details received from CP and start the actual work. There’s, however, at least one remaining implementation detail that is important to be aware of when designing this communication process. It’s called the continuation token and this is what we’ll look at in this post.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 8: the job agent communication process

May 8, 2016 Leave a comment

Introduction

In the previous post we went through the details behind the communication between your web site and CodePipeline when a CP user selects your third party action. The third party action developer will be responsible to set up and maintain an external configuration page where CP users can complete their action configuration. CP and the external page communicate with each other using HTTPS POST calls and URL-encoded JSON objects. CP sends a number of properties to the configuration page in the JSON object. These include the client ID and client token that you’ll need to save in your data store and map them to a user in your system. The configuration page redirects the user to the CP GUI with the list of key-value pairs that contain the configuration values for the test runner. These values will be available to the job agent when the job runs.

In this post we’ll look into the details of the communication between CP and the job agent. This post will go through the process at an overall level. The upcoming code examples will show a lot more details.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 7: the third party action user signup process

May 7, 2016 Leave a comment

Introduction

In the previous post we went through an overview of third party actions in Amazon CodePipeline. They are similar to custom plugins in other CI tools like TeamCity and Jenkins. However, their structure, architecture and deployment are very different. The third party action developer must be aware of the job agent and how it communicates with an Amazon CodePipeline endpoint to pull new jobs and process them in some way. The job agent is the connecting tissue between CodePipeline and your backend services. You can deploy multiple job agents for load balancing.

In this post we’ll continue looking at the communication flow between the third party action and CodePipeline.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 6: third party action overview

May 1, 2016 Leave a comment

Introduction

In the previous post we looked at the most important keywords related to AWS CodePipeline. A pipeline is a workflow of stages that together describe the delivery process for a piece of software. This piece of software is called an artifact and goes through various revisions as it is passed from one stage to another. Each stage can consist of one or more actions. An action is a task performed on an artifact. If all actions in a stage have completed without a failure then the stage transitions into the next stage in the pipeline. It’s possible to disable the transition e.g. if it’s necessary to enforce a manual activation.

In this post we’ll start looking at a global overview for third party action development.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 5: architecture key terms

April 30, 2016 Leave a comment

Introduction

In the previous post we looked at some key differences between TeamCity/Jenkins and AWS CodePipeline. There are a number of aspects where these CI tools differ such as the installation, the deployment and custom build runner development.

In this post we’ll start looking at the CP architecture. We’ll concentrate on the key terms to begin with.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 4: comparison with TeamCity and Jenkins

April 10, 2016 Leave a comment

Introduction

In the previous post we saw how to add a custom job runner of type Test to an existing pipeline. These runners cannot be added to a pipeline during the setup process, they can be applied when updating one instead. We went through an example with the Apica Loadtest job runner and saw how to specify the necessary inputs for the job that the runner executes.

In this post we’ll discuss some of the key differences between TeamCity/Jenkins and CodePipeline (CP). TeamCity (TC) and Jenkins are quite similar so I will treat them as a group for this discussion.

Read more of this post

Filed under Amazon, Java Tagged with , , , , ,

Introduction to Amazon Code Pipeline with Java part 3: adding custom job runners

April 9, 2016 Leave a comment

Introduction

In the previous post we went through the steps to set up a brand new pipeline in AWS Code Pipeline. If you are entirely new to the AWS tools then you might find it overwhelming at first as you have to learn about other AWS tools as well such as S3 and Elastic Beanstalk. However, you might not need all of it since a pipeline can be quite little and consist of only 2-3 steps. The pipeline will start executing as soon as it has been set up. The arrows connecting the steps enable us to cut the execution of the pipeline at a specific step.

In this post we’ll see how to update a pipeline. Updating the pipeline also makes it possible to add custom build runners to the pipeline. I’ll use the load test job runner I referred to in the first part. Note that you’ll need an Apica Loadtest account to fully follow along these steps. The main point is to demonstrate the process of adding a new custom job runner to an existing pipeline. The implementation details are not important at this moment.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

← Older posts

Newer posts →

Elliot Balynn's Blog

A directory of wonderful thoughts

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Once Upon a Camayoc

ARCHIVED: Bite-size insight on Cyber Security for the not too technical.