Creating an Amazon Beanstalk wrapper around a Kinesis application in Java


Suppose that you have a Java Amazon Kinesis application which handles messages from the Amazon message queue handler Kinesis. This means that you have a method that starts a Worker object in the library.

If you are developing an application that is meant to process messages from Amazon Kinesis and you don’t know what I mean by a Kinesis app then check out the Amazon documentation on the Kinesis Client Library (KCL) here.

The starting point for this post is that you have a KCL application and want to host it somewhere. One possibility is to deploy it on Amazon Elastic Beanstalk. You cannot simply deploy a KCL application as it is. You’ll need to wrap it within a special Kinesis Beanstalk worker wrapper.

Read more of this post

Big Data: a summary of various Amazon Big Data tools


We have gone through a lot of material about Big Data on this blog. This post summarises each Amazon Cloud component one by one, what they do and what their roles are in a Big Data architecture.

The components

Read more of this post

Introduction to Amazon Code Pipeline with Java part 24: summary


In the previous post we finalised the code for the job worker daemon thread. The job worker daemon is the central work horse of the Code Pipeline third party action. It includes all the necessary functions and objects that poll the CP endpoint for new jobs and act on each new job. The job worker is started as soon as the servlet context listener object has been executed. The daemon then sets up the polling action which is executed at specific intervals. As soon as there’s some new job the third party action will act upon it in some way. That part is entirely up to what your plug-in will accomplish.

The previous post was also the last part in this series. In this post we’ll quickly summarise what we’ve gone through in this series.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 23: job worker daemon wrap-up


In the previous post we finished discussing the job poller implementation. The job poller’s execute function runs every time the job agent polls Code Pipeline for new jobs. The job poller is responsible for pulling the new jobs and handing them over to the job service and job processor that will process them in some way. The poller will also send a signal to CP whether the job was a success or failure. The exact branching depends on the job status.

We’re now ready to return to the job worker daemon implementation. We’ve seen all the classes that the job worker calls upon either directly or indirectly so it’s time to connect the parts. We’ll do that in this post.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 22: the job poller implementation


In the previous post we looked at the job service interface and its implementation. The job service is responsible for polling for new jobs, acknowledging them and setting their final statuses to either success or failure. Much of the implementation uses the AWS client library to execute the work. We also know that each job polling thread will get a job service. The functions in the job service will be called by the job poller at a fixed interval.

We also started looking into the job poller implementation briefly but the job service took up most of the discussion. In this post we’ll finish the job poller class.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 21: the job service


In the previous post we looked at the job worker configuration class. It’s not a very complicated implementation. It basically constructs a job poller for each CP region the job agent will monitor. It also constructs an AWS client with the proper AWS credentials.

There are a couple more building blocks of the third party action before we have a functioning system. One of them is the implementation of the JobPoller interface which is the topic of this post. In particular we’ll be looking at one of the dependencies of the job poller, the job service class.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 20: the job worker configuration


In the previous part we started looking at the job worker initialisation details. The job worker takes the form of a deamon thread and implements the org.apache.commons.daemon.Daemon interface. The job worker daemon implementation introduces a range of new classes that we need to look at in order to get the full picture. One was the job poller interface which only has a single method called execute.

Another such element is the job worker configuration class which we’ll discuss in this post.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 19: the job worker initialisation and the job poller interface


In the previous post we first discussed the idea of a daemon thread. A daemon thread is most often used to initiate a long running task on its own thread so that the main executing thread is not blocked. A daemon thread is a good candidate to execute the Code Pipeline monitoring task. It will run in the background after executing the servlet initiator class which in turn is run automatically upon application start-up. We also saw how the daemon thread is started in code. However, we still know very little about the class that implements the daemon thread.

We’ll dive into that in this post.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 18: the job worker thread


In the previous post we looked at an implementation stub for the JobProcessor interface. The implementation reflects our real life Code Pipeline third party action job process. The business specific logic has been omitted of course as that is not crucial for understanding the code. Probably the most important bit was checking whether the continuation token was present in the incoming Code Pipeline job. That was the basis for distinguishing new load tests from ongoing ones. We also saw how to indicate to CP that a job is ongoing or whether it has finished with success or failure.

In this post we’ll start looking into how to start the job checking thread.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 17: the JobProcessor implementation


In the previous post we continued to explore the job processor interface of the job agent. In particular we looked at the WorkResult object which holds a number of properties that describe the job result. These include a summary message, the failure reason, the progress made and various others.

In this post we’ll discuss a possible implementation of the JobProcessor interface. Note that the exact implementation will depend on your business since this is where the actual Code Pipeline work item is processed. Our implementation the interface is also very tightly coupled with the load testing domain of our business. Hence I won’t present every single detail around that code as much of it irrelevant for Code Pipeline and the job agent.

Read more of this post


A great site

Elliot Balynn's Blog

A directory of wonderful thoughts

HarsH ReaLiTy

A Good Blog is Hard to Find

Softwarearchitektur in der Praxis

Wissenswertes zu Webentwicklung, Domain-Driven Design und Microservices

Technology Talks

on Microsoft technologies, Web, Android and others

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog


Once Upon a Camayoc

Bite-size insight on Cyber Security for the not too technical.

Guru N Guns's

OneSolution To dOTnET.

Johnny Zraiby

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

%d bloggers like this: