Creating an Amazon Beanstalk wrapper around a Kinesis application in Java

Introduction

Suppose that you have a Java Amazon Kinesis application which handles messages from the Amazon message queue handler Kinesis. This means that you have a method that starts a Worker object in the com.amazonaws.services.kinesis.clientlibrary.lib.worker library.

If you are developing an application that is meant to process messages from Amazon Kinesis and you don’t know what I mean by a Kinesis app then check out the Amazon documentation on the Kinesis Client Library (KCL) here.

The starting point for this post is that you have a KCL application and want to host it somewhere. One possibility is to deploy it on Amazon Elastic Beanstalk. You cannot simply deploy a KCL application as it is. You’ll need to wrap it within a special Kinesis Beanstalk worker wrapper.

Read more of this post

Big Data: a summary of various Amazon Big Data tools

Introduction

We have gone through a lot of material about Big Data on this blog. This post summarises each Amazon Cloud component one by one, what they do and what their roles are in a Big Data architecture.

The components

Read more of this post

Introduction to Amazon Code Pipeline with Java part 24: summary

Introduction

In the previous post we finalised the code for the job worker daemon thread. The job worker daemon is the central work horse of the Code Pipeline third party action. It includes all the necessary functions and objects that poll the CP endpoint for new jobs and act on each new job. The job worker is started as soon as the servlet context listener object has been executed. The daemon then sets up the polling action which is executed at specific intervals. As soon as there’s some new job the third party action will act upon it in some way. That part is entirely up to what your plug-in will accomplish.

The previous post was also the last part in this series. In this post we’ll quickly summarise what we’ve gone through in this series.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 23: job worker daemon wrap-up

Introduction

In the previous post we finished discussing the job poller implementation. The job poller’s execute function runs every time the job agent polls Code Pipeline for new jobs. The job poller is responsible for pulling the new jobs and handing them over to the job service and job processor that will process them in some way. The poller will also send a signal to CP whether the job was a success or failure. The exact branching depends on the job status.

We’re now ready to return to the job worker daemon implementation. We’ve seen all the classes that the job worker calls upon either directly or indirectly so it’s time to connect the parts. We’ll do that in this post.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 22: the job poller implementation

Introduction

In the previous post we looked at the job service interface and its implementation. The job service is responsible for polling for new jobs, acknowledging them and setting their final statuses to either success or failure. Much of the implementation uses the AWS client library to execute the work. We also know that each job polling thread will get a job service. The functions in the job service will be called by the job poller at a fixed interval.

We also started looking into the job poller implementation briefly but the job service took up most of the discussion. In this post we’ll finish the job poller class.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 21: the job service

Introduction

In the previous post we looked at the job worker configuration class. It’s not a very complicated implementation. It basically constructs a job poller for each CP region the job agent will monitor. It also constructs an AWS client with the proper AWS credentials.

There are a couple more building blocks of the third party action before we have a functioning system. One of them is the implementation of the JobPoller interface which is the topic of this post. In particular we’ll be looking at one of the dependencies of the job poller, the job service class.

Read more of this post

Introduction to Amazon Code Pipeline with Java part 20: the job worker configuration

Introduction

In the previous part we started looking at the job worker initialisation details. The job worker takes the form of a deamon thread and implements the org.apache.commons.daemon.Daemon interface. The job worker daemon implementation introduces a range of new classes that we need to look at in order to get the full picture. One was the job poller interface which only has a single method called execute.

Another such element is the job worker configuration class which we’ll discuss in this post.

Read more of this post

ultimatemindsettoday

A great WordPress.com site

iReadable { }

.NET Tips & Tricks

Robin Sedlaczek's Blog

Love to use Microsoft Technologies

HarsH ReaLiTy

A Good Blog is Hard to Find

Ricos Blog zu Softwaredesign- und architektur

Ideen und Gedanken rund um Softwaredesign und -architektur, Domain-Driven Design, C# und Windows Azure

the software architecture

thoughts, ideas, diagrams,enterprise code, design pattern , solution designs

Technology Talks

on Microsoft technologies, Web, Android and others

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

Anything around ASP.NET MVC,WEB API, WCF, Entity Framework & AngularJS

Cyber Matters

Bite-size insight on Cyber Security for the not too technical.

Guru N Guns's

OneSolution To dOTnET.

Johnny Zraiby

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

%d bloggers like this: