Introduction to Amazon Code Pipeline with Java part 11: starting with the job agent

May 22, 2016 Leave a comment

Introduction

In the previous post we looked a bit at the web pages that must be up and running for the third party action to work with code pipeline. There are 3 types of URL that must be set up. The external configuration URL defines a web page where the CP user can provide the necessary inputs for the job execution. The execution URL is where the CP user can follow the job execution, i.e. where you can show tables, graphs etc. that show the job progress. Lastly we have the entity URL which is a web page to show some resources or job summary page to the user.

In this post we’ll start looking into the job agent itself. It’s probably the largest component you’ll need to build for the CP integration.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to Amazon Code Pipeline with Java part 10: the client web pages

May 21, 2016 1 Comment

Introduction

In the previous post we looked at an important component that comes up during a job execution in CodePipeline, namely the continuation token. The continuation token can be supplied in the response to CP when a new job is processed. The token will then appear the next time the job agent gets a signal about a new job. That way the job agent will know that it’s a job that has already been initiated. The job is in effect divided up into multiple small partial jobs coming from CP. The partial jobs will keep coming until the job agent has indicated a 100% completion rate.

We’re almost at the point where we can start looking at the structure of the job agent itself. Before we do that I’d like to present you what type of web pages you’ll need to build for a third party action. We’ve already seen some pictures but I’d like to provide some further details.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to MongoDb with .NET part 31: the query plan and our first index

May 20, 2016 Leave a comment

Introduction

In the previous post we introduced a new sub-section in this series, namely how indexes work in MongoDb. We went through a general introduction of the topic without actually creating an index. We also looked at the stats() function that can be called on a collection to find some basic statistics such as the number of documents and index sizes. In addition we learnt a little bit about what a storage engine is and that the default storage engine that ships with MongoDb 3.2.4 is called WiredTiger.

In this post we’ll start creating indexes in the Mongo shell. Have both the mongo client and the mongo server ready in two different console windows. Remember, it’s the “mongo” and “mongod” commands.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 30: starting with indexing

May 19, 2016 Leave a comment

Introduction

In the previous post we finished our discussion of the aggregation framework in MongoDb. We now know the basics of building aggregation pipelines in both the mongo shell using JavaScript with JSON and using the MongoDb .NET driver. In particular we looked at how to write strongly typed aggregation queries with dedicated methods in the aggregation fluent API.

Starting with this post we’ll shift our focus towards a different topic within MongoDb: indexes and related topics in performance.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 29: aggregation in the .NET driver using strong typing and dedicated functions

May 18, 2016 1 Comment

Introduction

In the previous post we looked at how to build an aggregation pipeline in the MongoDb .NET driver using loosely typed BsonDocument objects and the AppendStage function. AppendStage can accept any type of aggregation stage and each stage is described with plain strings. An advantage with AppendStage is that we can express any kind of aggregation stage with it, including those for which there’s no suitable dedicated function, like $redact.

In this post we’ll look at the strongly typed alternative to the previous solution.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 28: aggregation in the .NET driver using loose typing and AppendStage

May 17, 2016 1 Comment

Introduction

In the previous post we started looking into MongoDb aggregations via the .NET driver. Our first example was a single $sort stage using the AppendStage function. The AppendStage function is generic in the sense that it can accept just any type of aggregation stage to be added to the pipeline. We also introduced the BsonDocument object which offers a way to describe MongoDb documents in a loosely typed way.

In this post we’ll continue our discussion and build the pipeline we’re targeting.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to Amazon Code Pipeline with Java part 9: the job agent continuation token

May 15, 2016 Leave a comment

Introduction

In the previous post we went through the details behind the communication between CodePipeline and the third part action. The job agent is responsible for for this communication on the side of the third party action. It must monitor a CodePipeline endpoint for new jobs and then acknowledge the job in some way, e.g. by sending a success or failure message. We also mentioned the role of AWS S3 which will store the artifact that enters the pipeline.

We’re at the point where the job agent will act upon the job details received from CP and start the actual work. There’s, however, at least one remaining implementation detail that is important to be aware of when designing this communication process. It’s called the continuation token and this is what we’ll look at in this post.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to MongoDb with .NET part 27: aggregation in the .NET driver

May 12, 2016 2 Comments

Introduction

In the previous post we looked at various other aggregation operators in MongoDb. With $first and $last we can find the highest and lowest member in a sorted collection. If you need to find e.g. the city with the highest population or the country with the lowest number of pets per capita then these operators will probably figure in the aggregation pipeline. The $unwind stage operator blows up the members of an array so that we can run a query on those members in an easier way. Without $unwind it’s difficult to access the individual members of an array. Note that $unwind can dramatically increase the result set of documents.

We’ve now gone through the most important aggregation operators in MongoDb. It’s time to turn our attention to how aggregations can be written in the .NET driver.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 26: more aggregation types and examples

May 10, 2016 Leave a comment

Introduction

In the previous post we looked at how to add multiple stages to a MongoDb aggregation pipeline. We saw that it was easy and straightforward to extend the stages array of the pipeline. It’s important to remember that each stage acts upon the set of documents produced by the previous stage. We can also have the same stage type appear more than once in the pipeline, i.e. there’s no limit saying we can only have one $group operation in a pipeline. The various stages can also appear in various order, we don’t have to follow some type of ordering either.

In this post we’ll continue where we left off and look at a couple more aggregation operators and examples.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 25: adding multiple stages to the aggregation pipeline

May 9, 2016 2 Comments

Introduction

In the previous post we continued our discussion of the grouping stage in the MongoDb aggregation pipeline. We looked at a couple of $ operators that can be used in aggregations, such as $min, $avg and $addToSet. We also saw how to use compound IDs to add multiple grouping keys. Compound IDs are also useful when we want to give an explanatory field to the compulsory _id property so that it’s clear what the grouping key is.

In this post we’ll start adding multiple stages to the pipeline. So far we’ve had simple aggregation pipelines with one stage but there’s nothing stopping us from adding more to the stage array. It’s also possible to add multiple instances of the same stage type.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

← Older posts

Newer posts →

Elliot Balynn's Blog

A directory of wonderful thoughts

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Once Upon a Camayoc

ARCHIVED: Bite-size insight on Cyber Security for the not too technical.