Introduction to Amazon Code Pipeline with Java part 10: the client web pages

May 21, 2016 1 Comment

Introduction

In the previous post we looked at an important component that comes up during a job execution in CodePipeline, namely the continuation token. The continuation token can be supplied in the response to CP when a new job is processed. The token will then appear the next time the job agent gets a signal about a new job. That way the job agent will know that it’s a job that has already been initiated. The job is in effect divided up into multiple small partial jobs coming from CP. The partial jobs will keep coming until the job agent has indicated a 100% completion rate.

We’re almost at the point where we can start looking at the structure of the job agent itself. Before we do that I’d like to present you what type of web pages you’ll need to build for a third party action. We’ve already seen some pictures but I’d like to provide some further details.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to MongoDb with .NET part 31: the query plan and our first index

May 20, 2016 Leave a comment

Introduction

In the previous post we introduced a new sub-section in this series, namely how indexes work in MongoDb. We went through a general introduction of the topic without actually creating an index. We also looked at the stats() function that can be called on a collection to find some basic statistics such as the number of documents and index sizes. In addition we learnt a little bit about what a storage engine is and that the default storage engine that ships with MongoDb 3.2.4 is called WiredTiger.

In this post we’ll start creating indexes in the Mongo shell. Have both the mongo client and the mongo server ready in two different console windows. Remember, it’s the “mongo” and “mongod” commands.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 30: starting with indexing

May 19, 2016 Leave a comment

Introduction

In the previous post we finished our discussion of the aggregation framework in MongoDb. We now know the basics of building aggregation pipelines in both the mongo shell using JavaScript with JSON and using the MongoDb .NET driver. In particular we looked at how to write strongly typed aggregation queries with dedicated methods in the aggregation fluent API.

Starting with this post we’ll shift our focus towards a different topic within MongoDb: indexes and related topics in performance.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 29: aggregation in the .NET driver using strong typing and dedicated functions

May 18, 2016 1 Comment

Introduction

In the previous post we looked at how to build an aggregation pipeline in the MongoDb .NET driver using loosely typed BsonDocument objects and the AppendStage function. AppendStage can accept any type of aggregation stage and each stage is described with plain strings. An advantage with AppendStage is that we can express any kind of aggregation stage with it, including those for which there’s no suitable dedicated function, like $redact.

In this post we’ll look at the strongly typed alternative to the previous solution.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 28: aggregation in the .NET driver using loose typing and AppendStage

May 17, 2016 1 Comment

Introduction

In the previous post we started looking into MongoDb aggregations via the .NET driver. Our first example was a single $sort stage using the AppendStage function. The AppendStage function is generic in the sense that it can accept just any type of aggregation stage to be added to the pipeline. We also introduced the BsonDocument object which offers a way to describe MongoDb documents in a loosely typed way.

In this post we’ll continue our discussion and build the pipeline we’re targeting.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to Amazon Code Pipeline with Java part 9: the job agent continuation token

May 15, 2016 Leave a comment

Introduction

In the previous post we went through the details behind the communication between CodePipeline and the third part action. The job agent is responsible for for this communication on the side of the third party action. It must monitor a CodePipeline endpoint for new jobs and then acknowledge the job in some way, e.g. by sending a success or failure message. We also mentioned the role of AWS S3 which will store the artifact that enters the pipeline.

We’re at the point where the job agent will act upon the job details received from CP and start the actual work. There’s, however, at least one remaining implementation detail that is important to be aware of when designing this communication process. It’s called the continuation token and this is what we’ll look at in this post.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

Introduction to MongoDb with .NET part 27: aggregation in the .NET driver

May 12, 2016 2 Comments

Introduction

In the previous post we looked at various other aggregation operators in MongoDb. With $first and $last we can find the highest and lowest member in a sorted collection. If you need to find e.g. the city with the highest population or the country with the lowest number of pets per capita then these operators will probably figure in the aggregation pipeline. The $unwind stage operator blows up the members of an array so that we can run a query on those members in an easier way. Without $unwind it’s difficult to access the individual members of an array. Note that $unwind can dramatically increase the result set of documents.

We’ve now gone through the most important aggregation operators in MongoDb. It’s time to turn our attention to how aggregations can be written in the .NET driver.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 26: more aggregation types and examples

May 10, 2016 Leave a comment

Introduction

In the previous post we looked at how to add multiple stages to a MongoDb aggregation pipeline. We saw that it was easy and straightforward to extend the stages array of the pipeline. It’s important to remember that each stage acts upon the set of documents produced by the previous stage. We can also have the same stage type appear more than once in the pipeline, i.e. there’s no limit saying we can only have one $group operation in a pipeline. The various stages can also appear in various order, we don’t have to follow some type of ordering either.

In this post we’ll continue where we left off and look at a couple more aggregation operators and examples.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to MongoDb with .NET part 25: adding multiple stages to the aggregation pipeline

May 9, 2016 2 Comments

Introduction

In the previous post we continued our discussion of the grouping stage in the MongoDb aggregation pipeline. We looked at a couple of $ operators that can be used in aggregations, such as $min, $avg and $addToSet. We also saw how to use compound IDs to add multiple grouping keys. Compound IDs are also useful when we want to give an explanatory field to the compulsory _id property so that it’s clear what the grouping key is.

In this post we’ll start adding multiple stages to the pipeline. So far we’ve had simple aggregation pipelines with one stage but there’s nothing stopping us from adding more to the stage array. It’s also possible to add multiple instances of the same stage type.

Read more of this post

Filed under .NET, MongoDb Tagged with ,

Introduction to Amazon Code Pipeline with Java part 8: the job agent communication process

May 8, 2016 Leave a comment

Introduction

In the previous post we went through the details behind the communication between your web site and CodePipeline when a CP user selects your third party action. The third party action developer will be responsible to set up and maintain an external configuration page where CP users can complete their action configuration. CP and the external page communicate with each other using HTTPS POST calls and URL-encoded JSON objects. CP sends a number of properties to the configuration page in the JSON object. These include the client ID and client token that you’ll need to save in your data store and map them to a user in your system. The configuration page redirects the user to the CP GUI with the list of key-value pairs that contain the configuration values for the test runner. These values will be available to the job agent when the job runs.

In this post we’ll look into the details of the communication between CP and the job agent. This post will go through the process at an overall level. The upcoming code examples will show a lot more details.

Read more of this post

Filed under Amazon, Java Tagged with , , , ,

← Older posts

Newer posts →

Elliot Balynn's Blog

A directory of wonderful thoughts

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Once Upon a Camayoc

ARCHIVED: Bite-size insight on Cyber Security for the not too technical.