Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 5: creating the IPv4 source file for DynamoDb

Introduction

In the previous post we went through our strategy to save the longitude-latitude coordinates in DynamoDb for our geo-spatial queries later on. We said that we would save the records in DynamoDb in a way so that it fits queries according to a library designed by AWS which in turn uses a geo-library from Google.

In this post we’ll finally see some action. We’re ready to format an upload the IP range to DynamoDb. Actually we’ll show the techniques using only a small subset of the MaxMind raw data source. I strongly recommend you follow the same strategy and not try to upload 10 million rows at once. Make sure the process works for a small subset from start to finish and then go for the real thing. The steps outlined in this series will also apply to the full, paid version of the CSV source.

Read more of this post

Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 4: lng/lat range strategy

Introduction

In the previous post we discussed our strategy to save the IP ranges in DynamoDb. We saw that it would be very inefficient to store the IPs as strings and run our queries based on some string manipulation. Instead we’ll store the IP ranges as lower and upper limit integers.

In this post we’ll discuss our strategy to save the longitude-latitude ranges for cities.

Read more of this post

Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 3: IPv4 range strategy

Introduction

In the previous post we went through the details of the CSV source files that show the IP and lng/lat ranges and the actual locations. We saw that the two source files are linked by the location ID.

The next task is to import the source into DynamoDb. Recall that we want to handle queries based on IPs and lng/lat pairs separately, those are the primary goals of this series. The way to query an IP database is very different from querying a lng/lat database. An IP will fit into some IP range and we’d like to find that record. Whereas if you have a lng/lat co-ordinate pair and would like to find the nearest city/school/hospital/etc. within a certain radius then that query will involve some complex maths instead.

Read more of this post

Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 2: MaxMind source files

Introduction

In the previous post we outlined the goals of this series and the tools that we’re going to use. We’ve got as far as downloading MaxMind’s free version of their geo-location source with IPs and longitude-latitude co-ordinates. We saw that the downloaded package had a number of CSV files.

In this post we’ll start off by looking at the structure of those files and how they are connected.

The source files

Read more of this post

Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 1: introduction and goals

Introduction

There are a lot of applications out there that involve finding a point on a map. Putting hotels, restaurants, metro stations etc. on a map on your mobile device has become commonplace. Queries that find the nearest hospital, theatre or school need to be executed in a fast and efficient manner.

In this series we’ll discuss a possible solution to the following geo-location related scenarios:

  • You have a pair of longitude (lng) and latitude (lat) co-ordinates and you’d like to find all locations in a circle around that point or just the nearest relevant location, e.g. the nearest city
  • You have an IP address and you’d like to find the location details of that address, such as New York or Sydney

The series is centred around Amazon cloud based tools. Even if you’re not familiar with Amazon Cloud but looking for a solution to questions similar to the ones outlined above I encourage you to read on – you might just find something useful.

Read more of this post

Using Amazon DynamoDb with the AWS .NET API Part 7: its place in Big Data

Introduction

In the previous post we looked at how to query our test table in Amazon DynamoDb. That post also completed our main discussion on how to use DynamoDb in .NET.

This post is dedicated to a larger picture: Big Data. This series is part of a large series on cloud-based Big Data infrastructure along with the message handler Amazon Kinesis and the raw data storage S3. Therefore the pre-requisite of following the discussion in this post is familiarity with what we discussed in those posts or at least some knowledge of Big Data. However, I’ll try to write in a way that those of you who’ve only come here for DynamoDb may get a taste of the role it can play in a cloud-based Big Data architecture.

Read more of this post

Using Amazon DynamoDb with the AWS .NET API Part 6: queries

Introduction

In the previous post we looked at how to delete and update records in Amazon DynamoDb. In this post we’ll investigate how to run queries against the data set in a DynamoDb table. Like in the two posts before this we’ll look at the document and data models separately.

We’ll extend the DynamoDbDemo console application in Visual Studio.

I’ve inserted a couple more records into the demo DynamoDb table in preparation for the queries:

Read more of this post

ultimatemindsettoday

A great WordPress.com site

Elliot Balynn's Blog

A directory of wonderful thoughts

HarsH ReaLiTy

A Good Blog is Hard to Find

Softwarearchitektur in der Praxis

Wissenswertes zu Webentwicklung, Domain-Driven Design und Microservices

Technology Talks

on Microsoft technologies, Web, Android and others

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Guru N Guns's

OneSolution To dOTnET.

Johnny Zraiby

Measuring programming progress by lines of code is like measuring aircraft building progress by weight.

%d bloggers like this: