Tips and tricks in C# .NET
September 3, 2016 Leave a comment
Say you want to fill a byte array from a source, e.g. a from a randomly simulated data of 50 bytes like the following:
byte[] simulatedSource = new byte[50]; Random random = new Random(); random.nextBytes(simulatedSource); InputStream inputStream = new ByteArrayInputStream(simulatedSource);
At a basic level we can read each byte one by one from the input stream as follows:
May 17, 2015 Leave a comment
Introduction
In the previous post we created the DynamoDb source file with a reduced set of geolocations from the full MaxMind data source. We saw how we could reuse the same process as before in the case of the IP and coordinate range tables.
In this final post of this series we’ll close the loop by actually extracting and geographic properties of a geoname ID. After all you’d like to know whether a visitor come from New York other than “geoname ID 3452334”.
May 16, 2015 Leave a comment
Introduction
In the previous post we successfully queried the coordinate range database in DynamoDb. We used the query endpoints that are built into the AWS geo-location library to find the data records within the radius around a centre point.
Where are we now?
We’ve got quite far with our project. We have the ability to query an IPv4 and coordinate range table in DynamoDb. We can extract a geoname ID that belongs to either an IP or a latitude-longitude pair. The next step is to dress up those IDs with real location data such as “Stockholm” or “Tehran”.
May 10, 2015 1 Comment
Introduction
In the previous post we loaded the limited lng/lat range records into DynamoDb. As we’re only talking about about 50 records we could have added them in code one by one. However, that strategy would never work for the full MaxMind data set even after discarding the duplicates. So instead we looked at the built-in Import/Export functionality in DynamoDb. You’ll be able to go through the same process when you’re ready to import the full data set.
In this post we’ll see how to query the lnglat range database to extract the ID of the nearest geolocation. We’ll get to use the AWS Java SDK.
May 9, 2015 Leave a comment
Introduction
In the previous post we successfully created the lng/lat import file that DynamoDb can understand and process.
In this post we’ll upload this file to DynamoDb. The process will be the same to what we saw in this post where we inserted the demo data into the IPv4 range table. If necessary then re-read that post to refresh your memory about the process. We’ll follow the strategy we laid out in this post.
May 3, 2015 Leave a comment
Introduction
In the previous post we successfully queried the limited IPv4 range table in DynamoDb and found the geoname ID that belongs to a single IP. We used 3 available integer properties in the table to narrow down the number of records that had to be scanned to reduce the query execution time and the risk for exceptions.
In this post we’ll start the same process for the lng/lat coordinate range. More specifically we’ll prepare the raw data file that can be uploaded into DynamoDb through S3. The process will be very similar to what we saw in this post where we created the IPv4 range source file. It is good idea to quickly re-scan that post to remind you of the process.
May 2, 2015 Leave a comment
Introduction
In the previous post we loaded the limited IP range records into DynamoDb. As we’re only talking about about 250 records we could have added them in code one by one. However, that strategy would never work for the full MaxMind IP data set of 10 million records. So instead we looked at the built-in Import/Export functionality in DynamoDb. You’ll be able to go through the same process when you’re ready to import the full data set.
In this post we’ll see how to query the IP range database to extract the ID of the nearest geolocation. We’ll get to use the AWS Java SDK.
April 26, 2015 3 Comments
Introduction
In the previous post we successfully created a limited IPv4 range file ready to be uploaded to DynamoDb. We saw how the relevant bits were extracted from the reduced subset of the MaxMind CSV source file and how the DynamoDb-specific input file was created.
In this post we’ll see how to upload the source file to DynamoDb using the bulk insertion tools available there. We’ll only import our limited test data but the same steps apply for large data sets as well.
April 25, 2015 Leave a comment
Introduction
In the previous post we went through our strategy to save the longitude-latitude coordinates in DynamoDb for our geo-spatial queries later on. We said that we would save the records in DynamoDb in a way so that it fits queries according to a library designed by AWS which in turn uses a geo-library from Google.
In this post we’ll finally see some action. We’re ready to format an upload the IP range to DynamoDb. Actually we’ll show the techniques using only a small subset of the MaxMind raw data source. I strongly recommend you follow the same strategy and not try to upload 10 million rows at once. Make sure the process works for a small subset from start to finish and then go for the real thing. The steps outlined in this series will also apply to the full, paid version of the CSV source.
April 19, 2015 Leave a comment
Introduction
In the previous post we discussed our strategy to save the IP ranges in DynamoDb. We saw that it would be very inefficient to store the IPs as strings and run our queries based on some string manipulation. Instead we’ll store the IP ranges as lower and upper limit integers.
In this post we’ll discuss our strategy to save the longitude-latitude ranges for cities.
A directory of wonderful thoughts
Web development
Various tidbits
WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS
ARCHIVED: Bite-size insight on Cyber Security for the not too technical.
Exercises in .NET with Andras Nemes 