Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 3: IPv4 range strategy

Introduction

In the previous post we went through the details of the CSV source files that show the IP and lng/lat ranges and the actual locations. We saw that the two source files are linked by the location ID.

The next task is to import the source into DynamoDb. Recall that we want to handle queries based on IPs and lng/lat pairs separately, those are the primary goals of this series. The way to query an IP database is very different from querying a lng/lat database. An IP will fit into some IP range and we’d like to find that record. Whereas if you have a lng/lat co-ordinate pair and would like to find the nearest city/school/hospital/etc. within a certain radius then that query will involve some complex maths instead.

Read more of this post

Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 2: MaxMind source files

Introduction

In the previous post we outlined the goals of this series and the tools that we’re going to use. We’ve got as far as downloading MaxMind’s free version of their geo-location source with IPs and longitude-latitude co-ordinates. We saw that the downloaded package had a number of CSV files.

In this post we’ll start off by looking at the structure of those files and how they are connected.

The source files

Read more of this post

Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 1: introduction and goals

Introduction

There are a lot of applications out there that involve finding a point on a map. Putting hotels, restaurants, metro stations etc. on a map on your mobile device has become commonplace. Queries that find the nearest hospital, theatre or school need to be executed in a fast and efficient manner.

In this series we’ll discuss a possible solution to the following geo-location related scenarios:

  • You have a pair of longitude (lng) and latitude (lat) co-ordinates and you’d like to find all locations in a circle around that point or just the nearest relevant location, e.g. the nearest city
  • You have an IP address and you’d like to find the location details of that address, such as New York or Sydney

The series is centred around Amazon cloud based tools. Even if you’re not familiar with Amazon Cloud but looking for a solution to questions similar to the ones outlined above I encourage you to read on – you might just find something useful.

Read more of this post

Insert a non-existent value into a Map in Java 8

Consider the following Employee class:

public class Employee
{
    private UUID id;
    private String name;
    private int age;

    public Employee(UUID id, String name, int age)
    {
        this.id = id;
        this.name = name;
        this.age = age;
    }
        
    public UUID getId()
    {
        return id;
    }

    public void setId(UUID id)
    {
        this.id = id;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }    
    
    public int getAge()
    {
        return age;
    }

    public void setAge(int age)
    {
        this.age = age;
    }
}

Let’s put some Employee objects into a hash map:

Read more of this post

Replacing a value in a Map in Java 8

The Java 8 SDK has a couple of interesting new default “replace” methods available on the Map interface.

Consider the following HashMap:

Map<String, String> sizes = new HashMap<>();
sizes.put("XS", "Extra small");
sizes.put("S", "Small");
sizes.put("M", "Medium");
sizes.put("L", "Large");
sizes.put("XL", "Extra large");
sizes.put("XXL", "Extra extra large");

Say we’d like to replace the value of key “S”:

String replacedValue = sizes.replace("S", "Small size");

The replace method returns the value of the replaced string. In the above case the key “S” will have a new value “Small size” and “replace” returns “Small” as it was the value of “S” before the replace operation.

Read more of this post

Conditionally remove elements from a List in Java 8

Java 8 introduces a new method available for Collection types: removeif(). It accepts a predicate which defines the condition on which the elements should be removed. It returns a boolean where a true response means that at least one item has been removed and false otherwise:

Collection<String> stringStack = new Stack<>();
stringStack.add("Hello");
stringStack.add("my");
stringStack.add("dear");
stringStack.add("world");
        
stringStack.removeIf(s -> s.contains("ll"));

The above example will remove “Hello” from the list stack.

Note that not all collections support item removal. In that case the method will throw an UnsupportedOperationException in case an attempt is made to remove a matching element. The ArrayList is one such collection:

Collection<String> asList = Arrays.asList("hello", "my", "dear", "world");
asList.removeIf(s -> s.contains("ll"));

This will throw an exception unfortunately as the Array.asList method returns an ArrayList of type java.util.Arrays.ArrayList (which is read only and fixed size) and not the classic java.util.ArrayList (resizable and item-removable) – based on a comment by Juanito below.

View all posts related to Java here.

Using the Comparator class in Java 8 to compare objects

Java 8 comes with a range of built-in implementations of the Comparator interface.

Consider the following Employee class:

public class Employee
{
    private UUID id;
    private String name;
    private int age;

    public Employee(UUID id, String name, int age)
    {
        this.id = id;
        this.name = name;
        this.age = age;
    }
        
    public UUID getId()
    {
        return id;
    }

    public void setId(UUID id)
    {
        this.id = id;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }    
    
    public int getAge()
    {
        return age;
    }

    public void setAge(int age)
    {
        this.age = age;
    }    
}

…and the following list of employees:

Read more of this post

Create a List using Arrays.asList in Java 8

Java 8 has a number of new methods on Collections. One such utility method is the static asList method with which you can quickly create a List of T.

Here’s how it works for a List of integers:

List<Integer> asList = Arrays.asList(1,2,3,4);

…and for a List of strings:

List<String> asList = Arrays.asList("hello", "my", "dear", "world");

View all posts related to Java here.

Return a default value from a Map in Java 8

Consider the following Employee class:

public class Employee
{
    private UUID id;
    private String name;
    private int age;

    public Employee(UUID id, String name, int age)
    {
        this.id = id;
        this.name = name;
        this.age = age;
    }
        
    public UUID getId()
    {
        return id;
    }

    public void setId(UUID id)
    {
        this.id = id;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }    
    
    public int getAge()
    {
        return age;
    }

    public void setAge(int age)
    {
        this.age = age;
    }
}

Let’s put some Employee objects into a hash map:

Map<Integer, Employee> employeeMap = new HashMap<>();
employeeMap.put(1, new Employee(UUID.randomUUID(), "Elvis", 50));
employeeMap.put(2, new Employee(UUID.randomUUID(), "Marylin", 18));
employeeMap.put(3, new Employee(UUID.randomUUID(), "Freddie", 25));
employeeMap.put(4, null);
employeeMap.put(5, new Employee(UUID.randomUUID(), "Mario", 43));
employeeMap.put(6, new Employee(UUID.randomUUID(), "John", 35));
employeeMap.put(7, new Employee(UUID.randomUUID(), "Julia", 55));

Note the null value for key 4. Let’s also define a default Employee object:

Employee defaultEmployee = new Employee(UUID.fromString("00000000-0000-0000-0000-000000000000"), "", -1);

Java 8 includes a new method called “getOrDefault” on the Map interface. It accepts a key, like the “get” method on the Map, but it also accepts a default object that will be returned if the key does not exist.

Can you guess what the below code will return?

Employee employee = employeeMap.getOrDefault(4, defaultEmployee);

“employee” will be null of course, as key 4 exists and its value is null. However, if you simply call the “get” method with 4 as the key input then you don’t know exactly how to interpret the null result: does 4 exist as key in the map and its value is null or does the key 4 not exist at all in the map? With getOrDefault returning 0 in this case you can be 100% sure that a null response is unambiguous.

Let’s see what the below bit of code returns:

Employee employee = employeeMap.getOrDefault(12, defaultEmployee);

This time it returns the default employee as the key 12 does not exist in the map.

View all posts related to Java here.

How to merge two Maps in Java 8

The Map interface has been extended with the “merge” function in Java 8. Let’s see an example on how to use it.

Consider the following Empolyee class:

public class Employee
{
    private UUID id;
    private String name;
    private int age;

    public Employee(UUID id, String name, int age)
    {
        this.id = id;
        this.name = name;
        this.age = age;
    }
        
    public UUID getId()
    {
        return id;
    }

    public void setId(UUID id)
    {
        this.id = id;
    }

    public String getName()
    {
        return name;
    }

    public void setName(String name)
    {
        this.name = name;
    }    
    
    public int getAge()
    {
        return age;
    }

    public void setAge(int age)
    {
        this.age = age;
    }
}

Let’s say that we have the following two maps where the key is an indicator of the employees’ performance and the value is the list of employees that fall into that category:

List<Employee> averageMapOne = new ArrayList<>();
averageMapOne.add(new Employee(UUID.randomUUID(), "Elvis", 50));
averageMapOne.add(new Employee(UUID.randomUUID(), "Marylin", 18));
List<Employee> poorMapOne = new ArrayList<>();
poorMapOne.add(new Employee(UUID.randomUUID(), "Mario", 43));
poorMapOne.add(new Employee(UUID.randomUUID(), "John", 35));
List<Employee> excellentMapOne = new ArrayList<>();
excellentMapOne.add(new Employee(UUID.randomUUID(), "Julia", 55));
List<Employee> okMapOne = new ArrayList<>();
okMapOne.add(new Employee(UUID.randomUUID(), "Nick", 43));
okMapOne.add(new Employee(UUID.randomUUID(), "Richard", 61));

Map<String, List<Employee>> employeeMapOne = new HashMap<>();
employeeMapOne.put("average", averageMapOne);
employeeMapOne.put("poor", poorMapOne);
employeeMapOne.put("excellent", excellentMapOne);
employeeMapOne.put("OK", okMapOne);

List<Employee> averageMapTwo = new ArrayList<>();
averageMapTwo.add(new Employee(UUID.randomUUID(), "Lotta", 52));
averageMapTwo.add(new Employee(UUID.randomUUID(), "Eva", 42));
averageMapTwo.add(new Employee(UUID.randomUUID(), "Mark", 24));
List<Employee> poorMapTwo = new ArrayList<>();
poorMapTwo.add(new Employee(UUID.randomUUID(), "Anna", 20));
List<Employee> excellentMapTwo = new ArrayList<>();
excellentMapTwo.add(new Employee(UUID.randomUUID(), "Bertil", 28));
excellentMapTwo.add(new Employee(UUID.randomUUID(), "Cecilia", 36));
excellentMapTwo.add(new Employee(UUID.randomUUID(), "Edit", 21));

Map<String, List<Employee>> employeeMapTwo = new HashMap<>();

employeeMapTwo.put("average", averageMapTwo);
employeeMapTwo.put("poor", poorMapTwo);
employeeMapTwo.put("excellent", excellentMapTwo);

Our goal is to merge map 2 into map 1 in way that all employees that fall into the same category will have all employees in a joined list. E.g. “average” will have 5 employees: Elvis, Marylin, Lotta, Eva and Mark.

The solution requires an understanding of the Java Stream API.

Here’s an explanation of the below bit of code:

  • We open a stream on the entry set of the first map
  • Then we iterate through the map using the forEach method
  • We pass a lambda expression into forEach where we want to do “something” with each entry in the entry set
  • This “something” is the actual merge operation of map 2
  • The merge operation accepts a key and a value which will be the key and value of map 1
  • Merge also accepts a BiFunction which will be used in case the key already exists in map 2
  • In this function we pass in the lists from map 1 and map 2, add the contents of map 1 into map 2 and return the new list

Here’s the code that will do the trick:

employeeMapOne.entrySet().stream()
        .forEach(entry -> employeeMapTwo.merge(entry.getKey(), entry.getValue(),
                (listTwo, listOne) ->
                        {
                            listOne.addAll(listTwo);
                            return listOne;
                        }));

View all posts related to Java here.

Elliot Balynn's Blog

A directory of wonderful thoughts

Software Engineering

Web development

Disparate Opinions

Various tidbits

chsakell's Blog

WEB APPLICATION DEVELOPMENT TUTORIALS WITH OPEN-SOURCE PROJECTS

Once Upon a Camayoc

ARCHIVED: Bite-size insight on Cyber Security for the not too technical.