Amazon cloud/Big Data
Amazon cloud
Big Data overall architecture
- Architecture of a Big Data messaging and aggregation system using Amazon Web Services part 1
- Architecture of a Big Data messaging and aggregation system using Amazon Web Services part 2
- Architecture of a Big Data messaging and aggregation system using Amazon Web Services part 3
- Architecture of a Big Data messaging and aggregation system using Amazon Web Services part 4
- Architecture of a Big Data messaging and aggregation system using Amazon Web Services part 5
Amazon Big Data components and code
The message handler: Kinesis
- Big Data: using Amazon Kinesis with the AWS.NET API Part 1: introduction
- Using Amazon Kinesis with the AWS.NET API Part 2: stream, NET SDK and domain setup
- Big Data: using Amazon Kinesis with the AWS.NET API Part 3: sending to the stream
- Big Data: using Amazon Kinesis with the AWS.NET API Part 4: reading from the stream
- Using Amazon Kinesis with the AWS.NET API Part 5: validation
- Using Amazon Kinesis with the AWS.NET API Part 6: storage
The raw data storage: S3
- Using Amazon S3 with the AWS.NET API Part 1: introduction
- Using Amazon S3 with the AWS.NET API Part 2: code basics
- Using Amazon S3 with the AWS.NET API Part 3: code basics cont’d
- Using Amazon S3 with the AWS.NET API Part 4: working with folders in code
- Using Amazon S3 with the AWS.NET API Part 5: S3 in Big Data
- Using Amazon S3 with the AWS.NET API Part 6: S3 in Big Data II
Data storage alternative: DynamoDb
- Using Amazon DynamoDb with the AWS.NET API Part 1: introduction
- Using Amazon DynamoDb with the AWS.NET API Part 2: code beginnings
- Using Amazon DynamoDb with the AWS.NET API Part 3: table operations
- Using Amazon DynamoDb with the AWS .NET API Part 4: record insertion
- Using Amazon DynamoDb with the AWS .NET API Part 5: updating and deleting records
- Using Amazon DynamoDb with the AWS .NET API Part 6: queries
- Using Amazon DynamoDb with the AWS .NET API Part 7: its place in Big Data
Data mining and analysis tool: Elastic MapReduce
- Using Amazon Elastic MapReduce with the AWS.NET API Part 1: introduction
- Using Amazon Elastic MapReduce with the AWS.NET API Part 2: the cluster startup GUI
- Using Amazon Elastic MapReduce with the AWS.NET API Part 3: starting and logging into a cluster
- Using Amazon Elastic MapReduce with the AWS.NET API Part 4: Hive basics with Hadoop
- Using Amazon Elastic MapReduce with the AWS .NET API Part 5: cluster-related code
- Using Amazon Elastic MapReduce with the AWS .NET API Part 6: Hive with Amazon S3 and DynamoDb
- Using Amazon Elastic MapReduce with the AWS .NET API Part 7: indirect Hive with .NET
- Using Amazon Elastic MapReduce with the AWS .NET API Part 8: connection to our Big Data demo
Data mining and analysis tool: RedShift
- Using Amazon RedShift with the AWS .NET API Part 1: introduction
- Using Amazon RedShift with the AWS .NET API Part 2: MPP definition and first cluster
- Using Amazon RedShift with the AWS .NET API Part 3: connecting to the master node
- Using Amazon RedShift with the AWS .NET API Part 4: code beginnings
- Using Amazon RedShift with the AWS .NET API Part 5: connecting to master node using ODBC
- Using Amazon RedShift with the AWS .NET API Part 6: Postgresql to master node using ODBC
- Using Amazon RedShift with the AWS .NET API Part 7: data warehousing and the star schema
- Using Amazon RedShift with the AWS .NET API Part 8: data warehousing and the star schema 2
- Using Amazon RedShift with the AWS .NET API Part 9: data warehousing and the star schema 3
- Using Amazon RedShift with the AWS .NET API Part 10: RedShift in Big Data
AWS Big Data summary
Amazon CodePipeline
- Introduction to Amazon Code Pipeline with Java part 1: basics of CI/CD
- Introduction to Amazon Code Pipeline with Java part 2: setup
- Introduction to Amazon Code Pipeline with Java part 3: adding custom job runners
- Introduction to Amazon Code Pipeline with Java part 4: comparison with TeamCity and Jenkins
- Introduction to Amazon Code Pipeline with Java part 5: architecture key terms
- Introduction to Amazon Code Pipeline with Java part 6: third party action overview
- Introduction to Amazon Code Pipeline with Java part 7: the third party action user signup process
- Introduction to Amazon Code Pipeline with Java part 8: the job agent communication process
- Introduction to Amazon Code Pipeline with Java part 9: the job agent continuation token
- Introduction to Amazon Code Pipeline with Java part 10: the client web pages
- Introduction to Amazon Code Pipeline with Java part 11: starting with the job agent
- Introduction to Amazon Code Pipeline with Java part 12: the job agent entry point in code
- Introduction to Amazon Code Pipeline with Java part 13: the client token lookup service
- Introduction to Amazon Code Pipeline with Java part 14: the loadtest executor service
- Introduction to Amazon Code Pipeline with Java part 15: the job processor interface and related objects
- Introduction to Amazon Code Pipeline with Java part 16: the WorkResult object
- Introduction to Amazon Code Pipeline with Java part 17: the JobProcessor implementation
- Introduction to Amazon Code Pipeline with Java part 18: the job worker thread
- Introduction to Amazon Code Pipeline with Java part 19: the job worker initialisation and the job poller interface
- Introduction to Amazon Code Pipeline with Java part 20: the job worker configuration
- Introduction to Amazon Code Pipeline with Java part 21: the job service
- Introduction to Amazon Code Pipeline with Java part 22: the job poller implementation
- Introduction to Amazon Code Pipeline with Java part 23: job worker daemon wrap-up
- Introduction to Amazon Code Pipeline with Java part 24: summary
Geo-spatial services
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 1: introduction and goals
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 2: MaxMind source files
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 3: IPv4 range strategy
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 4: lng/lat range strategy
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 5: creating the IPv4 source file for DynamoDb
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 6: uploading IPv4 range to DynamoDb
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 7: querying the IPv4 range table
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 8: creating the lng/lat coordinates source file for DynamoDb
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 9: uploading the co-ordinate range to DynamoDb
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 10: querying the coordinate range table
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 11: uploading the geolocation range to DynamoDb
- Using Amazon DynamoDb for IP and co-ordinate based geo-location services part 12: querying the geolocation range to DynamoDb
AMIs
- How to manage Amazon Machine Images with the .NET Amazon SDK Part 1: starting an image instance
- How to manage Amazon Machine Images with the .NET Amazon SDK Part 2: monitoring and terminating AMI instances, managing Security Groups
Data Pipeline
Elastic Beanstalk
Exercises in .NET with Andras Nemes 