Hi, I am Vladimir Chmuzh, Full Stack Web developer & Solution Architect at NIX United. I have been in web development for seven years, and I have experience interacting with cloud platforms. The attention to cloud technologies is growing every year. If you are planning to spend your life in software development, it is very likely that you will have to deal with one cloud platform or another. Amazon Web Services is by far the most popular of these.
This article will give you a basic understanding of the main AWS features and will be useful for those who are just starting to get acquainted with the platform. Let's go!
If One Day Amazon Didn't Exist, Every Internet User Would Feel It
Amazon Web Services (AWS) is an online platform that provides users with virtual computing resources, storage, infrastructure, and services that contain functionality that is ready to use. Instead of buying a physical server and installing it locally, you rent virtual resources online.
AWS services are trusted by major corporations that choose the platform to serve workloads, including mobile and web applications, game development, information processing, and storage. If AWS were to suddenly shut down, many internet users would be affected. Popular sites and services would cease to function as they make extensive use of AWS power.
The chart below shows that AWS has 32% of the market for all cloud platforms. In second place is Microsoft Azure, and in third place is Google.
AWS has almost half of the market for all cloud platforms / Canalys
Using the AWS platform, Coursera can handle about 500 Tbytes of traffic per month. In 2009, Netflix became the most prominent first user of AWS among large corporations and now uses over 100,000 server instances on AWS.
There is a long list of successful examples of using AWS’s capabilities. But before we proceed into a detailed analysis of these services, let's look at the main ways of interacting with the platform.
How to Manage the AWS Platform
AWS is accessed and managed through the AWS Console, a site where resources can be managed through a browser. This is one way to interact, but there are others:
SDK is a type of software interaction with the platform. SDKs are implemented for such popular programming languages as C++, Go, Java, JS, .NET, PHP, Python, Ruby. If there is no official SDK for the language you are writing code in; there is probably a library that is supported by the community. Or you can write your own SDK, which will be a kind of wrapper to the HTTP interface provided.
AWS CLI is a command-line utility that is useful for automating processes. It is also suitable for those who are more comfortable using the command line rather than a graphical interface.
The 2 Main Advantages of AWS
1. Doesn't require a big investment
If you want to launch your startup and create a messenger, you will need:
- Servers that will carry out the communication
- Server with a database that will store the history of messages
- Servers for storing files so that users can share images with each other
All of this can cost quite a lot of money. In this case, you will not have any guarantees that the business will be successful and you will not lose what you have invested.
If you buy a physical server and pay for it, you probably won't be using it 24/7 and at full capacity. If you use a cloud platform, however, your virtual machine can share physical hardware with other users' virtual machines. This ensures its lower cost.
In the case of AWS, you can rent only those resources and capacities that you need right now and pay for them by the hour or by the number of requests executed.
If you notice that something is going wrong, you can simply shut down all the servers and resources that have been engaged. This way you don't risk losing a huge investment.
2. Provides the largest infrastructure amongst all cloud providers
Today, the AWS infrastructure consists of 25 regions, each of which has several availability zones. These are one or more data centers.
AWS offers many cloud services that can be combined to meet your business and organizational needs.
You can easily disable cloud servers and resources you don't use
17 Basic AWS Services
AWS offers a huge number of services. These include services for various calculations, file storage systems and databases, virtual networks, services for machine learning and AI, container orchestration, and media services. Below I will describe the most basic ones, in my opinion, which are useful for web developers.
1. Identity Access Management in AWS
Identity Access Management (IAM) is a service without which teamwork in AWS is impossible. It allows you to create and manage user access levels.
When you sign up for AWS, you create an account. Within that account, you can form various resources. To ensure that your team members also have access to these resources, you create users and assign them to each other.
For example, you can combine them into separate groups—administrators, testers, or developers. You can then create policies that define access and permissions within your account. For example, you can create a policy that allows you to read and modify all resources and pass this policy to the Administrator group. Or you can allow all resources to be read but forbid them to be modified, and assign this policy to the testers group.
Another entity you can create is roles. You create users for people and roles for services. For example, you have a web server, and that means you can create a role for it that allows access to the database. This way you will allow access to the database, but other services won't have it.
2. AWS Simple Storage Service (S3)
Simple Storage Service (S3) is an object storage service that offers automatic scalability.
S3 stores copies of the data on different physical devices, which ensures high availability and reliability. The S3 provides an HTTP interface to access and manage data.
This is exactly the kind of service we would use if we created our own version of Dropbox or Google Drive. It can also be used to store static site files, for example. S3 provides limitless storage that expands automatically. You won't have to worry about the size available for files. You'll be able to store files up to five terabytes in size.
There are also different storage classes. For example, if you have a file and access it infrequently, you can put it in the infrequent access storage class and you will pay significantly less for storing data. The extra charge, in this case, depends on the number of requests made. The current prices can be seen at this link.
Files can be archived using the S3 glacier service. For example, if you are required by law to keep data for at least three years, this service is just what you need. It will reduce the cost of payment. But there is a disadvantage—it can take several hours to get access to the data.
S3 provides a versioning function. Here’s how it works: after changing files, you can always go back to previous versions and restore them. Lifecycle management allows you to assign a file that, once created, will go into the archive and, after two years, will be completely deleted. S3 supports multi-factor authentication for file deletion. This means that you can configure the repository so that the user can delete files only after passing multi-factor authentication.
3. AWS CloudFront
CloudFront is essentially the Content Delivery Network, a system of servers around the world that caches static content.
A user who wants to receive static content is directed by the system to the nearest server, which will provide the data promptly. Such servers are called edge locations. Their system exists independently of regions and availability zones. This is an additional layer of Amazon's infrastructure. The system works not only to give out files quickly but also to receive them.
If you have an S3 bucket in Europe and a user is trying to download a file from the U.S., the S3 Transfer Acceleration functionality will route it to the nearest edge location, allowing the file to upload to the platform faster, and then it will get to the right region via Amazon's dedicated, high availability, low-latency private network.
4. AWS Elastic Computer Cloud (EC2)
Elastic Computer Cloud (EC2) is a service that allows you to create virtual machines with different operating systems and install software on them.
It takes only a few minutes to create such a virtual machine. EC2 offers different types and configurations of servers. For example, they are customized for specific computing tasks with powerful processors. Another option is servers for working with large amounts of data.
EC2 allows you to connect different virtual storages, e.g. general-purpose or high-bandwidth ones, depending on your needs. You can read more about this here.
EC2 allows you to create bootstrap scripts that will be used during the virtual machine creation phase. You can use these scripts to make presets and install the necessary software. You can also make snapshots of the state of the virtual machine at a certain time. These are used to create the desired image of the virtual machine, and then make a copy with the software and settings already installed.
EC2 Offers Different Price Models:
- OnDemand server - you pay for servers by the hour and only when they are running.
- Reserved server - you enter into a contract with Amazon for one or three years to use the server. By doing so, you agree to use the server within the specified time frame. There are discounts depending on the term of use.
- Spot server - the price can vary, but you set a bar above which you are not willing to pay. This can be useful, for example, for calculations for which it is not very important that they be done continuously and strictly on time, but they are unprofitable at a price above some level.
- Dedicated server - allows you to install a virtual machine on the same physical hardware. This is useful if, for example, the license requires you to use the software on a specific machine. It could be that you purchased the software and registered it on a particular machine, and then you can no longer use the same license. New hardware requires a new license.
5. AWS Elastic Load Balancer
Elastic Load Balancer (ELB) is a service that allows you to adjust and distribute the load between existing servers.
Load Balancer can periodically check servers for availability. If a server fails and does not respond, Elastic Load Balancer stops sending traffic to it and distributes it among the remaining servers.
Amazon offers several types of ELBs:
- Application - allows flexible routing based on HTTP parameters.
- Network - Without advanced routing, allows you to distribute traffic equally. It executes requests quickly, handling millions of requests per second.
- Classic - an obsolete type of LB. For new systems, it is better to use Application LB or Network.
6. Auto Scaling with AWS
Auto Scaling - a service that allows you to create groups of servers and automatically manage their number.
Auto Scaling can support a minimal, yet the necessary number of instances. It allows you to scale manually or configure by the event. For example, if the average CPU utilization exceeds 60%, then you need to add servers. Scalability by schedule is also available. Imagine: you have launched a promotion in your online store. You believe that a huge influx of customers will be exactly on Sunday. This means that on this day it is necessary to scale twice.
7. AWS Cloud Watch
Cloud Watch is a service for monitoring your resources in AWS.
This service allows you to monitor hundreds of different predefined metrics (e.g., CPU, disk, network, and so on) and identify individual metrics. CloudWatch also allows you to collect and analyze logs of your entire infrastructure, applications, and services in one place.
Based on the information collected, you can define "alarms" for certain events. For example, create a rule like this: if the average CPU load on your server for a certain period of time does not fall below 80%, CloudWatch will notify about it with an e-mail or SMS to your phone. Or if you have a certain budget for using Amazon's resources, you can set an alarm in the same way, which will warn you that you have exceeded your limit.
8. AWS Relational Database Service
Relational Database Service (RDS) is a relational database service from Amazon that supports several engines: MySQL, MariaDB, PostgreSQL, Oracle, and Microsoft SQL.
If you use Amazon RDS, the administration of the software is not your responsibility. All necessary software for the database system is already installed, managed, and supported by the AWS platform. RDS provides Multi-Availability Zones functionality for disaster recovery. For example, you can create a copy of your database on a separate server in a different availability zone. This database will be constantly in sync with the primary instance. When the primary server goes down, RDS will automatically switch all requests to the copy located in the other zone. The performance of the application will not be affected.
There is also Read Replicas functionality, which allows you to create copies of the database to scale performance. Let's say there is one replica that can't handle the load. To solve this problem, you need to create one additional or multiple replicas that can distribute the read load among themselves and increase performance.
9. Amazon Aurora
Amazon Aurora is a managed, relational database engine compatible with PostgreSQL and MySQL that offers automatic scaling. Amazon claims that its performance is five times that of MySQL and three times that of PostgreSQL.
With automatic scaling, you don't have to constantly monitor your remaining space. For high availability and security, by default, all of your data is stored as six copies, two each, in three availability zones. If there is a fire in any of the data centers, Aurora will continue to operate.
Aurora Serverless additionally offers automatic scaling of instance power. For example, depending on the current load, the database server can use one vCPU or eight. The automatic start/stop functionality allows you to stop the server if, for example, there are no requests for five minutes.
10. DynamoDB by Amazon
DynamoDB is Amazon's primary NoSQL solution. DynamoDB is able to automatically scale up to huge sizes without performance loss.
Copies of the data are stored in three physically different locations, providing high availability out of the box. It is important to understand that despite its speed and easy scalability, NoSQL will not provide you with the kind of language power, query power, and data structure rigor that a relational database has. It might not be worth using DynamoDB as a database for finance or health applications.
11. Route53 within AWS infrastucture
Route53 is a DNS service that allows you to register a domain name and bind it to different targets within the AWS infrastructure—for example, EC2 instance, Load Balancer, or any other IP outside of Amazon. Route53 offers different types of routing:
- Emergency - if any of the tags does not respond, Route53 stops sending traffic
- By latency - the service redirects traffic to the target that responds the fastest to the user
- By geographic location - Route53 selects the target that is geographically closer to the user
You can read more about all kinds of routing at this link.
12. Amazon Cognito
Cognito is an authentication and authorization service.
It supports modern standards OAuth 2.0, OpenID Connect, and SAML 2.0, and offers registration functionality, multi-factor authentication via email/SMS, a ready-to-use user interface, integration with third-party providers (Facebook, Google, etc.), and Microsoft Active Directory.
13. Amazon Simple Queue Service (SQS)
Simple Queue Service (SQS) is a service for exchanging messages via queues.
Allows for elasticity and weak connectivity between components (services) of the system. RabbitMQ is a popular open source analog to SQS.
SQS offers two types of queues:
- Standard - faster and more scalable option, but the order of message delivery is not guaranteed
- FIFO (first in-first out) - guarantees the order of messages, but its speed is limited to 3000 messages per second
14. AWS Simple Notification Service
Simple Notification Service (SNS) is a messaging service that works on the publisher-subscriber model.
Suppose you have a publisher who sends a message to an SNS topic. SNS sends the message to several subscribers of the topic. The role of subscribers can be a SQL queue, e-mail, or push notification in a mobile app (you can read more about it here).
Here is an example of the use of SNS: Let's say that a new user is registered in the system. We send a message about this action to SNS. The service sends a message to the user with the text that the registration was successful. At the same time, the message is sent to the SQS-queue, where the data is consumed, processed, and sent to the repository for further analysis. The message is also sent in parallel to another SQS queue, the processor of which will try to obtain additional data about the user from a third-party service on the basis of the provided information.
15. AWS Lambda
Lambda is a service that allows you to execute certain code as a result of events, without worrying about configuring and maintaining infrastructure. In this case, the developer is abstracted from such a notion as a server and can focus more on the implementation of their business tasks.
The developer writes code, uploads it to AWS, and creates a Lambda function based on it. He then defines a trigger (event) that will trigger the created function. The code must contain the main handler function (entry point), which will receive details about the event that triggered the function as arguments.
Lambda supports different runtimes: Node.js, Python, Ruby, Java, Go, .NET. Custom runtime allows you to customize your runtime and use any other programming languages (more about that here).
16. AWS API Gateway
AWS API Gateway allows you to define HTTP RESTful APIs or websocket interfaces for other AWS resources.
You can define HTTP endpoints or websocket routes to different backends, such as AWS Lambda, SQS, or DynamoDB.
API Gateway is a managed service that requires no configuration or infrastructure support. At the same time, it provides scalability and fault tolerance. Logging, throttling of requests, and caching are also supported out-of-the-box. At the Gateway API level, authorization can be defined based on integration with the aforementioned AWS Cognito. And based on the Gateway and Lambda APIs, you can create a serverless application.
The point is that Lambda can load not only a simple function but also a full-fledged web application. In this case, the Gateway API will work as a proxy, redirecting requests to your application's pipeline.
17. AWS CloudFormation
CloudFormation is a service for automating infrastructure deployment.
You describe the resources that make up your system in a YML or JSON file. You can recreate copies of the system from this file. CloudFormation is convenient in that the entire system is managed as a single entity. You can delete all the system resources, update, or recreate a copy with almost one command.
Materials for AWS Self-study
There are several ways to learn about AWS. You can use technical descriptions, manuals, online courses, and videos. From personal experience, I've found that my preparation for AWS Certified Solutions Architect certification helped me get a fairly basic understanding of the platform. So I advise you to check out this course, even if you are not going to get certified in the near future. The course is an overview and will help you learn the basic components of AWS and prepare for testing for the AWS Solution Architect role in the future.
Also among other sources for training, I recommend ADV-IT, a YouTube channel where both beginners and experienced developers can find information about AWS services, the main ways to interact with the platform, as well as the infrastructure of cloud computing.
And in general, you can find many useful materials on the official Amazon Web Services site. Good luck!