AWS Series — EC2 Technical Essentials
We humans communicate using our senses. How do you think computers communicate?
Consider the Linux machine, we do SSH over the Port 22 to communicate with linux remotely. Windows we use RDP and port 3389 and HTTP which is not encrypted then using web browsing at Port 80. Finally, for HTTPS encrypted web browsing (SSL) we use the port 443.
This is how computers communicate.
Security Groups
Security groups are virtual firewalls for your EC2 instance. By default, everything is blocked. To let everything we use 0.0.0.0/0
In order to be able to communicate to your EC2 instances via SSH/RDP/HTTP, you will need to open up the correct ports.
Bootstrap Scripts
A script that runs when the instance first runs
Adding these tasks at boot time adds to the amount of time it takes to boot the instance. However, it allows you to automate the installation of applications.
When the Ec2 instance is first created and boots the server, it automatically loads all the applications in the bootstrap script and then give the machine.
Let us give for example a bootstrap script
This installs the apache to communicate, starts the apache and then create a basic HTML file called index.html.
Where can we provide these bootstrap scripts while launching EC2 instance?
On clicking the Launch the instance and we select the AMI from the list and choose Instance Type. Next we have to configure Instance Details.
Go to Advanced Details and paste the bootstrap scripts in the User data
Then we can have the settings same as default. Create a new Security Group. When we create > choose the Add Rule and add the HTTP and HTTPS and the port is auto populated. This will enable the communication using these protocols and saves sometime adding them later.
Launch the instance. In which the bootstrap scripts is already run with opening the ports.
This is how we automate the process of installing the services on start up.
EC2 Metadata
EC2 metadata is simply data about your EC2 instance. This can include information such as private IP address, public IP address, hostname, security groups etc.,
In order to retrieve metadata, we can use curl command, we can query metadata about our EC2 instance.
As you can see the curl command we can query information about EC2 instance.
Applications of EC2 Metadata
You might be wondering how this is useful. We already discussed about bootstrap scripts which will execute some scripts that are required for set up. What if we need some of EC2 instance data to be used in my HTML file. We can retrieve and pass the data to html in bootstrap scripts. Example is shown below.
In this simple bootstrap ( user data) script, we use the curl command to save our EC2 metadata. where it is extracting public IP address and passing to the html.
Networking with EC2
You can attach 3 different types of virtual networking cards to your EC2 Instances
ENI — Elastic Network Interface — For basic, day-to-day networking
An ENI is simply a virtual network card that allows:
- Private IPv4 Addresses
- Public IPv4 Address
- Many IPv6 Addresses
- MAC Address
- 1 or more Security Groups.
Common ENI Use Cases —
- Create a management network
- Use network and security applicances in your VPC
- Create dual-homed instances with workloads/roles on distinct subnets
- Create a low-budge, high- availability solution
EN — Enhanced Networking — Uses single root I/O virtualization (SR-IOV) to provide high performance
For High-performance networking between 10 Gbps — 100 Gbps
Single root I/O virtualization ( SR-IOV) — Provides higher I/O performance and lower CPU utilization
Performance — Provides higher bandwidth, higher packet per second (PPS) performance, and consistently lower inter-instance latencies.
Depending on your instance type, enhance networking can be enabled using:
Elastic Network Adapter(ENA) — Supports network speeds of up to 100 Gbps for supported instance types.
INTEL 2599 Virtual Function (VF) Interface — Supports network speeds of up to 10 Gbps for supported instance types. Typically used on older instances.
EFA — Elastic Fabric Adapter — Accelerates High Performance and Computing (HPC) and machine learning applications
A network device you can attach to your Amazon EC2 instance to accelerate High performance Computing(HPC) and machine learning applications
Provides lower and more consistent latency and higher throughput than the TCP transport traditionally used in cloud-based HPC systems.
EFA can use OS-Bypass — It makes it a lot faster with much lower latency. OS-bypass enables HPC and machine learning applications to bypass the operating system kernel and communicate directly with the EFA device. Not currently supported with Windows — Only linux.
EC2 Placement Groups
There are 3 types of Placement Groups
Cluster Placement Groups
Grouping of instances within a single availability zone. Recommended for applications that need low network latency, high network throughput, or both
Only certain instance types can be launched into a cluster placement group.
Spread Placement Groups
A spread placement group is a group of instances that are each placed on distinct underlying hardware.
Spread placement groups are recommended for applications that have a small number of critical instances that should be kept separate from each other.
Used for Individual instances
Partition Placement Group
Each partition placement group has its own set of racks. Each rack has its own network and power source. No two partitions within a placement group share the same racks, allowing you to isolate the impact of hardware failure within your application.
EC2 divides each group into logical segments called partitions.
Used for multiple instances which needs dedicated network and power source.
Solving Licensing issues with dedicated hosts
We have already looked at the use cases for each of the pricing options of Ec2 instances. Dedicated instances is one of the costly option. If you want to revisit go over this article.
If you want to choose for a situation where we have special licensing requirements, we should definitely go with Dedicated host.
An Amazon EC2 Dedicate Host is a physical server with EC2 instance capacity fully dedicated to your use. Dedicated hosts allow you to use your existing per-socket, per-core or per-VM software licenses, including Windows server, Microsoft SQL Server, and SUSE Linux Enterprise Server.
Points to remember —
Security Groups
- Changes to security groups take effect immediately
- You can have any number of EC2 instances within a security group
- You can have multiple security groups attached to EC2 instances
- All inbound traffic is blocked by default
- All outbound traffic is allowed.
Bootstrap Scripts
- A bootstrap script is a script that runs when the instance first runs.
- It passes user data to the EC2 instance and can be used to install applications ( like webs servers and databases) as well as do updates and more.
User Data Vs Metadata
- User data is simply bootstrap scripts
- Metadata is data about your EC2 instances
- You can use bootstrap scripts ( user data) to access metadata.
ENI
For basic networking. Perhaps you need a separate management network from your production network or a separate logging network, and you need to do this at a low cost. In this scenario, use multiple ENIs for each network
Enhanced Networking
For when you need speeds between 10 Gbps and 100 Gbps. Anywhere you need reliable, high throughput.
EFA
For when you need to accelerate High Performance Computing (HPC) and machine learning applications or if you need to do an OS-bypass.
Placement Groups
3 types of placement groups
Cluster Placement Group s— Low network latency, high network throughput
Spread Placement Groups — Individual critical EC2 instances on their own dedicated hardware
Partition Placement Groups — Multiple EC2 instances; HDFS, HBase and Cassandra
- A cluster placement group can’t span multiple Availability zones, whereas a spread placement group and partition plaement group can.
- Only certain types of instances can be launched in a placement group (compute optimized, GPU, memory optimized, storage optimized)
- AWS recommends homogeneous instances within cluster placement groups.
- You can’t merge placement groups
- You can move an existing instance into a placement group.Before you move the instance, the instance must be in stopped state. You can move or remove an instance using AWS CLI or AWS SDK, but can’t do it via console.
