AWS Compute
1) Elastic Compute Cloud (EC2)
- Cloud Computing Service. Choose your OS, Storage , Memory, Network Throughput. Launch and SSH into your server within minutes.
- Elastic compute Cloud (EC2) is a highly configurable server.
- EC2 is resizable compute capacity. It takes minutes to launch new instances.
- Anything and everything on AWS uses EC2 Instance underneath.
- Choose your OS via Amazon Machine Image (AMI)
* RedHat
* Ubuntu
* Windows
* Amazon Linux
* Suse
- Choose you Instance Type
* t2.nano - $0.0065/hour ($4.75/month), 1vCPU 0.5GB Mem
* c4.8xlarge - $1.591/hour ($1161.43/month), 36vCPU, 60GB Mem , 10 Gigabit performance
- Add Storage (EBS, EFS) - SSD, HDD, Virtual Magnetic Tape, Multiple Volumes
- Configure your Instance - Security Groups, Key Pairs, UserData, IAM Roles, Placement Group
-
a) EC2 - Instance Types and Usage
- General Purpose
* A1, T3, T3a, T2, M5, M5a, M4
* balance of compute, memory and networking resources
* Use-cases web servers and code repositories
- Compute Optimized
* C5, C5n, C4
* Ideal for compute bound applications,that benefit from high performance processor
* Use cases scientific modeling, dedicated gaming servers and ad server engines
- Memory Optimized
* R5, R5a, X1e, X1, High Memory, z1d
* fast performance for workloads that process large data sets in memory.
* Use-cases in-memory caches, in-memory databases, real time big data analytics
- Accelerated Optimized
* P3, P2, G3, F1
* hardware accelerators, or co-processors
* Use-cases Machine learning, computational finance, seismic analysis, speech recognition
- Storage Optimized
* I3, I3en, D2, H1
* high, sequential read and write access to very large data sets on local storage
* Use-cases NoSQL, in-memory or transactional databases , data warehousing
b) EC2 - Instance Sizes
- EC2 Instance Sizes generally double in price and key attributes
Name vCPU RAM (GIB). On-Demand per hour On-Demand per month
t2. small . 1 12 $0.023 $16.79
t2.medium 2 24 $0.0464 $33.87
t2.large 2 36 $0.0928 $67.74
t2.xlarge 4 54 $0.1856 $135.48
c) EC2 - Instance Profile
- Instead of embedding your AWS credentials (Access Key and Secret) in your code so your Instance has permissions to access certain services you can Attach a role to an instance via an Instance Profile
-You want to always avoid embedding your AWS credentials when possible.
- IAM Policy -> IAM Role -> Instance Profile <- EC2 Instance
- An Instance Profile holds a reference to a role. The EC2 instance is associated with the Instance Profile. When you select an IAM role when Launching an EC2 instance, AWS will automatically create the Instance Profile for you. Instance Profiles are not easily viewed via the AWS Console.
d) EC2 - Placement Groups
- Placement Groups let you to choose the logical placement of your instances to optimize for communication, performance or durability. Placement groups are free.
- Cluster
* packs instances close together inside an AZ
* low-latency network performance for tightly-coupled node-to-node communication
* well suited for High Performance Computing (HPC) applications
* Clusters cannot be multi-AZ
-Partition
* spreads instances across logical partitions
* each partition do not share the underlying hardware with each other (rack per partition)
* well suite for large distributed and replicated workloads (Hadoop, Cassandra, Kafka)
- Spread
* Each instance is placed on a different rack
* When critical instances should be keep separate from each other
* You can spread a max of 7 instances. Spreads can be multi-AZ
-
e) EC2 - UserData
- You can provide an EC2 with UserData which is a script that will be automatically run when launching an EC2 instance. You could install package, apply updates or anything you like.
-From within the EC2 instance, if you were to SSH in and CURL this special URL you can see the UserData script eg. curll http://169.254.169.254/latest/user-data
f) EC2 - MetaData
- From within your EC2 instance you can access information about the EC2 via a special url endpoint at 169.254.169.254.
- You would SSH into your EC2 instance and can use the CURL command: curl http://169.254.169.254/latest/meta-data
/public-ipv4 - get the current public IPv4 address
/ami-id - the AMI ID used to launch this EC2 instance
/instatnce-type - the Instance Type of this EC2 instance
-Combine metadata with userdata scripts to perform all sorts of advanced AWS staging automation
g) EC2 - CheatSheet
- Elastic Compute cloud (EC2) is a Cloud Computing Service
- Configure your EC2 by choosing your OS, Storage, Memory, Network Throughput
- Launch and SSH into your server within minutes.
- EC2 comes in variety Instance Types specialized for different roles:
* General Purpose - balance of compute, memory and networking resources
* Compute Optimized - ideal for compute bound appliations that benefit from high performance processor
* Memory Optimized - fast performance for workloads that process large data sets in memory
* Accelerated Optimized - hardware acceletrators, or co-processors
* Storage Optimized - high, sequential read and write access to very large data sets on local storage
- Instance Sizes generally double in price and key attributes
- Placement Groups let you to choose the logical placement of your instances to optimize for communication, performance or durability. Placement groups are free.
- UserData a script that will be automatically run when launching an EC2 instance.
- Metadata meta data about the current instance. You access this meta data via a local endpoint when SSH'd into the EC2 instance. eg. curl http://169.254.169.254/latest/meta-data meta data could be the instance type, current ip address etc..
- Instance Profiles a container for an IAM role that you can use to pass role information to an EC2 instance when the instance starts.
h)
i)
j)
k)
l)
m)
n)
2) EC2 - AMI
- Amazon Machine Image (AMI) provides the information required to launch an instance.
- You can turn your EC2 instances into AMIs so you can create copies of your servers
- An AMI holds the following information
* A template for the root volume for the instance (EBS Snapshot or Instance Store template) eg. an operating system, an application server, and applications
* Launch permissions that control which AWS accounts can use the AMI to launch instances.
* A block device mapping that specifies the volumes to attach to the instance when it's launched.
- AMIs are Region Specific
a) AMI - Use Cases
- AMIs help you keep incremental changes to your OS, application code and system packages.
- Using System Manager Automation you can routinely patch your AMIs with security updates and bake those AMIs
- AMIs are used with LaunchConfigurations. When you want to roll out updates to multiple instances you make a copy of your LaunchConfiguration with new AMI
b) AWS Marketplace
-
c) AMI - Choosing an AMI
- AWS has hundreds of AMIs you can search and select from.
- Community AMI are free AMIs maintained by the community.
- AWS Marketplace free or paid AMIs maintained by vendors.
- AMIs have an AMI ID. AMIs are region specific. Will have different AMI ID per region.
- Amazon Machine Images can be selected based on:
* Region
* Operating System
* Architecture (32-bit or 64-bit)
* Launch Permissions
* Root Device Volume
** Instance Store (Ephemeral Storage)
** EBS Backed Volumes
- AMIs are categorized as either backed by Amazon EBS, or backed by Instance Store.
d) AMI - Creating an AMI
- You can create an AMI from an existing EC2 instance that's either running or stopped.
e) AMI - Copying an AMI
- AMIs are region specific. If you want to use an AMI from another region. You need to Copy the AMI and then select the destination region.
f) AMI CheatSheet
- Amazon Machine Image (AMI) provides the information required do launch an instance
- AMIs are region specific, if you need to use an AMI in another region you can copy an AMI into the destination region via Copy AMI
- You can create an AMI from an existing EC2 instance that's either running or stopped.
- Community AMI are free AMIs maintained by the community
- AWS Marketplace free or paid subscription AMIs maintained by vendors
- AMIs have an AMI ID. The same AMI eg. (Amazon Linux 2) will vary in both AMI ID and options eg. Architecture options in different regions.
- An AMI holds the following information.
* A template for the root volume for the instance (EBS Snapshot or Instance Store template) eg. an operating system, an application server, and applications.
* Launch permissions that control which AWS accounts can use the AMI to launch instances.
* A block device mapping that specifies the volumes to attach to the instance when it's launched.
g)
h)
i)
j)
k)
l)
m)
n)
o)
p)
r)
s)
t)
u)
w)
x)
y)
z)
3) EC2 Autoscaling Groups (ASG)
- Set scaling rule which will automatically launch additional EC2 instance or shutdown instances to meet current demand
- Auto Scaling Group (ASG) contains a collection of EC2 instances that are treated as a group for the purposes of automatic scaling and management.
-Automatic scaling can occur via:
* 1. Capacity Settings
* 2. Health Check Replacements
* 3. Scaling Policies.
-
a) ASG - Capacity Settings
- The size of an Auto Scaling Group is based on Min, Max and Desired Capacity.
- Min is how many EC2 instances should at least be running.
- Max is number EC2 instances allowed to be running.
- Desired Capacity is how many EC2 instances you want to ideally run.
- ASG will always launch instances to meet minimum capacity.
b) ASG - Health Check Replacement
- EC2 Health Check Type
ASG will perform a health check on EC2 instances to determine if there is a software or hardware issue. This is based on the EC2 Status Checks. If an instance is considered unhealthy. ASG will terminate and launch a new instance.
- ELB Health Check Type
ASG will perform a health check based on the ELB health check. ELB can perform health checks by pinging an HTTP(S) endpoint with an expected response. If ELB determines a instance is unhealthy it forwards this information to ASG which will terminate the unhealthy instance.
c) ASG - Scaling Policy
- Scaling Out: Adding More Instances
- Scaling In: Removing Instances
- Target Tracking Scaling Policy
Maintains a specific metric at a target value.
eg. If Average CPU Utilization exceeds 75% then add another server.
- Simple Scaling Policy
Scales when an alarm is breached.
- Not recommended, legacy scaling policy. Use scaling policies with steps now.
- Scaling policies with steps
Scales when an alarm is breached , can escalates based on alarm value changing.
d) ASG - ELB Integration
- ASG can be associated with Elastic Load Balancers (ELB). When ASG is associated with ELB richer health checks can be set.
- Classic Load Balancers are associated directly to the ASG
- Application and Network Load Balancers are associated indirectly via their Target Groups.
ASG -> Target Group -> ALB, NLB
e) ASG - Use Case
- 1. Burst of traffic from the internet hits our domain.
- 2. Route53 points that traffic to our load balancer.
- 3. Our load balancer passes traffic to its target group.
- 4. The target group is associated with our ASG and sends the traffic to instances registered with our ASG
- 5. The ASG Scaling Policy will check if our instances are near capacity.
- 6. The Scaling Policy determines we need another instance, and it Launches and EC2 instance with the associated Launch Configuration to our ASG.
Internet -> Route53 -> LB -> Target Group -> ASG -> Scaling Policy -> Launch Configuration
f) Launch Configuration
- A launch configuration is an instance configuration template that an AutoScaling group uses to launch EC2 instances.
- A Launch Configuration is the same process as Launching an EC2 instance except you are saving that configuration to Launch an Instance for later. Hence Launch Configuration.
- Launch Configurations cannot be edited. When you need to update your Launch Configuration you create a new one or clone the existing configuration and then manually associate that new Launch Configuration.
- Launch Templates are Launch Configurations with Versioning, Everyone appears to still use Launch Configuration
g) EC2 AutoScaling Groups CheatSheet
- An ASG is a collection of EC2 instances grouped for scaling and management
- Scaling Out is when add servers
- Scaling I. is when you remove servers
- Scaling Up is when you increase the size of an instance (eg. updating Launching Configuration with larger size)
- Size of an ASG is based on a Min, Max and Desired Capacity
- Target Scaling Policy scales based on when a target value for a metric is breached eg. Average CPU Utilization exceed 75%
-Simple Scaling policy triggers a scaling when an alarm is breached
- Scaling Policy with Steps is the new version of Simple Scaling policy and allows you to create steps based on education alarm values.
- Desired Capacity is how many EC2 instances you want to ideally run
- An ASG will always launch instances to meet minimum capacity
- Health checks determine the current state of an instance in the ASG
- Health checks can be run against either an ELB or the EC2 instances
- When an Autoscaling launches a new instance it uses a Launch Configuration which holds the configuration values for that new instance eg. AMI, InstanceType, Role
- Launch Configurations cannot be edited and must be cloned or a new one created
- Launch Configurations must be manually updated in by editing the AutoScaling settings.
h)
i)
j)
k)
l)
m)
n)
o)
p)
r)
s)
t)
u)
w)
x)
y)
z)
5)
6)
7)
8)
9)
10)
11)
12)
13)
14)
15)
Komentarze
Prześlij komentarz