AWS Deployment and Provisioning
1) AWS CloudFormation
- A Templating Language that defines AWS resources to be provisioned. Automating the creation of resources via code. Infrastructure as a Code.
- What is Infrastructure As Code? (IaC)
The process of managing and provisioning computer data centers (eg. AWS) through machine-readable definition files (eg. YAML, JSON files) rather that physical hardware configuration or interactive configuration tools. (stop doing manual configuration)
- Use Case
* People pay a monthly subscription and we run a Minecraft server. They choose where they want and what size of server they want to run.
* We can use their inputs and use an AWS Lambda to create a new CloudFormation stack. We have a lambda send them the email of their new Minecraft Server Ip address and details
- cfn-lint - tool to validate templates
cfn validate template.yml
-
a) CloudFormation - Template Formats
- CloudFormation can be written in two different formats: JSON, YAML.
-
b) CloudFormation - Template Anatomy
- Template Sections:
* Metadata - Additional information about the template
* Description - A description of what this template is suppose to do
* Parameters - Values to pass your template at runtime
* Mappings - A lookup table, Maps keys to values so you change your values to something else
* Conditions - Whether resources are created or properties are assigned
* Transform - Applies marcos (like applying a mod which change the anatomy to be custom)
* Resources - A resources you want to create eg. IAM Role , EC2 Instance, Lambda , RdS CloudFormation Templates requires you to at least list one resources.
* Outputs - Values that returned eg. an ip-address of new server created.
c) AWS Quick Starts
- AWS Quick Starts are a collection of pre-built CloudFormation templates.
d) CloudFormation - Stack Updates
- When you need to make a change to stack instead of deleteing and recreating the stack. You can modify the CloudFormation template and push a stack update and CloudFormation will intelligently change or delete and recreate resources.
-There are two ways to perform a stack update
* 1. direct update
** You directly update a stack, you submit changes and AWS CloudFormation immediately deploys the.
** Use direct updates when you want to quickly deploy your updates.
* 2. executing change set
** You can preview the changes to CloudFormation will make to your stack, and then decide whether to apply those changes.
** Use change sets when you want to ensure that CloudFormation doesn't make unintentional changes.
- When you submit an update, CloudFormation updates differently depending on the state of you're resources and circumstances:
* Update with No Interruption
** 1. Updates the resource without disrupting operation
** 2. and without changing the resource's physical ID
* Updates with Some Interruption
** 1. Updates the resource with some interruption
** 2. and retains the physical ID.
* Replacement
** 1. recreates the resource during an update.
** 2. also generates new physical ID.
-
e) CloudFormation - Prevent Stack Updates
- You may want certain resources not to be updates.
- Prevent a stack-update could be to prevent data loss or interruption to service.
- Stack Policy - A stack policy is a JSON document that defines the update actions that can be performed on designated resources.
{
Statement: [
{
Effect: Deny,
Action: Update:Replace,
Principal: *,
Condition: {
StringEquals : {
ResourceType : [AWS::DynamDB::Table ...
}
}
},
{
Effect: Allow,
Action: Update:*,
Principal: *,
Resource: *
}
}
}
}
f) CloudFormation - Nested Stacks
- Nested Stacks allows you to reference CFN templates inside of another CFN templates:
* 1. create modular templates (reusability)
* 2. assemble large templates (reduce complexibility)
- You can nest stacks onto nested stacks.
- Parent stacks have immediate access to child stacks.
- The root stack is accessible by all nested stacks.
- AWS::CloudFormation::Stack is used to include the template
- The nested template needs to reside in an S3 Bucket.
AWSTemplateFormaVersion: '2010-09-09'
Resources:
myStack:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: https://s3.amazonaws.com/my-cfn/s3_bucket.template
TimeoutInMinutes: '60'
g) CloudFormation - Rollbacks
- When you create, update or destroy a stack you could encounter and error: eg. Your CloudFormation template has a syntax error, your stack is trying to delete a resource which no longer exists.
- CloudFormation will attempt to Rollback in case of an error put your stack back to its previous State.
Rollback are turned on my default. You can ignore rollback by using the --ignore-rollback flag via the AWS CLI.
- Rollback can fail. Sometimes you'll have to investigate and change resource configuration or possibly need . To reach out to paid AWS Support to resolve the failed rollback
- Rollback status
* When a rollback is in progress you'll see ROLLBACK_IN_PROGRESS
* When a rollback is succeeds you'll see UPDATE_ROLLBACK_COMPLETE
* When a rollback is fails you'll see UPDATE_ROLLBACK_FAILED
h) CloudFormation - Drift Detection
- What is Drift?
Drift is when your stack's actual configuration differs (has drifted) by what CloudFormation expects.
- Why does Drift happen?
When developers start making manual ad-hoc changes to the stack (most common example is deleting resources) Instead of changing configuration by updating and relaunching the CloudFormation template.
- CloudFormation "Detect Drift" Feature
CloudFormation can detect drift and tell you if resources deleted or have been modified
- Select actions -> Detect drift
- Select actions -> View drift results
- Nest Stacks and Drift Detection
When detecting drift on a stack, CloudFormation does not detect drift on any nested stacks that belong to that stack. Instead, you can initiate a drift detection operation directly on the nested stack.
- Stack drift status
* DELETED - The resource has been deleted
* MODIFIED - There has been a change to the resource
* NOT_CHECKED - CloudFormation hasn't checked resource
* IN_SYNC - Everything is as expected
i) CloudFormation - Pseudo Parameters
- parameters that are predefined by AWS CloudFormation.
- You do not declare them in your template.
- Use them the same way as you would a paramter, as the argument for the Ref function.
Outputs:
MyStacksRegion:
Value: !Ref "AWS::Region"
- Pseudo parameters
* AWS:: Partition - Returns the partition that the resource is in.
* AWS::Region - Returns a string representing the AWS Region in which the encompassing resource is being created
* AWS::StackId - Returns the ID of the stack as specified with the aws coludformation create-stack command
* AWS::StackName - Returns the name of the stack as specified with the aws cloudformation create-stack command.
* AWS::URLSuffix - Returns the suffix for a domain. The suffix is typically amazonaws.com, but might differ by region.
j) CloudFormation - Resource Attributes
- Creation Policy
* prevent its status from reaching create complete until AWS CloudFormation receives a specified number of success signals or the timeout period is exceeded.
Resources:
Type: AWS::AutoScaling::AutoScalingGroup
CreationPolicy:
ResourceSignal:
Count: 3
Timeout: PT15M
- Deletion Policy
* reserve or (in some cases) backup a resource when its stack is deleted
* Delete, Retain or Snapshot
Resources:
MyDbInstance:
Type: AWS::RDS::DBInstance
DeletionPolicy: Snapshot
- UpdatePolicy
* How to handle an update for ASG, ElastiCache, Domain or Lambda Alias
UpdatePolicy:
AutoscalingReplacingUpdate:
WillReplace: True
- UpdatedReplacePolicy
* To retain or (in some cases) backup the existing physical instance of a resource when it is replaced during a stack updated operation.
* Delete, Retain or Snapshot
Resources:
MyDbInstance:
Type: AWS::RDS::DBInstance
UpdateReplacePolicy: Retain
- DependsOn
* That resource is created only after the creation of the resource specified in the DependsOn atrribute
Resources:
MyEC2Instance:
Type: AWS::EC2::Instance
DependsOn: MyDbInstance
MyDbInstance:
Type: AWS::RDS::DBInstance
k) CloudFormation - Intrinsic Functions
- Use intrinsic functions in your templates to assign values to properties that are not available until runtime.
* Fn::Base64 - returns the Base64 representation of the input string.
* Fn::Cidr - returns an array of CIDR address blocks
* Condition Functions - Fn::And Fn::Equals Fn::If Fn::Not Fn::Or
* Fn::FindInMap - returns the value corresponding to keys in a two-level map that is declared in the Mappings section
* Fn::Transform - a macro to perform custom processing on part of a stack template
* Fn::GetAZs - returns an array that lists Availability Zones for a specified region
* Fn::ImportValue - returns the value of an output exported by another stack
* Fn::Join - appends a set of values into a single value, separated by the specified delimiter
* Fn::Split - split a string into a list of string values so that you can select an element from the resulting string list
* Fn::Sub - substitutes variables in an input string with values that you specify
* Fn::GetAtt - returns the value of an attribute from a resource in the template
* Ref - returns the value of the specified parameter or resource
- Most commonly used Intrinsic functions - Fn::GetAtt, Ref
-
l) CloudFormation - Intrinsic Functions - Ref
- Ref returns different things for different resources.
- You need to look up each resource in the AWS Docs to what it returns: eg. ARN, Resource Name, Physical ID
- Ref can reference parameters or other defined resources
- Ref in the case with return the CIDR block
AWSTemplateFormatVersion: 2010-09-09
Description: >-
Sets up Networking Intfrastructure
Parameters:
AddressVPC:
Description: 'CIDR for VPC'
Type: String
MinLength: '9'
MaxLength: '18'
AllowedPattern: '(\d{1,3}\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})/(\d{1,2})'
ConstraintDescription: must be a valid IP CIDR range of the form x.x.x.x/x
Resources:
VPC:
Type: AWS::EC2::VPC
Properties:
CidrBlock: !Ref AddressVPC
EnableDnsSupport: true
EnableDnsHostnames: true
InstanceTenancy: 'default'
- When you need a value for a resource and can't get it from Ref you can get it from Fn::GetAttr
m) CloudFormation - Intrinsic Functions - Fn::GetAttr
- GetAttr allows you to access many different variables in a resource.
- You'll need to check the AWS Docs to see what is available per resource.
- Here is the resource we. Want to get a value from.
- To get the GroupId we use GetAttr on the resource.
Resources:
SecurityGroup:
Type: 'AWS::EC2::SecurityGroup'
Properties:
VpcId: !Ref VpcId
GroupDescription: Enable HTTP and SSH access
SecurityGroupIngress:
- In Protocol: tcp
FromPort: 80
ToPort: 80
CidrIp: !Ref LocationHTTP
- In Protocol: tcp
FromPort: 22
ToPort: 22
CidrIp: !Ref LocationSSH
WebServer:
Type: 'AWS::EC2::Instance'
Properties:
Monitoring: false
ImageId: !Ref ImageId
InstanceType: !Ref InstanceType
KeyName: !Ref KeyName
IamInstanceProfile: !Ref InstanceProfile
SecurityGroupIds:
- !GetAtt SecurityGroup.GroupId
n) CloudFormation - Wait Conditions
- Wait Conditions wait for a condition
- They are used in two cases:
* 1. To coordinate stack resource creation with configuration actions that are external to the stack creation
* 2. To track the status of a configuration process
-WaitCondition is very similar to CreationPolicy AWS recommends using CreationPolicy for EC2 and ASG
* CreationPolicy waits on the dependent resource
* WaitCondition waits on the wait condition (external)
WebServerGroup:
Type: AWS::Autoscaling::AutoScalingGroup
Properties:
AvailabilityZones:
Fn::GetAZs: ""
LaunchConfigurationName:
Ref: "LaunchingConfig"
MinSize: "1"
MaxSize: "5"
DesiredCapacity:
Ref: "WebServerCapacity"
LoadBalancerNames:
-
Ref: "ElasticLoadBalancer"
WaitHandle:
Type: AWS::CloudFormation::WaitConditionHandle
WaitCondition:
Type: AWS::CloudFormation::WaitCondition
DependsOn: "WebServerGroup"
Properties:
Handle:
Ref: "WaitHandle"
Timeout: "300"
Count:
Ref: "WebServerCapacity"
o) CloudFormation CheatSheet
- When being asked to automate the provisioning of resources think CloudFormation
- When infrastructure as Code (IaC) is mentioned think CloudFormation
- CloudFormation can be written in either JSON or YAML
- When CloudFormation encounters an error it will rollback with ROLLBACK_IN_PROGRESS
- CloudFormation templates larger than 51,200 bytes (0,05 MB) are too large to upload directly, and must be imported into CloudFormation via an S3 bucket.
- NestedStacks helps you break up your CloudFormation template into smaller reusable templates that can be composed into larger templates
- At least one resource under resources: must be defined for a CloudFormation template to be valid
- CloudFormation Template Sections:
* MetaData extra information about your template
* Description a description of what template is suppose to do
* Parameters is how you get user inputs into templates
* Transforms Applies macros (like applying a mod which change the anatomy to be custom)
* Outputs are values you can use to import into other stacks
* Mapping maps keys to values, just like a lookup table
* Resources defines the resources you want to provision, at least one resource is required.
* Conditions are whether resources are created or properties are assigned.
- Stack Updates can be performed two different ways:
* Direct updates
** You directly update the stack
** You submit changes and AWS CloudFormation immediately deploys them
*Executing Change Sets
** You can preview the changes to CloudFormation will make to your stack (Change Set)
** Then decide whether you want to apply those changes
- Stack Updates will change state of your resources based on circumstances:
* Update with No Interruption - Updates the resource without disrupting operation and without changing the resource's physical ID
* Updates with Some Interruptions Updates the resource with some interruption and retains the physical ID.
* Replacement recreates the resource during an update, also generates new physical ID.
* You can use a StackPolicy to prevent stack updates on resources to prevent data loss or interruption to services.
- Drift Detection feature lets CloudFormation tell you when your expected configuration has changed to due manual overrides. eg. A CFN creates an SG but a Developer deletes it.
-Rollbacks occur when a CloudFormation encounters an error when you create, update or destroy a stack
* When a rollback is in progress you'll see ROLLBACK_IN_PROGRESS
* When a rollback succeeds you'll see UPDATE_ROLLBACK_COMPLETE
* When a rollback fails you'll see UPDATE_ROLLBACK_FAILED
- Pseudo Parameters are predefined parameters eg. !Ref AWS:Region return us-east-1
- Resource Attributes
* CreationPolicy - prevent its status from reaching create complete until aws CloudFormation receives a specified number of success signals or the timeout period is exceeded.
* DeletionPolicy - reserve or (in some cases) backup a resource when its stack is deleted Delete, Retain or Snapshot
* UpdatedPolicy - How to handle an update for ASG, ElastiCache, Domain or Lambda Alias
* UpdatedReplacePolicy - To retain or (in some cases) backup the existing physical instance of a resource when it is replaced during a stack update operation. Delete, Retain or Snapshot
* DependsOn - That resource is created only after the creation of the resource specified in DependsOn attribute
- Intrinsic Functions allow you to assign properties that are not availble during runtime most important two to know:
* Ref returns the value of the specified parameter or resource
* Fn:GetAttr returns the value of an attribute from resource in the template
- AWS CloudFormation create-stack - CLI command to create a stack
- AWS CloudFormation update-stack - CLI command to update a stack
- Serverless Application Model (SAM) is an extension of CloudFormation that lets you define serverless applications.
p)
r)
s)
t)
u)
w)
x)
y)
z)
2) Elastic Container Service (ECS)
- fully-managed container orchestration service. Highly secure , reliable, and scalable way.
a) Components of ECS
- Cluster - Multiple EC2 instances which will house the docker containers.
- Task definition - A JSON file that defines the configuration of (upto 10) containers you want to run
- Task - Launches containers defined in Task Definition. Tasks do not remaining running once workload is complete
- Service - Ensures tasks remaining running eg. web-app.
- Container Agent - Binary on each EC2 instance which monitors, starts and stops tasks
-
b) Creating an ECS Cluster
- Create Cluster- options:
* EC2 or Farget
* Use Spot or On Demand
* EC2 Instance Type
* Number of Instances
* EBS Storage Volume
* EC2 can be Amazon Linux 2 or Amazon Linux 1
* Choose a VPC or create a new VPC
* Assign an IAM Role
* Option to turn on CloudWatch Container Insights
* Choose a Key Pair
- You can ssh into an EC2 Container Instance and make changes but its not generally recommended.
c) ECS - Task Definition JSON File Example
- You can define multiple containers within a task definition.
- The (docker) images can be provided either via ECR or an official docker repository eg. Docker Hub
- You must have one essential container. If this container fails or stops than all other containers will be stopped.
- AWS had a wizard to create Task Definitions instead having to create a file by hand.
d) Elastic Container Registry (ECR)
- A fully-managed Docker container registery that makes it easy for developers to store, manage, and deploy Docker container images.
-
e)
f)
g)
h)
i)
j)
k)
l)
3) Elastic Beanstalk
- Quickly deploy and manage web-apps on AWS without worrying about the underlying infrastructure
- What is Platform as a Service? (Paas)
a platform allowing customer to develop , run, and manage applications without the complexity of building and maintaining the infrastructure typically associated with developing an launching app
- Think of Elastic Beanstalk (EB) as the Heroku of AWS
- Choose a platform, upload your code and it runs with little knowladge of the infrastructure. Not Recommended for "Production" applications. AWS is talking about enterprise, large companies.
- Elastic Beanstalk is powered by a CLoudFormation template setups for you:
* Elastic Load Balancer
* Autoscaling Groups
* RDS Database
* EC2 Instance preconfigured (or custom) platfroms
** Elastic Beanstalk Packer Builder
** Go
** .NET (Windows/IIS)
** Java
** Node.js
** Ruby
** PHP
** Python
** Tomcat
* Monitoring (CloudWatch, SNS)
* In-Place and Blue/Green deployment methodologies
* Security (Rotates passwords)
* Can run Dockerized enviorments
a) EB - Supported Languages
- Ruby -> Rails
- Python -> Django
- PHP -> Laravel
- Tomcat -> Spring
- NodeJS -> ExpressJs
-Go
b) EB - Web vs Worker Environment
- Web Environment
* comes in two variants
* Creates an ASG
* Creates an ELB
- Worker Environment
* Creates an ASG
* Creates an SQS Queue
* Installs the Sqsd deamon on the Ec2 instances
* Creates CloudWatch Alarm to dynamically scale instances based on health.
-
c) EB - Web Environment Types
- Load Balancer Env
* Uses ASG and set to scale
* Uses an ELB
* Designed to Scale
- Single-Instance Env
* Still uses an ASG but Desired Capacity set to 1 to ensure server is always running.
* No ELB to save on cost.
* Public IP Address has to be used to route traffic to server
d) EB - Deployment Policies
- These are the Deployment Policies available with Elastic Beanstalk:
Deployment Policy Load Balancer Env Single Instance Env
All at once + +
Rolling. + -
Rolling with additional batch + -
Immutable + +
- Rolling requires an ELB because it attaches and detaches from the ELB
- Why is In-Place and Blue/Green missing on this list? We'll explain shortly.
e) EB - All At Once Deployment
- 1. Deploys the new app version to all instances at the same time.
- 2. Takes all instances out of service while the deployment processes
- 3. Servers become available again.
- The fastest but also the most dangerous deployment method
- In Case of Failure - You need to roll back the changes by re-deploying the original version again to all instances.
f) EB - Rolling Deployment
- 1. Deploys the new app version to a batch of instances at a time.
- 2. Takes batch's instances out of service while the deployment processes.
- 3. Reattaches updated instances
- 4. Goes onto next batch , taking them out of service
- 5. Reattaches those instances (rinse and repeat)
- In Case of Failure - You need to perform an additional rolling update in order to roll back the changes.
g) EB - Rolling With Additional Batch Deployment
- 1. Launch new instance that will be used to replace a batch
- 2. Deploy update app version to new batch
- 3. Attach the new batch and terminate the existing batch
- Rolling with Additional Batch's ensure our capacity is never reduced. This is important for applications were a reduction in capacity could case availability issues for users.
- In Case of Failure - You need to perform an additional rolling update to roll back the changes.
h) EB - Immutable
- 1. Create a new ASG group with EC2 instances
- 2. Deploy the updated version of the app on the new EC2 instances
- 3. Point the ELB to the new ASG and delete the old ASG which will terminate the old EC2 instances.
- The safest way to deploy for critical applications
- In Case of Failure - You just terminate the new instances since the existing instances still remain.
i)
-
j)
k)
l)
m)
n)
o)
p)
4)
5)
6)
7)
8)
9)
10)
11)
12)
13)
14)
15)
16)
Komentarze
Prześlij komentarz