AWS Automation and Optimization
1. AWS OpsWorks
- configuration management service that provides fully-managed instances for Chef and Puppet
- What is Configuration Management?
The act of maintaining a specific software configuration and the ability to remotely make configuration changes to servers in way that they can be monitored , audited and automated.
- What is Software Provisioning ?
The act of configuration software packages on servers. Setting up all the software and configuration requirements to run a web-application.
- What is Application Deployment ?
The act of deploying new versions of web-application or rollback to a previous web-application version.
-What software does Configuration Managment, Software Provisioning, Application Deployment
* Chef
* Puppet
* Ansible
* Saltstack
- AWS OpsWorks provides managed instances for the automation platforms Chef and Puppet
-OpsWorks lets you use Chef and Puppet to automate how your EC2 or on-premise compute is:
* Configured
* Deployed
* Managed
-OpsWorks eliminates:
*operating your own configuration management systems
* worrying about maintaining its infrastructure.
-OpsWorks allows you to clearly define the layers of your stack eg.:
* Load Balancer
* Application
* Database
- AWS OpsWorks has three offerings:
* OpsWorks Stacks
* Chef Automate
* Puppet Enterprise
Stacks
- The container for the entire stack.
- Represents a set of instances that you want to manage collectively
- Top level configurations eg. Region, Default OS, Default SSH, Custom Cookbooks
Layers
- a blueprint for a set of Amazon EC2 instances specifies the instances's settings, associated resources, installed packages, and security groups.
- add recipes to lifecycle events of your instances eg. setup, configure, deploy, updeploy, shutdown
Instances
- An instance represents a server.
-It can belong to one or more layers that define the instance's settings, resources , installed packages , profiles and security groups.
Apps
- An app represents code stored in a repository that you want to install on application server instance. This is where you Deploy the app and configure Environment Variables
a) OpsWorks Stacks
- Managed version of Chef 11 and Chef 12
- Define your application as a stack which has layers, instances and app.
- Easily upgrade your OS (only for Amazon Linux 1)
- Can attach a Load Balancer (only Classic Load Balancer)
- Can defined Container Layers and RDS Layers
- Set IAM User Permissions to control stack operations
- Schedule EC2 instance to run on a schedule
- Set EC2 instances to run auto scaling base CPU usage
- Not supporting the latest version of Chef (v.14)
b) OpsWorks Stacks - Users and Permissions
- Add IAM users to your OpsWorks Stacks and manage their Permissions. You must be logged in as Administrative user or Manage user to modify any of the permissions settings
-Permission Levels
* Deny - Blocks the user's access to this stack
* IAM Policies - bases a user's permissions exclusively on policies attached to the user in IAM
* Show - Provide read-only access to the stack's resources.
* Deploy - Show permissions that let the user deploy new application versions
* Manage - Provide full control of this stack
-Instance Access
* SSH/RDP - Allowed to gain remote access
* sudo / admin - Allowed to use audo/admin privileges when gaining remote access.
c) OpsWorks Stacks - Lifecycle Events
- Each layer has a set of five lifecycle events, each of which has an associated set of Chef recipes that are specific to the layer.
Custom Chef Recipes
1. Setup - event occurs after a started instance has finished booting
2. Configure
-event occurs on all of the stack's instances when one of the following occurs:
* Instance enters/leaves the online state.
* Elastic IP address associated/disassociate from an instance.
* Elastic Load balancer attach/detached to a layer
3. Deploy
- event occurs when you run a deploy command
4. Undeploy
- event occurs when you delete an app or run an undeploy command to remove an app from a set of application server instances.
5. Shutdown
- event occurs after you direct OpsWorks Stacks to shut an instance down hut before the associated EC2 instance is actually terminated.
d) OpsWorks Stacks - Run Command
- Run commands allow you to perform manual operations on your Stack and underlying instances
- Update Custom Cookbooks - Updates the instances custom cookbooks with the current version from the repository.
- Execute Recipes - Executes a specified set of recipes on the instances.
- Setup - Runs the instances Setup recipes
- Configure - Runs the instances Configure reciepies
- Upgrade Operating System
* (Amazon Linux 1 Only) Upgrades the instances Amazon Linux operating systems to the latest version.
* You can set AWS OpsWorks to reboot the instance after installing the upgrade (default to yes)
e) OpsWorks Stacks - App Settings
- An App represents the Web Applications settings that will be applied to your instances running within your OpsWorks Layer.
-When you create an App in OpsWorks you can see the following options:
* You can set the Application Source to be from:
** Gir Repository
** HTTP Archive
** S3 Object
* You can pass Environment Variables to your web-app
* You can pass an SSL certificate to configure SSL that terminates at the instance (not load balancer)
* You can map a domain name to your app. You have to first:
** Create a subdomain with your DNS registrar
** Map it to your CLB Elastic IP address or your app server's public IP address
f) OpsWorks Stacks - Layers
- You can directly create 3 kinds of Layers:
* OpsWorks (EC2 Instances)
* ECS
* RDS
- To create an ECS Layer you have to first create the ECS Cluster and then associate the ECS cluster on creation of the Layer.
- To create an RDS Layer you have to first create the RDS instance and then associate the RDS instance on creation of the Layer.
g) OpsWorks Stacks - Stack Settings
- Support OS AMIs
* Amazon Linux 2
* Amazon Linux 1
* CentOS Linux 7
* Red Hat Enterprise Linux 7
* Ubuntu 14 / 16 / 18 LTS
* Windows Server 2012
** R2 Base
** R2 with SQL Server Express
** R2 with SQL Server Standard
** R2 with SQL Server Web
- Custom Chef Cookbooks Support
* HTTP Archive
* Git
* S3 Bucket
- You can provide a username and password for Repo access
- If you are using Chef 11 you can use Berkshelf to provide your Custom Chef cookbooks.
h) OpsWorks Stacks - Linux Security Updates
- Linux operating system provides supply regular updates, most of which are operating system security patches but can also include updates to installed packages.
- AWS OpsWorks Stacks (by default) automatically installs the latest updates during setup, after an instance finishes booting
- OpsWorks Stacks does not automatically install updates after an instance is online. To update your online instance you can
* create and start new instances to replace your current online instances
* then delete the current instances
- OpsWorks will perform these update for the support Linux Operation Systems:
* Amazon Linux
* Ubuntu
* CentOS
* Red Hat Enterprise Linux
- The way AWS OpsWorks update these OS's is by running the update command for Linux version.
i) OpsWorks Stacks - Load Balancer Layer
- With OpsWorks Stacks you can attach a Classic Load Balanacer directly to an instances Layer.
- You don't create a new Layer , you just directly associate the CLB with the Instance Layer.
- You can only attach a Classic Load Balancer, Application and Network are not supported.
-You have to create the Classic Load Balancer in the EC2 console first and then associate it to the layer
- When you associate the CLB it will automatically register all instances in that layer to load balancer ..
- Make sure you have Connection Draining turned on your CLB (because it is controlling Instance Shutdown Behavior)
- Instance Shutdown Behavior
* Wait for an instance's connections to drain from the load balancer before shutting it down
* Shut down an instance without waiting for connections to drain
j) OpsWorks Stacks - Time/Load Based Instances
- You can create two special types of instances:
* Time-based automatically starts and stops instances based on a specified schedule
** Choose hourly interval that show up green. These represent when the server should run
** You can set a schedule for everyday
** You can set different schedules for specific days of the week
* Load-based automatically starts and stops instances in response to CPU, memory, and application load changes across all the instances in a layer.
** Load-based Instances you allows you to scale based on Layer Averages or CloudWatch Alarms.
*** Scale based on Average Layers to:
**** Average CPU
**** Average Memory
**** Average Load
*** Set an Alarm for:
**** Up
**** Down
** Scaling Parameters determines:
*** how many servers should be added / removed when scaling.
*** how long the threshold has to be exceed before scaling
*** how long metrics should be ignored after scaling
Scale UP - means to Scale out ( to add servers)
Scale DOWN - means to Scale in (to remove servers)
k) OpsWorks Stacks - Monitoring Layers
- Monitoring Layers allows you to averaged stats across all your Instances within that layer.
- Memory Used is not tracked by CloudWatch for EC2 instances and requires installation of the CloudWatch Agent on the OS. By default OpsWorks is tracking Memory Used !!!!!!!!!!!!!!!!!!!!
-
l) OpsWorks Stacks - Time/Load Based Instances
- OpsWorks Stacks installs on every Linux instance the OpsWork Stacks Agent CLI
-When user has gained SSH access to an EC2 instanced managed by OpsWorks Stack you can use the OpsWorks Stacks Agent CLI to perform various commands:
sudo opsworks-agent-cli get_json - returns information about a Chef run as JSON object
sudo opsworks-agent-cli agent_raport - returns an agent report eg. Agent Status and Agent Version
sudo opsworks-agent-cli instance_report - returns an extended instance report. Eg Instance ID, Type, Architecture, Total Memory, CPU, Region, AZ
sudo opsworks-agent-cli list_commands - Lists the times for each activity that has been executed on this instance
sudo opsworks-agent-cli run_command - Runs an AWS OpsWorks Stacks command. You have to provide a JSON document containing a Chef run-list
sudo opsworks-agent-cli show_log - Tails the most recent command log file
sudo opsworks-agent-cli stack_state - Displays information that OpsWorks Stacks uses internally for the most recent Chef run
-
m) OpsWorks Stacks - Chef Automate
- Chef Automate . An Enterprise platform that collects data from 3 of Chef's opensource automation engines:
* InSpec - Automating Security and Compliance audits
* Chef - Automating your Infrastructure configuration
* Habitat - Automating application build and release
- AWS OpsWorks for Chef Automate is a fully managed configuration management service that hosts Chef Automate Server.
- OpsWorks automatically
* Patches
* Updates
* Backups
to you Chef Automate Server
n) OpsWorks Stacks - Puppet Enterprise
- Puppet Enterprise platform that:
* Automates all the things across your multi-cloud infrastructure at scale
* Enforce system desired state, automatically remediate unexpected changes, and trace intent
* Pre-defined workflows to build, test, and deploy infrastructure
- AWS OpsWorks for Puppet Enterprise is a fully managed configuration management service that hosts Puppet Enterprise Server
- OpsWorks automatically :
* Patches
* Updates
* Backes up
Your Puppet Enterprise Server
o) OpsWorks Stacks - What is Chef?
- Chef is a company and the name of a configuration management tool written in Ruby
- chef-repo - a directory on your workstation that stores everything you need to define your infrastructure with Chef Infra
- cookbooks - a cookbook is the fundamental unit of configuration and policy distribution. A cookbook defines a scenario and contains everything that is required to support that scenario
- recipes - a recipe is the most fundamental configuration element within the organization
- attributes - an attribute is a specific detail about a node
- nodes - a node is any device - physical, virtual, cloud, network device, etc. - that is under management by Chef Infra
- policy - policy maps business and operational requirements, process, and workflow to settings and objects stored on the Chef Infra Server
- Chef Infra Server acts as a hub for configuration data for:
* cookbooks
* the policies that are applied to nodes
* metadata that describes each registered node
- Knife is a CLI between a local chef repository and the Chef Infra Server
knife supermarket install chef-ruby
- Berkshelf
* Bershelf is a dependency manager for Chef cookbooks
* Similar to a Gemfile, requierments.txt or package.json but called Berkshelf
* This allows you to utilize community cookbooks or publish your own cookbooks
source 'https://supermarket.chef.io'
cookbook 'chef-ruby'
- What is Chef Recipe? Chef recipes are written in Ruby using a DSL (Domain Specific Language). A DSL is a language within a language. Here are some of the DSL commands available in ruby.
* execute - allows you to execute system commands
* apt_package - allows you to install software packages
* service - allows you to stop, start, enable, restart a service
* directory - allows you to create a folder
* cookbook_file - allows you to bring in a file that exists in cookbook
* template - allows you to bring in a template file and populate it with variables.
* attributes - is where you can get or set data on a node .eg Server
r) OpsWorks Stacks - What is Puppet?
- What is Puppet? Puppet is a server automation framework and application. Puppet has its only programming language for describing how servers should be configured.
- Manifest File - Holds all the resource declarations
- Resources - A resource describes something about the state of the system, such as a certain user or file should exist, or a package should be installed
* Exec - To execute commands, such as apt-get
* Package - To install packages via apt
* Service - To ensure that a service is running
* File - To ensure that certain files exist
s) OpsWorks Stacks - Cheatsheet
- OpsWorks is a configuration managment service for the following:
* OpsWorks Stacks - Deploy AWS apps using Chef 11 or 12 and a GUI that simplifies different stack layers
* OpsWork Chef Automate - fully managed server of Chef Automate
* OpsWork Chef Enterprise - fully managed server of Chef Enterprise
- OpsWorks Stacks
* A managed version of Chef 11 and Chef 12
* Can attach a Load Balancer (only Classic Load Balancer)
* Can defined Container Layers and RDS Layers
* OpsWorks Stacks is composed of the following:
** Stack - the container for the entire stack
** Layers - a blueprint for a set of Amazon EC2 Instances
** Instances - an instance represents a server
** Apps - an app represents code stored in a repository that you want to install on application server instances.
* You can grant IAM users access to as OpsWorks Users, and set thier permissions eg. Allowed to SSH into server.
* A layer has 5 lifecycle events that you can attach a chef recipe to:
1. Setup - after a started instance has finished booting
2. Configure - on all of the stack's instances when:
** Instance enters/leaves the online state
** Elastic IP address associated/disassociate from a instance
** Elastic Load balancer attach/detached to a layer
3. Deploy - when you run a deploy command
4. Undeployed - when you delete an app or run an undeploy command to remove an app from a set of application server instances
5. Shutdown - you direct OpsWorks Stacks to shut an instance down but before the associated EC2 instance is actually terminated
* Run Commands allows you to perform manual operations on your Stack and underlying instances.
** Update Custom Cookbooks - Updates the instances custom cookbooks with the current version from the repository.
** Execute Recipes - Executes a specified set of recieps on the instances
** Setup - Runs the instances setup recipes
** Configure - Runs the instances configure recipes
** Upgrade - Operating System (Amazon Linux 1 Only) Upgrades the instances Amazon Linux operating systems to the latest version.
*** You can set AWS OpsWorks to reboot the instance after installing the upgrade (default to yes)
* App Settings represents the web-app setting that will be applied to your running instances:
** Set the Application Source - the location of your application code on S3 , Git Repo , or an HTTP Archive
** Set Environment Variables -
** Pass an SSL Certificate (terminates at the instance, end-to-end encryption)
** Map a custom domains (uses Classic Load Balancer)
* Three Kinds of Layers:
** OpsWorks Layer (EC2 Instance)
** ECS Layer
*** You have to create the ECS cluster first than create and associate layer
** RDS Layer
*** You have to create the RDS instance first than create and associate layer
** Load Balancer Layer
*** You create a CLB and then you edit your OpsWorks layer and associate the Load Balancer
* Stack Settings you can change the default Operation System and Chef Cookbooks repo
** If you are using Chef 11 you can use Berkshelf to provide your Custom Chef cookbooks.
* You can create two special types of instances:
** Time-based - automatically starts and stops instances based on a specified schedule
** Load-based - autmatically starts and stops instances in response to CPU , memory , and application load changes across all the instances in a layer
* OpsWorks provides you a monitoiring dashboard called Monitoring Layers
** Monitoring Layers allows you to averaged stats across all your instances within that layer.
** OpsWorks is tracking Memory Used by default Normal EC2 instances require to manually install the CloduWatch Agent
* OpsWorks Stacks installs on every Linux instance the OpsWork Stacks Agent CLI
-Chef is a company and the name of a configuration managment tool written in Ruby
* A cookbook defines a scenario and contains everything that is required to su[[prt that scenario
* A recipe is the most fundamental configuration element within the organization
* Knife is a CLI between a local chef repository and the Chef Infra Server.
* Berskelf is a dependency manager for Chef cookbooks.
** Similiar to a Gemfile, requierments.txt or package.json but called Berkshelf
-Puppet is a server autmation framework and application
* Puppet has its only programming language for describing how servers should be configured
* Manifest File Holds all the resource declarations
* A resource describes something about the state of the system, such as certain user or file should exist, or package should be installed
t)
2. Service Catalog
- AWS Service Catalog enables organization to create and manage catalogs of products that are approved for use on AWS to achieve consistent governance and meet compliance requierments.
- The AWS Service Catalog is an alternative to granting direct access to AWS resources via the AWS Console
* Standarization
* Self-service discovery and launch
* Fine-grain access control
* Extensibility and version control
Anatomy of Service Control
b) Service Catalog - Administrator Portfolio
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
Komentarze
Prześlij komentarz