Posted on |

083.mp4 084.mp4 – Auto Scaling

  • can scale up / down

  • only scale horizontal

  • can scale accross AZ (can’t scale accross region !)

  • params: min size; max size; desired capacity (init size)

  • “Launch configuration”

    • AMI type; Instance Type; Key pair; Security Groups
    • Optional: spot instance bid pricing

  • Autoscaling group

    • unhealth instance will be terminted and replaced

  • Scaling plans (?)

  • Scaling Policy

    • how to trigger: Alarm + policy to decide how to scale
    • trigger what action: ChangeInCapacity; ExactCapacity; PercentChangeInCapacity

  • Scaling Policy Types:

    • Simple Scaling
    • Step Scaling (new feature): allow small changes,like 20% more capacity when 40%<CPU<70%

AutoScaling Margin

Scaling options
https://docs.aws.amazon.com/autoscaling/ec2/userguide/scaling_plan.html

Save cost
https://docs.aws.amazon.com/autoscaling/ec2/userguide/as-instance-termination.html

PercentChangeInCapacity
https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_PutScalingPolicy.html

The order of execution for scheduled actions ; time confliction
https://docs.aws.amazon.com/autoscaling/ec2/userguide/schedule_time.html

AutoScaling Lifecycle Hooks; Action Result: ABANDON / CONTINUE
自动起来的实例，可以给一定时间装软件搞配置，一切ready返回信号（abandon或者continue），如果是continue，这个实例就可以加入集群了。
https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html

health check grace period
autoscaling检查新加入实例的健康状态之前必须等待的时间。（确保新实例完全ready再检查）
https://docs.aws.amazon.com/autoscaling/ec2/userguide/healthcheck.html

Read more »

Posted on |

AWS Disaster Recovery

http://www.ecloudgate.com/Doc/DisasterRecovery_Overview

Backup and Restore vs Pilot Light vs Warm Standby
cheap —> Inexpensive
Slow ----> quick
RPO high --> Low which mean time to recover from high to low
RTO high --> Low which mean data loss time period from high to low

079.mp4 080.mp4 - Backup and disaster recovery

RPO and PTO

两个用来定义灾备需求的重要参数。用于指导灾备技术的选择。

RPO: Recovery Point Objective. The age of files that must be recovered from backup storage for normal operations to resume if a computer, system, or network goes down as a result of a hardware, program, or communications failure.
The recovery time objective (RTO) is the maximum tolerable length of time that a computer, system, network, or application can be down after a failure or disaster occurs.
https://whatis.techtarget.com/definition/recovery-point-objective-RPO
https://whatis.techtarget.com/definition/recovery-time-objective-RTO

Full backup vs Incremental Backup

  • Full backup, good RTO, bad RPO
  • Incremental backup , Good RPO, slower RTO

Redundant Array of Inexpensive Disks (RAID)

Read more »

Posted on |

077.mp4 078.mp4 - Deployment Overview

  • Infrastructure as code

    • Cloudformation templates;Cloudformation designer;

  • Continous Deployment

    • CodeCommit
    • CodePipeline
    • ElasticBeanStalk
    • OpsWorks
    • Elastic Container Service (ECS)

  • Application update: Prebaking AMI; in place update application; Disposable upgrade

  • Blue-Green upgrade

    • Staged roll out ; need doubled resource; make use of Route53 Service

ELB and ElasticBeanStalk work together
auto create ELB config
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/using-features.managing.elb.html

Read more »

Posted on |

073.mp4 074.mp4 – handson with S3 and Route53

  • purchase a domain
  • create a S3 bucket (use the domain name as bucket name)
  • Download a HTML5 website
  • update the website files to S3
  • Edit S3 configuration : “Static Website Hosting” --> select the index.html
  • Click the website , verify the origional host name endpoint works
  • Enable versioning
  • Delete index.html then use version control to revert the deletion (by deleting the deletion marker)
  • upload a different version of index.html and then revert the update(by deleting the new version)
  • Edite the life cycle rules: for revious version, after 30 days to Glacier and after another 30 days, delete permenently.
  • create another S3 bucket (using the subdomain name with www. prefix)
  • instead of hosting website, for this new S3 bucket, select re-direct all request to another host (the domain name we purchased)

075.mp4 – Bring in CloudFront

  • CloudFront can front web ; and RTMP (Real Time Message Protocol), 流媒体
  • Select CloudFront for Website
    • Source is S3 Bucket domain name with content files in it
    • Origin Path used to filter out which directory need to be cached
    • select allowed Protocol(Http/Https) and methods
    • TTL (set max,min,default)
    • Distribution Settings (how many edge location need to distributed to )
    • CNames: put puchased domain name
    • set Logging and comment
  • wait till status to “Deployed” , check the cached contents
  • use the CloudFront assigned domain name to visit the website
  • trigger a “Invalidate” request to certain file or folder that matched with your request list.

076.mp4 – Bring in Route53

Demo1,

  • Route53 --> Hosted zones
  • select the puchased domain and click “Create Record Set”
  • Because the website hosted inside aws, select “Use Alias”
  • Choose the target as the CloudFront url
  • hit the domain name from browser verify it works

Demo2

  • delete the existing record set
  • create a new recordset without using Alias, pointing to a public IP which is the EC2 hosting a wordpress web application

Demo3 : Failover

Read more »

Posted on |

Rout53 Overview

It’s the entry point of distributed infra at backend.

DNS in nutshell

History,

  1. static DNS
  2. dynamic DNS
    • same domain name, based on user source or other properites, assign different resolving result
  3. Policy based

Terminology

Traffic Policy : Rules routing to points
Traffic Policy Record : domain name with an applied traffic policy version.

Hosted Zones:
DNS Records: route request to the correct endpoint address

Steps to use Route53

  1. Register your domain name with Route53 or other registra (needs some process to update)
Read more »

Posted on |

overview

Cloudwatch Metrics is a time series data store.

Cloudwatch Alarms : based on Metrics ; Metrics Threshhold combined with Evaluation Period will decide wether to trigger alarm

  • integrated with SNS, Email etc.
  • state: OK;ALARM;INSUFFICIENT_DATA

Integrate Cloudwatch with 3rd party monitoring platform

Consider below carefully when integrate

  • IAM Permission
  • API : There is a limit of how many metrics being returned by one request. The number equals to if you have metrics with 1min period, then each time you can only retrive 1 day’s metrics. (24*60=1440 metrics).
1
2
3
4
5
6
7
aws cloudwatch list-metrics --metric-name EstimatedCharges
# period is in second, so it's 5 min
aws cloudwatch get-metric-statistics --namespace AWS/EC2 --metric-name CPUUtilization \
      --dimentions Name=InstanceId,Value=i-30c9605  \
      --start-time "2014-10-11T00:00:00Z" --end-time "2014-10-12T00:00:00Z" \
      --period 300 \
      --statistic  {"average","maximum"} | more
  • Request Through put: change the retrieving amount by adjusting start and stop and period make sure the response fit in requirement. (???)
  • Late Arriving data ( BackFill feature)

Cloudwatch logs

Read more »

Posted on |

069.mp4 – overview

  • Data Storage: Redshift; DynamoDB; S3 ; RDS
  • Data Analysis: EMR ; ElasticSearch; QuickSight BI; Amazon Machine Learning; Lambda
  • Data Streaming: Kinesis Streams

Redshift

  • Petabyte level
  • PostgreSQL based
  • continously backed up to S3 with snapshots (1-35 days)
  • quick recovery from snapshots

EMR (Elastic Map Reduce)

  • Fully managed hadoop service
  • Clusters can be automatically deleted upon task finish
  • Data processing framework: Hadoop Mapreduce & Spark
  • Storage options: HDFS / EMRFS (S3 based) / EC2 local file system

ElasticSearch

  • datasource: s3, Kinesis Streams, DynamoDB Streams, Cloudwatch logs, CloudTrail
  • Not suitable for Petabyte level storage

QuickSight

  • BI Reporting tools
  • SPICE (Super-fast , Parrallel , In-memory, Calculation Engine)
Read more »

Posted on |

061.mp4 062.mp4 – overview

  • Message being published to Topic via SDK/CLI/Console

  • Subscribed by : SQS (FIFO Queue Not supported); Email (format: Email, Email-JSON); Mobile; HTTP(s); Lambda; SMS

  • Message include,

    • MessageId, Timestamp, TopicArn, Type, UnsubscribeUrl, MessageBody, Subject,Signature, SignatureVersion

  • SNS Mobile Push Notification Steps,

    • Request Credential from Mobile platforms
    • Request Token ( ADM, GCM registration ID; APNS device Token)

  • ADM(Amazon Device Messaging) – push to kindle

  • APNS — push to iOS device

  • GCM — push to android

High-level Steps
https://docs.aws.amazon.com/sns/latest/dg/mobile-push-pseudo.html

Size limite for SMS (140byte per sms, 1600 byte per msg)
https://docs.aws.amazon.com/sns/latest/dg/sms_publish-to-phone.html

HTTP(s) protocole: user password
https://docs.aws.amazon.com/sns/latest/dg/SendMessageToHttp.html

Read more »

Posted on |

059.mp4 060.mp4 - SWF overview

SWF: Simple WorkFlow Service

  • long running process
  • interact with aws, user, on-promise infrastructure
  • WorkFlow Engine
  • Workflow and sub WorkFlow
  • SWF Domain : one or multiple workflows
  • Actor:
    • Starter
    • Decider
    • Worker
  • Task:
    • Register via console or CLI (RegisterActivityType)
    • Specify Queue for task
    • Use “Task Routing” for routing to specific worker
  • Implementation / Set up
    • Implementation : SDK; API Call ; framework (Java / Ruby)
    • Setup : CLI or console
  • A scenario :
    • worker upload video --> transform --> review --> online

Steps to develop and run a WorkFlow
https://docs.aws.amazon.com/amazonswf/latest/developerguide/swf-dg-intro-to-swf.html

SWF limitations (number of domains; request size; flow execution time)
https://docs.aws.amazon.com/amazonswf/latest/developerguide/swf-dg-limits.html

Read more »
0%