Terminology

Vault : Container for Archives. 1k Vualts per account
Archives: Basic Unit of backup. 40TB Max per archieve. No limit on number of archives.
Inventory: Cold index of archives (refresh every 24 hours)

Access Glacier

1. SDK / API
2. S3 Lifecycle (Bucket Level or Object Level)

• new feature: Archive S3 object by tag

3. 3rd party tools & Gateways

Upload to glacier

• Make use of description to persist metadata (in case local index is corrupted)
• Aggregate data into MBs , small data will have loads of overhead when persist into glacier
• Consider to persist file checksum with index locally ;
• Consider to persist file offset when files are aggregated, this helps to retrive data using range head
• Use multi-part upload

Data Management

• Vault Tag
  • View billing by tag; config security by tag
• Integrate with CloudTrail
• Vault access policies: easy to control access and share content with other account
• Vault Lock : 24hours cooling down / test period
• Vault Access Policy : give more flexibility compare to Vault lock. For example, make use of the Legal Hold tag on the vault

Athena Query on S3

• No data loading
• Serverless
  • support multi format (data lake)
• $5 for 5T data scanned from S3

https://youtu.be/x20Qx7lWSLQ

demo

• Create a new pipeline
  • source bucket (name, storage level,access)
  • target bucket (name, storage level,access)
  • thumbnails bucket(name, storage level,access)
• Create a new job
  • select pipeline
  • for one input can define multi output
  • define playlist

Free tier:
20 standard definition
10 HD definition

EBS definition

• Network Block Storage
• 5 9s availability
• Attached to EC2 in same AZ
• point-in-time backup to S3

Terminology

• iops : transaction /sec
• Throughput: read/write / sec
• Latency: delay between request response
• Capacity: Volumn size
• Block size: size of each i/o (kb)

![GP2 Bursting Diagram][https://github.com/racheliurui/markdown/blob/master/AWS/AWS2018/images/extra_EBS_gp2.png?raw=true]

• Scenario: boost time speed up for windows when using gp2 because of Bursting
  • which is important autosclaing
• Database ( you can use 2 gp2 volumn to archive 2*3k iops)

History

With EC2 -> Magnet Storage -> SSD Storage -> gp2 SSD Storage -> Volumn Encryption -> Larger, faster -> Boot volumn encryption -> st1 and sc1 (HDD) -> EBS Elastic Volumns (2017)

IOPS vs Throughput

Select the correct type
IOPS: gp2 io1
Throughput: st1 sc1

Background

Application Security Design goals

CIA (Confidentiality, Integrity, Availability)

• Confidentiality:
  • AWS is using the PARC Model
    • Principal, Action, Resource, Condition
• Integrity
• Availability: how long to encrypt/decrept the data, and how long the customer can stand if any of the system is not available and needs failover
  • For example, how long it take to encrypt the data and write to S3

Key Implement to meet CIA requirements

1. “Don’t store secret as plaintext on disk” and “Decrypt only happens in your instance”

• means encrypt and decrypt only happens inside your code inside your instance. (not aws service side)
• User AWS KMS client SDK; S3 encryption client ; DynamoDB encryption client
• Envolop Encryption : use random key to encrypt each piece of data, encrypted data and corresponding key stored together, the key will be encrypted using master key before being stored.

2. “keep cipher text of secret in multiple locations”

• make use of S3 --> 11 9s durability or DynamoDB (if you consider latency)

3. “Make sure secrets not being changed since last used”
4. "if instance can launch, secret should be accessible; <1 min to provision plaintext secret to instance "

Summary

DDOS

• Layer 3/4 DDoS
  • SYN/UDP Floods: A SYN flood attack works by not responding to the server with the expected ACK code， cause the server waiting for ACK for ever(timeout) and exhaust server resources.
  • reflection Attacks: trusted entities use shared mutual key, when faker is challenged to response using mutual key, he will send the orginal challenge to any server using same mutual key as a challenge, and get the response to respond to original request.
• Layer 7 DDoS

Key Features

AWS Shield

• Standard : layer 3/4 protection
  • Always on : heuristics-based anomal dectection; baseling
• Advanced : layer 7 protection
  • with AWS Shield, WAF is free
  • DDoS Scaling up free (report and refund)
  • Available when you have App ELB, Classic ELB, CloudFront, S3 and Rout53
  • Integrate with Cloudwatch to have metrics and report about the attack
  • Billing : multi accounts shared in one enterprise can share the service once enterprise bought this services

AWS WAF

Feature Summary,

• Filter traffic based on customized rules

• Malicious Request protection

  • SQL injection
  • Process encrpting (???)

• Active monitoring and tuning

• Less than 55 sec before the new rule is applied globally

• Less than 1 minisec inspection time when turned on

• API & SDK support when define the rules

• Pre-configured rules

VMware Software Defined Data Center (VMware SDDC)

AWS key advantages (Cloud advantages)

• Fexibility with Failover accross Availability Zones
  • Different data collection options & technique :
    • AWS Data Pipeline , AWS Import/Export Snowball , AWS Mobile Hub, AWS IOT, Kinesis Firehose

Comparison

Service Supported Data Volumn and antipatterns

Service Cost model

Senario

Move >10k records from MS SQL Database A to B (Different Table Design)

1
2
3
4
5
6
7
8

spool "C:\temp\data.csv"

select /*csv*/
    name as userName,
    address as userAddress
from customers;

spool off;

The records will be written into csv for furthur processing

• Suitable for temp solution or inital load
• Data exported with “” , which I haven’t found a way to get rid of; I am using Java to strip off the “” before the next processing steps

Shared Responsibility Model

不同的类型服务有不同的shared responsibility model

• Infrastructure Service
  • including EC2, EBS, VPC
• Container Service
  • including RDS, EMR, AWS Elastic Beanstalk
    • 跟Infrastructure service相比，aws会负责OS以及OS上部署的应用平台
• Abstracted Service
  • Serveless
    • 跟上一种相比，aws还负责服务端和网络的加密；用户基本只负责客户端

推荐使用“Trusted Advisor”

• 除非要求买专业服务，这个安全报告是免费的。
• 检查内容包含常见端口检查

设计AWS安全的步骤方法论

第一步： Define and categorize Assets on AWS

最佳时间，列表：

第二步： Design ISMS (Information Security Management System)