Hey Hey Hey


  • Home

  • Archives

  • Tags

  • Search

AWS - Design MQTT Topics for AWS IoT Core

Posted on 2020-01-02 |

References

https://d1.awsstatic.com/whitepapers/Designing_MQTT_Topics_for_AWS_IoT_Core.pdf

MQTT Communication Patterns

  • Point to Point
    • different devices subscribe to the topic relevant to itself
  • Broadcast
    • multiple devices subscribe to same topic
  • Fan-in
    • multiple devices publish to same topic
    • avoid using fan-in to a single end device (?); use fan-in to route a large fleet of messages via IoT Rules Engine.
      • because this routing may hit a non-adjustable limit on a single device MQTT connection (!!!)

MQTT Communication Patterns

  • device to device
  • device to cloud
  • cloud to device
    • include session information for tracking purpose
  • device to/from users

MQTT Design Best Practices

General Best Practices

  • topic level: lowercase letters, numbers and dashes
  • general to specific
  • include any relevant routing information in topic
  • prefix to distinguish data and command topics
  • document topic structure as operation practices
  • use IoT Thing name as MQTT client ID – easy to correlate for logging and policy purpose
  • including Thing Name in any MQTT message published by a thing or sending to a specific thing
  • review the limitations
    • https://docs.aws.amazon.com/general/latest/gr/iot-core.html
  • include contextual information in payload messages
  • avoid fan-in to a single device – do not allow a single device subscribe to a shared topic (!!!)
  • never allow device to subscribe to all topics (#); Use single level wildcard (+) for IoT Rules

Best Practices for Telemetry

Read more »

AWS - EC2

Posted on 2019-09-17 |

Reference

https://youtu.be/cb0KvqGjXRE

EC2

  • AWS’s vision of EC2 : compute platform for the world and keep innovation
  • EC2 new OS: Amazon Linux 2
    • 5 years support
    • You can use for on-premise
  • EC2 support Windows
    • Most windows on cloud runs on AWS
  • BYO License use AWS License Manager
  • Specifically optimized for SAP

Deep dive

  • AWS Nitro System, accelerate the hypervisor layer
  • AWS Firecracker, used by Lambda

Serverless

  • Lambda are triggered trillions of times / month

Storage

  • S3 Intelligent Tiering — Auto category the data
  • S3 Glacier Deep Archive — 70% cheaper than Glacier ; New product
Read more »

AWS - Handson Best Practice

Posted on 2019-09-13 |

CloudFormation

  • Define Security Group separated with Server
  • Otherwise the Server Stack is not able to be deleted when Security Group is referenced by Other Servers
Read more »

AWS - Keynotes

Posted on 2019-08-27 |

Reference

https://youtu.be/femopq3JWJg

Redesign of the DB architecture

History of Aurora

  • Cell based architectures
    • Shared storage
    • Easy plus one failure mode
  • The log is the database
    • Across the AZ and shard, it’s the log that being moved, not the data.
  • Change happens at storage layer, redesign to make the storage layer database awareness.

History of DynamoDB

  • Analysis show that 70% query to relational DB is just key value.

DYNAMO

  • Feature
    • automatic re-sharding
    • DB migration service (From Oracle to Dynamo)

Basic knowledge with Aurora sharding

Read more »

Gazebo

Posted on 2019-08-16 |

Gazebo

http://gazebosim.org/

Read more »

AWS - CloudMap

Posted on 2019-08-15 |

Reference

https://youtu.be/fMGd9IUaotE

Service registers

  • Zookeeper , Eureka, SmartStack, SkyDns, Doozerd, etcd, etc
  • CloudMap : dynamic map of your cloud

Issue try to solve

  • Attribute based service discovery under complex service environment

    • Multiple Stage
    • Multiple Version
    • Multiple Status
  • Handle partial failure

    • help you provision Route53 to help handle partial failure

Integrate with existing AWS service

  • Cloudformation
  • IAM

Demo

Read more »

AWS - DotNet

Posted on 2019-08-15 |

Reference

https://youtu.be/FteCJQcTDc4

Modern .NET applications on AWS

Mosaic image

  • Service being used: .net tool, lambda, xray, ecr fargate, dynamodb, cognito, s3, code pipeline, sqs, stepfunction, aws batch, ssm param, cloudformation

Demo : use visual studio to CICD

  • AWS Batch
    • Work as queue; ability to use EC2 Spot Instances
  • Use Visual Studio, you can directly publish the code to generate Docker image and publish to AWS ECR
    • The code logic is to download the pic, and upload to corresponding S3 Raw folder
  • Use Visual Studio, directly publish Lambda function
    • In lambda , register XRay will enable XRay drill down details of the invoke
  • Code Pipeline
  • Use step function to link all the functions
Read more »

AWS - MachineLearning

Posted on 2019-08-14 |

Hands-on

https://s3.amazonaws.com/solutions-reference/predictive-maintenance-using-machine-learning/latest/predictive-maintenance-using-machine-learning.pdf

1
Couldn't call 'describe_notebook_instance' to get the Role ARN of the instance PredictiveMaintenanceNotebookInstance.

Update the role attached to the sagemaker instance

1
ResourceLimitExceeded

Change to train_instance_type = ‘ml.p2.xlarge’

Reference

https://youtu.be/GW0Bktm55nI

AWS Machine Learning Stack

Read more »

AWS - Encryption

Posted on 2019-08-14 |

Reference - S3 SSE-KMS

https://youtu.be/jZYkJf-9yXI

Encryption_KMS.PNG

Encryption_KMS_1.PNG

Read more »

Server Hardware

Posted on 2019-08-13 |

HPE Ethernet 10Gb 2-port 562FLR-SFP+ Adapter

FLR: integrated on motherboard
SFP: fiber
SPF+ : single port fiber support 10G

Read more »
123…18
Rachel Rui Liu

Rachel Rui Liu

178 posts
193 tags
RSS
GitHub Linkedin
© 2021 Rachel Rui Liu
Powered by Hexo
Theme - NexT.Pisces
0%