Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmCache warming: Leveraging EBS for moving petabytes of data2021-12-01T10:09:14+00:00
https://netflixtechblog.medium.com/cache-warming-leveraging-ebs-for-moving-petabytes-of-data-adcf7a4a78c3
jmarchitecture ebs memcache caching networking ec2 netflixhttps://pinboard.in/https://pinboard.in/u:jm/b:1e40b505a3ec/Amazon EBS Multi-Attach now available on Provisioned IOPS io1 volumes2020-02-17T10:49:05+00:00
https://aws.amazon.com/about-aws/whats-new/2020/02/ebs-multi-attach-available-provisioned-iops-ssd-volumes/
jmebs ec2 filesystems networking opshttps://pinboard.in/https://pinboard.in/u:jm/b:2f99d09d89c4/Baseline EBS performance2019-08-08T09:52:42+00:00
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html
jmaws ebs performance planning ops baselineshttps://pinboard.in/https://pinboard.in/u:jm/b:9ae5f4671ff1/Deep Dive: Maximizing EC2 and EBS Performance2018-09-17T11:11:36+00:00
https://www.slideshare.net/AmazonWebServices/deep-dive-maximizing-ec2-and-ebs-performance
jmebs ec2 aws ops storage tipshttps://pinboard.in/https://pinboard.in/u:jm/b:8dc4551f948d/AWS re:Invent 2017 slides: STG306 - Deep Dive on Amazon EBS2017-12-14T10:23:04+00:00
https://www.slideshare.net/AmazonWebServices/stg306deep-dive-on-amazon-ebs
jmec2 ebs aws amazon ops tips slides reinventhttps://pinboard.in/https://pinboard.in/u:jm/b:043c7cccb5c5/AWS re:Invent 2017 slides: DAT302 - Deep Dive on Amazon Relational Database Service (RDS)2017-12-14T10:22:46+00:00
https://www.slideshare.net/AmazonWebServices/dat302deep-dive-on-amazon-relational-database-service-rds
jmec2 ebs aws amazon ops tips slides reinventhttps://pinboard.in/https://pinboard.in/u:jm/b:6bbf7316c9fb/AWS re:Invent 2017 slides: CMP301 - Deep Dive on Amazon EC2 Instances2017-12-14T10:22:27+00:00
https://www.slideshare.net/AmazonWebServices/cmp301deep-dive-on-amazon-ec2-instances
jmec2 ebs aws amazon ops tips slides reinventhttps://pinboard.in/https://pinboard.in/u:jm/b:4674fd1ed517/EBS gp2 I/O BurstBalance exhaustion2017-07-26T13:14:54+00:00
https://jeremyeder.com/2017/07/25/docker-operations-slowing-down-on-aws-this-time-its-not-dns/
jmperformance aws ebs ec2 burst-balance ops debugginghttps://pinboard.in/https://pinboard.in/u:jm/b:bef976e76060/Top 5 ways to improve your AWS EC2 performance2017-07-03T10:51:31+00:00
https://www.datadoghq.com/blog/top-5-ways-to-improve-your-aws-ec2-performance/
jmops ec2 performance datadog aws ebs stolen-cpu virtualization metrics tipshttps://pinboard.in/https://pinboard.in/u:jm/b:26efe6d76709/Deep Dive on Amazon EBS Elastic Volumes2017-03-30T16:03:06+00:00
https://www.slideshare.net/AmazonWebServices/deep-dive-on-amazon-ebs-elastic-volumes-a
jmaws ebs storage architecture ops slideshttps://pinboard.in/https://pinboard.in/u:jm/b:5c14f1b8a4a4/(SDD416) Amazon EBS Deep Dive | AWS re:Invent 20142014-11-24T11:14:58+00:00
http://www.slideshare.net/AmazonWebServices/sdd416-amazon-ebs-deep-dive-aws-reinvent-2014#22
jmebs ops aws reinvent slideshttps://pinboard.in/https://pinboard.in/u:jm/b:cbb6850db17b/Latest EBS tuning tips2014-07-11T13:54:10+00:00
https://twitter.com/AWSSummits/status/487280813780656129/photo/1
jm
Cheat sheet of EBS-optimized instances. http://t.co/vmTlhUtpWk
Optimize your queue depth to achieve lower latency & highest IOPS. http://t.co/EO48oa0D6X
When configuring your RAID, use a stripe size of 128KB or 256KB. http://t.co/N0ldtFJ4t6
Use larger block size to speed up the pre-warming process. http://t.co/8UoIeWE2px
]]>ebs aws amazon iops raid ops tuninghttps://pinboard.in/https://pinboard.in/u:jm/b:dbfd882c9ad6/Amazon EC2 Service Limits Report Now Available2014-06-27T14:22:36+00:00
http://aws.amazon.com/about-aws/whats-new/2014/06/19/amazon-ec2-service-limits-report-now-available/
jmaws ec2 vpc ebs autoscaling limits opshttps://pinboard.in/https://pinboard.in/u:jm/b:5eb986e4b5dc/Use of Formal Methods at Amazon Web Services2014-06-17T13:33:47+00:00
http://research.microsoft.com/en-us/um/people/lamport/tla/formal-methods-amazon.pdf
jm
The success with DynamoDB gave us enough evidence to present TLA+ to the broader engineering community at Amazon. This raised a challenge; how to convey the purpose and benefits of formal methods to an audience of software engineers? Engineers think in terms of debugging rather than ‘verification’, so we called the presentation “Debugging Designs”.
Continuing that metaphor, we have found that software engineers more readily grasp the concept and practical value of TLA+ if we dub it 'Exhaustively-testable pseudo-code'.
We initially avoid the words ‘formal’, ‘verification’, and ‘proof’, due to the widespread view that formal methods are impractical. We also initially avoid mentioning what the acronym ‘TLA’ stands for, as doing so would give an incorrect impression of complexity.
More slides at http://tla2012.loria.fr/contributed/newcombe-slides.pdf ; proggit discussion at http://www.reddit.com/r/programming/comments/277fbh/use_of_formal_methods_at_amazon_web_services/]]>formal-methods model-checking tla tla+ programming distsys distcomp ebs s3 dynamodb aws ec2 marc-brooker chris-newcombehttps://pinboard.in/https://pinboard.in/u:jm/b:8758e4d5f996/Basho LevelDB supports tiered storage2014-04-11T09:04:24+00:00
https://github.com/basho/leveldb/wiki/mv-tiered-options
jmThe justification for two types/speeds of storage arrays is simple. leveldb is extremely write intensive in its lower levels. The write intensity drops off as the level number increases. Similarly, current and frequently updated data tends to be in lower levels while archival data tends to be in higher levels. These leveldb characteristics create a desire to have faster, more expensive storage arrays for the high intensity lower levels. This branch allows the high intensity lower levels to be on expensive storage arrays while slower, less expensive storage arrays to hold the higher level data to reduce costs.
]]>caching tiered-storage storage ssds ebs leveldb basho patches riak iopshttps://pinboard.in/https://pinboard.in/u:jm/b:b1d225492ab3/Monitoring the Status of Your EBS Volumes2013-05-16T12:51:52+00:00
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/monitoring-volume-status.html
jmebs aws monitoring metrics ops documentation cloudwatchhttps://pinboard.in/https://pinboard.in/u:jm/b:bb20cf8505e0/AWS forum post on interpreting iostat output for EBS2013-05-15T21:32:58+00:00
https://forums.aws.amazon.com/thread.jspa?messageID=124044#124227
jmiostat ebs disks hardware aws opshttps://pinboard.in/https://pinboard.in/u:jm/b:f047266d25be/Measuring & Optimizing I/O Performance2013-05-15T17:16:02+00:00
http://www.igvita.com/2009/06/23/measuring-optimizing-io-performance/
jmio optimization sysadmin performance iostat ebs aws opshttps://pinboard.in/https://pinboard.in/u:jm/b:dc96db2c0f69/ec2-consistent-snapshot2013-05-15T16:41:10+00:00
https://github.com/alestic/ec2-consistent-snapshot
jmThis program creates an EBS snapshot for an Amazon EC2 EBS volume. To
help ensure consistent data in the snapshot, it tries to flush and
freeze the filesystem(s) first as well as flushing and locking the
database, if applicable.
Filesystems can be frozen during the snapshot. Prior to Linux kernel
2.6.29, XFS must be used for freezing support. While frozen, a
filesystem will be consistent on disk and all writes will block.
There are a number of timeouts to reduce the risk of interfering with
the normal database operation while improving the chances of getting a
consistent snapshot.
If you have multiple EBS volumes in a RAID configuration, you can
specify all of the volume ids on the command line and it will create
snapshots for each while the filesystem and database are locked. Note
that it is your responsibility to keep track of the resulting snapshot
ids and to figure out how to put these back together when you need to
restore the RAID setup.
Handy!]]>ubuntu ec2 aws linux ebs snapshots ops tools alestichttps://pinboard.in/https://pinboard.in/u:jm/b:a1b5b4279884/Understanding Elastic Block Store Availability and Performance [slides]2013-05-03T21:46:00+00:00
http://www.slideshare.net/AmazonWebServices/understanding-ebs-availabilityandperformance
jmpiops ebs performance aws ec2 ops storage amazon presentationshttps://pinboard.in/https://pinboard.in/u:jm/b:039188bb5a6e/Latency's Worst Nightmare: Performance Tuning Tips and Tricks [slides]2013-04-19T20:27:52+00:00
https://speakerdeck.com/mza/latencys-worst-nightmare-performance-tuning-tips-and-tricks
jmbenchmarks aws ec2 ebs piops services scaling scalability presentationshttps://pinboard.in/https://pinboard.in/u:jm/b:0dad472cef4b/Increasing EBS Performance - Amazon Elastic Compute Cloud2013-04-05T16:20:57+00:00
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSPerformance.html
jmec2 ebs performance piops docshttps://pinboard.in/https://pinboard.in/u:jm/b:d06cfa1702bf/High Performance MongoDB Clusters with Amazon EBS Provisioned IOPS2013-04-05T16:08:10+00:00
http://www.slideshare.net/AmazonWebServices/ebs-mongo-dbwebinarfinal-nn
jmebs piops aws performance tips ops ec2 mongodb presentationshttps://pinboard.in/https://pinboard.in/u:jm/b:86bb77c25ac9/Amazon EC2 outage: summary and lessons learned2011-04-25T21:19:28+00:00
http://blog.rightscale.com/2011/04/25/amazon-ec2-outage-summary-and-lessons-learned/
jmebs ec2 aws cloud availability slas rightscale amazonhttps://pinboard.in/https://pinboard.in/u:jm/b:8149694811fa/SmugMug's Don MacAskill on last week's EBS outage2011-04-25T09:14:34+00:00
http://don.blogs.smugmug.com/2011/04/24/how-smugmug-survived-the-amazonpocalypse/
jmEBS outages aws smugmughttps://pinboard.in/u:jm/b:8a7ec182a941/