Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmThe Three DynamoDB Limits You Need to Know2021-10-04T10:14:03+00:00
https://www.alexdebrie.com/posts/dynamodb-limits/
jmdynamodb limits aws coding opshttps://pinboard.in/https://pinboard.in/u:jm/b:ccb21b69ea39/The Three DynamoDB Limits You Need to Know2021-06-08T11:23:51+00:00
https://www.alexdebrie.com/posts/dynamodb-limits/#partition-throughput-limits
jmthere are a few limits you must understand to model properly in DynamoDB. If you’re not aware of them, you can run into a brick wall. But if you understand them and account for them, you remove the element of surprise once your app hits production.
Those limits are:
The item size limit;
The page size limit for Query and Scan operations; and
The partition throughput limits.
Notice how these limits build on each other. The first is about an individual item, whereas the second is about a collection of items that are read together in a single request. Finally, the partition throughput limit is about the number and size of concurrent requests in a single DynamoDB partition.
I just ran into the last one on a pretty massive table we own, so this is worth bookmarking...]]>dynamodb aws storage gotchas limits ops architecturehttps://pinboard.in/https://pinboard.in/u:jm/b:d6c9a66f1bb8/The secret life of DNS packets: investigating complex networks2019-05-23T15:04:48+00:00
https://stripe.com/gb/blog/secret-life-of-dns
jmOne more surprising detail we discovered in the tcpdump data was that the VPC resolver was not sending back responses to many of the queries. During one of the 60-second collection periods the DNS server sent 257,430 packets to the VPC resolver. The VPC resolver replied back with only 61,385 packets, which averages to 1,023 packets per second. We realized we may be hitting the AWS limit for how much traffic can be sent to a VPC resolver, which is 1,024 packets per second per interface.
]]>aws limits ops stripe vpc dns outages postmortems debugginghttps://pinboard.in/https://pinboard.in/u:jm/b:12605b81e624/jantman/awslimitchecker2017-05-15T16:49:25+00:00
https://github.com/jantman/awslimitchecker?__s=gf36pf8g1gjugcqh6ppo
jm
A script and python module to check your AWS service limits and usage, and warn when usage approaches limits.
Users building out scalable services in Amazon AWS often run into AWS' service limits - often at the least convenient time (i.e. mid-deploy or when autoscaling fails). Amazon's Trusted Advisor can help this, but even the version that comes with Business and Enterprise support only monitors a small subset of AWS limits and only alerts weekly. awslimitchecker provides a command line script and reusable package that queries your current usage of AWS resources and compares it to limits (hard-coded AWS defaults that you can override, API-based limits where available, or data from Trusted Advisor where available), notifying you when you are approaching or at your limits.
(via This Week in AWS)]]>aws amazon limits scripts opshttps://pinboard.in/https://pinboard.in/u:jm/b:d5a5dc1da1b8/Instapaper Outage Cause & Recovery2017-02-14T10:42:03+00:00
https://medium.com/making-instapaper/instapaper-outage-cause-recovery-3c32a7e9cc5f#.39bn4xjas
jmWithout knowledge of the pre-April 2014 file size limit, it was difficult to foresee and prevent this issue. As far as we can tell, there’s no information in the RDS console in the form of monitoring, alerts or logging that would have let us know we were approaching the 2TB file size limit, or that we were subject to it in the first place. Even now, there’s nothing to indicate that our hosted database has a critical issue.
]]>limits aws rds databases mysql filesystems ops instapaper riskshttps://pinboard.in/https://pinboard.in/u:jm/b:70ad5bf26582/youtube/doorman2016-07-14T10:03:36+00:00
https://github.com/youtube/doorman
jmDoorman is a solution for Global Distributed Client Side Rate Limiting. Clients that talk to a shared resource (such as a database, a gRPC service, a RESTful API, or whatever) can use Doorman to voluntarily limit their use (usually in requests per second) of the resource. Doorman is written in Go and uses gRPC as its communication protocol. For some high-availability features it needs a distributed lock manager. We currently support etcd, but it should be relatively simple to make it use Zookeeper instead.
From google -- very interesting to see they're releasing this as open source, and it doesn't rely on G-internal services]]>distributed distcomp locking youtube golang doorman rate-limiting rate-limits limits grpc etcdhttps://pinboard.in/https://pinboard.in/u:jm/b:ab2cacfda26b/DRUG PUMP’S SECURITY FLAW LETS HACKERS RAISE DOSE LIMITS2015-05-08T10:41:37+00:00
http://www.wired.com/2015/04/drug-pumps-security-flaw-lets-hackers-raise-dose-limits/
jmdrugs drug-pumps hospira exploits vulnerabilities security root dosage limitshttps://pinboard.in/https://pinboard.in/u:jm/b:5277dacbad2f/Amazon EC2 Service Limits Report Now Available2014-06-27T14:22:36+00:00
http://aws.amazon.com/about-aws/whats-new/2014/06/19/amazon-ec2-service-limits-report-now-available/
jmaws ec2 vpc ebs autoscaling limits opshttps://pinboard.in/https://pinboard.in/u:jm/b:5eb986e4b5dc/