Pinboard (jm)

Pinboard (jm) https://pinboard.in/u:jm/public/ recent bookmarks from jm More on the Coinbase 07-05-2026 outage 2026-06-03T09:26:47+00:00 https://www.coinbase.com/en-fr/blog/a-postmortem-of-our-may-7-2026-outage jmAWS's managed Kafka service failed silently. A significant portion of our event-streaming infrastructure runs on MSK, AWS's managed Kafka offering. The architectural promise of a managed Kafka service is that when individual brokers go down, the service automatically reelects partition leaders and continues to serve traffic out of the remaining brokers. The loss of an entire zone should result in reduced capacity, not unavailability. That is not what happened and this extended the outage. A defect in the AWS MSK control plane prevented automatic partition-leader reelection. Two of our MSK clusters became stuck in a "healing" state with producers unable to write. The cascading effect blocked our fee service, which blocked quoting, which is why most customers experienced this incident as broken trades and quotes rather than as a Kafka outage. Adjacent systems, including portions of our ledger pipeline, payments, and several data pipelines, were affected the same way. Additionally, one of our Kafka clusters was set up in a 2-AZ configuration that increased the blast radius and recovery time, but the MSK control plane defect impacted 2-AZ and 3-AZ Kafka clusters similarly. We worked the recovery in real time with AWS engineering, ultimately performing manual partition reassignments at 3:00 AM ET to migrate topics off the impaired brokers. Priority-zero and priority-one topics were back to full availability by 9:30 AM ET. The remainder cleared by 2:00 PM ET. In fairness, they also had a single-AZ point of failure in their architecture which they also describe there, but still, not great performance from MSK. Disappointing. ]]> msk reliability multi-az aws services kafka resiliency outages post-mortems postmortems coinbase https://pinboard.in/ https://pinboard.in/u:jm/b:7124695a3aec/ Best Practices for TCP Connection Management on EC2 2026-06-02T10:49:57+00:00 https://aws.amazon.com/blogs/networking-and-content-delivery/best-practices-for-tcp-connection-management-on-ec2/ jm networking protocols tcp idle-timeouts aws architecture nitro conntrack idle-connections https://pinboard.in/ https://pinboard.in/u:jm/b:d3963371f1ff/ Coinbase MSK outage post-mortem 2026-06-02T10:05:02+00:00 https://x.com/rwitoff/status/2052863502424133949 jm Root cause: a thermal event (cooling system failure) inside a subset of racks within a single building in AWS us-east-1. We run a primary replica of our exchange infrastructure in a single zone, consistent with industry standards to reduce latency. To prepare for failures like this, we maintain a distributed standby, but during this incident, failures in the primary zone that were designed to be isolated were not [...] Our primary managed Kafka partitions process many terabytes of data daily and are designed with resiliency guarantees for uninterrupted operation during a datacenter failure just like this. In this case, those guarantees failed and required manual recovery. [...] There is a hint here that MSK failed to have multi-AZ resiliency despite multiple replicas configured at the application level. It will be interesting to see what the full root-cause analysis looks like....]]> kafka resiliency coinbase multi-az az aws us-east-1 post-mortems postmortems https://pinboard.in/ https://pinboard.in/u:jm/b:828a1b3b7a2c/ Serverless Functions Post-Mortem 2026-06-02T09:59:24+00:00 https://matduggan.com/serverless-functions-post-mortem/ jm serverless cloud programming architecture aws https://pinboard.in/ https://pinboard.in/u:jm/b:5e47d50b4647/ Amazon Connect Talent vs. bias law 2026-04-30T10:04:52+00:00 https://www.linkedin.com/posts/coquinn_a-note-from-amazon-web-services-awss-share-7454936476359819264-8DqQ/?rcm=ACoAAAD7ciEBGVCp3m50-5sdPXL70GJw7TDNHVE jmAmazon Connect Talent was just announced. It conducts AI-powered conversational interviews with candidates, generates "anonymized competency scores," and surfaces ranked candidates to recruiters who "make the call." Fun fact: in New York City, that is an Automated Employment Decision Tool under Local Law 144. AEDTs require an annual independent bias audit with publicly posted results, plus at least ten business days of notice to candidates before use. Illinois, Colorado, and the EU AI Act impose adjacent obligations. The launch materials mention none of this. The compliance posture appears to be: candidate names are stripped from recruiter dashboards, therefore bias is solved. That is not how any of this works. Proxies for protected class -- speech patterns, zip codes, education history, the resume already sitting in your ATS -- are exactly what bias audits exist to measure. I don't think the product is bad. I think the announcement is conspicuously missing the guidance customers need before they can deploy it in NYC without violating Local Law 144 on day one. (The day's other news so far: Amazon Connect now ships as a four-SKU family, and there is a new design philosophy called "humorphism" with its own .com. Both feel small next to the above.) If you're selling automated hiring decisions in 2026, the bias-audit conversation belongs in the launch. ]]> bias law amazon aws recruiting regulation automation ai https://pinboard.in/ https://pinboard.in/u:jm/b:af7440a2d620/ russellromney/turbolite 2026-03-27T12:31:23+00:00 https://github.com/russellromney/turbolite jmIt also offers page-level compression (zstd) and encryption (AES-256) for efficiency and security at rests, which can be used separately from S3. Object storage is getting fast. S3 Express One Zone delivers single-digit millisecond GETs and Tigris is also extremely fast. The gap between local disk and cloud storage is shrinking, and turbolite exploits that. The design and name are inspired by turbopuffer's approach of ruthlessly architecting around cloud storage constraints. The project's initial goal was to beat Neon's 500ms+ cold starts. Goal achieved. If you have one database per server, use a volume. turbolite explores how to have hundreds or thousands of databases (one per tenant, one per workspace, one per device), don't want a volume for each one, and you're okay with a single write source. ]]> sqlite sql s3 aws gcp object-stores rust databases https://pinboard.in/ https://pinboard.in/u:jm/b:9c43d573d0f3/ hectorvent/floci 2026-03-23T17:30:08+00:00 https://github.com/hectorvent/floci jm floci aws emulation testing local coding https://pinboard.in/ https://pinboard.in/u:jm/b:f24a9f4a5a12/ 2 Ways to Correct the Financial Times at AWS (So Far) - Last Week in AWS Blog 2026-03-18T16:32:06+00:00 https://www.lastweekinaws.com/blog/2-ways-to-correct-the-financial-times-at-aws-so-far/?ck_subscriber_id=512829374 jm A healthy engineering culture, when confronted with "your AI tool contributed to a production incident," responds with: "Yeah, that tracks. Here's what we're changing so it doesn't happen again." An unhealthy one responds with a condescending press release explaining why the journalist is wrong and probably an idiot, and the human is at fault. The engineers building and operating these systems are talented people doing hard work under increasingly constrained conditions. They deserve leadership that backs them up when things go sideways, not leadership that throws them under the bus to protect a product launch narrative.]]> incidents production ai llms amazon aws communications pr https://pinboard.in/ https://pinboard.in/u:jm/b:34601cfa11ef/ Adrian Cockroft's take on the AWS outage 2025-10-23T15:09:59+00:00 https://www.linkedin.com/posts/adriancockcroft_summary-of-the-amazon-dynamodb-service-disruption-activity-7387117492135133184-WG9Y/ jm adrian-cockroft outages post-mortems aws amazon us-east-1 dynamodb circular-dependencies depe https://pinboard.in/ https://pinboard.in/u:jm/b:806d900eaa88/ Summary of the Amazon DynamoDB Service Disruption in Northern Virginia (US-EAST-1) Region 2025-10-23T15:08:30+00:00 https://aws.amazon.com/message/101925/ jm dynamodb dns aws ec2 nlb outages post-mortems cloud-computing amazon us-east-1 https://pinboard.in/ https://pinboard.in/u:jm/b:4a70bf22a463/ Today is when Amazon brain drain finally caught up with AWS (The Register) 2025-10-21T10:01:47+00:00 https://www.theregister.com/2025/10/20/aws_outage_amazon_brain_drain_corey_quinn/ jmAWS has given increasing levels of detail, as is their tradition, when outages strike, and as new information comes to light. Reading through it, one really gets the sense that it took them 75 minutes to go from "things are breaking" to "we've narrowed it down to a single service endpoint, but are still researching," which is something of a bitter pill to swallow. To be clear: I've seen zero signs that this stems from a lack of transparency, and every indication that they legitimately did not know what was breaking for a patently absurd length of time. [...] At the end of 2023, Justin Garrison left AWS and roasted them on his way out the door. He stated that AWS had seen an increase in Large Scale Events (or LSEs), and predicted significant outages in 2024. It would seem that he discounted the power of inertia, but the pace of senior AWS departures certainly hasn't slowed — and now, with an outage like this, one is forced to wonder whether those departures are themselves a contributing factor. You can hire a bunch of very smart people who will explain how DNS works at a deep technical level (or you can hire me, who will incorrect you by explaining that it's a database), but the one thing you can't hire for is the person who remembers that when DNS starts getting wonky, check that seemingly unrelated system in the corner, because it has historically played a contributing role to some outages of yesteryear. When that tribal knowledge departs, you're left having to reinvent an awful lot of in-house expertise that didn't want to participate in your RTO games, or play Layoff Roulette yet again this cycle. This doesn't impact your service reliability — until one day it very much does, in spectacular fashion. I suspect that day is today. Ouch. This is a very painful read and I'd say AWS are not happy to see it....]]> aws amazon layoffs tech how-we-work lses outages us-east-1 rto brain-drain work https://pinboard.in/ https://pinboard.in/u:jm/b:dfb5b53a1fe3/ Migrating to Hetzner 2025-10-17T11:00:28+00:00 https://digitalsociety.coop/posts/migrating-to-hetzner-cloud/ jm hetzner uptime k8s aws migration cloud infrastructure ops https://pinboard.in/ https://pinboard.in/u:jm/b:1581a65a93d6/ AWS in 2025: The Stuff You Think You Know That's Now Wrong 2025-08-25T16:25:47+00:00 https://www.lastweekinaws.com/blog/aws-in-2025-the-stuff-you-think-you-know-thats-now-wrong/ jm ebs dynamodb lambda history changelog aws us-east-1 https://pinboard.in/ https://pinboard.in/u:jm/b:f91256dc33bb/ Best practices for Google Cloud Storage 2025-04-25T17:12:59+00:00 https://cloud.google.com/storage/docs/best-practices jm gcs aws s3 storage google best-practices ops https://pinboard.in/ https://pinboard.in/u:jm/b:9d8107872c47/ Handling billions of invocations – best practices from AWS Lambda 2025-03-24T15:10:54+00:00 https://aws.amazon.com/blogs/compute/handling-billions-of-invocations-best-practices-from-aws-lambda/?ck_subscriber_id=512829374#415:%20Google%20Takes%20Wiz%20-%2017033042 jmDrawing inspiration from the “The Power of Two Random Choices” paper, the Lambda team explored the shuffle-sharding technique for its asynchronous invocations processing. Using this technique, you shuffle-shard tenants into several randomly assigned queues. Upon receiving an asynchronous invocation, you place the message in the queue with the smallest backlog to optimize load distribution. This approach helps to minimize the likelihood of assigning tenants to a busy queue. [....] The shuffle-sharding technique proved remarkably effective. By distributing tenants across shards, the approach ensures that only a very small subset of tenants could be affected by a noisy neighbor. The potential impact is also minimized since each affected tenant maintains access to unaffected queues. As your workloads grow, increasing the number of queues enhances resilience and further reduces the probability of multiple tenants being assigned to the same shard. This significantly lowers the risk of a single point of failure, making shuffle sharding a robust strategy for workload isolation and fault tolerance. Automated Isolation, covered in the next section, is also a neat trick. (via Last Week In AWS)]]> via:lwia sharding architecture services horizontal-scaling shuffle-sharding algorithms load-balancing async queues aws multitenant https://pinboard.in/ https://pinboard.in/u:jm/b:b407ba9caf17/ amarlearning/spot-optimizer 2025-03-14T10:51:20+00:00 https://github.com/amarlearning/spot-optimizer jm aws spot-instances python ops cli tools https://pinboard.in/ https://pinboard.in/u:jm/b:62552770139c/ Answers for AWS Survey for 2025 2025-03-03T17:56:20+00:00 https://answersforaws.com/2025/ jm aws sqs sns architecture cloud-computing surveys https://pinboard.in/ https://pinboard.in/u:jm/b:c64f7d6e7bda/ Inside an Amazon CoE 2025-03-03T15:40:49+00:00 https://beabetterdev.com/2025/02/23/why-amazon-never-makes-the-same-mistake-twice/#412:%20AWS%20%22Don't%20Mention%20TikTok%22%20for%20Containers%20-%2016799450 jm via:lwia aws amazon post-mortems coe incidents ops process https://pinboard.in/ https://pinboard.in/u:jm/b:8fe5888b75e3/ Mark Butcher on AWS sustainability claims 2025-02-27T13:05:11+00:00 https://www.linkedin.com/posts/markbutcher_real-datacenter-emissions-are-a-dirty-secret-activity-7287861693383098368-LUYh/?rcm=ACoAAAAHBmAB77TLKQn7I2B3LyyoX4-5CdLspW4 jm3 years after shouting about Amazons total lack of transparency with sustainability reporting, here's a list of what I think they've achieved: 1) They let you export a CSV for 3 lines of numbers showing your last months made up numbers that are up to 99% too low 2) Urmmm.... that's about it [....] I know of several very large enterprise orgs starting to proactively marginalise them (i.e. not move away 100%, but massively reducing consumption). The one's I know about will cost them around $1 billion of spend. Is that enough to make them pay attention? This article from Canalys in the Register says "Amazon doesn't provide AWS-specific, location-based data, meaning: "We don't really know how big AWS's footprint truly is, which I think is a bit worrying." They follow up with "Amazon has chosen not break out data on environmental stats such as greenhouse gas emissions for AWS from the rest of the company in its sustainability reports, making it almost impossible to determine whether these emissions are growing as they have been for its cloud rivals." Interesting isn't it... if they were actually as sustainable as they pretend, you'd expect them to share open and honest numbers, instead what we get are marketing puff pieces making what seem like invented PUE claims backed by zero evidence. Elsewhere he notes "AWS customers are still unable to natively measure actual power consumption, report on actual carbon emissions, report on water usage. This'll make life interesting for all those AI companies subject to legislation like the EU AI Act or needing to report to the EED and similar." (Via ClimateAction.tech)]]> climate-change aws sustainability pue reporting amazon cloud datacenters emissions https://pinboard.in/ https://pinboard.in/u:jm/b:61cdb9b627b4/ write hedging in Amazon DynamoDB 2025-02-24T11:29:13+00:00 https://aws.amazon.com/blogs/database/timestamp-writes-for-write-hedging-in-amazon-dynamodb/?ck_subscriber_id=512829374 jm Imagine you want a very low p99 read latency. One way to lower tail latencies is to hedge requests. You make a read request and then, if the response doesn’t come back quickly enough, make a second equivalent hedging request and let the two race. First response wins. If the first request suffered a dropped network packet, the second request will probably win. If things are just temporarily slow somewhere, the first request will probably win. Either way, hedging helps improve the p99 metrics, at the cost of some extra read requests. Write hedging has a little more complexity involved, since you want to avoid accidental overwrites during races; this blog post goes into some detail on a technique to do this in DynamoDB, using timestamps. Good stuff. (via Last Week In AWS)]]> via:lwia aws dynamodb write-hedging read-hedging p99 latencies tail-latencies optimization performance algorithms https://pinboard.in/ https://pinboard.in/u:jm/b:2c8e574b8330/ cur.vantage.sh 2025-02-05T10:01:43+00:00 https://cur.vantage.sh/ jm aws billing codes cost-explorer ec2 s3 finops https://pinboard.in/ https://pinboard.in/u:jm/b:55952b17d38d/ Long-running EKS bug 2025-01-22T10:16:29+00:00 https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/1064#issuecomment-1808000528 jm aws load-balancing alb eks kubernetes ops bugs https://pinboard.in/ https://pinboard.in/u:jm/b:b739fcca432e/ Principal Engineer Roles 2024-12-20T09:38:07+00:00 https://www.linkedin.com/pulse/principal-engineer-roles-framework-mai-lan-tomsen-bukovec-142df/ jm Sponsor: A Sponsor is a project/program lead, spanning multiple teams. Yes, this role can be played by a manager but it does not have to be (at least not at Amazon). If you are a Sponsor, you have to make sure decisions are made and that people aren’t stuck in analysis paralysis. This doesn’t mean that you yourself make those decisions (that’s often a Tie-breaker’s role which you may or may not be here). But you have to drive making sure decisions get made, which can mean owning those decisions, escalating to the right people, or whatever it takes to get it done. A Sponsor is constantly clearing obstacles and getting things moving. It is a time-consuming role. You shouldn’t have time to act as Guide or a Sponsor on more than two projects combined, and you don’t have to be a Sponsor every year. But if a few years go by, and you haven’t been a Sponsor, it might be time to think about where you can step in and play that role. It tends to build new skills because you have to operate in different dimensions to land the right outcomes for the project. Guide: Guides tend to be domain experts that are deeply involved in the architecture of a project. Guide will often drive the design but they’re not “The Architect.” A Guide often works through others to produce the designs, and themselves produce exemplary artifacts, like design docs or bodies of code. The code produced by a Guide is usually illustrative of a broader pattern or solving a difficult problem that the rest of the team will often run with afterwards. The difference between a Guide and a Sponsor is that the Guide focuses on the technical path for the project, and the Sponsor owns all aspects of project delivery, including product definition and organizational alignment. Guides influence teams. If you are influencing individuals, you’re likely being a mentor and not a Guide. A Guide is a time-consuming role. You shouldn’t have time to Guide more than two projects, and that drops to one project if you are a Sponsor at the same time. Catalyst: A Catalyst gets an idea off the ground, and it’s not always their idea. In my experience, the idea might not even come from the Catalyst—it can be something we’ve been talking about doing for years but never really got off the ground. Catalysts will create docs or prototypes and drive discussions with senior decision makers to think through the concept. Catalysts are not just “idea factories.” They take the time to develop the concept, drive buy-in for the idea, and work with the larger leadership team to assign engineers to deliver the project. A Catalyst is a time-consuming role because of all the work that needs to be done. At Amazon, that involves prototypes, docs and discussions. It is hard to effectively Catalyze more than one or two things at once. It is important to note that Catalysts, like Tie-breakers, are not permanent roles. Once a project is catalyzed (e.g., in engineering with a dedicated team working on the project), a Catalyst moves out of the role. The Catalyst might take on a Guide or Sponsor role on the project, or not. Not every project needs a Catalyst. A Catalyst is a very helpful (arguably critical) role for your most ambitious, complex, and/or ambiguous problems to solve in the organization. Tie Breaker: A Tie-Breaker makes a decision after a debate. At Amazon, that means deeply understanding the different positions, weighing in with a choice, and then formally closing it out with an email or a doc to the larger group. Not every project needs a Tie-Breaker. But if your project gets stuck in a consensus-seeking mode without making progress on hard decisions, a senior engineer might have to step in as a Tie-Breaker. Tie-breakers own breaking a log-jam on direction in the team by making a decision. Obviously, a Tie Breaker has to have great judgment. But, it is incredibly important that the Tie-Breaker listens well and understands all the nuances to the different positions as part of breaking the tie. When a Tie -Breaker drives a choice, they must bring other engineers into their thought process so that all the engineers in the debate understand the “why” behind the choice even if some are disappointed by the direction. A Tie-Breaker must have strong engineering and organizational acumen in this role. Sometimes an organization will depend on a small set of senior engineers to play the role of Tie-Breaker because they are so good at it. As a successful Tie-Breaker, you want to be careful not to set a tone that every decision, no matter how small, must go through you. You’ll quickly transition from Tie-Breaker to a “decision bottleneck” at that point—and that is not a role any team needs. If a team finds itself frequently seeking out a Tie-Breaker, it could be a sign that the team needs help understanding how to make decisions. That's a topic for a different time. The Tie-Breaker role is considered a “moment in time” role, versus Sponsor/Guide which are ongoing until you reach a milestone. Once the decision is made and closed out, you’re no longer the Tie-Breaker. Catcher: A Catcher gets a project back on track, often from a technical perspective. It requires high judgement because a Catcher drives prioritization and formulating a pragmatic plan under tight deadlines. Catchers must quickly do their own detailed analysis to understand the nuances of the problem and come up with the path forward in the right timeframe. As a comparison, a Tie-breaker tends to step in when the pros/cons of the different approaches are well known and the team needs to make a hard decision. Once “caught” (i.e., the project is back on track and moving forward), a project doesn’t need the Catcher anymore. Sometimes Principal Engineers can do too much catching. Don’t get me wrong, we are all Catchers sometimes—including me. Any fast-paced business needs Catchers in engineering and management. It teaches important skills about leadership in difficult moments and helps the business by landing deliverables. It also teaches you what not to do next time. However, it is better to generalize a Catcher skill set across more engineers and not depend on a small set to Principal Engineers as Catchers. If a Principal Engineer plays Catcher all the time through a succession of projects, it leaves no time to develop skills in other roles. Participant: A participant works on something without one of these explicitly assigned leadership roles. A Participant can be active or passive. Active participants are hands-on, and do things like spend a few days working through a design discussion or picking up a coding task occasionally on a project, etc. Passive participants offer up a few points in a meeting and move on. In general, if you're going to participate it's better to do so actively. Time-boxing some passive participation (e.g., office hours for engineers) can be a useful mechanism to stay connected to the team. However, keep in mind that it is easy for your time to get consumed by being a Participant in too many things. (via Marc Brooker)]]> roles principal-engineer work projects project-management amazon aws via:marc-brooker https://pinboard.in/ https://pinboard.in/u:jm/b:d8b7a78f5cce/ S3 now supports appending 2024-11-22T10:53:25+00:00 https://docs.aws.amazon.com/AmazonS3/latest/userguide/directory-buckets-objects-append.html jmYou can append data to the end of existing objects stored in the S3 Express One Zone storage class in directory buckets. We recommend that you use the ability to append data to an object if the data is written continuously over a period of time or if you need to read the object while you are writing to the object. Appending data to objects is common for use-cases such as adding new log entries to log files or adding new video segments to video files as they are trans-coded then streamed. By appending data to objects, you can simplify applications that previously combined data in local storage before copying the final object to Amazon S3. ]]> aws s3 storage cloud features https://pinboard.in/ https://pinboard.in/u:jm/b:0e07647a77d2/ Amazon S3 now supports strongly-consistent conditional writes 2024-09-03T15:24:11+00:00 https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3-conditional-writes/ jm s3 aws conditional-writes distcomp architecture storage consistency https://pinboard.in/ https://pinboard.in/u:jm/b:5cc7971399fa/ AWS region/service availability matrix 2024-07-25T09:57:54+00:00 https://aws-new-features.s3.amazonaws.com/html/aws_services.html jm via:lwia aws features releases regions ops ec2 vpc https://pinboard.in/ https://pinboard.in/u:jm/b:9504799b9f96/ CloudSnorkel/cdk-github-runners 2024-07-23T10:34:32+00:00 https://github.com/CloudSnorkel/cdk-github-runners jmUse this CDK construct to create ephemeral self-hosted GitHub runners on-demand inside your AWS account. 🧩 Easy to configure GitHub integration with a web-based interface 🧠 Customizable runners with decent defaults 🏃🏻 Multiple runner configurations controlled by labels 🔐 Everything fully hosted in your account 🔃 Automatically updated build environment with latest runner version Self-hosted runners in AWS are useful when: You need easy access to internal resources in your actions You want to pre-install some software for your actions You want to provide some basic AWS API access (but aws-actions/configure-aws-credentials has more security controls) You are using GitHub Enterprise Server Ephemeral (or on-demand) runners are the recommended way by GitHub for auto-scaling, and they make sure all jobs run with a clean image. Runners are started on-demand. You don't pay unless a job is running. ]]> github github-actions aws hosting elastic-scaling autoscaling cdk builds https://pinboard.in/ https://pinboard.in/u:jm/b:2ee5fffb2d7b/ turbopuffer 2024-07-10T10:07:46+00:00 https://turbopuffer.com/blog/turbopuffer jm aws s3 storage search vectors vector-search fuzzy-search lsm databases via:hn https://pinboard.in/ https://pinboard.in/u:jm/b:bc1c2f833151/ AWS Maniac's History of AWS Outages 2024-05-31T10:33:27+00:00 https://awsmaniac.com/aws-outages/ jm aws cloud outages downtime s3 ec2 dynamodb incidents post-mortems https://pinboard.in/ https://pinboard.in/u:jm/b:1b6a3252e8c2/ Ex-Amazon AI exec claims she was asked to ignore IP law 2024-04-24T09:14:43+00:00 https://www.theregister.com/2024/04/22/ghaderi_v_amazon/ jmAccording to Ghaderi's account in the complaint, she returned to work after giving birth in January 2023, inheriting a large language model project. Part of her role was flagging violations of Amazon's internal copyright policies and escalating these concerns to the in-house legal team. In March 2023, the filing claims, her team director, Andrey Styskin, challenged Ghaderi to understand why Amazon was not meeting its goals on Alexa search quality. The filing alleges she met with a representative from the legal department to explain her concerns and the tension they posed with the "direction she had received from upper management, which advised her to violate the direction from legal." According to the complaint, Styskin rejected Ghaderi's concerns, allegedly telling her to ignore copyright policies to improve the results. Referring to rival AI companies, the filing alleges he said: "Everyone else is doing it." Move fast and break laws!]]> aws amazon llms alexa maternity-leave parenting parental-leave work dont-be-evil copyright ip ai https://pinboard.in/ https://pinboard.in/u:jm/b:0b221724b3a8/ AWS told to pay $525M in cloud storage patent suit - The Register 2024-04-11T17:09:00+00:00 https://www.theregister.com/2024/04/11/aws_lawsuit_kove_io/ jm swpats software patents patent-trolls kove aws s3 algorithms distributed-hash-tables consistent-hashing ip https://pinboard.in/ https://pinboard.in/u:jm/b:d2dede16285a/ Answers for AWS survey results for 2024 2024-03-04T10:28:26+00:00 https://answersforaws.com/blog/2024-02-29-results/ jm aws services ops infrastructure architecture sqs sns dynamodb github-actions ecs via:lastweekinaws https://pinboard.in/ https://pinboard.in/u:jm/b:04ea95c6ae80/ Amazon Employees Fear Increased 'Quiet Firing' 2024-01-16T23:06:51+00:00 https://www.businessinsider.com/amazon-jobs-layoffs-quiet-firing-tactics-employees-2024-1?r=US&IR=T jmOne manager told [Business Insider] they were told to target 10% of all [their team's] employees for performance improvement plans. [...] Another manager said their ["unregretted employee attrition"] target is now as high as 12%. Senior staff are predicting that this will soon have externally-visible impact on system stability:

The loss of senior engineers who can lead in crisis situations is a growing risk, these people said. One person who works on Amazon's cloud infrastructure service told BI that they lost a third of their team following the layoffs, leaving them with more junior engineers in charge. If a large-scale outage happens, for example, those engineers will have to learn how to be in crisis mode on the job. Another AWS employee told BI they feel like they are "doing the job of three people." A similar question was also raised during a recent internal all-hands meeting, BI previously reported.

yikes.]]> amazon quiet-firing how-we-work ura pips work grim aws working hr https://pinboard.in/ https://pinboard.in/u:jm/b:4917b26b3c4e/ Signs that it’s time to leave a company… | by adrian cockcroft 2024-01-05T11:19:29+00:00 https://adrianco.medium.com/signs-that-its-time-to-leave-a-company-5f8759ad018e jm Founder led companies often have problems maintaining their innovation culture when the founder moves on. I think this is part of the problem at Amazon, and I was happy to be leaving as Andy Jassy took over from Jeff Bezos and Adam Selipsky took over AWS. Jeff Bezos was always focused on keeping the “Day 1” culture at Amazon, and everyone I talk to there is clear that it’s now “Day 2”. Politics and micromanagement have taken over, and HR processes take up far too much of everyone’s time. There’s another red flag for me when large real estate construction projects take up too much management attention. [...] We now have the situation that Amazon management care more about real estate than product. Where is the customer obsession in that? There’s lessons to be learned, and that the delusion that they can roll back work from home and enforce RTO without killing off innovation is a big problem that will increasingly hurt them over time. I personally hired a bunch of people into AWS, in my own team and by encouraging people to join elsewhere. Nowadays I’d say a hard no to anyone thinking of working there. Try and get a job at somewhere like NVIDIA instead. See also https://justingarrison.com/blog/2023-12-30-amazons-silent-sacking/ -- Justin Garrison's post about Amazon's Return-To-Office strategy really being "silent sacking" to downsize Amazon's staff, which has been confirmed by other AWS insiders.]]> aws amazon adrian-cockcroft how-we-work culture rto silent-sacking downsizing https://pinboard.in/ https://pinboard.in/u:jm/b:bd4683ce8d26/ Inside AWS: AI Fatigue, Sales Issues, and the Problem of Getting Big 2023-12-01T09:21:36+00:00 https://www.businessinsider.com/amazon-aws-ai-fatigue-sales-challenges-2023-11?r=US&IR=T jm One employee said their team is instructed to always try to sell AWS's coding assistant app, CodeWhisperer, even if the customer doesn't necessarily need it [....] Amazon is also scrambling internally to brainstorm generative AI projects, and CEO Andy Jassy said in a recent call that "every one of our businesses" is working on something in the space. [...] Late last month, one AWS staffer unleashed a rant about this in an internal Slack channel with more than 21,000 people, according to screenshots viewed by [Business Insider]. "All of the conversations from our leadership are around GenAI, all of the conferences are about GenAI, all of the trainings are about GenAI…it's too much," the employee wrote. "I'm starting to not even want to have conversations with customers about it because it's starting to become one big buzzword. Anyone have any ideas for how to combat this burn out or change my mindset?" Archive.is nag-free copy: https://archive.is/pUP2p]]> aws amazon generative-ai ai llms cloud-computing https://pinboard.in/ https://pinboard.in/u:jm/b:988cb67509c5/ Creating a Correction Of Errors document 2023-11-14T12:41:22+00:00 https://aws.amazon.com/blogs/mt/creating-a-correction-of-errors-document/?ck_subscriber_id=512829374 jm coes ops processes aws amazon work outages post-mortems operational-excellence best-practices https://pinboard.in/ https://pinboard.in/u:jm/b:7f5a95ec03a2/ Should you use a Lambda Monolith, aka Lambdalith, for your API? 2023-11-14T12:39:20+00:00 https://rehanvdm.com/blog/should-you-use-a-lambda-monolith-lambdalith-for-the-api jm The argument to limit the blast radius on a per route level by default is too fine-grained, adds bloat and optimizes too early. The boundary of the blast radius should be on the whole API/service level, just as it is and always has been for traditional software. Use a Lambdalith if you are not using any advance features of AWS REST API Gateway and you want the highest level of portability to other AWS gateways or compute layer. There are also many escape hatches to fill some of the promises that single-purpose functions offer. ]]> lambda monolith api design architecture aws serverless https://pinboard.in/ https://pinboard.in/u:jm/b:ccf4ade953bc/ Summary of the AWS Service Event in the Northern Virginia (US-EAST-1) Region 2023-10-31T10:59:34+00:00 https://aws.amazon.com/message/061323/?ck_subscriber_id=512829374 jm aws outages fail lambda sts security us-east-1 https://pinboard.in/ https://pinboard.in/u:jm/b:a9ab0d0bca2f/ AWS ALB returns 503 for Istio enabled pods 2023-10-25T10:38:31+00:00 https://domagalski-j.medium.com/aws-alb-returns-503-for-istio-enabled-pods-a6942383143c jm istio aws alb bugs networking tcp fail k8s https://pinboard.in/ https://pinboard.in/u:jm/b:2e144dfcee66/ Lessons Learned from 1TB DynamoDB Import 2023-10-24T17:07:10+00:00 https://kichik.com/2022/08/31/lessons-learned-from-1tb-dynamodb-import/ jm backfills dynamodb batch scaling aws https://pinboard.in/ https://pinboard.in/u:jm/b:b117711019c1/ AWS Reliability Pillar Single-Region scenarios 2023-10-12T13:29:48+00:00 https://docs.aws.amazon.com/wellarchitected/latest/reliability-pillar/single-region-scenarios.html jm via:singer aws reliability architecture availability uptime services ops high-availability https://pinboard.in/ https://pinboard.in/u:jm/b:f9f9b0a1c1c2/ Cell-Based Architecture 2023-08-29T08:29:58+00:00 https://aws.amazon.com/solutions/guidance/cell-based-architecture-on-aws/ jm architecture aws infrastructure slack cell-based-architecture cellular-architecture fault-tolerance fault-isolation https://pinboard.in/ https://pinboard.in/u:jm/b:5ea4da05300f/ Mountpoint for S3 2023-08-09T15:58:39+00:00 https://aws.amazon.com/s3/features/mountpoint/ jm s3 clients oss linux aws filesystems unix https://pinboard.in/ https://pinboard.in/u:jm/b:2f96c68c04ef/ AWS JSON 1.0 protocol - Smithy 2.0 2023-07-28T20:07:47+00:00 https://smithy.io/2.0/aws/protocols/aws-json-1_0-protocol.html jm aws json protocols wire sqs networking https://pinboard.in/ https://pinboard.in/u:jm/b:aed717e73f79/ Container Loading in AWS Lambda 2023-05-23T22:10:31+00:00 https://brooker.co.za/blog/2023/05/23/snapshot-loading.html jmThis system gets performance by doing as little work as possible (deduplication, caching, lazy loading), and then gets resilience by doing slightly more work than needed (erasure coding, salted deduplication, etc). This is a tension worth paying attention to in all system designs. ]]> architecture aws lambda marc-brooker performance storage caching containers caches https://pinboard.in/ https://pinboard.in/u:jm/b:18da95343673/ Kafka vs Redpanda Performance 2023-05-15T16:19:55+00:00 https://jack-vanlightly.com/blog/2023/5/15/kafka-vs-redpanda-performance-do-the-claims-add-up jm kafka redpanda benchmarks ec2 aws ssd optimization performance ops https://pinboard.in/ https://pinboard.in/u:jm/b:c155a58ab563/ Exploring performance differences between Amazon Aurora and vanilla MySQL | Plaid 2023-04-12T08:28:02+00:00 https://plaid.com/blog/exploring-performance-differences-between-amazon-aurora-and-vanilla-mysql/ jmbecause Aurora MySQL primary and replica instances share a storage layer, they share a set of undo logs. This means that, for a REPEATABLE READ isolation level, the storage instance must maintain undo logs at least as far back as could be required to satisfy transactional guarantees for the primary or any read replica instance. Long-running replica transactions can negatively impact writer performance in Aurora MySQL—finally, an explanation for the incident that spawned this investigation. The same scenario plays out differently in vanilla MySQL because of its different model for undo logs. Vanilla MYSQL: there are two undo logs – one on the writer, and one on the reader. The performance impact of an operation that prevents the garbage collection of undo log records will be isolated to either the writer or the reader. Aurora MySQL: there is a single undo log that is shared between the writer and reader. The performance impact of an operation that prevents the garbage collection of undo log records will affect the entire cluster. ]]> aurora aws mysql performance databases isolation-levels https://pinboard.in/ https://pinboard.in/u:jm/b:06b1342ee8c4/ AWS' proprietary SRD protocol 2023-02-16T11:48:02+00:00 https://aws.amazon.com/about-aws/whats-new/2023/02/ena-express-15-new-ec2-instances/ jm srd networking protocols ip ena-express aws amazon multi-pathing congestion-control nitro https://pinboard.in/ https://pinboard.in/u:jm/b:adb42574c16a/ ddbsh 2023-01-30T16:08:20+00:00 https://hypecycles.com/2023/01/25/hello-dynamodb-shell/?ck_subscriber_id=512829374 jm aws dynamodb cli tools unix sql https://pinboard.in/ https://pinboard.in/u:jm/b:ab7f3bc25bcd/ AWS Fault Isolation Boundaries 2022-11-22T12:30:51+00:00 https://docs.aws.amazon.com/whitepapers/latest/aws-fault-isolation-boundaries/abstract-and-introduction.html?ck_subscriber_id=512829374 jm aws dependencies uptime reliability iam https://pinboard.in/ https://pinboard.in/u:jm/b:f17c80e57a39/ Cloud Jewels 2022-11-15T12:47:56+00:00 https://www.etsy.com/codeascraft/cloud-jewels-estimating-kwh-in-the-cloud jm We thought about how we might be able to estimate our energy consumption in Google Cloud using the data we do have: Google provides us with usage data that shows us how many virtual CPU (Central Processing Unit) seconds we used, how much memory we requested for our servers, how many terabytes of data we have stored for how long, and how much networking traffic we were responsible for. Our supposition was that if we could come up with general estimates for how many watt-hours (Wh) compute, storage and networking draw in a cloud environment, particularly based on public information, then we could apply those coefficients to our usage data to get at least a rough estimate of our cloud computing energy impact. We are calling this set of estimated conversion factors Cloud Jewels. Other cloud computing consumers can look at this and see how it might work with their own energy usage across providers and usage data. The goal is to help cloud users across the industry to help refine our estimates, and ultimately help us encourage cloud providers to empower their customers with more accurate cloud energy consumption data. This is a good interim step, but it's disappointing how inaccurate the CO2 data exposed by cloud providers is. IMO this needs to be fixed]]> climate co2 google cloud aws etsy estimation cloud-jewels sustainability https://pinboard.in/ https://pinboard.in/u:jm/b:e6277c4d2cc9/ Serverless Messaging: Latency Compared 2022-09-07T08:46:07+00:00 https://bitesizedserverless.com/bite/serverless-messaging-latency-compared/ jm aws latency messaging queues architecture ops sqs sns kinesis https://pinboard.in/ https://pinboard.in/u:jm/b:0fd2c91e94ef/ How Google Cloud and AWS Approach Customer Carbon Emissions - Last Week in AWS Blog 2022-09-01T08:30:22+00:00 https://www.lastweekinaws.com/blog/How-Google-Cloud-and-AWS-Approach-Customer-Carbon-Emissions/?ck_subscriber_id=512829374 jmOn a larger scale, Google Cloud is already at 100% carbon neutrality, apparently via offsets and a few other accounting approaches, with a goal to move to 100% renewable energy for all cloud regions by 2030. Meanwhile, AWS’s carbon footprint tool is an embarrassment to AWS and its stated goal of reaching 100% renewable energy usage by 2025. The bottom line: One of these carbon neutrality approaches is indicative of a thoughtful approach to partnering with customers to lead to a better climate story around cloud usage. The other appears to have been phoned in by clowns the night before it was due. ]]> aws google carbon climate-change co2 hosting https://pinboard.in/ https://pinboard.in/u:jm/b:44b9a53ef1b0/ AWS EC2 Carbon Footprint Dataset - Google Sheets 2022-08-29T09:13:46+00:00 https://docs.google.com/spreadsheets/d/1DqYgQnEDLQVQm5acMAhLgHLD8xXCG9BIrk-_Nv6jF3k/edit#gid=504755275 jm aws carbon emissions ec2 hosting https://pinboard.in/ https://pinboard.in/u:jm/b:1c85b1d03555/ Key Takeaways from the DynamoDB Paper 2022-08-08T10:21:17+00:00 https://www.alexdebrie.com/posts/dynamodb-paper/ jm scalability scaling dynamodb aws storage services architecture https://pinboard.in/ https://pinboard.in/u:jm/b:3f183f470778/ DynamoDB's metastable cache load workaround 2022-07-14T10:27:37+00:00 https://brooker.co.za/blog/2022/07/12/dynamodb.html jm 'When a router received a request for a table it had not seen before, it downloaded the routing information for the entire table and cached it locally. Since the configuration information about partition replicas rarely changes, the cache hit rate was approximately 99.75 percent.' What's not to love about a 99.75% cache hit rate? The failure modes! 'The downside is that caching introduces bimodal behavior. In the case of a cold start where request routers have empty caches, every DynamoDB request would result in a metadata lookup, and so the service had to scale to serve requests at the same rate as DynamoDB' So this metadata table needs to scale from handling 0.25% of requests, to handling 100% of requests. A 400x potential increase in traffic! Designing and maintaining something that can handle rare 400x increases in traffic is super hard. To address this, the DynamoDB team introduced a distributed cache called MemDS. 'A new partition map cache was deployed on each request router host to avoid the bi-modality of the original request router caches.' Which leads to more background work, but less amplification in the failure cases. The constant traffic to the MemDS fleet increases the load on the metadata fleet compared to the conventional caches where the traffic to the backend is determined by cache hit ratio, but prevents cascading failures to other parts of the system when the caches become ineffective. ]]> aws dynamodb metastability caching caches production failure outages load memds marc-brooker papers usenix https://pinboard.in/ https://pinboard.in/u:jm/b:fd8e30bcc625/ AWS Lambda Function URLs 2022-04-08T10:46:45+00:00 https://aws.amazon.com/blogs/aws/announcing-aws-lambda-function-urls-built-in-https-endpoints-for-single-function-microservices/ jm aws http lambda serverless microservices web-services https://pinboard.in/ https://pinboard.in/u:jm/b:373bac9edd30/ aws-sdk-core > 3.78.0 slows down the fetching of credentials · Issue #2177 · aws/aws-sdk-ruby 2022-04-05T08:37:58+00:00 https://github.com/aws/aws-sdk-ruby/issues/2177#issuecomment-562772431 jm aws aws-sdk containers docker eks ec2 bugs fail imdsv2 https://pinboard.in/ https://pinboard.in/u:jm/b:fea302b0ba13/ What Exactly are AWS VPC Endpoints 2022-03-07T17:34:20+00:00 https://itnext.io/what-exactly-are-vpc-endpoints-and-why-they-need-real-inter-region-support-283a9987fe51 jm aws networking vpc vpc-endpoints architecture ops s3 dynamodb security https://pinboard.in/ https://pinboard.in/u:jm/b:ada6e9069db1/ New – Customer Carbon Footprint Tool | AWS News Blog 2022-03-02T14:39:20+00:00 https://aws.amazon.com/blogs/aws/new-customer-carbon-footprint-tool/ jm aws carbon emissions climate-change sustainability climate infrastructure ops ec2 s3 https://pinboard.in/ https://pinboard.in/u:jm/b:6fb815c75d98/ Why you should develop a correction of error (COE) | AWS Cloud Operations & Migrations Blog 2022-03-02T14:33:45+00:00 https://aws.amazon.com/blogs/mt/why-you-should-develop-a-correction-of-error-coe/ jmWhen documenting times, be sure to include a time zone, and make sure that you’re using it correctly (e.g., PDT vs. PST). Better yet, either use UTC or omit the middle letter of the time zone (e.g., “PT”). As Brian Scanlan sez: "A good 1/4 of the neurons in my brain were wired to quickly add and subtract 8 hours from timestamps by the time I left there" Just. Use. UTC.]]> amazon aws timezones coe process postmortems dates pdt pst utc https://pinboard.in/ https://pinboard.in/u:jm/b:66f5a94c1203/ Using load shedding to avoid overload 2021-10-12T12:10:25+00:00 https://aws.amazon.com/builders-library/using-load-shedding-to-avoid-overload/ jm error-handling distributed-systems http aws services soa david-yanacek load load-shedding uptime reliability https://pinboard.in/ https://pinboard.in/u:jm/b:c8ea792bb37a/ The Three DynamoDB Limits You Need to Know 2021-10-04T10:14:03+00:00 https://www.alexdebrie.com/posts/dynamodb-limits/ jm dynamodb limits aws coding ops https://pinboard.in/ https://pinboard.in/u:jm/b:ccb21b69ea39/ doctorray117/minecraft-ondemand 2021-09-07T08:27:16+00:00 https://github.com/doctorray117/minecraft-ondemand jm Instead of paying a minecraft hosting service for a private server for you and your friends, host it yourself. By utilizing several AWS services, a minecraft server can automatically start when you're ready to use it, and shut down when you are done. The final cost will depend on use but can be as little as a a dollar or two per month. The cost estimate breakdown is below. This is a reasonably cost effective solution for someone that doesn't need their server running 24/7. If that's you, read on! The process works as follows: Open Minecraft Multiplayer, let it look for our server, it will time out. The DNS lookup query is logged in Route 53 on our public hosted zone. CloudWatch forwards the query to a Lambda function. The Lambda function modifies an existing ECS Fargate service to a desired task count of 1. Fargate launches two containers, Minecraft and a watchdog, which updates the DNS record to the new IP The watchdog optionally sends a text message through Twilio when the server is ready. Refresh Minecraft server list, server is ready to connect. After 10 minutes without a connection or 20 minutes after the last client disconnects (customizable) the watchdog sets the desired task count to zero and shuts down. This is a very neat hack, actually quite potentially usable, and a good illustration of how viable Fargate+EFS are at hosting transient but not transitory workloads! ]]> minecraft fargate efs aws lambda hacks hosting https://pinboard.in/ https://pinboard.in/u:jm/b:4e1150b1f48c/ Lambda vs EC2 costs 2021-08-10T09:10:29+00:00 https://www.bbva.com/en/economics-of-serverless/ jm With traffic profiles where requests arrive in at periodic intervals, and a low total amount of requests, serverless architecture seems to be a great architecture in terms of cost, speed of delivery and effort. Thus, Lambda is probably the way to go if our application has sufficiently large periods of inactivity. Once the break-even point is reached, when EC2 is more cost-effective than Lambda, the cost difference grows rapidly, making Lambda less and less attractive in terms of cost. Thus, it is of great importance to know if the expected amount of traffic will be around the break-even point. Be aware of the CPU throttling you will get with the smaller memory flavors of Lambda. If your code is CPU-bound, choosing the smaller memory flavors might not be an option, since execution times, and thus latency, might grow beyond your requirements. On the other hand, if your code is I/O bound, the CPU throttling might not affect you significantly. Break-even point (if there is one, that is) strongly depends on the application itself. Without measuring the target application code, knowing the intended usage of the service, the SLA and the capabilities of the team in charge of building the application it is almost impossible to know for sure which service, Lambda or EC2, is more convenient. IMO there are still significant costs in organisational and infrastructure terms around replacing a working EC2 infrastructure with a Lambda-based one; deployment and other integration points with AWS are extremely tricky to deal with. But this is good data on the $ point alone. ]]> serverless aws lambda costs cost-control ec2 hosting architecture https://pinboard.in/ https://pinboard.in/u:jm/b:2ff2f72827fa/ Interesting AWS performance tip 2021-06-27T20:41:55+00:00 https://www.reddit.com/r/aws/comments/o7oxnl/strange_performance_decline_aws_clocksource_change/ jmwe determined that calls to time.Now() or time.Since(time.Time) in Golang were taking about 200 times longer on new servers than they were on old servers. We found similar impacts in Python, which lead us to the clocksource. It turns out that in our Grub boot parameters for AL2, we had added clocksource=hpet about two years ago, but all of our servers launched prior to 6/17 had a clocksource of kvm-clock, apparently ignoring the Grub config. Servers launched after 6/17 (with the same AMI, Grub config, everything) were honoring the specified clocksource, which caused our performance issue. ]]> ec2 aws clocksource linux kvm https://pinboard.in/ https://pinboard.in/u:jm/b:68b3230eeae8/ Amazon S3 is quietly deprecating BitTorrent support 2021-06-14T15:40:39+00:00 https://github.com/awsdocs/amazon-s3-userguide/commit/0d1759880ccb1818ab0f14129ba1321c519d2ac1#diff-72be9d82d9be9bda6a297a4fbd11aca66ecde97e4f90de6f86bdf95c5f6b72c0 jm bittorrent aws s3 deprecation features https://pinboard.in/ https://pinboard.in/u:jm/b:19ac597c23ba/ The Three DynamoDB Limits You Need to Know 2021-06-08T11:23:51+00:00 https://www.alexdebrie.com/posts/dynamodb-limits/#partition-throughput-limits jmthere are a few limits you must understand to model properly in DynamoDB. If you’re not aware of them, you can run into a brick wall. But if you understand them and account for them, you remove the element of surprise once your app hits production. Those limits are: The item size limit; The page size limit for Query and Scan operations; and The partition throughput limits. Notice how these limits build on each other. The first is about an individual item, whereas the second is about a collection of items that are read together in a single request. Finally, the partition throughput limit is about the number and size of concurrent requests in a single DynamoDB partition. I just ran into the last one on a pretty massive table we own, so this is worth bookmarking...]]> dynamodb aws storage gotchas limits ops architecture https://pinboard.in/ https://pinboard.in/u:jm/b:d6c9a66f1bb8/ The 17 Ways to Run Containers on AWS - Last Week in AWS 2021-05-31T14:36:45+00:00 https://www.lastweekinaws.com/blog/the-17-ways-to-run-containers-on-aws/?ck_subscriber_id=512829374 jm containers aws ec2 eks k8s docker architecture https://pinboard.in/ https://pinboard.in/u:jm/b:0efe3719a580/ Using shared memory for low-latency, intra-node communication in AWS Batch | AWS Compute Blog 2021-05-21T08:46:49+00:00 https://aws.amazon.com/blogs/compute/using-shared-memory-for-low-latency-intra-node-communication-in-aws-batch/ jmHPC workloads use algorithms that require parallelization and a low latency communication between the different processes. The two main technologies used for the parallel communications are message-passing with distributed memory and shared memory. Message Passing Interface (MPI) is a message-passing standard used for the communication in a parallel distributed environment. Elastic Fabric Adapter (EFA) enables your MPI applications to use low-latency, inter-node communication. The shared memory paradigm allows multiple processors in the same system to communicate using a memory (RAM) portion that is shared between the processes. This method takes advantage of the high-speed memory bus. ]]> shared-memory hpc mpi shmem ram coding efa aws ec2 low-latency https://pinboard.in/ https://pinboard.in/u:jm/b:6dd0427bd3ed/