Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmpodmansh2023-08-14T19:03:58+00:00
https://blog.podman.io/2023/08/podman-v4-6-introduces-podmansh-a-revolutionary-login-shell/
jmpodmansh podman containers shells unix ops securityhttps://pinboard.in/https://pinboard.in/u:jm/b:9c8339c085dc/Container Loading in AWS Lambda2023-05-23T22:10:31+00:00
https://brooker.co.za/blog/2023/05/23/snapshot-loading.html
jmThis system gets performance by doing as little work as possible (deduplication, caching, lazy loading), and then gets resilience by doing slightly more work than needed (erasure coding, salted deduplication, etc). This is a tension worth paying attention to in all system designs.
]]>architecture aws lambda marc-brooker performance storage caching containers cacheshttps://pinboard.in/https://pinboard.in/u:jm/b:18da95343673/Every Cloud Architecture2023-01-10T16:54:37+00:00
https://www.goodtechthings.com/every-cloud-architecture/
jmarchitecture cloud comics containers event-bus funny databaseshttps://pinboard.in/https://pinboard.in/u:jm/b:0e59f109b7f3/Docker build cache sharing on multi-hosts with BuildKit and buildx2022-06-01T14:43:44+00:00
https://medium.com/titansoft-engineering/docker-build-cache-sharing-on-multi-hosts-with-buildkit-and-buildx-eb8f7005918e
jmdocker building compilation ci tweaks containershttps://pinboard.in/https://pinboard.in/u:jm/b:ff2e0294d357/Bringing emulation into the 21st century2022-04-25T17:03:05+00:00
https://blog.davetcode.co.uk/post/21st-century-emulator/
jmAn 8080 microprocessor utilising a modern, containerised, microservices-based architecture running on Kubernetes with frontends for a CP/M test harness and a full implementation of the original Space Invaders arcade machine. The full project can be found as a github organisation https://github.com/21st-century-emulation which contains ~60 individual repositories each implementing an individual microservice or providing the infrastructure.
Needless to say this monster runs at approximately 1KHz, instead of the required 2MHz. A good demo of how some deliberately obtuse and inappropriate architectural decisions can really make a mess of things.]]>emulation kubernetes satire k8s containers microservices yikeshttps://pinboard.in/https://pinboard.in/u:jm/b:c176a7cd23a6/aws-sdk-core > 3.78.0 slows down the fetching of credentials · Issue #2177 · aws/aws-sdk-ruby2022-04-05T08:37:58+00:00
https://github.com/aws/aws-sdk-ruby/issues/2177#issuecomment-562772431
jmaws aws-sdk containers docker eks ec2 bugs fail imdsv2https://pinboard.in/https://pinboard.in/u:jm/b:fea302b0ba13/The CFS quota container throttling problem2021-12-19T22:01:40+00:00
https://danluu.com/cgroup-throttling/
jmAlmost all services at Twitter run on Linux with the CFS scheduler, using CFS bandwidth control quota for isolation, with default parameters. The intention is to allow different services to be colocated on the same boxes without having one service's runaway CPU usage impact other services and to prevent services on empty boxes from taking all of the CPU on the box, resulting in unpredictable performance, which service owners found difficult to reason about before we enabled quotas. The quota mechanism limits the amortized CPU usage of each container, but it doesn't limit how many cores the job can use at any given moment. Instead, if a job "wants to" use more than that many cores over a quota timeslice, it will use more cores than its quota for a short period of time and then get throttled, i.e., basically get put to sleep, in order to keep its amortized core usage below the quota, which is disastrous for tail latency1.
Since the vast majority of services at Twitter use thread pools that are much larger than their mesos core reservation, when jobs have heavy load, they end up requesting and then using more cores than their reservation and then throttling. This causes services that are provisioned based on load test numbers or observed latency under load to over provision CPU to avoid violating their SLOs. They either have to ask for more CPUs per shard than they actually need or they have to increase the number of shards they use.
Note that Kubernetes uses CFS to implement CPU quotas by default, too.
In the twitter thread about this post, a commenter noted: "'By shrinking the CFS period, the worst case time between quota exhaustion causing throttling and the process group being able to run again is reduced proportionately'. Our P99s at previous gig reduced in line after I petitioned cloud provider to adjust setting." --- this at least seems like a relatively easy setting to tune.
]]>cgroups kubernetes linux k8s cfs scheduling containers quotashttps://pinboard.in/https://pinboard.in/u:jm/b:c6f83c36887b/Fly.io demo using Pi-Hole2021-11-25T10:48:23+00:00
https://fly.io/blog/stuff-your-pi-hole-from-anywhere/
jmfly.io ops deployment cdn docker containershttps://pinboard.in/https://pinboard.in/u:jm/b:cdcabf745817/The 17 Ways to Run Containers on AWS - Last Week in AWS2021-05-31T14:36:45+00:00
https://www.lastweekinaws.com/blog/the-17-ways-to-run-containers-on-aws/?ck_subscriber_id=512829374
jmcontainers aws ec2 eks k8s docker architecturehttps://pinboard.in/https://pinboard.in/u:jm/b:0efe3719a580/Fly.io2021-05-27T10:00:50+00:00
https://fly.io/docs/introduction/
jmcdn serverless docker containers fly.io hosting internet ops platformshttps://pinboard.in/https://pinboard.in/u:jm/b:f9434d355348/Testcontainers2021-03-18T09:49:47+00:00
https://www.testcontainers.org/
jmTestcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
]]>docker java junit testing containers testshttps://pinboard.in/https://pinboard.in/u:jm/b:ab95f5fb86ea/hexops/dockerfile2020-12-14T15:11:20+00:00
https://github.com/hexops/dockerfile
jmdocker best-practices advice tips containers opshttps://pinboard.in/https://pinboard.in/u:jm/b:ec4e95d4e19a/Google Cloud Run2020-01-13T12:47:49+00:00
https://alexolivier.me/posts/deploy-container-stateless-cheap-google-cloud-run-serverless
jmAfter using Cloud Run for over a year now I have never had to touch a server, VM, cluster or anything else. This is truly a deploy and forget service.
Due to it being fully managed there are a few requirements to make your application compatible. The only real one you have to worry about it to ensure your application runs an HTTP server and listens on the port set in the PORT environment variable that is present at runtime of your container.
The beauty of Cloud Run is that it is only ‘running’ your container when it gets traffic. The pricing model is setup that you only pay for the CPU/memory/network bandwidth used when your app is getting requests.
It achieves this by deploying your app on demand when traffic hits the domain name they give you, it hangs around for a bit (undetermined) until after traffic stops, and the the app is torn down. The other way to look at this is autoscaling - when there is no traffic, it scales to 0.
There is a cost in the form of time - if you app takes time to ‘setup’ when it starts up, you will be making your users wait as Cloud Run scales up your application from 0 to 1+ instances. From my own use I’ve found this to be negligible though.
Due to this pricing model I’ve never paid more than a few cents - yes CENTS - a month for all my side projects (10+ deployed currently). This is a factor of the little traffic I get to them so you may need to do the maths for yours - the pricing page is here.
As Cloud Run takes any container image and deploys it, you can use any language you want. Be it Node, Go, Java, PHP or something entirely obscure, as long as it speaks HTTP and listens on the port defined in the PORT environment variable, Cloud Run doesn’t care what you do inside the container.
I've become increasingly impressed by Google's cloud offerings, and this sounds great. Gonna have to give it a go.]]>hosting cloud docker google containers cloud-runhttps://pinboard.in/https://pinboard.in/u:jm/b:baeecc41934e/Nixery2019-05-24T10:47:21+00:00
https://nixery.appspot.com/
jmNixery provides the ability to pull ad-hoc container images from a Docker-compatible registry server. The image names specify the contents the image should contain, which are then retrieved and built by the Nix package manager. Nix is also responsible for the creation of the container images themselves.
e.g. "docker run -ti nixery.appspot.com/shell/htop bash"
]]>docker containers nix nixpkgs packaging deployment opshttps://pinboard.in/https://pinboard.in/u:jm/b:415edb4724cb/Argo Workflows & Pipelines2019-02-28T10:44:37+00:00
https://argoproj.github.io/argo
jmk8s kubernetes docker containers workflow pipelines architecture batch nightly-jobs opshttps://pinboard.in/https://pinboard.in/u:jm/b:0cd7daacad31/aws-lambda-container-image-converter2019-01-28T15:56:02+00:00
https://github.com/awslabs/aws-lambda-container-image-converter/
jmlambda docker aws layers filesystem coding containers serverlesshttps://pinboard.in/https://pinboard.in/u:jm/b:b673d7cf4082/Makisu2018-12-10T11:37:22+00:00
https://eng.uber.com/makisu/
jmwe built our own image building tool, Makisu, a solution that allows for more flexible, faster container image building at scale. Specifically, Makisu:
requires no elevated privileges, making the build process portable.
uses a distributed layer cache to improve performance across a build cluster.
provides flexible layer generation, preventing unnecessary files in images.
is Docker-compatible, supporting multi-stage builds and common build commands.
]]>makisu docker containers ops build mesos kubernetes buildinghttps://pinboard.in/https://pinboard.in/u:jm/b:49f4fc69d9f5/The JVM in Docker 20182018-11-19T11:57:33+00:00
http://batey.info/docker-jvm-k8s.html
jm
Later JDK versions have made it far easier to run a JVM application in a Linux container. The memory support means that if you relied on JVM ergonomics before than you can do the same inside a container where as previously you had to override all memory related settings. The CPU support for containers needs to be carefully evaluated for your application and environment. If you’ve previously set low cpu_shares in environments like Kubernetes to increase utilisation while relying on using up unused cycles then you might get a shock.
]]>jvm docker kubernetes linux containers opshttps://pinboard.in/https://pinboard.in/u:jm/b:521b7b5a67e4/Another reason why your Docker containers may be slow2018-04-06T11:33:20+00:00
https://hackernoon.com/another-reason-why-your-docker-containers-may-be-slow-d37207dec27f
jmlinux fadvise filesystems performance docker containers opshttps://pinboard.in/https://pinboard.in/u:jm/b:9015116d6c92/Containers Patterns2018-02-19T11:01:41+00:00
https://l0rd.github.io/containerspatterns/#1
jmdocker containers design-patterns coding packaging deployment via:fanfhttps://pinboard.in/https://pinboard.in/u:jm/b:0e99583f73a7/Hardening Docker containers and hosts against vulnerabilities: a security toolkit | StackRox: Security built in2018-01-18T17:02:02+00:00
https://www.stackrox.com/post/2017/08/hardening-docker-containers-and-hosts-against-vulnerabilities-a-security-toolkit/
jmdocker security containers linux hardeninghttps://pinboard.in/https://pinboard.in/u:jm/b:8066373d01a3/Securing Docker Containers on AWS | nearForm2018-01-18T17:00:33+00:00
https://www.nearform.com/blog/securing-docker-containers-on-aws/
jmdocker aws security nearform containers linux packerhttps://pinboard.in/https://pinboard.in/u:jm/b:81c78e3e1ed8/Learning to operate Kubernetes reliably2017-12-20T22:44:09+00:00
https://stripe.com/blog/operating-kubernetes
jmstripe kubernetes cron distributed-cron jobs docker containers ops julia-evanshttps://pinboard.in/https://pinboard.in/u:jm/b:639810fe8ecd/AWS re:invent 2017: Container Networking Deep Dive with Amazon ECS (CON401) // Practical Applications2017-12-04T22:06:17+00:00
https://www.awsgeek.com/posts/reinvent2017-con401-notes/
jmreinvent aws containers docker ecs networking sdn opshttps://pinboard.in/https://pinboard.in/u:jm/b:e8c1ee59a9d2/Introducing AWS Fargate – Run Containers without Managing Infrastructure2017-11-29T17:09:02+00:00
https://aws.amazon.com/blogs/aws/aws-fargate/?sc_channel=sm&sc_campaign=AWS_reInvent&sc_publisher=TWITTER&sc_country=re:Invent&sc_geo=GLOBAL&sc_outcome=awareness&trk=sm_EC2_Container_Service_9625c5f9_Fargate_TWITTER&sc_content=EC2_Container_Service_9625c5f9_Fargate&sc_category=Amazon_Fargate&linkId=45301695
jmeks ecs fargate aws services ops containers dockerhttps://pinboard.in/https://pinboard.in/u:jm/b:ea5d427cbdf3/'Lambda and serverless is one of the worst forms of proprietary lock-in we've ever seen in the history of humanity' • The Register2017-11-07T10:25:09+00:00
https://www.theregister.co.uk/2017/11/06/coreos_kubernetes_v_world/
jmThat doesn't mean Polvi is a fan. "Lambda and serverless is one of the worst forms of proprietary lock-in that we've ever seen in the history of humanity," said Polvi, only partly in jest, referring to the most widely used serverless offering, AWS Lambda. "It's seriously as bad as it gets."
He elaborated: "It's code that tied not just to hardware – which we've seen before – but to a data center, you can't even get the hardware yourself. And that hardware is now custom fabbed for the cloud providers with dark fiber that runs all around the world, just for them. So literally the application you write will never get the performance or responsiveness or the ability to be ported somewhere else without having the deployment footprint of Amazon."
Absolutely agreed...]]>lambda amazon aws containers coreos deployment lockin proprietary serverless alex-polvi kuberneteshttps://pinboard.in/https://pinboard.in/u:jm/b:8417fb02298a/Kubernetes Best Practices // Speaker Deck2017-07-24T13:35:34+00:00
https://speakerdeck.com/thesandlord/kubernetes-best-practices
jmk8s kubernetes devops ops containers docker best-practices tips packaginghttps://pinboard.in/https://pinboard.in/u:jm/b:b5d166ee5797/Instead of containerization, give me strong config & deployment primitives2017-04-28T12:42:17+00:00
https://abe-winter.github.io/blues/2017/04/27/config-vs-containers.html
jmdocker containers complaining whinge networking swarm deployment architecture build packaginghttps://pinboard.in/https://pinboard.in/u:jm/b:d1e61e83e357/Capturing all the flags in BSidesSF CTF by pwning Kubernetes/Google Cloud2017-04-24T11:26:35+00:00
https://hackernoon.com/capturing-all-the-flags-in-bsidessf-ctf-by-pwning-our-infrastructure-3570b99b4dd0
jmgce google-cloud kubernetes security docker containers gke ctf hacking exploitshttps://pinboard.in/https://pinboard.in/u:jm/b:95f5883ee483/ctop2017-03-12T23:07:40+00:00
https://bcicen.github.io/ctop/
jmdocker containers top ops go monitoring cpuhttps://pinboard.in/https://pinboard.in/u:jm/b:d168e83ae1a7/pachyderm2017-02-20T10:48:24+00:00
https://github.com/pachyderm/pachyderm
jmThere are two bold new ideas in Pachyderm:
Containers as the core processing primitive
Version Control for data
These ideas lead directly to a system that's much more powerful, flexible and easy to use.
To process data, you simply create a containerized program which reads and writes to the local filesystem. You can use any tools you want because it's all just going in a container! Pachyderm will take your container and inject data into it. We'll then automatically replicate your container, showing each copy a different chunk of data. With this technique, Pachyderm can scale any code you write to process up to petabytes of data (Example: distributed grep).
Pachyderm also version controls all data using a commit-based distributed filesystem (PFS), similar to what git does with code. Version control for data has far reaching consequences in a distributed filesystem. You get the full history of your data, can track changes and diffs, collaborate with teammates, and if anything goes wrong you can revert the entire cluster with one click!
Version control is also very synergistic with our containerized processing engine. Pachyderm understands how your data changes and thus, as new data is ingested, can run your workload on the diff of the data rather than the whole thing. This means that there's no difference between a batched job and a streaming job, the same code will work for both!
]]>analytics data containers golang pachyderm tools data-science docker version-controlhttps://pinboard.in/https://pinboard.in/u:jm/b:29d3b1dc41d5/How-to Debug a Running Docker Container from a Separate Container2017-02-20T10:34:01+00:00
https://medium.com/@rothgar/how-to-debug-a-running-docker-container-from-a-separate-container-983f11740dc6#.almdvzn67
jmstrace docker ops debugging containershttps://pinboard.in/https://pinboard.in/u:jm/b:aca1d05c70a3/Comparing Amazon Elastic Container Service and Google Kubernetes – Medium2017-02-10T11:47:31+00:00
https://medium.com/@betz.mark/comparing-amazon-elastic-container-service-and-google-kubernetes-1c63fbf19ccd#.2hqmy149l
jmkubernetes containers docker opshttps://pinboard.in/https://pinboard.in/u:jm/b:df03888f5b44/Measuring Docker IO overhead - Percona Database Performance Blog2016-11-01T11:59:27+00:00
https://www.percona.com/blog/2016/02/11/measuring-docker-io-overhead/
jmdocker percona overhead mysql deployment performance ops containershttps://pinboard.in/https://pinboard.in/u:jm/b:c324d2498cee/J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required"2016-09-13T10:05:50+00:00
http://www.slideshare.net/dbryant_uk/j1-2015-debugging-java-apps-in-containers-no-heavy-welding-gear-required
jmjava docker ops containershttps://pinboard.in/https://pinboard.in/u:jm/b:80ba8c32a215/Skyliner2016-09-06T13:52:51+00:00
https://blog.skyliner.io/the-happy-genius-of-my-household-2f76efba535a#.19cvn4bj3
jmdocker aws packaging ops devops containers skylinerhttps://pinboard.in/https://pinboard.in/u:jm/b:53b2faf8043f/Some thoughts on operating containers2016-06-15T15:56:41+00:00
https://www.devco.net/archives/2015/03/30/some-thoughts-on-operating-containers.php
jmops containers docker ripienaar packaginghttps://pinboard.in/https://pinboard.in/u:jm/b:0d022492864f/grammarly/rocker2016-05-30T09:23:22+00:00
https://github.com/grammarly/rocker
jmbackward compatible replacement for Dockerfile. Yes, you can take any Dockerfile, rename it to Rockerfile and use rocker build instead of docker build. ... Rocker aims to solve the following use cases, which are painful with plain Docker:
Mount reusable volumes on build stage, so dependency management tools may use cache between builds.
Share ssh keys with build (for pulling private repos, etc.), while not leaving them in the resulting image.
Build and run application in different images, be able to easily pass an artifact from one image to another, ideally have this logic in a single Dockerfile.
Tag/Push images right from Dockerfiles.
Pass variables from shell build command so they can be substituted to a Dockerfile.
And more. These are the most critical issues that were blocking our adoption of Docker at Grammarly.
The most challenging part is caching. While implementing those features seems to be not a big deal, it's not trivial to do that just by utilising Docker’s image cache (the one that docker build does). Actually, it is the main reason why those features are still not in Docker. With Rocker we achieve this by introducing a set of trade-offs. Search this page for "trade-off" to find out more details.
]]>docker rocker build containers dockerfileshttps://pinboard.in/https://pinboard.in/u:jm/b:bc02eb9dcc1d/fiunchinho/dockerize-me2016-05-30T09:19:43+00:00
https://github.com/fiunchinho/dockerize-me
jmdocker dockerfile images build best-practices alpine containershttps://pinboard.in/https://pinboard.in/u:jm/b:f1c3e07194ae/GitLab Container Registry2016-05-24T07:17:17+00:00
https://about.gitlab.com/2016/05/23/gitlab-container-registry/
jmgitlab github git ci cd containers docker deployment codinghttps://pinboard.in/https://pinboard.in/u:jm/b:5378add3a162/The Challenges of Container Configuration // Speaker Deck2016-05-03T14:34:45+00:00
https://speakerdeck.com/garethr/the-challenges-of-container-configuration
jmdocker metadata configuration build devops dev containers slidfeshttps://pinboard.in/https://pinboard.in/u:jm/b:0098762f74b7/A Decade Of Container Control At Google2016-04-04T11:03:42+00:00
http://www.nextplatform.com/2016/03/22/decade-container-control-google/
jmThe big thing that can be gleaned from the latest paper out of Google on its container controllers is that the shift from bare metal to containers is a profound one – something that may not be obvious to everyone seeking containers as a better way – and we think cheaper way – of doing server virtualization and driving up server utilization higher. Everything becomes application-centric rather than machine-centric, which is the nirvana that IT shops have been searching for. The workload schedulers, cluster managers, and container controllers work together to get the right capacity to the application when it needs it, whether it is a latency-sensitive job or a batch job that has some slack in it, and all that the site recovery engineers and developers care about is how the application is performing and they can easily see that because all of the APIs and metrics coming out of them collect data at the application level, not on a per-machine basis. To do this means adopting containers, period. There is no bare metal at Google, and let that be a lesson to HPC shops or other hyperscalers or cloud builders that think they need to run in bare metal mode.
]]>google containers kubernetes borg bare-metal opshttps://pinboard.in/https://pinboard.in/u:jm/b:05be814669ba/10 things to avoid in docker containers2016-03-01T11:43:23+00:00
http://developerblog.redhat.com/2016/02/24/10-things-to-avoid-in-docker-containers/
jmbest-practices docker ops containers tipshttps://pinboard.in/https://pinboard.in/u:jm/b:353cf8817195/Unikernels are unfit for production - Blog - Joyent2016-01-22T17:48:12+00:00
https://www.joyent.com/blog/unikernels-are-unfit-for-production
jmunikernels flavour-of-the-month devops joyent bryan-cantrill docker containers opshttps://pinboard.in/https://pinboard.in/u:jm/b:fe36c0ed0b66/About Microservices, Containers and their Underestimated Impact on Network Performance2016-01-12T21:57:04+00:00
http://www.slideshare.net/i21aneka/about-microservices-containers-and-their-underestimated-impact-on-network-performance
jmdocker performance network containers sdn ops networking microserviceshttps://pinboard.in/https://pinboard.in/u:jm/b:673549b1a296/Introducing dumb-init, an init system for Docker containers2016-01-07T12:20:55+00:00
http://engineeringblog.yelp.com/2016/01/dumb-init-an-init-for-docker.html
jmdocker tools yelp init containers signals unix linux dumb-inithttps://pinboard.in/https://pinboard.in/u:jm/b:507024699c64/Amazon EC2 Container Registry2015-12-22T10:45:06+00:00
https://aws.amazon.com/ecr/
jmecs docker registry ops containers awshttps://pinboard.in/https://pinboard.in/u:jm/b:fbd4bd4a0cbb/Why We Chose Kubernetes Over ECS2015-12-14T16:23:34+00:00
https://railsadventures.wordpress.com/2015/12/06/why-we-chose-kubernetes-over-ecs/
jm3 months ago when we, at nanit.com, came to evaluate which Docker orchestration framework to use, we gave ECS the first priority. We were already familiar with AWS services, and since we already had our whole infrastructure there, it was the default choice. After testing the service for a while we had the feeling it was not mature enough and missing some key features we needed (more on that later), so we went to test another orchestration framework: Kubernetes. We were glad to discover that Kubernetes is far more comprehensive and had almost all the features we required. For us, Kubernetes won ECS on ECS’s home court, which is AWS.
]]>kubernetes ecs docker containers aws ec2 opshttps://pinboard.in/https://pinboard.in/u:jm/b:9426fd8edac6/Petabyte-Scale Data Pipelines with Docker, Luigi and Elastic Spot Instances — AdRoll2015-09-28T20:07:14+00:00
http://tech.adroll.com/blog/data/2015/09/22/data-pipelines-docker.html
jmdata-pipelines docker luigi containers workflowhttps://pinboard.in/https://pinboard.in/u:jm/b:c60ffe872d06/Anatomy of a Modern Production Stack2015-09-18T16:37:49+00:00
http://www.eightypercent.net/post/layers-in-the-stack.html
jmarchitecture ops stack docker containerization deployment containers rkt coreos prod monitoring xooglershttps://pinboard.in/https://pinboard.in/u:jm/b:6929918b25c5/Why Docker is Not Yet Succeeding Widely in Production2015-07-29T12:52:39+00:00
http://sirupsen.com/production-docker/
jmdocker containers devops deployment releases linux opshttps://pinboard.in/https://pinboard.in/u:jm/b:f942eafba4ab/Deep Dive Into Docker Storage Drivers2015-07-20T17:08:55+00:00
http://jpetazzo.github.io/assets/2015-07-01-deep-dive-into-docker-storage-drivers.html#43
jmdocker overlayfs aufs btrfs filesystems ops linux containershttps://pinboard.in/https://pinboard.in/u:jm/b:8e7e75a8ed4e/Google Cloud Platform announces new Container Registry2015-06-23T11:21:50+00:00
http://googlecloudplatform.blogspot.ie/2015/06/Container-Engine-Container-Registry-Updates-New-Features-Pricing.html
jmGoogle Container Registry helps make it easy for you to store your container images in a private and encrypted registry, built on Cloud Platform. Pricing for storing images in Container Registry is simple: you only pay Google Cloud Storage costs. Pushing images is free, and pulling Docker images within a Google Cloud Platform region is free (Cloud Storage egress cost when outside of a region).
Container Registry is now ready for production use:
* Encrypted and Authenticated - Your container images are encrypted at rest, and access is authenticated using Cloud Platform OAuth and transmitted over SSL
* Fast - Container Registry is fast and can handle the demands of your application, because it is built on Cloud Storage and Cloud Networking.
* Simple - If you’re using Docker, just tag your image with a gcr.io tag and push it to the registry to get started. Manage your images in the Google Developers Console.
* Local - If your cluster runs in Asia or Europe, you can now store your images in ASIA or EU specific repositories using asia.gcr.io and eu.gcr.io tags.
]]>docker registry google gcp containers cloud-storage ops deploymenthttps://pinboard.in/https://pinboard.in/u:jm/b:dd814f15bb49/Kubernetes for developers2015-05-24T08:18:52+00:00
https://medium.com/fabric8-io/kubernetes-for-developers-2a9c7202fcd3
jmkubernetes ops docker containers rocket deployment packaginghttps://pinboard.in/https://pinboard.in/u:jm/b:2bae4eb96313/Deploying Elastic Beanstalk Applications from Docker Containers - Elastic Beanstalk2015-05-19T10:19:06+00:00
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker.html
jmdocker aws ec2 beanstalk deployment ops containershttps://pinboard.in/https://pinboard.in/u:jm/b:892d57be519a/Eric Brewer interview on Kubernetes2015-05-18T21:46:37+00:00
https://medium.com/s-c-a-l-e/google-systems-guru-explains-why-containers-are-the-future-of-computing-87922af2cf95
jmWhat is the relationship between Kubernetes, Borg and Omega (the two internal resource-orchestration systems Google has built)?
I would say, kind of by definition, there’s no shared code but there are shared people.
You can think of Kubernetes — especially some of the elements around pods and labels — as being lessons learned from Borg and Omega that are, frankly, significantly better in Kubernetes. There are things that are going to end up being the same as Borg — like the way we use IP addresses is very similar — but other things, like labels, are actually much better than what we did internally.
I would say that’s a lesson we learned the hard way.
]]>google architecture kubernetes docker containers borg omega deployment opshttps://pinboard.in/https://pinboard.in/u:jm/b:7ab8ae0d6508/Kubernetes compared to Borg2015-04-24T13:57:45+00:00
http://blog.kubernetes.io/2015/04/borg-predecessor-to-kubernetes.html
jmgoogle ops kubernetes borg containers docker networkinghttps://pinboard.in/https://pinboard.in/u:jm/b:fdfe53933320/Cluster-Based Architectures Using Docker and Amazon EC2 Container Service2015-04-24T13:56:08+00:00
https://medium.com/aws-activate-startup-blog/cluster-based-architectures-using-docker-and-amazon-ec2-container-service-f74fa86254bf
jmIn this post, we’re going to take a deeper dive into the architectural concepts underlying cluster computing using container management frameworks such as ECS. We will show how these frameworks effectively abstract the low-level resources such as CPU, memory, and storage, allowing for highly efficient usage of the nodes in a compute cluster. Building on some of the concepts detailed in the earlier posts, we will discover why containers are such a good fit for this type of abstraction, and how the Amazon EC2 Container Service fits into the larger ecosystem of cluster management frameworks.
]]>docker aws ecs ec2 ops hosting containers mesos clustershttps://pinboard.in/https://pinboard.in/u:jm/b:5c48004c7952/Microservices and elastic resource pools with Amazon EC2 Container Service2015-04-16T16:06:38+00:00
http://www.slideshare.net/nathariel/microservices-and-elastic-resource-pools-with-amazon-ec2-container-service
jmversioning microservices hailo aws ec2 ecs docker containers scheduling allocation deployment provisioning qoshttps://pinboard.in/https://pinboard.in/u:jm/b:6721d7ee3f45/Massive thumbs-down Docker review2015-02-12T21:49:16+00:00
http://iops.io/blog/docker-hype/
jmcontainers devops docker ops lxc bugs reviewshttps://pinboard.in/https://pinboard.in/u:jm/b:c1f5de54f4e2/EC2 Container Service Hands On2014-12-31T00:10:48+00:00
http://blog.bwhaley.com/ec2-container-service-first-impressions
jmThere is no native integration with Autoscaling or ELBs.
]]>ec2 containers docker ecs opshttps://pinboard.in/https://pinboard.in/u:jm/b:4dd2bde89044/CoreOS is building a container runtime, Rocket2014-12-02T10:15:54+00:00
https://coreos.com/blog/rocket/
jmWhen Docker was first introduced to us in early 2013, the idea of a “standard container” was striking and immediately attractive: a simple component, a composable unit, that could be used in a variety of systems. The Docker repository included a manifesto of what a standard container should be. This was a rally cry to the industry, and we quickly followed. Brandon Philips, co-founder/CTO of CoreOS, became a top Docker contributor, and now serves on the Docker governance board. CoreOS is one of the most widely used platforms for Docker containers, and ships releases to the community hours after they happen upstream. We thought Docker would become a simple unit that we can all agree on.
Unfortunately, a simple re-usable component is not how things are playing out. Docker now is building tools for launching cloud servers, systems for clustering, and a wide range of functions: building images, running images, uploading, downloading, and eventually even overlay networking, all compiled into one monolithic binary running primarily as root on your server. The standard container manifesto was removed. We should stop talking about Docker containers, and start talking about the Docker Platform. It is not becoming the simple composable building block we had envisioned.
]]>coreos docker linux containers open-source politics rockethttps://pinboard.in/https://pinboard.in/u:jm/b:0202e73e2671/Docker at Shopify: How we built containers that power over 100,000 onl2014-11-25T12:03:56+00:00
http://www.shopify.com/technology/15934308-docker-at-shopify-how-we-built-containers-that-power-over-100-000-online-shops
jmdocker devops deployment ops shopify containers productionhttps://pinboard.in/https://pinboard.in/u:jm/b:14b1174b98f7/veggiemonk/awesome-docker2014-11-06T16:49:12+00:00
https://github.com/veggiemonk/awesome-docker
jmlinux sysadmin docker ops devops containers hostinghttps://pinboard.in/https://pinboard.in/u:jm/b:3869f1d68d0a/Jonathan Bergknoff: Building good docker images2014-10-28T17:11:09+00:00
http://jonathan.bergknoff.com/journal/building-good-docker-images
jmdevops reference docker tips ops containers linuxhttps://pinboard.in/https://pinboard.in/u:jm/b:d040093ed03e/"Linux Containers And The Future Cloud" [slides]2014-10-07T15:47:02+00:00
http://media.wix.com/ugd/295986_d5059f95a78e451db5de3d54f711e45d.pdf
jmlsx docker criu namespaces cgroups linux via:lusis ops containers rami-rosen presentationshttps://pinboard.in/https://pinboard.in/u:jm/b:ceb59e520a8d/AWS Elastic Beanstalk for Docker2014-04-24T09:41:58+00:00
http://aws.typepad.com/aws/2014/04/aws-elastic-beanstalk-for-docker.html
jmaws amazon docker ec2 beanstalk ops containers linuxhttps://pinboard.in/https://pinboard.in/u:jm/b:8a34d5efadca/Docker all the things at Atlassian: automation and wiring2013-11-28T17:42:04+00:00
https://blogs.atlassian.com/2013/11/docker-all-the-things-at-atlassian-automation-and-wiring/
jmdocker infrastructure devops ops deployment lxc containers linuxhttps://pinboard.in/https://pinboard.in/u:jm/b:cabd127a5fe5/Is Google building a hulking floating data center in SF Bay?2013-10-25T23:23:38+00:00
http://news.cnet.com/8301-1023_3-57608585-93/is-google-building-a-hulking-floating-data-center-in-sf-bay/
jmgoogle data-centers bay-area ships containers shipping sea wave-power treasure-islandhttps://pinboard.in/https://pinboard.in/u:jm/b:c6bfad3ef93d/Dockerize2013-10-25T20:47:32+00:00
http://blog.howareyou.com/post/65048170054/continuous-delivery-with-docker-and-jenkins-part-ii
jmdocker provisioning vms containers dockerize jenkins continuous-delivery continuous-integrationhttps://pinboard.in/https://pinboard.in/u:jm/b:45b614651b69/