Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmDocker build cache sharing on multi-hosts with BuildKit and buildx2022-06-01T14:43:44+00:00
https://medium.com/titansoft-engineering/docker-build-cache-sharing-on-multi-hosts-with-buildkit-and-buildx-eb8f7005918e
jmdocker building compilation ci tweaks containershttps://pinboard.in/https://pinboard.in/u:jm/b:ff2e0294d357/Automating safe, hands-off deployments2020-06-23T22:53:17+00:00
https://aws.amazon.com/builders-library/automating-safe-hands-off-deployments/
jmEach team needs to balance the safety of small-scoped deployments with the speed at which we can deliver changes to customers in all Regions. Deploying changes to 24 Regions or 76 Availability Zones through the pipeline one at a time has the lowest risk of causing broad impact, but it could take weeks for the pipeline to deliver a change to customers globally. We have found that grouping deployments into “waves” of increasing size, as seen in the previous sample prod pipeline, helps us achieve a good balance between deployment risk and speed. Each wave’s stage in the pipeline orchestrates deployments to a group of Regions, with changes being promoted from wave to wave. New changes can enter the production phase of the pipeline at any time. After a set of changes is promoted from the first step to the second step in wave 1, the next set of changes from gamma is promoted into the first step of wave 1, so we don’t end up with large bundles of changes waiting to be deployed to production.
The first two waves in the pipeline build the most confidence in the change: The first wave deploys to a Region with a low number of requests to limit the possible impact of the first production deployment of the new change. The wave deploys to only one Availability Zone (or cell) at a time within that Region to cautiously deploy the change across the Region. The second wave then deploys to one Availability Zone (or cell) at a time in a Region with a high number of requests where it is highly likely that customers will exercise all the new code paths and where we get good validation of the changes.
After we have higher confidence in the safety of the change from the initial pipeline waves’ deployments, we can deploy to more and more Regions in parallel in the same wave. For example, the previous sample prod pipeline deploys to three Regions in wave 3, then to up to 12 Regions in wave 4, then to the remaining Regions in wave 5. The exact number and choice of Regions in each of these waves and the number of waves in a service team’s pipeline depend on the individual service’s usage patterns and scale. The later waves in the pipeline still help us achieve our objective to prevent negative impact to multiple Availability Zones in the same Region. When a wave deploys to multiple Regions in parallel, it follows the same cautious rollout behavior for each Region that was used in the initial waves. Each step in the wave only deploys to a single Availability Zone or cell from each Region in the wave.
]]>automation ops devops amazon aws deployment waves az multi-region ci cdhttps://pinboard.in/https://pinboard.in/u:jm/b:7ecaa0ea15ea/AWS CodeBuild Plugin - Jenkins - Jenkins Wiki2017-12-13T11:48:59+00:00
https://wiki.jenkins.io/display/JENKINS/AWS+CodeBuild+Plugin
jmjenkins hacks aws codebuild build coding cihttps://pinboard.in/https://pinboard.in/u:jm/b:747553640ba8/LambCI — a serverless build system2017-09-27T10:41:32+00:00
https://medium.com/@hichaelmart/lambci-4c3e29d6599b
jmLambCI is a tool I began building over a year ago to run tests on our pull requests and branches at Uniqlo Mobile. Inspired at the inaugural ServerlessConf a few weeks ago, I recently put some work into hammering it into shape for public consumption.
It was borne of a dissatisfaction with the two current choices for automated testing on private projects. You can either pay for it as a service (Travis, CircleCI, etc) — where 3 developers needing their own build containers might set you back a few hundred dollars a month. Or you can setup a system like Jenkins, Strider, etc and configure and manage a database, a web server and a cluster of build servers .
In both cases you’ll be under- or overutilized, waiting for servers to free up or paying for server power you’re not using. And this, for me, is where the advantage of a serverless architecture really comes to light: 100% utilization, coupled with instant invocations.
]]>lambci ci builds jenkins testing codinghttps://pinboard.in/https://pinboard.in/u:jm/b:441b1b37cc67/Pipeline Development Tools2017-07-27T09:32:26+00:00
https://jenkins.io/blog/2017/05/18/pipeline-dev-tools/
jmpipelines jenkins ci coding devhttps://pinboard.in/https://pinboard.in/u:jm/b:f5e98072bc88/The Push Train2017-05-10T11:06:53+00:00
http://pushtrain.club/
jmIt’s notable that almost all of the hard things we dealt with were social problems. Some of these solutions involved writing code, but the hard part was the human organization. The hard parts in were maintaining a sense of community ownership over the state of the whole system.
]]>etsy ci cd deployment devops deploys dan-mckinley mcfunley presentationshttps://pinboard.in/https://pinboard.in/u:jm/b:023c5d8b0b45/GitLab Container Registry2016-05-24T07:17:17+00:00
https://about.gitlab.com/2016/05/23/gitlab-container-registry/
jmgitlab github git ci cd containers docker deployment codinghttps://pinboard.in/https://pinboard.in/u:jm/b:5378add3a162/Jenkins 2.02016-03-27T10:20:43+00:00
https://jenkins.io/2.0/#pipelines
jmvia:eric jenkins ci cd deployment pipelines testing automation buildhttps://pinboard.in/https://pinboard.in/u:jm/b:435381483deb/remind101/conveyor2015-10-13T11:04:10+00:00
https://github.com/remind101/conveyor
jmbuild ci docker github gohttps://pinboard.in/https://pinboard.in/u:jm/b:ac78931ba0b0/Docker image creation, tagging and traceability in Shippable2015-08-25T20:47:44+00:00
http://blog.shippable.com/docker-image-creation-tagging-traceability
jmshippable docker ci ops dev continuous-integrationhttps://pinboard.in/https://pinboard.in/u:jm/b:aa573e53951e/'Continuous Deployment: The Dirty Details'2015-04-22T10:08:00+00:00
http://www.slideshare.net/mikebrittain/mbrittain-continuous-deploymentalm3public
jmcd deploy etsy slides migrations database schema ops ci version-control feature-flagshttps://pinboard.in/https://pinboard.in/u:jm/b:7b09e64e7d8f/[Nix-dev] Pulling a programs source code from a git repo2015-03-23T13:46:08+00:00
http://lists.science.uu.nl/pipermail/nix-dev/2014-January/012422.html
jmnix packaging build dev cihttps://pinboard.in/https://pinboard.in/u:jm/b:79c49287b925/Managing private Nix packages outside the Nixpkgs tree2015-03-23T12:25:15+00:00
http://sandervanderburg.blogspot.ie/2014/07/managing-private-nix-packages-outside.html
jmnix packaging deployment ops build ci nixpkgshttps://pinboard.in/https://pinboard.in/u:jm/b:fe5585e44425/Hydra2015-03-23T12:05:16+00:00
http://nixos.org/hydra/
jma Nix-based continuous build system, released under the terms of the GNU GPLv3 or (at your option) any later version. It continuously checks out sources of software projects from version management systems to build, test and release them. The build tasks are described using Nix expressions. This allows a Hydra build task to specify all the dependencies needed to build or test a project. It supports a number of operating systems, such as various GNU/Linux flavours, Mac OS X, and Windows.
]]>nixos nix hydra ci cd gplv3 continuous-integrationhttps://pinboard.in/https://pinboard.in/u:jm/b:8d28e6c1fdf8/Concourse2015-03-11T23:51:36+00:00
http://concourse.ci/
jmConcourse is a CI system composed of simple tools and ideas. It can express entire pipelines, integrating with arbitrary resources, or it can be used to execute one-off builds, either locally or in another CI system.
]]>ci concourse-ci build deployment continuous-integration continuous-deployment devopshttps://pinboard.in/https://pinboard.in/u:jm/b:ae3dccf71ef0/Try Server2015-03-07T21:05:11+00:00
http://www.chromium.org/developers/testing/try-server-usage/design
jmThe try server runs a similar configuration to the continuous integration server, except that it is triggered not on commits but on "try job request", in order to test code pre-commit.
See also https://wiki.mozilla.org/ReleaseEngineering/TryServer for the Moz take on it.]]>build ci integration try-server jenkins buildbot chromium developmenthttps://pinboard.in/https://pinboard.in/u:jm/b:2d5c59dad02c/How Etsy Does Continuous Integration for Mobile Apps2014-12-15T15:57:27+00:00
http://www.infoq.com/news/2014/11/continuous-integration-mobile
jmdojos testing ci cd builds etsy mobile ios shenzen trylib jenkins tester-dojoshttps://pinboard.in/https://pinboard.in/u:jm/b:c4b38e7ac959/Hudson's future2011-01-11T16:22:30+00:00
http://www.hudson-labs.org/content/hudsons-future
jmoracle hudson exodus stupid ci via:jameschttps://pinboard.in/u:jm/b:45574261e5c7/Kohsuke "Hudson" Kawaguchi leaving Sun2010-04-21T12:56:24+00:00
http://nighthacks.com/roller/jag/entry/hudson_s_new_life
jmhudson ci sun oracle acquisitions startups buildhttps://pinboard.in/u:jm/b:8ff943993235/Hudson at PyCon | the official hudson weblog2010-03-16T23:35:55+00:00
http://blog.hudson-ci.org/content/hudson-pycon?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+ContinuousBlog+%28continuous+blog%29
jmhudson buildbot python ci junithttps://pinboard.in/u:jm/b:655b2c9c0439/