https://pinboard.in/u:jm/public/ recent bookmarks from jm 2025-09-16T13:59:07+00:00 https://tanelpoder.com/posts/xcapture-v3-alpha-ebpf-performance-analysis-with-duckdb/ jm os debugging sysadmin ops monitoring xtop xcapture ebpf linux duckdb syscalls https://pinboard.in/ https://pinboard.in/u:jm/b:b730ad37aa33/ 2024-07-23T08:49:30+00:00 https://github.com/remuslazar/homeassistant-carwings jm todo homeassistant nissan leaf cars automation monitoring smarthome home https://pinboard.in/ https://pinboard.in/u:jm/b:9bd9c5690f8b/ 2024-02-26T12:11:22+00:00 https://github.com/louislam/uptime-kuma jm monitoring uptime network-monitoring networking ops via:itc via:tristam https://pinboard.in/ https://pinboard.in/u:jm/b:01d82b0fa274/ 2023-10-12T17:21:00+00:00 https://berthub.eu/articles/posts/client-side-scanning-dutch-parliament/ jmnow we are talking about 500 million Europeans, and saying, “Let’s just apply those scanners!” That is incredible. ... If we approve this as a country, if we as the Netherlands vote in favour of this in Europe and say, “Do it,” we will cross a threshold that we have never crossed before. Namely, every European must be monitored with a computer program, with a technology [...] of which the vast, overwhelming majority of scientists have said, “It is not finished.” I mentioned earlier the example that the Dutch National Forensic Institute says, “We cannot do this by hand.” The EU has now said, “Our computer can do that.” 420 scientists have signed a petition saying, “We know this technology, some of us invented it, we just can’t do it.” We can’t even make a reliable spam filter. Making a spam filter is exactly the same technology, by the way, but then much easier. It just doesn’t work that well, but the consequences aren’t that scary for a spam filter. Nevertheless, there are now MPs who say, “Well, I feel this is going to work. I have confidence in this.” While the scientists, including the real scientists who came here tonight, say, “Well, we don’t see how this could work well enough”. And then government then says, “Let’s start this experiment with those 500 million Europeans.” ]]> eu scanning css chatcontrol internet monitoring surveillance bert-hubert https://pinboard.in/ https://pinboard.in/u:jm/b:b8823261c362/ 2022-11-21T11:22:30+00:00 https://www.esbnetworks.ie/existing-connections/meters-and-readings/my-smart-data jm smart-meters esb electricity consumption monitoring ireland https://pinboard.in/ https://pinboard.in/u:jm/b:279cc28c14fe/ 2022-10-03T17:07:00+00:00 https://hveem.no/visualizing-latency-variance-with-grafana jm grafana influxdb monitoring smokeping dataviz https://pinboard.in/ https://pinboard.in/u:jm/b:d4455d46fd68/ 2022-09-07T12:06:04+00:00 https://brunty.me/post/energy-monitoring-2022-edition/ jm home energy monitoring tasmota plugs electricity homeassistant https://pinboard.in/ https://pinboard.in/u:jm/b:7ed8494cc907/ 2022-09-01T11:37:13+00:00 https://pyroscope.io/ jmLightning Fast - Doesn't matter if you're looking at 10 seconds or 10 months of profiling data — the queries are always fast thanks to our custom designed storage engine. Minimum Overhead - Pyroscope doesn't affect performance of your applications, thanks to the use of sampling profiling technology. Cost-Effective - Pyroscope uses a custom data storage engine and stores profiling data very efficiently, making it economically viable to store profiling data from all of your apps for years. ]]> monitoring observability profiling pyroscope continuous-profiling performance https://pinboard.in/ https://pinboard.in/u:jm/b:3e4fd17a7bf9/ 2022-08-10T17:07:19+00:00 https://www.browse.ai/ jm scraping web monitoring http https://pinboard.in/ https://pinboard.in/u:jm/b:d287c7100183/ 2022-05-31T08:11:25+00:00 https://www.sciencedirect.com/science/article/pii/S0048969722029254 jm dublin ireland diseases monitoring sars-cov-2 covid-19 water poop sewage https://pinboard.in/ https://pinboard.in/u:jm/b:8c6b4f4274f5/ 2022-01-21T09:48:05+00:00 https://nelsonslog.wordpress.com/2022/01/20/my-custom-solar-monitoring-system-pvs6/ jm solar-power solar home graphs grafana dashboards monitoring telegraf via:nelson https://pinboard.in/ https://pinboard.in/u:jm/b:e52ca6a193d1/ 2022-01-06T10:30:53+00:00 https://visualping.io/ jm monitoring tools web alerting change-detection diffs https://pinboard.in/ https://pinboard.in/u:jm/b:f9991be67017/ 2021-01-12T09:46:55+00:00 https://github.com/hubblo-org/scaphandre/ jmElectrical power consumption metrology agent. Let scaph dive and bring back the metrics that will help you make your systems and applications more sustainable! ]]> monitoring metrics power energy sustainability green ops https://pinboard.in/ https://pinboard.in/u:jm/b:63d4f4e3bb76/ 2020-08-09T22:04:28+00:00 https://aws.amazon.com/builders-library/building-dashboards-for-operational-visibility/ jm dashboards aws monitoring metrics alerts amazon https://pinboard.in/ https://pinboard.in/u:jm/b:bf2682c8188c/ 2020-02-25T16:04:17+00:00 https://www.hpsc.ie/a-z/respiratory/influenza/seasonalinfluenza/surveillance/influenzasurveillancereports/20192020season/ jm hse monitoring diseases outbreaks flu influenza health medicine ireland https://pinboard.in/ https://pinboard.in/u:jm/b:a7fd5ff86566/ 2020-02-10T12:06:19+00:00 https://square.github.io/cubism/ jmHorizon charts reduce vertical space without losing resolution. Larger values are overplotted in successively darker colors, while negative values are offset to descend from the top. As you increase the number of colors, you reduce the required vertical space [...] . By combining position and color, horizon charts improve perception: position is highly effective at discriminating small changes, while color differentiates large changes. To further increase data density, Cubism favors per-pixel metrics where each pixel encodes a distinct point in time. Cubism also includes thoughtful default colors by Cynthia Brewer. ]]> charts javascript visualization d3 charting graphs horizon-charts ui monitoring https://pinboard.in/ https://pinboard.in/u:jm/b:9282a22da2ab/ 2020-02-10T11:59:56+00:00 https://arxiv.org/pdf/2001.06561.pdf jm histograms aggregation quantiles percentiles measurement graphs data-structures summaries latency monitoring approximation papers https://pinboard.in/ https://pinboard.in/u:jm/b:763bb560c156/ 2020-01-09T17:19:22+00:00 https://sysdig.com/blog/golden-signals-kubernetes/ jm kubernetes monitoring sysdig golden-data k8s golden-signals metrics latency errors https://pinboard.in/ https://pinboard.in/u:jm/b:c8f356dfeae1/ 2020-01-06T11:43:49+00:00 https://www.theguardian.com/info/developer-blog/2016/dec/05/testing-in-production-how-we-combined-tests-with-monitoring jm testing monitoring ops devops the-guardian prod production releases https://pinboard.in/ https://pinboard.in/u:jm/b:083ed1350efa/ 2019-01-21T14:31:56+00:00 https://nickcraver.com/blog/2018/11/29/stack-overflow-how-we-do-monitoring/ jm logging monitoring stack-overflow dotnet ops metrics https://pinboard.in/ https://pinboard.in/u:jm/b:1fc7afb1f694/ 2018-10-23T13:23:40+00:00 http://www.livpi.com/store/c1/Featured_Products.html jm co2 air quality monitoring metrics health home raspberry-pi hardware to-get https://pinboard.in/ https://pinboard.in/u:jm/b:b2bfce088278/ 2018-05-18T21:54:18+00:00 https://improbable.io/games/blog/thanos-prometheus-at-scale jm devops monitoring tools prometheus ops metrics https://pinboard.in/ https://pinboard.in/u:jm/b:3588acf20d03/ 2018-01-29T10:46:24+00:00 https://www.datadoghq.com/blog/rabbitmq-monitoring/ jm rabbitmq mq monitoring metrics graphite ops https://pinboard.in/ https://pinboard.in/u:jm/b:e70e84e4c6aa/ 2018-01-18T14:24:25+00:00 https://opensource.googleblog.com/2018/01/opencensus.html jm google monitoring service-metrics metrics census opencensus open-source tracing zipkin prometheus https://pinboard.in/ https://pinboard.in/u:jm/b:65886e149118/ 2018-01-04T23:35:19+00:00 https://www.reddit.com/r/diabetes/comments/7nvufy/my_bedroom_lights_turn_on_when_my_blood_sugar/ jm iot via:fp via:eatpaste blood health diabetes monitoring home https://pinboard.in/ https://pinboard.in/u:jm/b:a09a476be990/ 2017-12-18T11:34:39+00:00 https://sourcegraph.github.io/checkup/ jm go ops monitoring uptime health-checks status-pages status golang s3 https://pinboard.in/ https://pinboard.in/u:jm/b:cb7582683d84/ 2017-11-14T23:05:56+00:00 https://blog.hostedgraphite.com/2017/07/06/continuous-self-testing-at-hosted-graphite-why-we-send-external-canaries-every-second/ jm graphite hostedgraphite monitoring canaries udp alerting ops https://pinboard.in/ https://pinboard.in/u:jm/b:398772f5b2a3/ 2017-09-25T09:51:26+00:00 https://drive.google.com/file/d/0BzRE_fwreoDQNzUybnRFOHpWZTA/view jm metrics monitoring alerting prometheus grafana nagios https://pinboard.in/ https://pinboard.in/u:jm/b:7a0ed0cf1b59/ 2017-08-21T13:17:17+00:00 http://www.brendangregg.com/blog/2017-08-08/linux-load-averages.html jmIn 1993, a Linux engineer found a nonintuitive case with load averages, and with a three-line patch changed them forever from "CPU load averages" to what one might call "system load averages." His change included tasks in the uninterruptible state, so that load averages reflected demand for disk resources and not just CPUs. These system load averages count the number of threads working and waiting to work, and are summarized as a triplet of exponentially-damped moving sum averages that use 1, 5, and 15 minutes as constants in an equation. This triplet of numbers lets you see if load is increasing or decreasing, and their greatest value may be for relative comparisons with themselves. ]]> load monitoring linux unix performance ops brendan-gregg history cpu https://pinboard.in/ https://pinboard.in/u:jm/b:00a0f4ee73d7/ 2017-03-12T23:07:40+00:00 https://bcicen.github.io/ctop/ jm docker containers top ops go monitoring cpu https://pinboard.in/ https://pinboard.in/u:jm/b:d168e83ae1a7/ 2017-02-06T12:08:26+00:00 https://code.facebook.com/posts/952820474848503/beringei-a-high-performance-time-series-storage-engine/ jmBeringei is different from other in-memory systems, such as memcache, because it has been optimized for storing time series data used specifically for health and performance monitoring. We designed Beringei to have a very high write rate and a low read latency, while being as efficient as possible in using RAM to store the time series data. In the end, we created a system that can store all the performance and monitoring data generated at Facebook for the most recent 24 hours, allowing for extremely fast exploration and debugging of systems and services as we encounter issues in production. Data compression was necessary to help reduce storage overhead. We considered several existing compression schemes and rejected the techniques that applied only to integer data, used approximation techniques, or needed to operate on the entire dataset. Beringei uses a lossless streaming compression algorithm to compress points within a time series with no additional compression used across time series. Each data point is a pair of 64-bit values representing the timestamp and value of the counter at that time. Timestamps and values are compressed separately using information about previous values. Timestamp compression uses a delta-of-delta encoding, so regular time series use very little memory to store timestamps. From analyzing the data stored in our performance monitoring system, we discovered that the value in most time series does not change significantly when compared to its neighboring data points. Further, many data sources only store integers (despite the system supporting floating point values). Knowing this, we were able to tune previous academic work to be easier to compute by comparing the current value with the previous value using XOR, and storing the changed bits. Ultimately, this algorithm resulted in compressing the entire data set by at least 90 percent. ]]> beringei compression facebook monitoring tsd time-series storage architecture https://pinboard.in/ https://pinboard.in/u:jm/b:3d15d674f951/ 2016-07-20T10:14:40+00:00 http://ispokemongodownornot.com/ jm datadog games monitoring pokemon-go pokemon uptime https://pinboard.in/ https://pinboard.in/u:jm/b:a69910d88699/ 2016-07-14T14:53:47+00:00 https://docs.google.com/document/d/199PqyG3UsyXlwieHaqbGiWVa8eMWi8zzAn0YfcApr8Q/edit jm. Seem pretty reasonable]]> monitoring sysadmin alerting alerts nagios pager ops sre rob-ewaschuk https://pinboard.in/ https://pinboard.in/u:jm/b:6c245de62c9b/ 2016-07-06T09:54:16+00:00 https://blog.raintank.io/dear-graphite-users-and-developers/ jm graphite metrics monitoring ops open-source grafana raintank https://pinboard.in/ https://pinboard.in/u:jm/b:f5504b4e51fb/ 2016-05-13T11:02:51+00:00 https://aws.amazon.com/blogs/apn/key-metrics-for-amazon-aurora/?adbsc=APN_Blog_20160512_61520166&adbid=730767018157170688&adbpl=tw&adbpr=509089629 jm datadog metrics aurora aws rds monitoring ops https://pinboard.in/ https://pinboard.in/u:jm/b:06d10a828d7e/ 2016-05-01T09:19:07+00:00 https://coreos.com/blog/coreos-and-prometheus-improve-cluster-monitoring.html jmEnabling GIFEE — Google Infrastructure for Everyone Else — is a primary mission at CoreOS, and open source is key to that goal. [....] Prometheus was initially created to handle monitoring and alerting in modern microservice architectures. It steadily grew to fit the wider idea of cloud native infrastructure. Though it was not intentional in the original design, Prometheus and Kubernetes conveniently share the key concept of identifying entities by labels, making the semantics of monitoring Kubernetes clusters simple. As we discussed previously on this blog, Prometheus metrics formed the basis of our analysis of Kubernetes scheduler performance, and led directly to improvements in that code. Metrics are essential not just to keep systems running, but also to analyze and improve application behavior. All things considered, Prometheus was an obvious choice for the next open source project CoreOS wanted to support and improve with internal developers committed to the code base. ]]> monitoring coreos prometheus metrics clustering ops gifee google kubernetes https://pinboard.in/ https://pinboard.in/u:jm/b:c041f696601f/ 2016-03-30T13:00:25+00:00 https://github.com/firehol/netdata jm via:markkenny linux monitoring ops netdata sysstat metrics graphing ui https://pinboard.in/ https://pinboard.in/u:jm/b:871c10a92d95/ 2016-03-29T11:10:48+00:00 https://blog.twitter.com/2016/observability-at-twitter-technical-overview-part-ii jm“Pull” vs “push” in metrics collection: At the time of our previous blog post, all our metrics were collected by “pulling” from our collection agents. We discovered two main issues: * There is no easy way to differentiate service failures from collection agent failures. Service response time out and missed collection request are both manifested as empty time series. * There is a lack of service quality insulation in our collection pipeline. It is very difficult to set an optimal collection time out for various services. A long collection time from one single service can cause a delay for other services that share the same collection agent. In light of these issues, we switched our collection model from “pull” to “push” and increased our service isolation. Our collection agent on each host only collects metrics from services running on that specific host. Additionally, each collection agent sends separate collection status tracking metrics in addition to the metrics emitted by the services. We have seen a significant improvement in collection reliability with these changes. However, as we moved to self service push model, it becomes harder to project the request growth. In order to solve this problem, we plan to implement service quota to address unpredictable/unbounded growth. ]]> pull push metrics tcp stacks monitoring agents twitter fault-tolerance https://pinboard.in/ https://pinboard.in/u:jm/b:5023179ebada/ 2016-03-24T13:21:13+00:00 http://prometheus.io/blog/2016/03/23/interview-with-life360/ jm'Prometheus has been known to us for a while, and we have been tracking it and reading about the active development, and at a point (a few months back) we decided to start evaluating it for production use. The PoC results were incredible. The monitoring coverage of MySQL was amazing, and we also loved the JMX monitoring for Cassandra, which had been sorely lacking in the past.' ]]> metrics monitoring time-series prometheus testimonials life360 cassandra jmx mysql https://pinboard.in/ https://pinboard.in/u:jm/b:c4405c9925a7/ 2015-12-09T11:35:37+00:00 https://labs.spotify.com/2015/11/17/monitoring-at-spotify-introducing-heroic/ jm cassandra elasticsearch spotify monitoring metrics heroic https://pinboard.in/ https://pinboard.in/u:jm/b:cdb427d86615/ 2015-12-09T11:26:47+00:00 https://www.vividcortex.com/blog/why-percentiles-dont-work-the-way-you-think jm performance percentiles quantiles statistics metrics monitoring baron-schwartz vividcortex https://pinboard.in/ https://pinboard.in/u:jm/b:c441c328a979/ 2015-11-20T13:51:52+00:00 https://en.wikipedia.org/wiki/Floating_car_data jmFloating car data (FCD), also known as floating cellular data, is a method to determine the traffic speed on the road network. It is based on the collection of localization data, speed, direction of travel and time information from mobile phones in vehicles that are being driven. These data are the essential source for traffic information and for most intelligent transportation systems (ITS). This means that every vehicle with an active mobile phone acts as a sensor for the road network. Based on these data, traffic congestion can be identified, travel times can be calculated, and traffic reports can be rapidly generated. In contrast to traffic cameras, number plate recognition systems, and induction loops embedded in the roadway, no additional hardware on the road network is necessary. ]]> surveillance cars driving mobile-phones phones travel gsm monitoring anpr alpr traffic https://pinboard.in/ https://pinboard.in/u:jm/b:c0e3462dec97/ 2015-11-05T22:22:21+00:00 https://www.vividcortex.com/blog/2015/11/05/nobody-loves-graphite-anymore/ jmGraphite has a place in our current monitoring stack, and together with StatsD will always have a special place in the hearts of DevOps practitioners everywhere, but it’s not representative of state-of-the-art in the last few years. Graphite is where the puck was in 2010. If you’re skating there, you’re missing the benefits of modern monitoring infrastructure. The future I foresee is one where time series capabilities (the raw power needed, which I described in my time series requirements blog post, for example) are within everyone’s reach. That will be considered table stakes, whereas now it’s pretty revolutionary. Like I've been saying -- we need Time Series As A Service! This should be undifferentiated heavy lifting.]]> graphite tsd time-series vividcortex statsd ops monitoring metrics https://pinboard.in/ https://pinboard.in/u:jm/b:f81dace94111/ 2015-09-23T12:48:44+00:00 https://aws.amazon.com/message/5467D2/ jm aws postmortem outages dynamodb ec2 post-mortems circuit-breakers monitoring https://pinboard.in/ https://pinboard.in/u:jm/b:f804599be513/ 2015-09-18T16:37:49+00:00 http://www.eightypercent.net/post/layers-in-the-stack.html jm architecture ops stack docker containerization deployment containers rkt coreos prod monitoring xooglers https://pinboard.in/ https://pinboard.in/u:jm/b:6929918b25c5/ 2015-08-07T10:01:09+00:00 https://blogs.aws.amazon.com/bigdata/post/Tx3ET30EGDKUUI2/Implementing-Efficient-and-Reliable-Producers-with-the-Amazon-Kinesis-Producer-L jm kpl aws kinesis tips java batching streaming production cloudwatch monitoring coding https://pinboard.in/ https://pinboard.in/u:jm/b:20af3897603a/ 2015-08-04T12:52:06+00:00 https://engineering.linkedin.com/sre/introducing-nurse-auto-remediation-linkedin jm nurse auto-remediation outages linkedin ops monitoring https://pinboard.in/ https://pinboard.in/u:jm/b:f862fe9da2a1/ 2015-07-15T09:38:28+00:00 https://www.datadoghq.com/blog/how-to-monitor-nginx/ jm nginx monitoring metrics howto datadog ops https://pinboard.in/ https://pinboard.in/u:jm/b:159b82cf2424/ 2015-06-25T16:49:38+00:00 https://github.com/aragozin/jvm-tools jma command line tool for JVM diagnostic troubleshooting and profiling. ]]> java jvm monitoring commandline jmx sjk tools ops https://pinboard.in/ https://pinboard.in/u:jm/b:8ecc9f104166/ 2015-06-23T13:27:18+00:00 http://githubengineering.com/brubeck/ jm github monitoring statsd c rewrites ops metrics https://pinboard.in/ https://pinboard.in/u:jm/b:d824ce3cab14/ 2015-04-21T11:39:07+00:00 https://gist.github.com/acolyer/95ef23802803cb8b4eb5 jm james-hamilton checklists ops internet-scale architecture operability monitoring reliability availability uptime aspirations https://pinboard.in/ https://pinboard.in/u:jm/b:827aeebe6695/ 2015-04-08T16:11:22+00:00 http://techblog.netflix.com/2015/04/introducing-vector-netflixs-on-host.html jm vector netflix performance monitoring sysstat top iostat netstat metrics ops dashboards real-time linux https://pinboard.in/ https://pinboard.in/u:jm/b:d0c08977fb73/ 2015-04-04T09:32:24+00:00 http://www.slideshare.net/jallspaw/etsy-codeascraft-allspaw1 jm devops monitoring ops five-whys allspaw slides etsy codeascraft incident-response incidents severity root-cause postmortems outages reliability techops tier-one-support https://pinboard.in/ https://pinboard.in/u:jm/b:1b513074b706/ 2015-03-13T11:39:13+00:00 http://hekad.readthedocs.org/en/v0.9.0/ jman open source stream processing software system developed by Mozilla. Heka is a “Swiss Army Knife” type tool for data processing, useful for a wide variety of different tasks, such as: Loading and parsing log files from a file system. Accepting statsd type metrics data for aggregation and forwarding to upstream time series data stores such as graphite or InfluxDB. Launching external processes to gather operational data from the local system. Performing real time analysis, graphing, and anomaly detection on any data flowing through the Heka pipeline. Shipping data from one location to another via the use of an external transport (such as AMQP) or directly (via TCP). Delivering processed data to one or more persistent data stores. Via feylya on twitter. Looks potentially nifty]]> heka mozilla monitoring metrics via:feylya ops statsd graphite stream-processing https://pinboard.in/ https://pinboard.in/u:jm/b:6bb38322d5e4/ 2015-02-19T22:21:45+00:00 http://influxdb.com/blog/2014/09/26/one-year-of-influxdb-and-the-road-to-1_0.html jmhalf of the [Monitorama] attendees were employees and entrepreneurs at monitoring, metrics, DevOps, and server analytics companies. Most of them had a story about how their metrics API was their key intellectual property that took them years to develop. The other half of the attendees were developers at larger organizations that were rolling their own DevOps stack from a collection of open source tools. Almost all of them were creating a “time series database” with a bunch of web services code on top of some other database or just using Graphite. When everyone is repeating the same work, it’s not key intellectual property or a differentiator, it’s a barrier to entry. Not only that, it’s something that is hindering innovation in this space since everyone has to spend their first year or two getting to the point where they can start building something real. It’s like building a web company in 1998. You have to spend millions of dollars and a year building infrastructure, racking servers, and getting everything ready before you could run the application. Monitoring and analytics applications should not be like this. ]]> graphite monitoring metrics tsd time-series analytics influxdb open-source https://pinboard.in/ https://pinboard.in/u:jm/b:cbe8b281afb6/ 2015-02-02T00:49:50+00:00 https://developers.soundcloud.com/blog/prometheus-monitoring-at-soundcloud jm monitoring soundcloud prometheus metrics service-metrics graphite alerting https://pinboard.in/ https://pinboard.in/u:jm/b:acbb62360d38/ 2014-11-11T16:49:30+00:00 https://www.statuspage.io/ jm monitoring server status outages uptime saas infrastructure https://pinboard.in/ https://pinboard.in/u:jm/b:177d01ae1b08/ 2014-09-01T09:34:20+00:00 http://fractio.nl/2014/08/26/cardiac-alarms-and-ops/ jm ops monitoring sysadmin alerts alarms nagios alarm-fatigue false-positives pages https://pinboard.in/ https://pinboard.in/u:jm/b:89a98e187218/ 2014-08-13T16:00:30+00:00 http://blog.factual.com/profiling-hadoop-jobs-with-riemann jmI’ve built a very simple distributed profiler for soft-real-time telemetry from hundreds to thousands of JVMs concurrently. It’s nowhere near as comprehensive in its analysis as, say, Yourkit, but it can tell you, across a distributed system, which functions are taking the most time, and what their dominant callers are. Potentially useful. ]]> riemann profiling aphyr hadoop emr performance monitoring https://pinboard.in/ https://pinboard.in/u:jm/b:b55131749671/ 2014-07-28T22:24:37+00:00 http://www.ebaytechblog.com/2014/03/11/rest-commander-scalable-web-server-management-and-monitoring/#.U9bM0YBdXno jmWe dynamically monitor and manage a large and rapidly growing number of web servers deployed on our infrastructure and systems. However, existing tools present major challenges when making REST/SOAP calls with server-specific requests to a large number of web servers, and then performing aggregated analysis on the responses. We therefore developed REST Commander, a parallel asynchronous HTTP client as a service to monitor and manage web servers. REST Commander on a single server can send requests to thousands of servers with response aggregation in a matter of seconds. And yes, it is open-sourced at http://www.restcommander.com. Feature highlights: Click-to-run with zero installation; Generic HTTP request template supporting variable-based replacement for sending server-specific requests; Ability to send the same request to different servers, different requests to different servers, and different requests to the same server; Maximum concurrency control (throttling) to accommodate server capacity; Commander itself is also “as a service”: with its powerful REST API, you can define ad-hoc target servers, an HTTP request template, variable replacement, and a regular expression all in a single call. In addition, intuitive step-by-step wizards help you achieve the same functionality through a GUI. ]]> rest http clients load-testing ebay soap async testing monitoring https://pinboard.in/ https://pinboard.in/u:jm/b:5ca168566fa5/ 2014-07-24T09:05:58+00:00 https://github.com/mariusaeriksen/heapster jmHeapster provides an agent library to do heap profiling for JVM processes with output compatible with Google perftools. The goal of Heapster is to be able to do meaningful (sampled) heap profiling in a production setting. Used by Twitter in production, apparently.]]> heap monitoring memory jvm java performance https://pinboard.in/ https://pinboard.in/u:jm/b:5a7c8a71eb82/ 2014-07-22T09:20:12+00:00 http://blog.librato.com/posts/2014/7/16/metrics-driven-development jmwe believe MDD is equal parts engineering technique and cultural process. It separates the notion of monitoring from its traditional position of exclusivity as an operations thing and places it more appropriately next to its peers as an engineering process. Provided access to real-time production metrics relevant to them individually, both software engineers and operations engineers can validate hypotheses, assess problems, implement solutions, and improve future designs. Broken down into the following principles: 'Instrumentation-as-Code', 'Single Source of Truth', 'Developers Curate Visualizations and Alerts', 'Alert on What You See', 'Show me the Graph', 'Don’t Measure Everything (YAGNI)'. We do all of these at Swrve, naturally (a technique I happily stole from Amazon).]]> metrics coding graphite mdd instrumentation yagni alerting monitoring graphs https://pinboard.in/ https://pinboard.in/u:jm/b:2bbaac35dc32/ 2014-07-15T09:14:05+00:00 http://go.boundary.com/10freeserversprereg jm boundary monitoring network ops metrics alarms tcp ip netstat https://pinboard.in/ https://pinboard.in/u:jm/b:175d993187e3/ 2014-05-06T15:00:22+00:00 http://www.site-reliability-engineering.info/2014/04/what-is-site-reliability-engineering.html jm sre ops devops google monitoring interviews ben-treynor https://pinboard.in/ https://pinboard.in/u:jm/b:c2afc10c4f2c/ 2014-04-28T11:20:43+00:00 http://riltsken.github.io/devops/infrastructure/monitoring/2014/04/19/making-runbooks-more-useful-by-exposing-them-through-monitoring.html jm runbooks ops devops monitoring sysadmin documentation wiki https://pinboard.in/ https://pinboard.in/u:jm/b:39c34d3c51f1/ 2014-04-08T14:05:41+00:00 https://deadmanssnitch.com/ jma cron job monitoring tool that keeps an eye on your periodic processes and notifies you when something doesn't happen. Daily backups, monthly emails, or cron jobs you need to monitor? Dead Man's Snitch has you covered. Know immediately when one of these processes doesn't work. via Marc.]]> alerts cron monitoring sysadmin ops backups alarms https://pinboard.in/ https://pinboard.in/u:jm/b:a15af0101f4f/ 2014-02-20T17:43:41+00:00 https://www.google.com/transparencyreport/traffic/explorer/?r=US&l=EVERYTHING jm transparency filtering web google http graphs monitoring syria https://pinboard.in/ https://pinboard.in/u:jm/b:903ee1300110/ 2014-01-29T15:00:43+00:00 http://word.bitly.com/post/74839060954/ten-things-to-monitor jm nagios metrics ops monitoring systems ntp bitly https://pinboard.in/ https://pinboard.in/u:jm/b:8d0b22e66fef/ 2013-12-09T12:44:56+00:00 http://www.youtube.com/watch?v=Yg6_ulhwLw0&feature=youtu.be jmIn this talk Kaushik Srenevasan describes a new, low overhead, full-stack tool (based on the Linux perf profiler and infrastructure built into the Hotspot JVM) we've built at Twitter to solve the problem of dynamically profiling and tracing the behavior of applications (including managed runtimes) in production. Looks very interesting. Haven't watched it yet though]]> twitter tech-talks video presentations java jvm profiling testing monitoring service-metrics performance production hotspot perf https://pinboard.in/ https://pinboard.in/u:jm/b:ee6e08004f6c/ 2013-10-15T16:08:14+00:00 https://vividcortex.com/blog/2013/10/14/what-should-i-monitor/ jm slides monitoring metrics ops devops baron-schwartz pdf capacity https://pinboard.in/ https://pinboard.in/u:jm/b:3a3dc0e93b75/