https://pinboard.in/u:jm/public/ recent bookmarks from jm 2025-02-25T12:19:14+00:00 https://github.com/Netflix/hollow jm Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access. Hollow focuses narrowly on its prescribed problem set: keeping an entire, read-only dataset in-memory on consumers. It circumvents the consequences of updating and evicting data from a partial cache. Due to its performance characteristics, Hollow shifts the scale in terms of appropriate dataset sizes for an in-memory solution. Datasets for which such liberation may never previously have been considered can be candidates for Hollow. For example, Hollow may be entirely appropriate for datasets which, if represented with json or XML, might require in excess of 100GB. Interesting approach, though possibly a bit scary in terms of circumventing the "keep things simple and boring" rule... still, a useful tool to have.]]> cache caching netflix java jvm memory hollow read-only architecture systems https://pinboard.in/ https://pinboard.in/u:jm/b:3f732f05411d/ 2025-01-23T14:45:35+00:00 https://www.infoq.com/presentations/optimizing-java-app-kubernetes/ jm kubernetes java eks resources ops scaling scalability gc optimization jvm https://pinboard.in/ https://pinboard.in/u:jm/b:2c9522ab8d41/ 2023-01-05T15:53:26+00:00 https://krzysztofslusarski.github.io/2022/12/12/async-manual.html jm async-profiler java jvm profiling tracing performance tuning https://pinboard.in/ https://pinboard.in/u:jm/b:8ec959433ae2/ 2022-09-05T10:57:00+00:00 https://twitter.com/nitsanw/status/1566677642748874753 jmIf you are profiling for a CPU bottleneck [in java], DO NOT RELY ON JVM FLIGHT RECORDER METHOD PROFILING. Not even a little bit. Use `async-profiler` for profiling(`-e cpu,lock,alloc`), with `--jfrsync default/profile` for extra JVM/JDK events. ]]> profiling java performance optimization jvm async-profiler via:twitter https://pinboard.in/ https://pinboard.in/u:jm/b:acdfaf080f4b/ 2022-04-20T09:29:49+00:00 https://neilmadden.blog/2022/04/19/psychic-signatures-in-java/ jmRecent releases of Java were vulnerable to a similar kind of trick, in the implementation of widely-used ECDSA signatures. If you are running one of the vulnerable versions then an attacker can easily forge some types of SSL certificates and handshakes (allowing interception and modification of communications), signed JWTs, SAML assertions or OIDC id tokens, and even WebAuthn authentication messages. All using the digital equivalent of a blank piece of paper. It’s hard to overstate the severity of this bug. If you are using ECDSA signatures for any of these security mechanisms, then an attacker can trivially and completely bypass them if your server is running any Java 15, 16, 17, or 18 version before the April 2022 Critical Patch Update (CPU). For context, almost all WebAuthn/FIDO devices in the real world (including Yubikeys*) use ECDSA signatures and many OIDC providers use ECDSA-signed JWTs. ]]> java jvm crypto security ecdsa webauthn saml jwt fail https://pinboard.in/ https://pinboard.in/u:jm/b:19326f7fa161/ 2020-12-04T10:57:55+00:00 https://gceasy.io/ jm java gc tuning performance jvm logs ops https://pinboard.in/ https://pinboard.in/u:jm/b:d3d8327d2376/ 2020-03-11T13:21:50+00:00 https://medium.com/thron-tech/lessons-learned-about-monitoring-the-jvm-in-the-era-of-containers-47e7fe0b77dc jm jvm g1 g1gc gc tuning metrics ops https://pinboard.in/ https://pinboard.in/u:jm/b:44d5b27b29bf/ 2019-11-06T14:00:14+00:00 https://github.com/apple/servicetalk/ jma JVM network application framework with APIs tailored to specific protocols (e.g. HTTP/1.x, HTTP/2.x, etc…​) and supports multiple programming paradigms. It is built on Netty and is designed to provide most of the performance/scalability benefits of Netty for common networking protocols used in service to service communication. ServiceTalk provides server support and "smart client" like features such as client-side load balancing and service discovery integration. Open source from Apple.]]> apple servicetalk netty libraries java jvm coding http async https://pinboard.in/ https://pinboard.in/u:jm/b:d97f22fd741a/ 2018-11-19T11:57:33+00:00 http://batey.info/docker-jvm-k8s.html jm Later JDK versions have made it far easier to run a JVM application in a Linux container. The memory support means that if you relied on JVM ergonomics before than you can do the same inside a container where as previously you had to override all memory related settings. The CPU support for containers needs to be carefully evaluated for your application and environment. If you’ve previously set low cpu_shares in environments like Kubernetes to increase utilisation while relying on using up unused cycles then you might get a shock. ]]> jvm docker kubernetes linux containers ops https://pinboard.in/ https://pinboard.in/u:jm/b:521b7b5a67e4/ 2018-11-19T11:30:30+00:00 http://www.evanjones.ca/java-bytebuffer-leak.html jmThe Java NIO APIs use ByteBuffers as the source and destination of I/O calls, and come in two flavours. Heap ByteBuffers wrap a byte[] array, allocated in the garbage collected Java heap. Direct ByteBuffers wrap memory allocated outside the Java heap using malloc. Only "native" memory can be passed to operating system calls, so it won't be moved by the garbage collector. This means that when you use a heap ByteBuffer for I/O, it is copied into a temporary direct ByteBuffer. The JDK caches one temporary buffer per thread, without any memory limits. As a result, if you call I/O methods with large heap ByteBuffers from multiple threads, your process can use a huge amount of additional native memory, which looks like a native memory leak. This can cause your process to unexpectedly run into memory limits and get killed. ]]> jvm performance java memory leaks bytebuffers netty threads coding bugs https://pinboard.in/ https://pinboard.in/u:jm/b:94c60cf2a090/ 2018-06-06T15:27:52+00:00 https://confluence.atlassian.com/kb/how-to-change-jvm-arguments-at-runtime-to-avoid-application-restart-816682109.html jmjava -XX:+PrintFlagsFinal -version|grep manageable ]]> jvm ops switches cli java heap-dumps memory debugging memory-leaks https://pinboard.in/ https://pinboard.in/u:jm/b:b591a243dc1d/ 2017-09-20T14:36:55+00:00 https://groups.google.com/forum/#!msg/mechanical-sympathy/gchG_oQ_kQM/59BDMOdUAwAJ jmLittle's law can be used to describe a system in steady state from a queuing perspective, i.e. arrival and leaving rates are balanced. In this case it is a crude way of modelling a system with a contention percentage of 100% under Amdahl's law, in that throughput is one over latency. However this is an inaccurate way to model a system with locks. Amdahl's law does not account for coherence costs. For example, if you wrote a microbenchmark with a single thread to measure the lock cost then it is much lower than in a multi-threaded environment where cache coherence, other OS costs such as scheduling, and lock implementations need to be considered. Universal Scalability Law (USL) accounts for both the contention and the coherence costs. http://www.perfdynamics.com/Manifesto/USLscalability.html When modelling locks it is necessary to consider how contention and coherence costs vary given how they can be implemented. Consider in Java how we have biased locking, thin locks, fat locks, inflation, and revoking biases which can cause safe points that bring all threads in the JVM to a stop with a significant coherence component. ]]> usl scaling scalability performance locking locks java jvm amdahls-law littles-law system-dynamics modelling systems caching threads schedulers contention https://pinboard.in/ https://pinboard.in/u:jm/b:d64fb1279a0b/ 2017-09-18T09:35:48+00:00 https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html jm java jvm memory native-memory malloc debugging coding nmt java-8 jcmd https://pinboard.in/ https://pinboard.in/u:jm/b:d2f2bae02fb3/ 2017-09-11T10:48:52+00:00 http://psy-lob-saw.blogspot.ie/2017/02/flamegraphs-intro-fire-for-everyone.html jm profiling java safepoints jvm flame-graphs perf measurement benchmarking testing https://pinboard.in/ https://pinboard.in/u:jm/b:1a6f8c8d9b75/ 2017-08-24T16:22:31+00:00 https://github.com/AdoptOpenJDK/jitwatch jmLog analyser and visualiser for the HotSpot JIT compiler. Inspect inlining decisions, hot methods, bytecode, and assembly. View results in the JavaFX user interface. ]]> analysis java jvm performance tools debugging optimization jit https://pinboard.in/ https://pinboard.in/u:jm/b:404ca57f5933/ 2017-07-12T10:53:58+00:00 https://plasma-umass.github.io/doppio-demo/ jm jvm java javascript js hacks browser emulation websockets https://pinboard.in/ https://pinboard.in/u:jm/b:15ebe75eae91/ 2017-01-10T16:39:09+00:00 http://www.evanjones.ca/java-native-leak-bug.html jm debugging memory jvm java leaks memory-leaks leak-checking jemalloc malloc native heap off-heap gzipinputstream https://pinboard.in/ https://pinboard.in/u:jm/b:5206aa65dc43/ 2016-06-18T09:46:44+00:00 https://blog.buoyant.io/2016/06/17/small-memory-jvm-techniques-for-microservice-sidecars/ jmWe found two specific techniques to be the most beneficial: turning off one of the two JIT compilers enabled by default (the “C2” compiler), and using a 32-bit, rather than a 64-bit, JVM. ]]> 32bit jvm java ops memory tuning jit linkerd https://pinboard.in/ https://pinboard.in/u:jm/b:ade3b2e45f61/ 2016-06-01T11:55:13+00:00 https://github.com/ztellman/dirigiste jm async jvm dirigiste java threadpools concurrency utilization capacity executors object-pools object-pooling latency https://pinboard.in/ https://pinboard.in/u:jm/b:5092c7f0c12e/ 2016-04-14T11:39:20+00:00 https://gdstechnology.blog.gov.uk/2015/12/11/using-jemalloc-to-get-to-the-bottom-of-a-memory-leak/ jm debugging java jvm memory jemalloc off-heap https://pinboard.in/ https://pinboard.in/u:jm/b:e8651207606c/ 2016-03-14T17:32:46+00:00 https://github.com/JCTools/JCTools/pull/93 jm longadder doug-lea mechanical-sympathy lock-free performance atomic multithreading java jvm https://pinboard.in/ https://pinboard.in/u:jm/b:28819375e2f4/ 2016-01-08T10:04:43+00:00 http://www.infoq.com/presentations/apple-netty jm apple netty norman-maurer java jvm async talks presentations https://pinboard.in/ https://pinboard.in/u:jm/b:09aa5e33f478/ 2016-01-04T10:42:16+00:00 https://www.voxxed.com/blog/2015/12/importance-tuning-thread-pools/ jm hystrix threadpools concurrency java jvm backpressure littles-law capacity https://pinboard.in/ https://pinboard.in/u:jm/b:d8e4a0101363/ 2015-09-24T10:11:48+00:00 http://byteman.jboss.org/ jma tool which simplifies tracing and testing of Java programs. Byteman allows you to insert extra Java code into your application, either as it is loaded during JVM startup or even after it has already started running. The injected code is allowed to access any of your data and call any application methods, including where they are private. You can inject code almost anywhere you want and there is no need to prepare the original source code in advance nor do you have to recompile, repackage or redeploy your application. In fact you can remove injected code and reinstall different code while the application continues to execute. The simplest use of Byteman is to install code which traces what your application is doing. This can be used for monitoring or debugging live deployments as well as for instrumenting code under test so that you can be sure it has operated correctly. By injecting code at very specific locations you can avoid the overheads which often arise when you switch on debug or product trace. Also, you decide what to trace when you run your application rather than when you write it so you don't need 100% hindsight to be able to obtain the information you need. ]]> tracing java byteman injection jvm ops debugging testing https://pinboard.in/ https://pinboard.in/u:jm/b:542b80c1b242/ 2015-09-18T09:15:02+00:00 http://steveloughran.blogspot.co.uk/2015/09/time-on-multi-core-multi-socket-servers.html jmThe conclusion I've reached is that except for the special case of using nanoTime() in micro benchmarks, you may as well stick to currentTimeMillis() —knowing that it may sporadically jump forwards or backwards. Because if you switched to nanoTime(), you don't get any monotonicity guarantees, it doesn't relate to human time any more —and may be more likely to lead you into writing code which assumes a fast call with consistent, monotonic results. ]]> java time monotonic sequencing nanotime timers jvm multicore distributed-computing https://pinboard.in/ https://pinboard.in/u:jm/b:7d7dedf4484e/ 2015-08-14T20:15:57+00:00 http://gary-rowe.com/agilestack/2013/07/03/preventing-dependency-chain-attacks-in-maven/ jm security whitelisting dependencies coding jar maven java jvm https://pinboard.in/ https://pinboard.in/u:jm/b:cded21345b09/ 2015-06-25T16:49:38+00:00 https://github.com/aragozin/jvm-tools jma command line tool for JVM diagnostic troubleshooting and profiling. ]]> java jvm monitoring commandline jmx sjk tools ops https://pinboard.in/ https://pinboard.in/u:jm/b:8ecc9f104166/ 2015-06-08T10:26:15+00:00 https://github.com/electronicarts/orbit/tree/master/async jmOrbit Async implements async-await methods in the JVM. It allows programmers to write asynchronous code in a sequential fashion. It was developed by BioWare, a division of Electronic Arts. Open source, BSD-licensed.]]> async await java jvm bioware coding threading https://pinboard.in/ https://pinboard.in/u:jm/b:93545dab47e5/ 2015-06-06T07:02:50+00:00 https://databricks.com/blog/2015/05/28/tuning-java-garbage-collection-for-spark-applications.html jm g1gc gc java jvm performance spark ops tuning https://pinboard.in/ https://pinboard.in/u:jm/b:431050bae8c9/ 2015-05-11T14:33:27+00:00 https://github.com/caplogic/mappedbus jma Java based low latency, high throughput message bus, built on top of a memory mapped file; inspired by Java Chronicle with the main difference that it's designed to efficiently support multiple writers – enabling use cases where the order of messages produced by multiple processes are important. MappedBus can be also described as an efficient IPC mechanism which enable several Java programs to communicate by exchanging messages. ]]> ipc java jvm mappedbus low-latency mmap message-bus data-structures queue message-passing https://pinboard.in/ https://pinboard.in/u:jm/b:2fd14b0d1310/ 2015-05-09T08:19:41+00:00 http://www.slideshare.net/MonicaBeckwith/garbage-first-garbage-collector-g1-gc-migration-to-expectations-and-advanced-tuning jm gc g1gc jvm java tuning performance ops migration https://pinboard.in/ https://pinboard.in/u:jm/b:1dbd33f78572/ 2015-05-06T16:34:59+00:00 https://timharris.uk/papers/2015-hotos.pdf jm blade via:adriancolyer papers gc distsys algorithms distributed java jvm latency spark cassandra https://pinboard.in/ https://pinboard.in/u:jm/b:bf80614a2fb9/ 2015-04-29T15:42:34+00:00 https://issues.apache.org/jira/browse/CASSANDRA-7486 jm cassandra tuning ops g1gc cms gc java jvm production performance memory https://pinboard.in/ https://pinboard.in/u:jm/b:0b4b3d1abe38/ 2015-04-11T20:21:41+00:00 http://engineering.linkedin.com/java/optimizing-java-cms-garbage-collections-its-difficulties-and-using-jtune-solution jm java jvm tuning gc cms linkedin performance ops https://pinboard.in/ https://pinboard.in/u:jm/b:717dfe95f072/ 2015-03-26T22:16:21+00:00 http://www.evanjones.ca/jvm-mmap-pause.html jmThe JVM by default exports statistics by mmap-ing a file in /tmp (hsperfdata). On Linux, modifying a mmap-ed file can block until disk I/O completes, which can be hundreds of milliseconds. Since the JVM modifies these statistics during garbage collection and safepoints, this causes pauses that are hundreds of milliseconds long. To reduce worst-case pause latencies, add the -XX:+PerfDisableSharedMem JVM flag to disable this feature. This will break tools that read this file, like jstat. ]]> bugs gc java jvm disk mmap latency ops jstat https://pinboard.in/ https://pinboard.in/u:jm/b:a8b7f0defa11/ 2015-03-09T23:35:31+00:00 https://blogs.oracle.com/dave/entry/biased_locking_in_hotspot jmYou can avoid biased locking on a per-object basis by calling System.identityHashCode(o). If the object is already biased, assigning an identity hashCode will result in revocation, otherwise, the assignment of a hashCode() will make the object ineligible for subsequent biased locking. ]]> hashcode jvm java biased-locking locking mutex synchronization locks performance https://pinboard.in/ https://pinboard.in/u:jm/b:f50fdd3cde18/ 2015-02-26T15:49:43+00:00 http://www.jclarity.com/2015/02/24/why-we-built-illuminate-where-apm-is-going-next/ jmUsers download and install an Illuminate Daemon using a simple installer which starts up a small stand alone Java process. The Daemon sits quietly unless it is asked to start gathering SLA data and/or to trigger a diagnosis. Users can set SLA’s via the dashboard and can opt to collect latency measurements of their transactions manually (using our library) or by asking Illuminate to automatically instrument their code (Servlet and JDBC based transactions are currently supported). SLA latency data for transactions is collected on a short cycle. When the moving average of latency measurements goes above the SLA value (e.g. 150ms), a diagnosis is triggered. The diagnosis is very quick, gathering key data from O/S, JVM(s), virtualisation and other areas of the system. The data is then run through the machine learned algorithm which will quickly narrow down the possible causes and gather a little extra data if needed. Once Illuminate has determined the root cause of the performance problem, the diagnosis report is sent back to the dashboard and an alert is sent to the user. That alert contains a link to the result of the diagnosis which the user can share with colleagues. Illuminate has all sorts of backoff strategies to ensure that users don’t get too many alerts of the same type in rapid succession! ]]> illuminate jclarity java jvm scala latency gc tuning performance https://pinboard.in/ https://pinboard.in/u:jm/b:d8f4dcc75538/ 2015-02-19T23:38:03+00:00 https://sysdigcloud.com/gain-insights-jvms-sysdig-clouds-jmx-metrics/ jmSysdig Cloud users have the ability to view and analyze Java Management Extensions (JMX) metrics out of the box with no additional configuration or setup required. ]]> sysdig jmx java jvm https://pinboard.in/ https://pinboard.in/u:jm/b:4c2a773bfd9d/ 2015-02-16T11:17:41+00:00 http://openjdk.java.net/projects/code-tools/jol/ jm java jvm tools scala memory estimation ram object-layout debugging via:nitsan https://pinboard.in/ https://pinboard.in/u:jm/b:e605db4598c2/ 2015-02-03T22:09:12+00:00 http://tobert.github.io/tldr/cassandra-java-huge-pages.html jm jvm performance tuning huge-pages vm ops cassandra java https://pinboard.in/ https://pinboard.in/u:jm/b:d448b5d53e34/ 2015-01-25T23:43:59+00:00 https://speakerdeck.com/elasticsearch/maintaining-performance-in-distributed-systems jm performance elasticsearch java jvm ops tuning https://pinboard.in/ https://pinboard.in/u:jm/b:99d8451bc962/ 2015-01-25T23:32:11+00:00 http://rdiyewar-tech.blogspot.de/2013/02/outofmemoryerror-because-of-default.html jm jvm java threads ops stack-size memory https://pinboard.in/ https://pinboard.in/u:jm/b:159a50a87d53/ 2015-01-25T08:50:22+00:00 https://github.com/JCTools/JCTools jmJava Concurrency Tools for the JVM. This project aims to offer some concurrent data structures currently missing from the JDK: Bounded lock free queues SPSC/MPSC/SPMC/MPMC variations for concurrent queues Alternative interfaces for queues (experimental) Offheap concurrent ring buffer for ITC/IPC purposes (experimental) Executor (planned) ]]> concurrency lock-free data-structures queues jvm java https://pinboard.in/ https://pinboard.in/u:jm/b:48920e47e7f3/ 2015-01-05T17:56:26+00:00 http://tech.stolsvik.com/2010/01/linux-java-thread-priorities-workaround.html jmif you just set the "ThreadPriorityPolicy" to something else than the legal values 0 or 1, [...] a slight logic bug in Sun's JVM code kicks in, and thus sets the policy to be as if running with root - thus you get exactly what one desire. The operating system, Linux, won't allow priorities to be heightened above "Normal" (negative nice value), and thus just ignores those requests (setting it to normal instead, nice value 0) - but it lets through the requests to set it lower (setting the nice value to some positive value). ]]> cassandra thread-priorities threads java jvm linux nice hacks https://pinboard.in/ https://pinboard.in/u:jm/b:abed46e6d700/ 2014-11-25T12:28:16+00:00 http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html jm concurrency java jvm threading threads executors coding https://pinboard.in/ https://pinboard.in/u:jm/b:35ca20533835/ 2014-11-19T11:51:12+00:00 http://stackoverflow.com/questions/219585/setting-multiple-jars-in-java-classpath/219801#219801 jm java jvm globbing classpath uberjars jars deployment https://pinboard.in/ https://pinboard.in/u:jm/b:985e11ba3303/ 2014-10-08T09:45:50+00:00 https://groups.google.com/forum/#!msg/project-voldemort/Y52UyHQ8tBA/9Ei79_RvS3EJ jm kafka metrics dropwizard java scala jvm timers ewma statistics measurement latency sampling tehuti voldemort linkedin jay-kreps https://pinboard.in/ https://pinboard.in/u:jm/b:b56664c1a098/ 2014-09-17T22:39:46+00:00 http://noahlz.roughdraft.io/865cc30e0fd93ad48369-troubleshooting-production-jvms-with-jcmd jm jvm jcmd debugging ops java gc heap troubleshooting https://pinboard.in/ https://pinboard.in/u:jm/b:cc90aa62a26d/ 2014-08-20T13:07:51+00:00 http://vanillajava.blogspot.it/2014/08/try-optimising-memory-consumption-first.html jm yourkit memory java jvm allocation gc bottlenecks performance https://pinboard.in/ https://pinboard.in/u:jm/b:ba674f43806e/ 2014-08-05T20:50:09+00:00 https://github.com/OpenHFT/hftc jmThis is a yet another Java collections library of primitive specializations. Java 6+. Apache 2.0 license. Currently only hash sets and hash maps are implemented. ]]> openhft performance java jvm collections asl hashsets hashmaps data-structures https://pinboard.in/ https://pinboard.in/u:jm/b:3918981e9d4d/ 2014-07-24T09:05:58+00:00 https://github.com/mariusaeriksen/heapster jmHeapster provides an agent library to do heap profiling for JVM processes with output compatible with Google perftools. The goal of Heapster is to be able to do meaningful (sampled) heap profiling in a production setting. Used by Twitter in production, apparently.]]> heap monitoring memory jvm java performance https://pinboard.in/ https://pinboard.in/u:jm/b:5a7c8a71eb82/ 2014-05-16T08:56:30+00:00 http://boundary.com/blog/2014/05/15/dynamic-tuple-performance-on-the-jvm/ jmgenerates heterogeneous collections of primitive values and ensures as best it can that they will be laid out adjacently in memory. The individual values in the tuple can either be accessed from a statically bound interface, via an indexed accessor, or via reflective or other dynamic invocation techniques. FastTuple is designed to deal with a large number of tuples therefore it will also attempt to pool tuples such that they do not add significantly to the GC load of a system. FastTuple is also capable of allocating the tuple value storage entirely off-heap, using Java’s direct memory capabilities. ]]> jvm java gc off-heap storage boundary memory https://pinboard.in/ https://pinboard.in/u:jm/b:bac458affd01/ 2014-05-16T08:54:53+00:00 https://github.com/RuedigerMoeller/fast-serialization/wiki/Structs jmstorage of structured data in a continuous block of memory. The memory can be allocated on the heap using a byte[] array or can be allocated off the java heap in native memory. [...] Use cases: store/cache huge amounts of data records without impact on GC duration; high performance data transfer in a cluster or in between processes handy OSS from Ruediger Moeller]]> structs java jvm memory off-heap storage reference https://pinboard.in/ https://pinboard.in/u:jm/b:714aaf030806/ 2014-05-14T13:09:42+00:00 http://shipilev.net/blog/2014/exceptional-performance/ jm java jvm exceptions benchmarking performance optimization coding https://pinboard.in/ https://pinboard.in/u:jm/b:7d2c8fedbc9e/ 2014-05-07T14:06:27+00:00 http://shipilev.net/blog/2014/nanotrusting-nanotime/ jmSystem.nanoTime is as bad as String.intern now: you can use it, but use it wisely. The latency, granularity, and scalability effects introduced by timers may and will affect your measurements if done without proper rigor. This is one of the many reasons why System.nanoTime should be abstracted from the users by benchmarking frameworks, monitoring tools, profilers, and other tools written by people who have time to track if the underlying platform is capable of doing what we want it to do. In some cases, there is no good solution to the problem at hand. Some things are not directly measurable. Some things are measurable with unpractical overheads. Internalize that fact, weep a little, and move on to building the indirect experiments. This is not the Wonderland, Alice. Understanding how the Universe works often needs side routes to explore. In all seriousness, we should be happy our $1000 hardware can measure 30 nanosecond intervals pretty reliably. This is roughly the time needed for the Internet packets originating from my home router to leave my apartment. What else do you want, you spoiled brats? ]]> benchmarking jdk java measurement nanoseconds nsecs nanotime jvm alexey-shipilev jmh https://pinboard.in/ https://pinboard.in/u:jm/b:6068e09f6496/ 2014-04-24T09:16:58+00:00 http://comcast.github.io/sirius/overview.html jmAt Comcast, our applications need convenient, low-latency access to important reference datasets. For example, our XfinityTV websites and apps need to use entertainment-related data to serve almost every API or web request to our datacenters: information like what year Casablanca was released, or how many episodes were in Season 7 of Seinfeld, or when the next episode of the Voice will be airing (and on which channel!). We traditionally managed this information with a combination of relational databases and RESTful web services but yearned for something simpler than the ORM, HTTP client, and cache management code our developers dealt with on a daily basis. As main memory sizes on commodity servers continued to grow, however, we asked ourselves: How can we keep this reference data entirely in RAM, while ensuring it gets updated as needed and is easily accessible to application developers? The Sirius distributed system library is our answer to that question, and we're happy to announce that we've made it available as an open source project. Sirius is written in Scala and uses the Akka actor system under the covers, but is easily usable by any JVM-based language. Also includes a Paxos implementation with "fast follower" read-only slave replication. ASL2-licensed open source. The only thing I can spot to be worried about is speed of startup; they note that apps need to replay a log at startup to rebuild state, which can be slow if unoptimized in my experience. Update: in a twitter conversation at https://twitter.com/jon_moore/status/459363751893139456 , Jon Moore indicated they haven't had problems with this even with 'datasets consuming 10-20GB of heap', and have 'benchmarked a 5-node Sirius ingest cluster up to 1k updates/sec write throughput.' That's pretty solid!]]> open-source comcast paxos replication read-only datastores storage memory memcached redis sirius scala akka jvm libraries https://pinboard.in/ https://pinboard.in/u:jm/b:54736b003116/ 2014-04-08T21:54:13+00:00 http://engineering.linkedin.com/garbage-collection/garbage-collection-optimization-high-throughput-and-low-latency-java-applications jm performance optimization linkedin java jvm gc tuning https://pinboard.in/ https://pinboard.in/u:jm/b:77cabe371f7c/ 2014-03-19T12:12:51+00:00 http://java-is-the-new-c.blogspot.ie/2013/07/impact-of-large-primitive-arrays-blobs.html jm cms java jvm performance optimization tuning off-heap-storage memory https://pinboard.in/ https://pinboard.in/u:jm/b:026572f118c8/ 2014-02-05T17:35:26+00:00 http://highscalability.com/blog/2014/2/5/littles-law-scalability-and-fault-tolerance-the-os-is-your-b.html jm jvm java quasar pulsar comsat littles-law scalability async erlang https://pinboard.in/ https://pinboard.in/u:jm/b:bb3e77510a90/ 2014-01-24T10:14:24+00:00 http://tech.shift.com/post/74311817513/cassandra-tuning-the-jvm-for-read-heavy-workloads jmThe cluster we tuned is hosted on AWS and is comprised of 6 hi1.4xlarge EC2 instances, with 2 1TB SSDs raided together in a raid 0 configuration. The cluster’s dataset is growing steadily. At the time of this writing, our dataset is 341GB, up from less than 200GB a few months ago, and is growing by 2-3GB per day. The workload on this cluster is very read heavy, with quorum reads making up 99% of all operations. Some careful GC tuning here. Probably not applicable to anyone else, but good approach in general.]]> java performance jvm scaling gc tuning cassandra ops https://pinboard.in/ https://pinboard.in/u:jm/b:bc4cff6803b7/ 2014-01-20T10:28:01+00:00 https://groups.google.com/forum/#!msg/mechanical-sympathy/4EDCX0F_3ow/FxJ9nOvZqYUJ jm What you are looking to do is enforce a barrier between your initializing stores and your publishing store, without that publishing store being made to a volatile field. This can be done by using volatile access to other fields in the publication path, without using those variables in the later access paths to the published object. ]]> volatile atomic java jvm gil-tene synchronization performance threading jmm memory-barriers https://pinboard.in/ https://pinboard.in/u:jm/b:a75f0eb82bd3/ 2013-12-30T21:29:25+00:00 http://jpbempel.blogspot.ie/2013/10/hardware-performance-counters-atomic-vs.html jm metrics hpc libpfm java jvm via:normanmaurer l2 dram llc cpu https://pinboard.in/ https://pinboard.in/u:jm/b:b7f225c67024/ 2013-12-09T12:44:56+00:00 http://www.youtube.com/watch?v=Yg6_ulhwLw0&feature=youtu.be jmIn this talk Kaushik Srenevasan describes a new, low overhead, full-stack tool (based on the Linux perf profiler and infrastructure built into the Hotspot JVM) we've built at Twitter to solve the problem of dynamically profiling and tracing the behavior of applications (including managed runtimes) in production. Looks very interesting. Haven't watched it yet though]]> twitter tech-talks video presentations java jvm profiling testing monitoring service-metrics performance production hotspot perf https://pinboard.in/ https://pinboard.in/u:jm/b:ee6e08004f6c/ 2013-11-16T21:36:48+00:00 https://github.com/mgodave/barge jmA JVM Implementation of the Raft Consensus Protocol ]]> via:sbtourist raft jvm java consensus distributed-computing https://pinboard.in/ https://pinboard.in/u:jm/b:564c92229ffb/ 2013-11-12T13:39:16+00:00 https://spring.io/blog/2013/11/12/it-can-t-just-be-big-data-it-has-to-be-fast-data-reactor-1-0-goes-ga jm Reactor provides the necessary abstractions to build high-throughput, low-latency--what we now call "fast data"--applications that absolutely must work with thousands, tens of thousands, or even millions of concurrent requests per second. Modern JVM applications must be built on a solid foundation of asynchronous and reactive components that efficiently manage the execution of a very large number of tasks on a very small number of system threads. Reactor is specifically designed to help you build these kinds of applications without getting in your way or forcing you to work within an opinionated pattern. Featuring the LMAX Disruptor ringbuffer, the JavaChronicle fast persistent message-passing queue, Groovy closures, and Netty 4.0. This looks very handy indeed....]]> disruptor reactive-programming reactor async libraries java jvm frameworks spring netty fast-data https://pinboard.in/ https://pinboard.in/u:jm/b:40b56ea65db9/ 2013-11-10T00:10:17+00:00 http://blog.ragozin.info/2013/11/hotspot-jvm-garbage-collection-options.html jm jvm hotspot java tuning ops gc https://pinboard.in/ https://pinboard.in/u:jm/b:a6b132013693/ 2013-10-25T10:46:56+00:00 http://hg.openjdk.java.net/lambda/lambda/jdk/file/b55f1ce224fb/src/share/classes/java/util/stream/SpinedBuffer.java jmAn ordered collection of elements. Elements can be added, but not removed. Goes through a building phase, during which elements can be added, and a traversal phase, during which elements can be traversed in order but no further modifications are possible. ]]> spinedbuffer data-structures algorithms java jdk jvm java-8 arrays lists https://pinboard.in/ https://pinboard.in/u:jm/b:79ed0824f439/ 2013-10-08T11:15:46+00:00 http://java-is-the-new-c.blogspot.de/2013/07/what-drives-full-gc-duration-its.html jmFull GC duration depends on the number of objects allocated and the locality of their references. It does not depend that much on actual heap size. Reference locality has a surprisingly high effect.]]> java jvm data gc tuning performance cms g1 https://pinboard.in/ https://pinboard.in/u:jm/b:71efc8abad7c/ 2013-10-07T09:51:25+00:00 http://psy-lob-saw.blogspot.ie/2013/10/spsc-revisited-part-iii-fastflow-sparse.html jm nitsanw optimization concurrency java jvm cas spsc queues data-structures algorithms https://pinboard.in/ https://pinboard.in/u:jm/b:76f60b5fe219/ 2013-10-01T09:30:44+00:00 https://speakerdeck.com/headius/down-the-rabbit-hole jmAn adventure that takes you through several popular Java language features and shows how they compile to bytecode and eventually JIT to assembly code. ]]> charles-nutter java jvm compilation reversing talks slides https://pinboard.in/ https://pinboard.in/u:jm/b:bf667f5b0e93/