Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmsimdjson/simdjson-java2023-10-09T08:06:01+00:00
https://github.com/simdjson/simdjson-java
jmsimd java json parsing formats performance librarieshttps://pinboard.in/https://pinboard.in/u:jm/b:587e2c3aab0d/The Black Magic of (Java) Method Dispatch2023-04-11T21:03:19+00:00
https://shipilev.net/blog/2015/black-magic-method-dispatch/#_cheating_the_runtime_2
jmProgramming languages like Java provide the facilities for subtyping/polymorphism as one of the ways to construct modular and reusable software. This language choice naturally comes at a price, since there is no hardware support for virtual calls, and therefore runtimes have to emulate this behavior. In many, many cases the performance of method dispatch is not important. Actually, in a vast majority of cases, the low-level performance concerns are not the real concerns.
However, there are cases when method dispatch performance is important, and there you need to understand how dispatch works, what runtimes optimize for you, and what you can do to cheat and/or emulate similar behavior in your code. For example, in the course of String Compression work, we were faced with the problem of selecting the coder for a given String. The obvious and highly maintainable approach of creating a Coder interface, a few implementations, and dispatching the virtual calls over it, had met some performance problems on the very tiny benchmarks. Therefore, we needed to contemplate something better. After a few experiments, this post was born as a reference for others who might try to do the same. This post also tangentially touches the inlining of virtual calls, as the natural thing during the optimization.
Discovered via this amazing commit: https://github.com/quarkusio/quarkus/commit/65dd4d43e2644db1c87726139280f9704140167c
]]>optimization performance java oo jit coding polymorphismhttps://pinboard.in/https://pinboard.in/u:jm/b:fcc32a99ea26/Async-profiler manual by use cases2023-01-05T15:53:26+00:00
https://krzysztofslusarski.github.io/2022/12/12/async-manual.html
jmasync-profiler java jvm profiling tracing performance tuninghttps://pinboard.in/https://pinboard.in/u:jm/b:8ec959433ae2/don't use JVM Flight Recorder profiling2022-09-05T10:57:00+00:00
https://twitter.com/nitsanw/status/1566677642748874753
jmIf you are profiling for a CPU bottleneck [in java], DO NOT RELY ON JVM FLIGHT RECORDER METHOD PROFILING. Not even a little bit.
Use `async-profiler` for profiling(`-e cpu,lock,alloc`), with `--jfrsync default/profile` for extra JVM/JDK events.
]]>profiling java performance optimization jvm async-profiler via:twitterhttps://pinboard.in/https://pinboard.in/u:jm/b:acdfaf080f4b/ULID2022-06-13T10:06:43+00:00
https://github.com/ulid/spec
jmulid uuid unique-ids java algorithms via:nelson unique idshttps://pinboard.in/https://pinboard.in/u:jm/b:2c3939ba08e2/CVE-2022-21449: Psychic Signatures in Java2022-04-20T09:29:49+00:00
https://neilmadden.blog/2022/04/19/psychic-signatures-in-java/
jmRecent releases of Java were vulnerable to a similar kind of trick, in the implementation of widely-used ECDSA signatures. If you are running one of the vulnerable versions then an attacker can easily forge some types of SSL certificates and handshakes (allowing interception and modification of communications), signed JWTs, SAML assertions or OIDC id tokens, and even WebAuthn authentication messages. All using the digital equivalent of a blank piece of paper.
It’s hard to overstate the severity of this bug. If you are using ECDSA signatures for any of these security mechanisms, then an attacker can trivially and completely bypass them if your server is running any Java 15, 16, 17, or 18 version before the April 2022 Critical Patch Update (CPU). For context, almost all WebAuthn/FIDO devices in the real world (including Yubikeys*) use ECDSA signatures and many OIDC providers use ECDSA-signed JWTs.
]]>java jvm crypto security ecdsa webauthn saml jwt failhttps://pinboard.in/https://pinboard.in/u:jm/b:19326f7fa161/Resilience4J2022-03-30T22:18:54+00:00
https://github.com/resilience4j/resilience4j
jmfp hystrix java fault-tolerance retries timeouts architecture patterns circuit-breakershttps://pinboard.in/https://pinboard.in/u:jm/b:3c49a9299497/SWAR indexOf byte search2022-01-24T10:08:22+00:00
https://github.com/netty/netty/pull/10737/files
jmsimd swar indexof bytebuffer java optimization performance search netty hackshttps://pinboard.in/https://pinboard.in/u:jm/b:b1bb44f3ef08/Distributed Unique Time Stamp Identifiers2022-01-06T10:28:00+00:00
https://blog.vanillajava.blog/2022/01/distributed-unique-time-stamp.html
jmjava ids uniqueness identifiers algorithms peter-lawrey performancehttps://pinboard.in/https://pinboard.in/u:jm/b:fc53ee7af57c/Overhead of Returning Optional Values in Java and Rust2021-10-16T21:46:58+00:00
https://pkolaczk.github.io/overhead-of-optional/
jmjava rust optimization performance hotspots benchmarks optional nullhttps://pinboard.in/https://pinboard.in/u:jm/b:950a505b2bef/SIMD is coming to the JVM2021-09-13T11:31:58+00:00
https://www.morling.dev/blog/fizzbuzz-simd-style/
jmsimd performance java vectorization aarch64 x64https://pinboard.in/https://pinboard.in/u:jm/b:7e62e6169542/gProfiler2021-06-28T09:31:58+00:00
https://github.com/Granulate/gprofiler
jmgprofiler profiling performance testing measurement coding ruby java linux perf via:reddithttps://pinboard.in/https://pinboard.in/u:jm/b:6772a8b88a40/Defer disabling TLS 1.0/1.1 by default?2021-04-06T10:03:30+00:00
https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2021-April/005577.html
jm
The short version is this: we think about 1% of applications and
traffic "out there" are still using TLS1.0/TLS1.1. Given where
browsers are at, I think this percentage is an under-estimate of the
usage on Java applications - I suspect it's even higher there. When we
dig in with customers "Why are you still using TLS1.0 or TLS1.1" the
most common reasons are legacy appliances and applications. Think of
hardware load balancers that were never updated, or can't be, to
support TLS1.2 or better. Compliance mandated traffic inspection
devices that force TLS1.0 in certain industries are another reason.
For these applications, the change will break them, and they'll get a
low-level exception. The users can re-enable TLS1.0 and TLS1.1, but
they may suffer an outage because they likely weren't expecting a
breaking change low in the networking stack.
]]>colmmacc java jdk tls ssl versioning backwards-compatibility internethttps://pinboard.in/https://pinboard.in/u:jm/b:7e92ac9a3e78/Testcontainers2021-03-18T09:49:47+00:00
https://www.testcontainers.org/
jmTestcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.
]]>docker java junit testing containers testshttps://pinboard.in/https://pinboard.in/u:jm/b:ab95f5fb86ea/GCEasy.io2020-12-04T10:57:55+00:00
https://gceasy.io/
jmjava gc tuning performance jvm logs opshttps://pinboard.in/https://pinboard.in/u:jm/b:d3d8327d2376/AWS CRT HTTP Client in the AWS SDK for Java 2.x2020-09-30T21:12:21+00:00
https://aws.amazon.com/about-aws/whats-new/2020/09/aws-crt-http-client-in-aws-sdk-for-java-2x/
jmjava libraries aws http http-1.1 clientshttps://pinboard.in/https://pinboard.in/u:jm/b:21056a150feb/ServiceTalk2019-11-06T14:00:14+00:00
https://github.com/apple/servicetalk/
jma JVM network application framework with APIs tailored to specific protocols (e.g. HTTP/1.x, HTTP/2.x, etc…) and supports multiple programming paradigms.
It is built on Netty and is designed to provide most of the performance/scalability benefits of Netty for common networking protocols used in service to service communication. ServiceTalk provides server support and "smart client" like features such as client-side load balancing and service discovery integration.
Open source from Apple.]]>apple servicetalk netty libraries java jvm coding http asynchttps://pinboard.in/https://pinboard.in/u:jm/b:d97f22fd741a/Moving From Apache Thrift to gRPC: A Perspective From Alluxio2019-06-27T09:21:18+00:00
https://www.alluxio.io/blog/moving-from-apache-thrift-to-grpc-a-perspective-from-alluxio/
jmThrift served well as a fast and reliable RPC framework powering the metadata operations in Alluxio 1.x. Its limitation in handling streamed data has led us to a journey in search of better alternatives. gRPC provides some nice features that help us in building a simpler, more unified API layer. In this post, we discussed some lessons learned to move from Thrift to gRPC, including performance tuning tips that helped us achieve comparable performance for both one-off RPC calls as well as data streams. We hope this helps if you are looking at gRPC as an option for building high-performance services. Check out our blog for more articles on how we build Alluxio.
]]>thrift alluxio java grpc protocols coding nettyhttps://pinboard.in/https://pinboard.in/u:jm/b:5b7ff8c205dc/jCenter is the new default repository used with Android's gradle plugin, I haven... | Hacker News2019-06-24T10:10:07+00:00
https://news.ycombinator.com/item?id=8099840
jmI am a developer Advocate with JFrog, the company behind Bintray.
So, jcenter is a Java repository in Bintray (https://bintray.com/bintray/jcenter), which is the largest repo in the world for Java and Android OSS libraries, packages and components. All the content is served over a CDN, with a secure https connection. JCenter is the default repository in Groovy Grape
(http://groovy.codehaus.org/Grape), built-in in Gradle (the jcenter() repository) and very easy to configure in every other build tool (maybe except Maven) and will become even easer very soon.
Bintray has a different approach to package identification than the legacy Maven Central. We don't rely on self-issued key-pairs (which can be generated to represent anyone, actually and never verified in Maven Central). Instead, similar to GitHub, Bintray gives a strong personal identity to any contributed library.
If you really need to get your package to Maven Central (for supporting legacy tools) you can do it from Bintray as well, in a click of a button or even automatically.
]]>jars maven gradle java bintray via:lemire packaging distributionhttps://pinboard.in/https://pinboard.in/u:jm/b:5fc8d137cd4f/Spark memory tuning on EMR2019-04-10T09:48:34+00:00
https://aws.amazon.com/blogs/big-data/best-practices-for-successfully-managing-memory-for-apache-spark-applications-on-amazon-emr/
jmspark emr aws tuning memory ooms javahttps://pinboard.in/https://pinboard.in/u:jm/b:7d538a70d00d/Troubleshooting Problems With Native (Off-Heap) Memory in Java Applications2019-03-22T15:03:45+00:00
https://dzone.com/articles/troubleshooting-problems-with-native-off-heap-memo
jmjava off-heap buffers memory memory-leaks gc jdk opshttps://pinboard.in/https://pinboard.in/u:jm/b:cc36a7f2cf9b/[1902.04023] Computing Extremely Accurate Quantiles Using t-Digests2019-02-18T11:05:52+00:00
https://arxiv.org/abs/1902.04023
jmjava go python open-source quantiles percentiles approximation statistics sketching algorithms via:fanfhttps://pinboard.in/https://pinboard.in/u:jm/b:6c84ec8a0947/Java's ByteBuffer native memory "leak"2018-11-19T11:30:30+00:00
http://www.evanjones.ca/java-bytebuffer-leak.html
jmThe Java NIO APIs use ByteBuffers as the source and destination of I/O calls, and come in two flavours. Heap ByteBuffers wrap a byte[] array, allocated in the garbage collected Java heap. Direct ByteBuffers wrap memory allocated outside the Java heap using malloc. Only "native" memory can be passed to operating system calls, so it won't be moved by the garbage collector. This means that when you use a heap ByteBuffer for I/O, it is copied into a temporary direct ByteBuffer. The JDK caches one temporary buffer per thread, without any memory limits. As a result, if you call I/O methods with large heap ByteBuffers from multiple threads, your process can use a huge amount of additional native memory, which looks like a native memory leak. This can cause your process to unexpectedly run into memory limits and get killed.
]]>jvm performance java memory leaks bytebuffers netty threads coding bugshttps://pinboard.in/https://pinboard.in/u:jm/b:94c60cf2a090/What to Look for in Java 8 Code2018-10-05T12:13:30+00:00
https://blog.jetbrains.com/upsource/2016/08/03/what-to-look-for-in-java-8-code/
jmjava8 java code-review codinghttps://pinboard.in/https://pinboard.in/u:jm/b:4bdd8929bf47/Do not fall into Oracle's Java 11 trap2018-09-26T15:29:06+00:00
https://blog.joda.org/2018/09/do-not-fall-into-oracles-java-11-trap.html
jmThe key part of the terms is as follows:
You may not: use the Programs for any data processing or any commercial, production, or internal business purposes other than developing, testing, prototyping, and demonstrating your Application;
The trap is as follows:
Download Oracle JDK (because that is what you've always done, and it is what the web-search tells you);
Use it in production (because you didn't realise the license changed);
Get a nasty phone call from Oracle's license enforcement teams demanding lots of money
In other words, Oracle can rely on inertia from Java developers to cause them to download the wrong (commercial) release of Java. Unless you read the text/warnings/legalese very carefully you might not even realise Oracle JDK is now commercial, and that you are therefore liable to pay Oracle for Java.
]]>java licensing openjdk open-source oracle software jdk jrehttps://pinboard.in/https://pinboard.in/u:jm/b:f8b03b43714b/Google Online Security Blog: Introducing the Tink cryptographic software library2018-08-30T21:25:04+00:00
https://security.googleblog.com/2018/08/introducing-tink-cryptographic-software.html
jmTink aims to provide cryptographic APIs that are secure, easy to use correctly, and hard(er) to misuse. Tink is built on top of existing libraries such as BoringSSL and Java Cryptography Architecture, but includes countermeasures to many weaknesses in these libraries, which were discovered by Project Wycheproof, another project from our team.
With Tink, many common cryptographic operations such as data encryption, digital signatures, etc. can be done with only a few lines of code.
]]>tink google java c++ boringssl ssl jca cryptohttps://pinboard.in/https://pinboard.in/u:jm/b:6a051e37ba48/spotify/dockerfile-maven: A set of Maven tools for dealing with Dockerfiles2018-08-22T09:14:31+00:00
https://github.com/spotify/dockerfile-maven
jmjava docker maven build coding packaginghttps://pinboard.in/https://pinboard.in/u:jm/b:6e9446908fe4/Google Cloud Platform Blog: Introducing Jib2018-07-10T16:04:31+00:00
https://cloudplatform.googleblog.com/2018/07/introducing-jib-build-java-docker-images-better.html
jmJib takes advantage of layering in Docker images and integrates with your build system to optimize Java container image builds in the following ways:
Simple - Jib is implemented in Java and runs as part of your Maven or Gradle build. You do not need to maintain a Dockerfile, run a Docker daemon, or even worry about creating a fat JAR with all its dependencies. Since Jib tightly integrates with your Java build, it has access to all the necessary information to package your application. Any variations in your Java build are automatically picked up during subsequent container builds.
Fast - Jib takes advantage of image layering and registry caching to achieve fast, incremental builds. It reads your build config, organizes your application into distinct layers (dependencies, resources, classes) and only rebuilds and pushes the layers that have changed. When iterating quickly on a project, Jib can save valuable time on each build by only pushing your changed layers to the registry instead of your whole application.
Reproducible - Jib supports building container images declaratively from your Maven and Gradle build metadata, and as such can be configured to create reproducible build images as long as your inputs remain the same.
]]>build google java docker maven gradle coding builds jars fat-jars packaginghttps://pinboard.in/https://pinboard.in/u:jm/b:b5d92c0dff8b/airlift/jvmkill2018-07-05T09:29:02+00:00
https://github.com/airlift/jvmkill/blob/master/README.md
jma simple JVMTI agent that forcibly terminates the JVM when it is unable to allocate memory or create a thread. This is important for reliability purposes: an OutOfMemoryError will often leave the JVM in an inconsistent state. Terminating the JVM will allow it to be restarted by an external process manager.
This is apparently still useful despite the existence of '-XX:ExitOnOutOfMemoryError' as of java 8, since that may somehow still fail occasionally.]]>oom java reliability uptime memory opshttps://pinboard.in/https://pinboard.in/u:jm/b:db2c1986f022/How to change JVM arguments at runtime to avoid application restart2018-06-06T15:27:52+00:00
https://confluence.atlassian.com/kb/how-to-change-jvm-arguments-at-runtime-to-avoid-application-restart-816682109.html
jmjava -XX:+PrintFlagsFinal -version|grep manageable
]]>jvm ops switches cli java heap-dumps memory debugging memory-leakshttps://pinboard.in/https://pinboard.in/u:jm/b:b591a243dc1d/lemire/JavaFastPFOR: A simple integer compression library in Java2018-04-09T15:39:37+00:00
https://github.com/lemire/JavaFastPFOR
jm
a library to compress and uncompress arrays of integers very fast. The assumption is that most (but not all) values in your array use much less than 32 bits, or that the gaps between the integers use much less than 32 bits. These sort of arrays often come up when using differential coding in databases and information retrieval (e.g., in inverted indexes or column stores).
Please note that random integers are not compressible, by this library or by any other means. If you ever had the means of systematically compressing random integers, you could compress any data source to nothing, by recursive application of your technique.
This library can decompress integers at a rate of over 1.2 billions per second (4.5 GB/s). It is significantly faster than generic codecs (such as Snappy, LZ4 and so on) when compressing arrays of integers.
The library is used in LinkedIn Pinot, a realtime distributed OLAP datastore. Part of this library has been integrated in Parquet (http://parquet.io/). A modified version of the library is included in the search engine Terrier (http://terrier.org/). This libary is used by ClueWeb Tools (https://github.com/lintool/clueweb). It is also used by Apache NiFi.
]]>compression java pfor encoding integers algorithms storagehttps://pinboard.in/https://pinboard.in/u:jm/b:79294351fa68/Iterating over hash sets quickly in Java2018-03-13T22:21:26+00:00
https://lemire.me/blog/2018/03/13/iterating-over-hash-sets-quickly-in-java/
jmMy numbers are clear: in my tests, it is three times faster to sum up the values in a LinkedHashSet [than a HashSet].
]]>performance java locality memory cache-friendly data-structures hashsets linkedhashsets setshttps://pinboard.in/https://pinboard.in/u:jm/b:18a5574bcbd0/airlift/aircompressor: A port of Snappy, LZO and LZ4 to Java2018-01-30T17:35:19+00:00
https://github.com/airlift/aircompressor
jmThis library contains implementations of LZ4, Snappy, and LZO written in pure Java. They are typically 10-40% faster than the JNI wrapper for the native libraries.
]]>lz4 lzo lzop snappy java libraries airlift compression performancehttps://pinboard.in/https://pinboard.in/u:jm/b:ff75bb45fa1f/Intel pcj library for persistent memory-oriented data structures2017-10-03T09:25:17+00:00
https://github.com/pmem/pcj/blob/master/README.md
jmThis is a "pilot" project to develop a library for Java objects stored in persistent memory. Persistent collections are being emphasized because many applications for persistent memory seem to map well to the use of collections. One of this project's goals is to make programming with persistent objects feel natural to a Java developer, for example, by using familiar Java constructs when incorporating persistence elements such as data consistency and object lifetime.
The breadth of persistent types is currently limited and the code is not performance-optimized. We are making the code available because we believe it can be useful in experiments to retrofit existing Java code to use persistent memory and to explore persistent Java programming in general.
(via Mario Fusco)]]>persistent-memory data-structures storage persistence java coding futurehttps://pinboard.in/https://pinboard.in/u:jm/b:3fa9ddfacc22/Locking, Little's Law, and the USL2017-09-20T14:36:55+00:00
https://groups.google.com/forum/#!msg/mechanical-sympathy/gchG_oQ_kQM/59BDMOdUAwAJ
jmLittle's law can be used to describe a system in steady state from a queuing perspective, i.e. arrival and leaving rates are balanced. In this case it is a crude way of modelling a system with a contention percentage of 100% under Amdahl's law, in that throughput is one over latency.
However this is an inaccurate way to model a system with locks. Amdahl's law does not account for coherence costs. For example, if you wrote a microbenchmark with a single thread to measure the lock cost then it is much lower than in a multi-threaded environment where cache coherence, other OS costs such as scheduling, and lock implementations need to be considered.
Universal Scalability Law (USL) accounts for both the contention and the coherence costs.
http://www.perfdynamics.com/Manifesto/USLscalability.html
When modelling locks it is necessary to consider how contention and coherence costs vary given how they can be implemented. Consider in Java how we have biased locking, thin locks, fat locks, inflation, and revoking biases which can cause safe points that bring all threads in the JVM to a stop with a significant coherence component.
]]>usl scaling scalability performance locking locks java jvm amdahls-law littles-law system-dynamics modelling systems caching threads schedulers contentionhttps://pinboard.in/https://pinboard.in/u:jm/b:d64fb1279a0b/Native Memory Tracking2017-09-18T09:35:48+00:00
https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html
jmjava jvm memory native-memory malloc debugging coding nmt java-8 jcmdhttps://pinboard.in/https://pinboard.in/u:jm/b:d2f2bae02fb3/Java Flame Graphs Introduction: Fire For Everyone!2017-09-11T10:48:52+00:00
http://psy-lob-saw.blogspot.ie/2017/02/flamegraphs-intro-fire-for-everyone.html
jmprofiling java safepoints jvm flame-graphs perf measurement benchmarking testinghttps://pinboard.in/https://pinboard.in/u:jm/b:1a6f8c8d9b75/AdoptOpenJDK/jitwatch2017-08-24T16:22:31+00:00
https://github.com/AdoptOpenJDK/jitwatch
jmLog analyser and visualiser for the HotSpot JIT compiler. Inspect inlining decisions, hot methods, bytecode, and assembly. View results in the JavaFX user interface.
]]>analysis java jvm performance tools debugging optimization jithttps://pinboard.in/https://pinboard.in/u:jm/b:404ca57f5933/DoppioJVM2017-07-12T10:53:58+00:00
https://plasma-umass.github.io/doppio-demo/
jmjvm java javascript js hacks browser emulation websocketshttps://pinboard.in/https://pinboard.in/u:jm/b:15ebe75eae91/How does language, memory and package size affect cold starts of AWS Lambda?2017-06-19T15:32:19+00:00
https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of-aws-lambda-a15e26d12c76?__s=gf36pf8g1gjugcqh6ppo
jmjava cold-start lambda serverless awshttps://pinboard.in/https://pinboard.in/u:jm/b:4bb0f99fa5ce/usl4j And You | codahale.com2017-06-01T10:08:29+00:00
https://codahale.com/usl4j-and-you/
jmusl scalability java performance optimization benchmarking measurement ops coda-halehttps://pinboard.in/https://pinboard.in/u:jm/b:c184c035e80a/don't use String.intern() in Java2017-05-15T10:23:27+00:00
https://shipilev.net/jvm-anatomy-park/10-string-intern/
jmString.intern is the gateway to native JVM String table, and it comes with caveats: throughput, memory footprint, pause time problems will await the users. Hand-rolled deduplicators/interners to reduce memory footprint are working much more reliably, because they are working on Java side, and also can be thrown away when done. GC-assisted String deduplication does alleviate things even more. In almost every project we were taking care of, removing String.intern from the hotpaths was the very profitable performance optimization. Do not use it without thinking, okay?
]]>strings interning java performance tipshttps://pinboard.in/https://pinboard.in/u:jm/b:936351988ed0/Hadoop Internals2017-02-17T14:39:34+00:00
http://ercoppa.github.io/HadoopInternals/
jmhadoop map-reduce architecture coding java distcomphttps://pinboard.in/https://pinboard.in/u:jm/b:5e96fe6a92a4/Debugging Java Native Memory Leaks (evanjones.ca)2017-01-10T16:39:09+00:00
http://www.evanjones.ca/java-native-leak-bug.html
jmdebugging memory jvm java leaks memory-leaks leak-checking jemalloc malloc native heap off-heap gzipinputstreamhttps://pinboard.in/https://pinboard.in/u:jm/b:5206aa65dc43/tdunning/t-digest2016-12-12T12:28:16+00:00
https://github.com/tdunning/t-digest
jmA new data structure for accurate on-line accumulation of rank-based statistics such as quantiles and trimmed means. The t-digest algorithm is also very parallel friendly making it useful in map-reduce and parallel streaming applications.
The t-digest construction algorithm uses a variant of 1-dimensional k-means clustering to product a data structure that is related to the Q-digest. This t-digest data structure can be used to estimate quantiles or compute other rank statistics. The advantage of the t-digest over the Q-digest is that the t-digest can handle floating point values while the Q-digest is limited to integers. With small changes, the t-digest can handle any values from any ordered set that has something akin to a mean. The accuracy of quantile estimates produced by t-digests can be orders of magnitude more accurate than those produced by Q-digests in spite of the fact that t-digests are more compact when stored on disk.
Super-nice feature is that it's mergeable, so amenable to parallel usage across multiple hosts if required. Java implementation, ASL licensing.]]>data-structures algorithms java t-digest statistics quantiles percentiles aggregation digests estimation rankinghttps://pinboard.in/https://pinboard.in/u:jm/b:aaf9fb613f21/J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required"2016-09-13T10:05:50+00:00
http://www.slideshare.net/dbryant_uk/j1-2015-debugging-java-apps-in-containers-no-heavy-welding-gear-required
jmjava docker ops containershttps://pinboard.in/https://pinboard.in/u:jm/b:80ba8c32a215/Lessons Learned from Using Regexes At Scale2016-09-06T14:40:15+00:00
https://www.loggly.com/blog/lessons-learned-from-using-regexes-at-scale/
jmloggly regexp regexes java dfa nfa architecturehttps://pinboard.in/https://pinboard.in/u:jm/b:ac8a514b7bc3/Unchecked exceptions for IO considered harmful - Google Groups2016-08-16T16:23:09+00:00
https://groups.google.com/forum/#!topic/mechanical-sympathy/Ko8osOYLq4I
jmPeter Lawrey: Our view is that Checked Exception makes more sense for library writers as they can explicitly pass off errors to the caller. As a caller, especially if you are new to a product, you don't understand the exceptions or what you can do about them. They add confusion.
For this reason we use checked exceptions internally in the lower layers and try to avoid passing them in our higher level interfaces. Note: A high percentage of our fall backs are handling iOExceptons and recovering from them. [....]
My experience is that the more complex and layered your libraries the more essential checked exceptions become. I see them as essential for scalability of your software.
]]>exceptions java style coding checked-exceptions ioexceptions io error-handlinghttps://pinboard.in/https://pinboard.in/u:jm/b:53173a73b67b/Squeezing blood from a stone: small-memory JVM techniques for microservice sidecars2016-06-18T09:46:44+00:00
https://blog.buoyant.io/2016/06/17/small-memory-jvm-techniques-for-microservice-sidecars/
jmWe found two specific techniques to be the most beneficial: turning off one of the two JIT compilers enabled by default (the “C2” compiler), and using a 32-bit, rather than a 64-bit, JVM.
]]>32bit jvm java ops memory tuning jit linkerdhttps://pinboard.in/https://pinboard.in/u:jm/b:ade3b2e45f61/ztellman/dirigiste2016-06-01T11:55:13+00:00
https://github.com/ztellman/dirigiste
jmasync jvm dirigiste java threadpools concurrency utilization capacity executors object-pools object-pooling latencyhttps://pinboard.in/https://pinboard.in/u:jm/b:5092c7f0c12e/Koloboke Collections2016-05-31T14:54:10+00:00
https://koloboke.com/
jmcollections java koloboke performance codinghttps://pinboard.in/https://pinboard.in/u:jm/b:9a6a988b787c/Why the Very Silly Oracle v. Google Trial Actually Matters2016-05-26T15:19:01+00:00
http://motherboard.vice.com/read/why-the-very-silly-oracle-v-google-trial-actually-matters
jmIf it’s illegal to write clean room implementations of APIs, then no one has clean hands. The now-shelved open source project Apache Harmony, like Android, reimplemented Java SE, and tech giant IBM contributed code to that project. Oracle itself built its business off a proprietary implementation of SQL, which was created by IBM. The proposition “Reimplementations of APIs are infringements” creates a recursive rabbit hole of liability that spans across the industry. Even the very 37 Java APIs at issue in this trial contain reimplementations of other APIs. Google witness Joshua Bloch—who, while at Sun Microsystems, wrote many of the Java APIs—testified that specific Java APIs are reimplementations of other APIs from Perl 5 and the C programming language.
]]>apis fair-use copyright ip android java google oracle lawhttps://pinboard.in/https://pinboard.in/u:jm/b:dfc223222b1d/uber-common/infer-plugin2016-05-17T13:11:47+00:00
https://github.com/uber-common/infer-plugin
jminfer java static-analysis bugs coding nullhttps://pinboard.in/https://pinboard.in/u:jm/b:445c127fd3da/In Oracle v. Google, a Nerd Subculture Is on Trial2016-05-13T14:26:51+00:00
http://motherboard.vice.com/read/in-google-v-oracle-the-nerds-are-getting-owned
jm“The G part stands for GNU?” Alsup asked in disbelief.
“Yes,” said Schwartz on the stand.
“That doesn’t make any sense,” said the 71-year-old Clinton appointee.
]]>law gnu gpl licensing java oracle sun apis iphttps://pinboard.in/https://pinboard.in/u:jm/b:122f28325fe9/Using jemalloc to get to the bottom of an off-heap Java memory leak2016-04-14T11:39:20+00:00
https://gdstechnology.blog.gov.uk/2015/12/11/using-jemalloc-to-get-to-the-bottom-of-a-memory-leak/
jmdebugging java jvm memory jemalloc off-heaphttps://pinboard.in/https://pinboard.in/u:jm/b:e8651207606c/JitPack2016-04-04T09:53:25+00:00
https://jitpack.io/
jmbuild github java maven gradle dependencies packaging librarieshttps://pinboard.in/https://pinboard.in/u:jm/b:0c48d5efec3f/Hashed Wheel Timer2016-03-29T12:02:43+00:00
https://github.com/ifesdjeen/hashed-wheel-timer
jmscalability java timers hashed-wheel-timers algorithms data-structureshttps://pinboard.in/https://pinboard.in/u:jm/b:6139ec69af2a/Wait and lock free alternatives to LongAdder and AtomicLong by qwwdfsad2016-03-14T17:32:46+00:00
https://github.com/JCTools/JCTools/pull/93
jmlongadder doug-lea mechanical-sympathy lock-free performance atomic multithreading java jvmhttps://pinboard.in/https://pinboard.in/u:jm/b:28819375e2f4/Conversant ConcurrentQueue and Disruptor BlockingQueue2016-03-07T11:24:20+00:00
https://github.com/conversant/disruptor
jmdisruptor blocking-queues queues queueing data-structures algorithms java conversant concurrency performancehttps://pinboard.in/https://pinboard.in/u:jm/b:502056282b58/Argon2 code audits - part one - Infer2016-02-23T17:24:29+00:00
https://www.lolware.net/2016/02/12/argon2-code-review.html
jminfer facebook java clang errors static-analysis lint dataflow dockerhttps://pinboard.in/https://pinboard.in/u:jm/b:5e2f76232714/Netty @Apple: Large Scale Deployment/Connectivity [video]2016-01-08T10:04:43+00:00
http://www.infoq.com/presentations/apple-netty
jmapple netty norman-maurer java jvm async talks presentationshttps://pinboard.in/https://pinboard.in/u:jm/b:09aa5e33f478/The Importance of Tuning Your Thread Pools2016-01-04T10:42:16+00:00
https://www.voxxed.com/blog/2015/12/importance-tuning-thread-pools/
jmhystrix threadpools concurrency java jvm backpressure littles-law capacityhttps://pinboard.in/https://pinboard.in/u:jm/b:d8e4a0101363/raskasa/metrics-okhttp2015-12-31T10:27:43+00:00
https://github.com/raskasa/metrics-okhttp
jmokhttp java clients http metrics dropwizardhttps://pinboard.in/https://pinboard.in/u:jm/b:5576ec805c35/Introducing Netty-HTTP from Cask2015-12-03T12:52:22+00:00
http://blog.cask.co/2015/12/introducing-netty-http-from-cask/
jmnetty-http library solves [Netty usability issues] by using JAX-RS annotations to build a HTTP path routing layer on top of netty. In addition, the library implements a guava service to manage the HTTP service. netty-http allows users of the library to just focus on writing the business logic in HTTP handlers without having to worry about the complexities of path routing or learning netty pipeline internals to build the HTTP service.
We've written something very similar, although I didn't even bother supporting JAX-RS annotations -- just a simple code-level DSL.]]>jax-rs netty http cask java services codinghttps://pinboard.in/https://pinboard.in/u:jm/b:c5c91942c05c/What Do WebLogic, WebSphere, JBoss, Jenkins, OpenNMS, and Your Application Have in Common? This Vulnerability.2015-11-09T12:31:56+00:00
http://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/
jmjava serialization security exploits jenkinshttps://pinboard.in/https://pinboard.in/u:jm/b:c0dc116c6113/Google is Maven Central's New Best Friend2015-10-29T11:24:01+00:00
http://takari.io/2015/10/28/google-maven-central.html
jmgoogle maven maven-central jars hosting java packages buildhttps://pinboard.in/https://pinboard.in/u:jm/b:aa09406818b9/Open-sourcing PalDB, a lightweight companion for storing side data2015-10-28T15:35:31+00:00
https://engineering.linkedin.com/blog/2015/10/open-sourcing-paldb--a-lightweight-companion-for-storing-side-da
jmlinkedin open-source storage side-data data config paldb java apache databaseshttps://pinboard.in/https://pinboard.in/u:jm/b:5f9ff3f038de/How Netty is used at Layer2015-10-22T08:26:14+00:00
https://medium.com/@mubarak.seyed/netty-at-layer-powering-client-api-2a6ca3617d48#.91mjjdwh2
jmnetty http api-services coding java servershttps://pinboard.in/https://pinboard.in/u:jm/b:2a9e57061ee5/librato/disco-java2015-10-12T09:32:23+00:00
https://github.com/librato/disco-java
jmzookeeper service-discovery librato java open-source load-balancinghttps://pinboard.in/https://pinboard.in/u:jm/b:1bc6194a9eef/Byteman2015-09-24T10:11:48+00:00
http://byteman.jboss.org/
jma tool which simplifies tracing and testing of Java programs. Byteman allows you to insert extra Java code into your application, either as it is loaded during JVM startup or even after it has already started running. The injected code is allowed to access any of your data and call any application methods, including where they are private. You can inject code almost anywhere you want and there is no need to prepare the original source code in advance nor do you have to recompile, repackage or redeploy your application. In fact you can remove injected code and reinstall different code while the application continues to execute. The simplest use of Byteman is to install code which traces what your application is doing. This can be used for monitoring or debugging live deployments as well as for instrumenting code under test so that you can be sure it has operated correctly. By injecting code at very specific locations you can avoid the overheads which often arise when you switch on debug or product trace. Also, you decide what to trace when you run your application rather than when you write it so you don't need 100% hindsight to be able to obtain the information you need.
]]>tracing java byteman injection jvm ops debugging testinghttps://pinboard.in/https://pinboard.in/u:jm/b:542b80c1b242/