Pinboard (jm) https://pinboard.in/u:jm/public/ recent bookmarks from jm Reverse engineering my cloud-connected e-scooter and finding the master key to unlock all scooters 2026-01-16T15:10:07+00:00 https://blog.nns.ee/2026/01/06/aike-ble/ jmAndroid exposes the Java classes android.bluetooth.BluetoothGatt and android.bluetooth.BluetoothGattCallback that apps are expected to use to use GATT characteristics. We can use Frida to hook into these and override many of the interesting functions. I was mostly interested in reads, writes and GATT notifications, so I whipped up a Frida script to hook into these and print all comms to the console [...] The 20-byte value had me suspecting that SHA-1 was somehow being used. To confirm, I wrote another Frida script that hooks Android hashing functions exposed by the Java class java.security.MessageDigest [...] The app uses Firebase for most of its cloud functionality. When signing in and pairing your scooter, the server sends the app a secret key. This is stored on the Android device, and can be read with root access. ]]> frida reverse-engineering android firebase java kotlin gatt bluetooth react-native https://pinboard.in/ https://pinboard.in/u:jm/b:30faa9bbc820/ UltraLogLog 2025-03-25T10:21:11+00:00 https://arxiv.org/abs/2308.16862 jmSince its invention HyperLogLog has become the standard algorithm for approximate distinct counting. Due to its space efficiency and suitability for distributed systems, it is widely used and also implemented in numerous databases. This work presents UltraLogLog, which shares the same practical properties as HyperLogLog. It is commutative, idempotent, mergeable, and has a fast guaranteed constant-time insert operation. At the same time, it requires 28% less space to encode the same amount of distinct count information, which can be extracted using the maximum likelihood method. Alternatively, a simpler and faster estimator is proposed, which still achieves a space reduction of 24%, but at an estimation speed comparable to that of HyperLogLog. In a non-distributed setting where martingale estimation can be used, UltraLogLog is able to reduce space by 17%. Moreover, its smaller entropy and its 8-bit registers lead to better compaction when using standard compression algorithms. All this is verified by experimental results that are in perfect agreement with the theoretical analysis which also outlines potential for even more space-efficient data structures. A production-ready Java implementation of UltraLogLog has been released as part of the open-source Hash4j library. (via Tony Finch)]]> via:fanf algorithms data-structures hyperloglog ultraloglog counting count-distinct distinct approximation counts java https://pinboard.in/ https://pinboard.in/u:jm/b:a54e7c3c0b7d/ Netflix/hollow 2025-02-25T12:19:14+00:00 https://github.com/Netflix/hollow jm Hollow is a java library and toolset for disseminating in-memory datasets from a single producer to many consumers for high performance read-only access. Hollow focuses narrowly on its prescribed problem set: keeping an entire, read-only dataset in-memory on consumers. It circumvents the consequences of updating and evicting data from a partial cache. Due to its performance characteristics, Hollow shifts the scale in terms of appropriate dataset sizes for an in-memory solution. Datasets for which such liberation may never previously have been considered can be candidates for Hollow. For example, Hollow may be entirely appropriate for datasets which, if represented with json or XML, might require in excess of 100GB. Interesting approach, though possibly a bit scary in terms of circumventing the "keep things simple and boring" rule... still, a useful tool to have.]]> cache caching netflix java jvm memory hollow read-only architecture systems https://pinboard.in/ https://pinboard.in/u:jm/b:3f732f05411d/ LLM-Driven Code Completion in JetBrains IDEs 2025-01-27T11:34:28+00:00 https://blog.jetbrains.com/blog/2024/04/04/full-line-code-completion-in-jetbrains-ides-all-you-need-to-know/#under-the-hood jm The length of the completion suggestions is a trade-off. While longer suggestions do tend to reduce how many keystrokes you have to make, which is good, they also increase the number of reviews required on your end. Taking the above into account, we decided that completing a single line of code would be a fair compromise. Some key features:
- It works locally and is available offline. This means you can take advantage of the feature even if you aren’t connected to the internet. - It doesn’t send any data from your machine over the internet. The language models that power full line code completion run locally, which is great for two reasons. First, your code remains safe, as it never leaves your machine. Second, there are no additional cloud-related expenses – that’s why this feature comes at no additional cost.
Also, customer code is never used for training. I've used this (in RubyMine), and found it fairly useful; it's good for generating the obvious next line, but is easily ignored when that's not what's needed. Not bad at all.]]> coding code-completion jetbrains ides java ruby llms ai code-generation rubymine intellij https://pinboard.in/ https://pinboard.in/u:jm/b:87af374c576d/ Optimizing Java Apps on Kubernetes 2025-01-23T14:45:35+00:00 https://www.infoq.com/presentations/optimizing-java-app-kubernetes/ jm kubernetes java eks resources ops scaling scalability gc optimization jvm https://pinboard.in/ https://pinboard.in/u:jm/b:2c9522ab8d41/ simdjson/simdjson-java 2023-10-09T08:06:01+00:00 https://github.com/simdjson/simdjson-java jm simd java json parsing formats performance libraries https://pinboard.in/ https://pinboard.in/u:jm/b:587e2c3aab0d/ The Black Magic of (Java) Method Dispatch 2023-04-11T21:03:19+00:00 https://shipilev.net/blog/2015/black-magic-method-dispatch/#_cheating_the_runtime_2 jmProgramming languages like Java provide the facilities for subtyping/polymorphism as one of the ways to construct modular and reusable software. This language choice naturally comes at a price, since there is no hardware support for virtual calls, and therefore runtimes have to emulate this behavior. In many, many cases the performance of method dispatch is not important. Actually, in a vast majority of cases, the low-level performance concerns are not the real concerns. However, there are cases when method dispatch performance is important, and there you need to understand how dispatch works, what runtimes optimize for you, and what you can do to cheat and/or emulate similar behavior in your code. For example, in the course of String Compression work, we were faced with the problem of selecting the coder for a given String. The obvious and highly maintainable approach of creating a Coder interface, a few implementations, and dispatching the virtual calls over it, had met some performance problems on the very tiny benchmarks. Therefore, we needed to contemplate something better. After a few experiments, this post was born as a reference for others who might try to do the same. This post also tangentially touches the inlining of virtual calls, as the natural thing during the optimization. Discovered via this amazing commit: https://github.com/quarkusio/quarkus/commit/65dd4d43e2644db1c87726139280f9704140167c ]]> optimization performance java oo jit coding polymorphism https://pinboard.in/ https://pinboard.in/u:jm/b:fcc32a99ea26/ Async-profiler manual by use cases 2023-01-05T15:53:26+00:00 https://krzysztofslusarski.github.io/2022/12/12/async-manual.html jm async-profiler java jvm profiling tracing performance tuning https://pinboard.in/ https://pinboard.in/u:jm/b:8ec959433ae2/ don't use JVM Flight Recorder profiling 2022-09-05T10:57:00+00:00 https://twitter.com/nitsanw/status/1566677642748874753 jmIf you are profiling for a CPU bottleneck [in java], DO NOT RELY ON JVM FLIGHT RECORDER METHOD PROFILING. Not even a little bit. Use `async-profiler` for profiling(`-e cpu,lock,alloc`), with `--jfrsync default/profile` for extra JVM/JDK events. ]]> profiling java performance optimization jvm async-profiler via:twitter https://pinboard.in/ https://pinboard.in/u:jm/b:acdfaf080f4b/ ULID 2022-06-13T10:06:43+00:00 https://github.com/ulid/spec jm ulid uuid unique-ids java algorithms via:nelson unique ids https://pinboard.in/ https://pinboard.in/u:jm/b:2c3939ba08e2/ CVE-2022-21449: Psychic Signatures in Java 2022-04-20T09:29:49+00:00 https://neilmadden.blog/2022/04/19/psychic-signatures-in-java/ jmRecent releases of Java were vulnerable to a similar kind of trick, in the implementation of widely-used ECDSA signatures. If you are running one of the vulnerable versions then an attacker can easily forge some types of SSL certificates and handshakes (allowing interception and modification of communications), signed JWTs, SAML assertions or OIDC id tokens, and even WebAuthn authentication messages. All using the digital equivalent of a blank piece of paper. It’s hard to overstate the severity of this bug. If you are using ECDSA signatures for any of these security mechanisms, then an attacker can trivially and completely bypass them if your server is running any Java 15, 16, 17, or 18 version before the April 2022 Critical Patch Update (CPU). For context, almost all WebAuthn/FIDO devices in the real world (including Yubikeys*) use ECDSA signatures and many OIDC providers use ECDSA-signed JWTs. ]]> java jvm crypto security ecdsa webauthn saml jwt fail https://pinboard.in/ https://pinboard.in/u:jm/b:19326f7fa161/ Resilience4J 2022-03-30T22:18:54+00:00 https://github.com/resilience4j/resilience4j jm fp hystrix java fault-tolerance retries timeouts architecture patterns circuit-breakers https://pinboard.in/ https://pinboard.in/u:jm/b:3c49a9299497/ SWAR indexOf byte search 2022-01-24T10:08:22+00:00 https://github.com/netty/netty/pull/10737/files jm simd swar indexof bytebuffer java optimization performance search netty hacks https://pinboard.in/ https://pinboard.in/u:jm/b:b1bb44f3ef08/ Distributed Unique Time Stamp Identifiers 2022-01-06T10:28:00+00:00 https://blog.vanillajava.blog/2022/01/distributed-unique-time-stamp.html jm java ids uniqueness identifiers algorithms peter-lawrey performance https://pinboard.in/ https://pinboard.in/u:jm/b:fc53ee7af57c/ Overhead of Returning Optional Values in Java and Rust 2021-10-16T21:46:58+00:00 https://pkolaczk.github.io/overhead-of-optional/ jm java rust optimization performance hotspots benchmarks optional null https://pinboard.in/ https://pinboard.in/u:jm/b:950a505b2bef/ SIMD is coming to the JVM 2021-09-13T11:31:58+00:00 https://www.morling.dev/blog/fizzbuzz-simd-style/ jm simd performance java vectorization aarch64 x64 https://pinboard.in/ https://pinboard.in/u:jm/b:7e62e6169542/ gProfiler 2021-06-28T09:31:58+00:00 https://github.com/Granulate/gprofiler jm gprofiler profiling performance testing measurement coding ruby java linux perf via:reddit https://pinboard.in/ https://pinboard.in/u:jm/b:6772a8b88a40/ Defer disabling TLS 1.0/1.1 by default? 2021-04-06T10:03:30+00:00 https://mail.openjdk.java.net/pipermail/jdk-updates-dev/2021-April/005577.html jm The short version is this: we think about 1% of applications and traffic "out there" are still using TLS1.0/TLS1.1. Given where browsers are at, I think this percentage is an under-estimate of the usage on Java applications - I suspect it's even higher there. When we dig in with customers "Why are you still using TLS1.0 or TLS1.1" the most common reasons are legacy appliances and applications. Think of hardware load balancers that were never updated, or can't be, to support TLS1.2 or better. Compliance mandated traffic inspection devices that force TLS1.0 in certain industries are another reason. For these applications, the change will break them, and they'll get a low-level exception. The users can re-enable TLS1.0 and TLS1.1, but they may suffer an outage because they likely weren't expecting a breaking change low in the networking stack. ]]> colmmacc java jdk tls ssl versioning backwards-compatibility internet https://pinboard.in/ https://pinboard.in/u:jm/b:7e92ac9a3e78/ Testcontainers 2021-03-18T09:49:47+00:00 https://www.testcontainers.org/ jmTestcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container. ]]> docker java junit testing containers tests https://pinboard.in/ https://pinboard.in/u:jm/b:ab95f5fb86ea/ GCEasy.io 2020-12-04T10:57:55+00:00 https://gceasy.io/ jm java gc tuning performance jvm logs ops https://pinboard.in/ https://pinboard.in/u:jm/b:d3d8327d2376/ AWS CRT HTTP Client in the AWS SDK for Java 2.x 2020-09-30T21:12:21+00:00 https://aws.amazon.com/about-aws/whats-new/2020/09/aws-crt-http-client-in-aws-sdk-for-java-2x/ jm java libraries aws http http-1.1 clients https://pinboard.in/ https://pinboard.in/u:jm/b:21056a150feb/ ServiceTalk 2019-11-06T14:00:14+00:00 https://github.com/apple/servicetalk/ jma JVM network application framework with APIs tailored to specific protocols (e.g. HTTP/1.x, HTTP/2.x, etc…​) and supports multiple programming paradigms. It is built on Netty and is designed to provide most of the performance/scalability benefits of Netty for common networking protocols used in service to service communication. ServiceTalk provides server support and "smart client" like features such as client-side load balancing and service discovery integration. Open source from Apple.]]> apple servicetalk netty libraries java jvm coding http async https://pinboard.in/ https://pinboard.in/u:jm/b:d97f22fd741a/ Moving From Apache Thrift to gRPC: A Perspective From Alluxio 2019-06-27T09:21:18+00:00 https://www.alluxio.io/blog/moving-from-apache-thrift-to-grpc-a-perspective-from-alluxio/ jmThrift served well as a fast and reliable RPC framework powering the metadata operations in Alluxio 1.x. Its limitation in handling streamed data has led us to a journey in search of better alternatives. gRPC provides some nice features that help us in building a simpler, more unified API layer. In this post, we discussed some lessons learned to move from Thrift to gRPC, including performance tuning tips that helped us achieve comparable performance for both one-off RPC calls as well as data streams. We hope this helps if you are looking at gRPC as an option for building high-performance services. Check out our blog for more articles on how we build Alluxio. ]]> thrift alluxio java grpc protocols coding netty https://pinboard.in/ https://pinboard.in/u:jm/b:5b7ff8c205dc/ jCenter is the new default repository used with Android's gradle plugin, I haven... | Hacker News 2019-06-24T10:10:07+00:00 https://news.ycombinator.com/item?id=8099840 jmI am a developer Advocate with JFrog, the company behind Bintray. So, jcenter is a Java repository in Bintray (https://bintray.com/bintray/jcenter), which is the largest repo in the world for Java and Android OSS libraries, packages and components. All the content is served over a CDN, with a secure https connection. JCenter is the default repository in Groovy Grape (http://groovy.codehaus.org/Grape), built-in in Gradle (the jcenter() repository) and very easy to configure in every other build tool (maybe except Maven) and will become even easer very soon. Bintray has a different approach to package identification than the legacy Maven Central. We don't rely on self-issued key-pairs (which can be generated to represent anyone, actually and never verified in Maven Central). Instead, similar to GitHub, Bintray gives a strong personal identity to any contributed library. If you really need to get your package to Maven Central (for supporting legacy tools) you can do it from Bintray as well, in a click of a button or even automatically. ]]> jars maven gradle java bintray via:lemire packaging distribution https://pinboard.in/ https://pinboard.in/u:jm/b:5fc8d137cd4f/ Spark memory tuning on EMR 2019-04-10T09:48:34+00:00 https://aws.amazon.com/blogs/big-data/best-practices-for-successfully-managing-memory-for-apache-spark-applications-on-amazon-emr/ jm spark emr aws tuning memory ooms java https://pinboard.in/ https://pinboard.in/u:jm/b:7d538a70d00d/ Troubleshooting Problems With Native (Off-Heap) Memory in Java Applications 2019-03-22T15:03:45+00:00 https://dzone.com/articles/troubleshooting-problems-with-native-off-heap-memo jm java off-heap buffers memory memory-leaks gc jdk ops https://pinboard.in/ https://pinboard.in/u:jm/b:cc36a7f2cf9b/ [1902.04023] Computing Extremely Accurate Quantiles Using t-Digests 2019-02-18T11:05:52+00:00 https://arxiv.org/abs/1902.04023 jm java go python open-source quantiles percentiles approximation statistics sketching algorithms via:fanf https://pinboard.in/ https://pinboard.in/u:jm/b:6c84ec8a0947/ Java's ByteBuffer native memory "leak" 2018-11-19T11:30:30+00:00 http://www.evanjones.ca/java-bytebuffer-leak.html jmThe Java NIO APIs use ByteBuffers as the source and destination of I/O calls, and come in two flavours. Heap ByteBuffers wrap a byte[] array, allocated in the garbage collected Java heap. Direct ByteBuffers wrap memory allocated outside the Java heap using malloc. Only "native" memory can be passed to operating system calls, so it won't be moved by the garbage collector. This means that when you use a heap ByteBuffer for I/O, it is copied into a temporary direct ByteBuffer. The JDK caches one temporary buffer per thread, without any memory limits. As a result, if you call I/O methods with large heap ByteBuffers from multiple threads, your process can use a huge amount of additional native memory, which looks like a native memory leak. This can cause your process to unexpectedly run into memory limits and get killed. ]]> jvm performance java memory leaks bytebuffers netty threads coding bugs https://pinboard.in/ https://pinboard.in/u:jm/b:94c60cf2a090/ What to Look for in Java 8 Code 2018-10-05T12:13:30+00:00 https://blog.jetbrains.com/upsource/2016/08/03/what-to-look-for-in-java-8-code/ jm java8 java code-review coding https://pinboard.in/ https://pinboard.in/u:jm/b:4bdd8929bf47/ Do not fall into Oracle's Java 11 trap 2018-09-26T15:29:06+00:00 https://blog.joda.org/2018/09/do-not-fall-into-oracles-java-11-trap.html jmThe key part of the terms is as follows: You may not: use the Programs for any data processing or any commercial, production, or internal business purposes other than developing, testing, prototyping, and demonstrating your Application; The trap is as follows: Download Oracle JDK (because that is what you've always done, and it is what the web-search tells you); Use it in production (because you didn't realise the license changed); Get a nasty phone call from Oracle's license enforcement teams demanding lots of money In other words, Oracle can rely on inertia from Java developers to cause them to download the wrong (commercial) release of Java. Unless you read the text/warnings/legalese very carefully you might not even realise Oracle JDK is now commercial, and that you are therefore liable to pay Oracle for Java. ]]> java licensing openjdk open-source oracle software jdk jre https://pinboard.in/ https://pinboard.in/u:jm/b:f8b03b43714b/ Google Online Security Blog: Introducing the Tink cryptographic software library 2018-08-30T21:25:04+00:00 https://security.googleblog.com/2018/08/introducing-tink-cryptographic-software.html jmTink aims to provide cryptographic APIs that are secure, easy to use correctly, and hard(er) to misuse. Tink is built on top of existing libraries such as BoringSSL and Java Cryptography Architecture, but includes countermeasures to many weaknesses in these libraries, which were discovered by Project Wycheproof, another project from our team. With Tink, many common cryptographic operations such as data encryption, digital signatures, etc. can be done with only a few lines of code. ]]> tink google java c++ boringssl ssl jca crypto https://pinboard.in/ https://pinboard.in/u:jm/b:6a051e37ba48/ spotify/dockerfile-maven: A set of Maven tools for dealing with Dockerfiles 2018-08-22T09:14:31+00:00 https://github.com/spotify/dockerfile-maven jm java docker maven build coding packaging https://pinboard.in/ https://pinboard.in/u:jm/b:6e9446908fe4/ Google Cloud Platform Blog: Introducing Jib 2018-07-10T16:04:31+00:00 https://cloudplatform.googleblog.com/2018/07/introducing-jib-build-java-docker-images-better.html jmJib takes advantage of layering in Docker images and integrates with your build system to optimize Java container image builds in the following ways: Simple - Jib is implemented in Java and runs as part of your Maven or Gradle build. You do not need to maintain a Dockerfile, run a Docker daemon, or even worry about creating a fat JAR with all its dependencies. Since Jib tightly integrates with your Java build, it has access to all the necessary information to package your application. Any variations in your Java build are automatically picked up during subsequent container builds. Fast - Jib takes advantage of image layering and registry caching to achieve fast, incremental builds. It reads your build config, organizes your application into distinct layers (dependencies, resources, classes) and only rebuilds and pushes the layers that have changed. When iterating quickly on a project, Jib can save valuable time on each build by only pushing your changed layers to the registry instead of your whole application. Reproducible - Jib supports building container images declaratively from your Maven and Gradle build metadata, and as such can be configured to create reproducible build images as long as your inputs remain the same. ]]> build google java docker maven gradle coding builds jars fat-jars packaging https://pinboard.in/ https://pinboard.in/u:jm/b:b5d92c0dff8b/ airlift/jvmkill 2018-07-05T09:29:02+00:00 https://github.com/airlift/jvmkill/blob/master/README.md jma simple JVMTI agent that forcibly terminates the JVM when it is unable to allocate memory or create a thread. This is important for reliability purposes: an OutOfMemoryError will often leave the JVM in an inconsistent state. Terminating the JVM will allow it to be restarted by an external process manager. This is apparently still useful despite the existence of '-XX:ExitOnOutOfMemoryError' as of java 8, since that may somehow still fail occasionally.]]> oom java reliability uptime memory ops https://pinboard.in/ https://pinboard.in/u:jm/b:db2c1986f022/ How to change JVM arguments at runtime to avoid application restart 2018-06-06T15:27:52+00:00 https://confluence.atlassian.com/kb/how-to-change-jvm-arguments-at-runtime-to-avoid-application-restart-816682109.html jmjava -XX:+PrintFlagsFinal -version|grep manageable ]]> jvm ops switches cli java heap-dumps memory debugging memory-leaks https://pinboard.in/ https://pinboard.in/u:jm/b:b591a243dc1d/ lemire/JavaFastPFOR: A simple integer compression library in Java 2018-04-09T15:39:37+00:00 https://github.com/lemire/JavaFastPFOR jm a library to compress and uncompress arrays of integers very fast. The assumption is that most (but not all) values in your array use much less than 32 bits, or that the gaps between the integers use much less than 32 bits. These sort of arrays often come up when using differential coding in databases and information retrieval (e.g., in inverted indexes or column stores). Please note that random integers are not compressible, by this library or by any other means. If you ever had the means of systematically compressing random integers, you could compress any data source to nothing, by recursive application of your technique. This library can decompress integers at a rate of over 1.2 billions per second (4.5 GB/s). It is significantly faster than generic codecs (such as Snappy, LZ4 and so on) when compressing arrays of integers. The library is used in LinkedIn Pinot, a realtime distributed OLAP datastore. Part of this library has been integrated in Parquet (http://parquet.io/). A modified version of the library is included in the search engine Terrier (http://terrier.org/). This libary is used by ClueWeb Tools (https://github.com/lintool/clueweb). It is also used by Apache NiFi. ]]> compression java pfor encoding integers algorithms storage https://pinboard.in/ https://pinboard.in/u:jm/b:79294351fa68/ Iterating over hash sets quickly in Java 2018-03-13T22:21:26+00:00 https://lemire.me/blog/2018/03/13/iterating-over-hash-sets-quickly-in-java/ jmMy numbers are clear: in my tests, it is three times faster to sum up the values in a LinkedHashSet [than a HashSet]. ]]> performance java locality memory cache-friendly data-structures hashsets linkedhashsets sets https://pinboard.in/ https://pinboard.in/u:jm/b:18a5574bcbd0/ airlift/aircompressor: A port of Snappy, LZO and LZ4 to Java 2018-01-30T17:35:19+00:00 https://github.com/airlift/aircompressor jmThis library contains implementations of LZ4, Snappy, and LZO written in pure Java. They are typically 10-40% faster than the JNI wrapper for the native libraries. ]]> lz4 lzo lzop snappy java libraries airlift compression performance https://pinboard.in/ https://pinboard.in/u:jm/b:ff75bb45fa1f/ Intel pcj library for persistent memory-oriented data structures 2017-10-03T09:25:17+00:00 https://github.com/pmem/pcj/blob/master/README.md jmThis is a "pilot" project to develop a library for Java objects stored in persistent memory. Persistent collections are being emphasized because many applications for persistent memory seem to map well to the use of collections. One of this project's goals is to make programming with persistent objects feel natural to a Java developer, for example, by using familiar Java constructs when incorporating persistence elements such as data consistency and object lifetime. The breadth of persistent types is currently limited and the code is not performance-optimized. We are making the code available because we believe it can be useful in experiments to retrofit existing Java code to use persistent memory and to explore persistent Java programming in general. (via Mario Fusco)]]> persistent-memory data-structures storage persistence java coding future https://pinboard.in/ https://pinboard.in/u:jm/b:3fa9ddfacc22/ Locking, Little's Law, and the USL 2017-09-20T14:36:55+00:00 https://groups.google.com/forum/#!msg/mechanical-sympathy/gchG_oQ_kQM/59BDMOdUAwAJ jmLittle's law can be used to describe a system in steady state from a queuing perspective, i.e. arrival and leaving rates are balanced. In this case it is a crude way of modelling a system with a contention percentage of 100% under Amdahl's law, in that throughput is one over latency. However this is an inaccurate way to model a system with locks. Amdahl's law does not account for coherence costs. For example, if you wrote a microbenchmark with a single thread to measure the lock cost then it is much lower than in a multi-threaded environment where cache coherence, other OS costs such as scheduling, and lock implementations need to be considered. Universal Scalability Law (USL) accounts for both the contention and the coherence costs. http://www.perfdynamics.com/Manifesto/USLscalability.html When modelling locks it is necessary to consider how contention and coherence costs vary given how they can be implemented. Consider in Java how we have biased locking, thin locks, fat locks, inflation, and revoking biases which can cause safe points that bring all threads in the JVM to a stop with a significant coherence component. ]]> usl scaling scalability performance locking locks java jvm amdahls-law littles-law system-dynamics modelling systems caching threads schedulers contention https://pinboard.in/ https://pinboard.in/u:jm/b:d64fb1279a0b/ Native Memory Tracking 2017-09-18T09:35:48+00:00 https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr007.html jm java jvm memory native-memory malloc debugging coding nmt java-8 jcmd https://pinboard.in/ https://pinboard.in/u:jm/b:d2f2bae02fb3/ Java Flame Graphs Introduction: Fire For Everyone! 2017-09-11T10:48:52+00:00 http://psy-lob-saw.blogspot.ie/2017/02/flamegraphs-intro-fire-for-everyone.html jm profiling java safepoints jvm flame-graphs perf measurement benchmarking testing https://pinboard.in/ https://pinboard.in/u:jm/b:1a6f8c8d9b75/ AdoptOpenJDK/jitwatch 2017-08-24T16:22:31+00:00 https://github.com/AdoptOpenJDK/jitwatch jmLog analyser and visualiser for the HotSpot JIT compiler. Inspect inlining decisions, hot methods, bytecode, and assembly. View results in the JavaFX user interface. ]]> analysis java jvm performance tools debugging optimization jit https://pinboard.in/ https://pinboard.in/u:jm/b:404ca57f5933/ DoppioJVM 2017-07-12T10:53:58+00:00 https://plasma-umass.github.io/doppio-demo/ jm jvm java javascript js hacks browser emulation websockets https://pinboard.in/ https://pinboard.in/u:jm/b:15ebe75eae91/ How does language, memory and package size affect cold starts of AWS Lambda? 2017-06-19T15:32:19+00:00 https://read.acloud.guru/does-coding-language-memory-or-package-size-affect-cold-starts-of-aws-lambda-a15e26d12c76?__s=gf36pf8g1gjugcqh6ppo jm java cold-start lambda serverless aws https://pinboard.in/ https://pinboard.in/u:jm/b:4bb0f99fa5ce/ usl4j And You | codahale.com 2017-06-01T10:08:29+00:00 https://codahale.com/usl4j-and-you/ jm usl scalability java performance optimization benchmarking measurement ops coda-hale https://pinboard.in/ https://pinboard.in/u:jm/b:c184c035e80a/ don't use String.intern() in Java 2017-05-15T10:23:27+00:00 https://shipilev.net/jvm-anatomy-park/10-string-intern/ jmString.intern is the gateway to native JVM String table, and it comes with caveats: throughput, memory footprint, pause time problems will await the users. Hand-rolled deduplicators/interners to reduce memory footprint are working much more reliably, because they are working on Java side, and also can be thrown away when done. GC-assisted String deduplication does alleviate things even more. In almost every project we were taking care of, removing String.intern from the hotpaths was the very profitable performance optimization. Do not use it without thinking, okay? ]]> strings interning java performance tips https://pinboard.in/ https://pinboard.in/u:jm/b:936351988ed0/ Hadoop Internals 2017-02-17T14:39:34+00:00 http://ercoppa.github.io/HadoopInternals/ jm hadoop map-reduce architecture coding java distcomp https://pinboard.in/ https://pinboard.in/u:jm/b:5e96fe6a92a4/ Debugging Java Native Memory Leaks (evanjones.ca) 2017-01-10T16:39:09+00:00 http://www.evanjones.ca/java-native-leak-bug.html jm debugging memory jvm java leaks memory-leaks leak-checking jemalloc malloc native heap off-heap gzipinputstream https://pinboard.in/ https://pinboard.in/u:jm/b:5206aa65dc43/ tdunning/t-digest 2016-12-12T12:28:16+00:00 https://github.com/tdunning/t-digest jmA new data structure for accurate on-line accumulation of rank-based statistics such as quantiles and trimmed means. The t-digest algorithm is also very parallel friendly making it useful in map-reduce and parallel streaming applications. The t-digest construction algorithm uses a variant of 1-dimensional k-means clustering to product a data structure that is related to the Q-digest. This t-digest data structure can be used to estimate quantiles or compute other rank statistics. The advantage of the t-digest over the Q-digest is that the t-digest can handle floating point values while the Q-digest is limited to integers. With small changes, the t-digest can handle any values from any ordered set that has something akin to a mean. The accuracy of quantile estimates produced by t-digests can be orders of magnitude more accurate than those produced by Q-digests in spite of the fact that t-digests are more compact when stored on disk. Super-nice feature is that it's mergeable, so amenable to parallel usage across multiple hosts if required. Java implementation, ASL licensing.]]> data-structures algorithms java t-digest statistics quantiles percentiles aggregation digests estimation ranking https://pinboard.in/ https://pinboard.in/u:jm/b:aaf9fb613f21/ J1 2015 "Debugging Java Apps in Containers: No Heavy Welding Gear Required" 2016-09-13T10:05:50+00:00 http://www.slideshare.net/dbryant_uk/j1-2015-debugging-java-apps-in-containers-no-heavy-welding-gear-required jm java docker ops containers https://pinboard.in/ https://pinboard.in/u:jm/b:80ba8c32a215/ Lessons Learned from Using Regexes At Scale 2016-09-06T14:40:15+00:00 https://www.loggly.com/blog/lessons-learned-from-using-regexes-at-scale/ jm loggly regexp regexes java dfa nfa architecture https://pinboard.in/ https://pinboard.in/u:jm/b:ac8a514b7bc3/ Unchecked exceptions for IO considered harmful - Google Groups 2016-08-16T16:23:09+00:00 https://groups.google.com/forum/#!topic/mechanical-sympathy/Ko8osOYLq4I jmPeter Lawrey: Our view is that Checked Exception makes more sense for library writers as they can explicitly pass off errors to the caller. As a caller, especially if you are new to a product, you don't understand the exceptions or what you can do about them.  They add confusion. For this reason we use checked exceptions internally in the lower layers and try to avoid passing them in our higher level interfaces. Note: A high percentage of our fall backs are handling iOExceptons and recovering from them. [....] My experience is that the more complex and layered your libraries the more essential checked exceptions become. I see them as essential for scalability of your software. ]]> exceptions java style coding checked-exceptions ioexceptions io error-handling https://pinboard.in/ https://pinboard.in/u:jm/b:53173a73b67b/ Squeezing blood from a stone: small-memory JVM techniques for microservice sidecars 2016-06-18T09:46:44+00:00 https://blog.buoyant.io/2016/06/17/small-memory-jvm-techniques-for-microservice-sidecars/ jmWe found two specific techniques to be the most beneficial: turning off one of the two JIT compilers enabled by default (the “C2” compiler), and using a 32-bit, rather than a 64-bit, JVM. ]]> 32bit jvm java ops memory tuning jit linkerd https://pinboard.in/ https://pinboard.in/u:jm/b:ade3b2e45f61/ ztellman/dirigiste 2016-06-01T11:55:13+00:00 https://github.com/ztellman/dirigiste jm async jvm dirigiste java threadpools concurrency utilization capacity executors object-pools object-pooling latency https://pinboard.in/ https://pinboard.in/u:jm/b:5092c7f0c12e/ Koloboke Collections 2016-05-31T14:54:10+00:00 https://koloboke.com/ jm collections java koloboke performance coding https://pinboard.in/ https://pinboard.in/u:jm/b:9a6a988b787c/ Why the Very Silly Oracle v. Google Trial Actually Matters 2016-05-26T15:19:01+00:00 http://motherboard.vice.com/read/why-the-very-silly-oracle-v-google-trial-actually-matters jmIf it’s illegal to write clean room implementations of APIs, then no one has clean hands. The now-shelved open source project Apache Harmony, like Android, reimplemented Java SE, and tech giant IBM contributed code to that project. Oracle itself built its business off a proprietary implementation of SQL, which was created by IBM. The proposition “Reimplementations of APIs are infringements” creates a recursive rabbit hole of liability that spans across the industry. Even the very 37 Java APIs at issue in this trial contain reimplementations of other APIs. Google witness Joshua Bloch—who, while at Sun Microsystems, wrote many of the Java APIs—testified that specific Java APIs are reimplementations of other APIs from Perl 5 and the C programming language. ]]> apis fair-use copyright ip android java google oracle law https://pinboard.in/ https://pinboard.in/u:jm/b:dfc223222b1d/ uber-common/infer-plugin 2016-05-17T13:11:47+00:00 https://github.com/uber-common/infer-plugin jm infer java static-analysis bugs coding null https://pinboard.in/ https://pinboard.in/u:jm/b:445c127fd3da/ In Oracle v. Google, a Nerd Subculture Is on Trial 2016-05-13T14:26:51+00:00 http://motherboard.vice.com/read/in-google-v-oracle-the-nerds-are-getting-owned jm“The G part stands for GNU?” Alsup asked in disbelief. “Yes,” said Schwartz on the stand. “That doesn’t make any sense,” said the 71-year-old Clinton appointee. ]]> law gnu gpl licensing java oracle sun apis ip https://pinboard.in/ https://pinboard.in/u:jm/b:122f28325fe9/ Using jemalloc to get to the bottom of an off-heap Java memory leak 2016-04-14T11:39:20+00:00 https://gdstechnology.blog.gov.uk/2015/12/11/using-jemalloc-to-get-to-the-bottom-of-a-memory-leak/ jm debugging java jvm memory jemalloc off-heap https://pinboard.in/ https://pinboard.in/u:jm/b:e8651207606c/ JitPack 2016-04-04T09:53:25+00:00 https://jitpack.io/ jm build github java maven gradle dependencies packaging libraries https://pinboard.in/ https://pinboard.in/u:jm/b:0c48d5efec3f/ Hashed Wheel Timer 2016-03-29T12:02:43+00:00 https://github.com/ifesdjeen/hashed-wheel-timer jm scalability java timers hashed-wheel-timers algorithms data-structures https://pinboard.in/ https://pinboard.in/u:jm/b:6139ec69af2a/ Wait and lock free alternatives to LongAdder and AtomicLong by qwwdfsad 2016-03-14T17:32:46+00:00 https://github.com/JCTools/JCTools/pull/93 jm longadder doug-lea mechanical-sympathy lock-free performance atomic multithreading java jvm https://pinboard.in/ https://pinboard.in/u:jm/b:28819375e2f4/ Conversant ConcurrentQueue and Disruptor BlockingQueue 2016-03-07T11:24:20+00:00 https://github.com/conversant/disruptor jm disruptor blocking-queues queues queueing data-structures algorithms java conversant concurrency performance https://pinboard.in/ https://pinboard.in/u:jm/b:502056282b58/ Argon2 code audits - part one - Infer 2016-02-23T17:24:29+00:00 https://www.lolware.net/2016/02/12/argon2-code-review.html jm infer facebook java clang errors static-analysis lint dataflow docker https://pinboard.in/ https://pinboard.in/u:jm/b:5e2f76232714/ Netty @Apple: Large Scale Deployment/Connectivity [video] 2016-01-08T10:04:43+00:00 http://www.infoq.com/presentations/apple-netty jm apple netty norman-maurer java jvm async talks presentations https://pinboard.in/ https://pinboard.in/u:jm/b:09aa5e33f478/ The Importance of Tuning Your Thread Pools 2016-01-04T10:42:16+00:00 https://www.voxxed.com/blog/2015/12/importance-tuning-thread-pools/ jm hystrix threadpools concurrency java jvm backpressure littles-law capacity https://pinboard.in/ https://pinboard.in/u:jm/b:d8e4a0101363/ raskasa/metrics-okhttp 2015-12-31T10:27:43+00:00 https://github.com/raskasa/metrics-okhttp jm okhttp java clients http metrics dropwizard https://pinboard.in/ https://pinboard.in/u:jm/b:5576ec805c35/ Introducing Netty-HTTP from Cask 2015-12-03T12:52:22+00:00 http://blog.cask.co/2015/12/introducing-netty-http-from-cask/ jmnetty-http library solves [Netty usability issues] by using JAX-RS annotations to build a HTTP path routing layer on top of netty. In addition, the library implements a guava service to manage the HTTP service. netty-http allows users of the library to just focus on writing the business logic in HTTP handlers without having to worry about the complexities of path routing or learning netty pipeline internals to build the HTTP service. We've written something very similar, although I didn't even bother supporting JAX-RS annotations -- just a simple code-level DSL.]]> jax-rs netty http cask java services coding https://pinboard.in/ https://pinboard.in/u:jm/b:c5c91942c05c/ What Do WebLogic, WebSphere, JBoss, Jenkins, OpenNMS, and Your Application Have in Common? This Vulnerability. 2015-11-09T12:31:56+00:00 http://foxglovesecurity.com/2015/11/06/what-do-weblogic-websphere-jboss-jenkins-opennms-and-your-application-have-in-common-this-vulnerability/ jm java serialization security exploits jenkins https://pinboard.in/ https://pinboard.in/u:jm/b:c0dc116c6113/