Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmHow Discord Supercharges Network Disks for Extreme Low Latency2022-08-16T10:07:47+00:00
https://discord.com/blog/how-discord-supercharges-network-disks-for-extreme-low-latency
jmdiscord google disks performance optimization scylladb md raid opshttps://pinboard.in/https://pinboard.in/u:jm/b:f663d3221476/Scylla compression benchmarks2019-10-08T12:06:43+00:00
https://www.scylladb.com/2019/10/07/compression-in-scylla-part-two/
jmUse compression. Unless you are using a really (but REALLY) fast hard drive, using the default compression settings will be even faster than disabling compression, and the space savings are huge.
When running a data warehouse where data is mostly being read and only rarely updated, consider using DEFLATE. It provides very good compression ratios while maintaining high decompression speeds; compression can be slower, but that might be unimportant for your workload.
If your workload is write-heavy but you really care about saving disk space, consider using ZStandard on level 1. It provides a good middle-ground between LZ4/Snappy and DEFLATE in terms of compression ratios and keeps compression speeds close to LZ4 and Snappy. Be careful however: if you often want to read cold data (from the SSTables on disk, not currently stored in memory, so for example data that was inserted a long time ago), the slower decompression might become a problem.
]]>compression scylladb storage deflate zstd zstandard lz4 snappy gzip benchmarks tests performancehttps://pinboard.in/https://pinboard.in/u:jm/b:af5d6a010327/Isolating workloads with Systemd slices2019-09-26T10:41:48+00:00
https://www.scylladb.com/2019/09/25/isolating-workloads-with-systemd-slices/
jmsystemd cgroups process-isolation linux containerisation scylladb opshttps://pinboard.in/https://pinboard.in/u:jm/b:e0ea78ae0b3f/Taming the Beast: How Scylla Leverages Control Theory to Keep Compactions Under Control - ScyllaDB2018-06-14T10:29:17+00:00
https://www.scylladb.com/2018/06/12/scylla-leverages-control-theory/
jm
At any given moment, a database like ScyllaDB has to juggle the admission of foreground requests with background processes like compactions, making sure that the incoming workload is not severely disrupted by compactions, nor that the compaction backlog is so big that reads are later penalized.
In this article, we showed that isolation among incoming writes and compactions can be achieved by the Schedulers, yet the database is still left with the task of determining the amount of shares of the resources incoming writes and compactions will use.
Scylla steers away from user-defined tunables in this task, as they shift the burden of operation to the user, complicating operations and being fragile against changing workloads. By borrowing from the strong theoretical background of industrial controllers, we can provide an Autonomous Database that adapts to changing workloads without operator intervention.
]]>scylladb storage settings compaction automation thresholds control-theory ops cassandra feedbackhttps://pinboard.in/https://pinboard.in/u:jm/b:0924f70f896e/CharybdeFS: a new fault-injecting filesystem for software testing2016-02-17T14:40:52+00:00
http://www.scylladb.com/2016/02/16/fault-injection-filesystem-software-testing/
jmfuse software testing scylladb filesystems disk charybdefs fault-injection testshttps://pinboard.in/https://pinboard.in/u:jm/b:f13d512cee10/