https://pinboard.in/u:jm/public/ recent bookmarks from jm 2024-11-08T13:21:19+00:00 https://transactional.blog/how-to-learn/disk-io jm databases filesystems linux macos fsync durability coding https://pinboard.in/ https://pinboard.in/u:jm/b:95771a126b07/ 2023-08-09T15:58:39+00:00 https://aws.amazon.com/s3/features/mountpoint/ jm s3 clients oss linux aws filesystems unix https://pinboard.in/ https://pinboard.in/u:jm/b:2f96c68c04ef/ 2020-09-23T10:43:42+00:00 https://github.com/Feh/nocache jm The nocache tool tries to minimize the effect an application has on the Linux file system cache. This is done by intercepting the open and close system calls and calling posix_fadvise() with the POSIX_FADV_DONTNEED parameter. Because the library remembers which pages (ie., 4K-blocks of the file) were already in file system cache when the file was opened, these will not be marked as "don't need", because other applications might need that, although they are not actively used (think: hot standby). ]]> cache linux memory performance filesystems backup k8s unix fadvise https://pinboard.in/ https://pinboard.in/u:jm/b:bd12d7ea46ac/ 2020-09-21T10:38:32+00:00 https://www.jeffgeerling.com/blog/2020/raspberry-pi-usb-boot-uasp-trim-and-performance jmIn the past few weeks, I reviewed USB drive performance on the Raspberry Pi 4, and the importance of UASP support for USB drive performance. Both posts generated great discussion, and there were three things I wanted to cover in this follow-up, namely: Which drives support UASP; Real-world performance benchmarks; TRIM support. ]]> raspberry-pi optimization linux trim ssd filesystems usb https://pinboard.in/ https://pinboard.in/u:jm/b:36695789d680/ 2020-04-15T12:03:35+00:00 https://github.com/8051Enthusiast/regex2fat jm insane stupid funny fat32 drivers filesystems regexps regex https://pinboard.in/ https://pinboard.in/u:jm/b:44aedb42f788/ 2020-02-17T10:49:05+00:00 https://aws.amazon.com/about-aws/whats-new/2020/02/ebs-multi-attach-available-provisioned-iops-ssd-volumes/ jm ebs ec2 filesystems networking ops https://pinboard.in/ https://pinboard.in/u:jm/b:2f99d09d89c4/ 2018-04-06T11:33:20+00:00 https://hackernoon.com/another-reason-why-your-docker-containers-may-be-slow-d37207dec27f jm linux fadvise filesystems performance docker containers ops https://pinboard.in/ https://pinboard.in/u:jm/b:9015116d6c92/ 2017-02-14T10:42:03+00:00 https://medium.com/making-instapaper/instapaper-outage-cause-recovery-3c32a7e9cc5f#.39bn4xjas jmWithout knowledge of the pre-April 2014 file size limit, it was difficult to foresee and prevent this issue. As far as we can tell, there’s no information in the RDS console in the form of monitoring, alerts or logging that would have let us know we were approaching the 2TB file size limit, or that we were subject to it in the first place. Even now, there’s nothing to indicate that our hosted database has a critical issue. ]]> limits aws rds databases mysql filesystems ops instapaper risks https://pinboard.in/ https://pinboard.in/u:jm/b:70ad5bf26582/ 2016-06-07T10:42:01+00:00 http://pl.atyp.us/2016-06-torus.html jm jeff-darcy distcomp filesystems coreos torus storage https://pinboard.in/ https://pinboard.in/u:jm/b:9e388f6ae979/ 2016-02-17T14:40:52+00:00 http://www.scylladb.com/2016/02/16/fault-injection-filesystem-software-testing/ jm fuse software testing scylladb filesystems disk charybdefs fault-injection tests https://pinboard.in/ https://pinboard.in/u:jm/b:f13d512cee10/ 2015-12-14T11:15:12+00:00 http://danluu.com/file-consistency/ jm filesystems linux unix files operating-systems posix fsync osdi papers reliability https://pinboard.in/ https://pinboard.in/u:jm/b:a449535f1673/ 2015-12-09T14:58:39+00:00 http://epickrram.blogspot.co.uk/2015/12/journalling-revisited.html jm linux xfs ext3 ext4 filesystems lmax performance latency journalling ops https://pinboard.in/ https://pinboard.in/u:jm/b:9b225ebb15c4/ 2015-09-01T14:34:18+00:00 https://bitbucket.org/nikratio/s3ql jma file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X. S3QL is a standard conforming, full featured UNIX file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees and snapshotting which make it especially suitable for online backup and archival. S3QL is designed to favor simplicity and elegance over performance and feature-creep. Care has been taken to make the source code as readable and serviceable as possible. Solid error detection and error handling have been included from the very first line, and S3QL comes with extensive automated test cases for all its components. ]]> filesystems aws s3 storage unix google-storage openstack https://pinboard.in/ https://pinboard.in/u:jm/b:7f4f3290c9ff/ 2015-07-30T21:54:08+00:00 https://github.com/danilop/yas3fs jmYAS3FS (Yet Another S3-backed File System) is a Filesystem in Userspace (FUSE) interface to Amazon S3. It was inspired by s3fs but rewritten from scratch to implement a distributed cache synchronized by Amazon SNS notifications. A web console is provided to easily monitor the nodes of a cluster. ]]> aws s3 s3fs yas3fs filesystems fuse sns https://pinboard.in/ https://pinboard.in/u:jm/b:fee4884e5c7f/ 2015-07-20T17:08:55+00:00 http://jpetazzo.github.io/assets/2015-07-01-deep-dive-into-docker-storage-drivers.html#43 jm docker overlayfs aufs btrfs filesystems ops linux containers https://pinboard.in/ https://pinboard.in/u:jm/b:8e7e75a8ed4e/ 2015-07-15T15:44:26+00:00 http://blog.cloud66.com/docker-with-overlayfs-first-impression/ jm overlayfs docker filesystems ops linux https://pinboard.in/ https://pinboard.in/u:jm/b:abef57e1c7ee/ 2014-11-14T11:17:18+00:00 http://en.wikipedia.org/wiki//dev/full jm dev /dev/full filesystems devices linux testing enospc error-handling https://pinboard.in/ https://pinboard.in/u:jm/b:169c08da2b2d/ 2014-09-15T09:05:14+00:00 https://clusterhq.com/blog/state-zfs-on-linux/ jmLinux users familiar with other filesystems or ZFS users from other platforms will often ask whether ZFS on Linux (ZoL) is “stable”. The short answer is yes, depending on your definition of stable. The term stable itself is somewhat ambiguous. Oh dear. that's not a good start. Good reference page, though]]> zfs linux filesystems ops solaris https://pinboard.in/ https://pinboard.in/u:jm/b:15912aa744a6/ 2014-09-01T22:26:39+00:00 http://0pointer.net/blog/revisiting-how-we-put-together-linux-systems.html jmInstantiating a new system or OS container (which is exactly the same in this scheme) just consists of creating a new appropriately named root sub-volume. Completely naturally you can share one vendor OS copy in one specific version with a multitude of container instances. Everything is double-buffered (or actually, n-fold-buffered), because usr, runtime, framework, app sub-volumes can exist in multiple versions. Of course, by default the execution logic should always pick the newest release of each sub-volume, but it is up to the user keep multiple versions around, and possibly execute older versions, if he desires to do so. In fact, like on ChromeOS this could even be handled automatically: if a system fails to boot with a newer snapshot, the boot loader can automatically revert back to an older version of the OS. (via Tony Finch)]]> via:fanf linux docker btrfs filesystems unionfs copy-on-write os hacking unix https://pinboard.in/ https://pinboard.in/u:jm/b:b9563993a8c5/ 2014-03-18T21:29:53+00:00 https://bitbucket.org/nikratio/s3ql/overview jma file system that stores all its data online using storage services like Google Storage, Amazon S3, or OpenStack. S3QL effectively provides a hard disk of dynamic, infinite capacity that can be accessed from any computer with internet access running Linux, FreeBSD or OS-X. S3QL is a standard conforming, full featured UNIX file system that is conceptually indistinguishable from any local file system. Furthermore, S3QL has additional features like compression, encryption, data de-duplication, immutable trees and snapshotting which make it especially suitable for online backup and archival. ]]> s3 s3ql backup aws filesystems linux freebsd osx ops https://pinboard.in/ https://pinboard.in/u:jm/b:ebbb82ef8807/ 2011-12-04T22:51:37+00:00 http://hoytech.com/vmtouch/ jm vmtouch vm linux unix fs filesystems instagram https://pinboard.in/ https://pinboard.in/u:jm/b:9b2b286e0d35/ 2010-03-15T23:30:21+00:00 http://glinden.blogspot.com/2010/03/gfs-and-its-evolution.html jm google gfs filesystems networking https://pinboard.in/u:jm/b:38ba2cbced7c/ 2009-08-01T10:31:58+00:00 http://lwn.net/Articles/342892/ jm btrfs history zfs linux open-source licensing storage sysadmin b-trees b+trees algorithms fs filesystems https://pinboard.in/u:jm/b:eabded4de881/