https://pinboard.in/u:jm/public/ recent bookmarks from jm 2015-06-19T22:52:50+00:00 https://blog.cloudflare.com/how-to-receive-a-million-packets/ jm To sum up, if you want a perfect performance you need to: Ensure traffic is distributed evenly across many RX queues and SO_REUSEPORT processes. In practice, the load usually is well distributed as long as there are a large number of connections (or flows). You need to have enough spare CPU capacity to actually pick up the packets from the kernel. To make the things harder, both RX queues and receiver processes should be on a single NUMA node. ]]> linux networking performance cloudflare packets numa so_reuseport sockets udp https://pinboard.in/ https://pinboard.in/u:jm/b:7c3b21a94945/ 2015-02-13T11:10:44+00:00 http://vaurien.readthedocs.org/en/1.8/index.html jmVaurien is basically a Chaos Monkey for your TCP connections. Vaurien acts as a proxy between your application and any backend. You can use it in your functional tests or even on a real deployment through the command-line. Vaurien is a TCP proxy that simply reads data sent to it and pass it to a backend, and vice-versa. It has built-in protocols: TCP, HTTP, Redis & Memcache. The TCP protocol is the default one and just sucks data on both sides and pass it along. Having higher-level protocols is mandatory in some cases, when Vaurien needs to read a specific amount of data in the sockets, or when you need to be aware of the kind of response you’re waiting for, and so on. Vaurien also has behaviors. A behavior is a class that’s going to be invoked everytime Vaurien proxies a request. That’s how you can impact the behavior of the proxy. For instance, adding a delay or degrading the response can be implemented in a behavior. Both protocols and behaviors are plugins, allowing you to extend Vaurien by adding new ones. Last (but not least), Vaurien provides a couple of APIs you can use to change the behavior of the proxy live. That’s handy when you are doing functional tests against your server: you can for instance start to add big delays and see how your web application reacts. ]]> proxy tcp vaurien chaos-monkey testing functional-testing failures sockets redis memcache http https://pinboard.in/ https://pinboard.in/u:jm/b:c2b6d44f05b0/ 2015-01-25T23:50:05+00:00 https://github.com/amscanne/huptime jm linux ops servers uptime restarting libc bind accept sockets https://pinboard.in/ https://pinboard.in/u:jm/b:8dc351b013f1/ 2014-06-17T11:19:59+00:00 http://aws.amazon.com/articles/3604 jm aws sdk java logging ec2 s3 dynamodb sockets tuning https://pinboard.in/ https://pinboard.in/u:jm/b:6e67b33f2691/ 2013-08-24T20:33:19+00:00 http://blog.netherlabs.nl/articles/2009/01/18/the-ultimate-so_linger-page-or-why-is-my-tcp-not-reliable jm If we look at the HTTP protocol, there data is usually sent with length information included, either at the beginning of an HTTP response, or in the course of transmitting information (so called ‘chunked’ mode). And they do this for a reason. Only in this way can the receiving end be sure it received all information that it was sent. Using the shutdown() technique above really only tells us that the remote closed the connection. It does not actually guarantee that all data was received correctly by program B. The best advice is to send length information, and to have the remote program actively acknowledge that all data was received. ]]> SO_LINGER sockets tcp ip networking linux protocols shutdown FIN RST https://pinboard.in/ https://pinboard.in/u:jm/b:b6db924cd294/ 2011-10-26T12:16:10+00:00 http://www.cyberciti.biz/files/ss.html jm via:scanlan ss linux sockets networking tools cli https://pinboard.in/ https://pinboard.in/u:jm/b:de5d4f7fcd68/