Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmHow Uber Engineering Evaluated JSON Encoding and Compression Algorithms to Put the Squeeze on Trip Data2019-12-16T11:32:03+00:00
https://eng.uber.com/trip-data-squeeze/
jmKey conclusions:
Simply compressing JSON with zlib would yield a reasonable tradeoff in size and speed. The result would be just a little bigger, but execution was much faster than using BZ2 on JSON.
Going with IDL-based protocols, Thrift and Protocol Buffers compressed with zlib or Snappy would give us the best gain in size and/or speed.
]]>compression json performance python serialization protobuf zlib snappy cbor messagepack thrift bz2https://pinboard.in/https://pinboard.in/u:jm/b:41b3c6225764/