Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmEvolution of Babbel’s data pipeline on AWS: from SQS to Kinesis2015-09-08T12:56:09+00:00
http://bytes.babbel.com/en/articles/2015-09-01-aws-data-pipeline.html
jm
Our new data pipeline with Kinesis in place allows us to plug new consumers without causing any damage to the current system, so it’s possible to rewrite all Queue Workers one by one and replace them with Kinesis Workers. In general, the transition to Kinesis was smooth and there were not so tricky parts.
Another outcome was significantly reduced costs – handling almost the same amount of data as SQS, Kinesis appeared to be many times cheaper than SQS.
]]>aws kinesis kafka streaming data-pipelines streams sqs queues architecture kclhttps://pinboard.in/https://pinboard.in/u:jm/b:7e9df1f932f5/