Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmQuerying OpenStreetMap Buildings with AWS Athena – door2door Engineering2018-10-12T14:27:39+00:00
https://engineering.door2door.io/querying-openstreetmap-buildings-with-aws-athena-ae50bf1bc5f0
jmAWS not only made OpenStreetMap planet data available on S3, but it also made it query-able with Athena. Pretty cool, no? Now, in theory, you can just construct an SQL query, send it to Athena, and then do whatever you want with the results. No more:
Updating OSM planet data yourself; it gets updated on AWS whenever OSM publishes it, once a week.
Transforming the data into a query-able format; Athena handles that for you.
Query/request frequency limits (it’s still AWS though, so other limitations might apply 💸 💸)
At door2door, we had a pretty straight-forward use-case for this: we needed to get buildings in specific regions based only on where they were, and transform those buildings into GeoJSON that we can attach our data to, and visualize on the front-end on top of our base map.
]]>athena osm buildings aws geodata mapping maps door2door cool hackshttps://pinboard.in/https://pinboard.in/u:jm/b:1507124525ff/Mythology about security…2018-04-09T22:59:59+00:00
https://gettys.wordpress.com/2018/04/09/mythology-about-security/
jmGovernment export controls crippled Internet security and the design of Internet protocols from the very beginning: we continue to pay the price to this day. Getting security right is really, really hard, and current efforts towards “back doors”, or other access is misguided. We haven’t even recovered from the previous rounds of government regulations, which has caused excessive complexity in an already difficult problem and many serious security problems. Let us not repeat this mistake…
I remember the complexity of navigating crypto export controls. As noted here, it was generally easier just not to incorporate security features.]]>security crypto export-control jim-gettys x11 history x-windows mit athena kerberoshttps://pinboard.in/https://pinboard.in/u:jm/b:62fb4f0e8ae2/S3 Inventory Adds Apache ORC output format and Amazon Athena Integration2017-11-20T10:22:59+00:00
https://aws.amazon.com/about-aws/whats-new/2017/11/s3-inventory-adds-apache-orc-output-format-and-amazon-athena-integration/
jmorc formats data interchange s3 athena outputhttps://pinboard.in/https://pinboard.in/u:jm/b:3ffe967cccc0/