Pinboard (jm)

Software Licenses and Workers' Rights · Agent IO

jm — 2026-04-07T11:26:40+00:00

Huh, this is a thought-provoking blog post about OSS licensing.

It is observably and objectively bad for society when investors own closed-source software. That starts by being bad for tech workers, creators lose the right to the value that they create, and users are still harmed because they don’t get the protection from spying and abuse that open source promised them. [...] The open source movement is a ladder that leans on the wall of users’ rights. We’ve spent forty years climbing that ladder. Where are we now? Our world is controlled by moguls who’ve built empires using open source software that they’ve locked behind proprietary barriers. Those empires exploit workers and harm the users that the open source movement was supposed to protect. Our ladder is leaning on the wrong wall.

How StrongDM’s AI team build serious software without even looking at the code

jm — 2026-02-09T10:46:10+00:00

This is really thought-provoking: StrongDM's AI team are apparently trying a new model of software engineering where there is _no_ human code review:

In kōan or mantra form: - Why am I doing this? (implied: the model should be doing this instead) In rule form: - Code must not be written by humans - Code must not be reviewed by humans Finally, in practical form: - If you haven’t spent at least $1,000 on tokens today per human engineer, your software factory has room for improvement

Frankly, I'm not there yet. There's a load of questions about how viable that level of spend is, and how much slop code is going to come out the other side. Particularly concerning when it's a security product! But I did find this bit interesting:

StrongDM’s answer was inspired by Scenario testing (Cem Kaner, 2003). As StrongDM describe it: We repurposed the word scenario to represent an end-to-end “user story”, often stored outside the codebase (similar to a “holdout” set in model training), which could be intuitively understood and flexibly validated by an LLM. [The Digital Twin Universe is] behavioral clones of the third-party services our software depends on. We built twins of Okta, Jira, Slack, Google Docs, Google Drive, and Google Sheets, replicating their APIs, edge cases, and observable behaviors. With the DTU, we can validate at volumes and rates far exceeding production limits. We can test failure modes that would be dangerous or impossible against live services. We can run thousands of scenarios per hour without hitting rate limits, triggering abuse detection, or accumulating API costs.

We actually did this in Swrve! Our end-to-end system tests for the push notifications system obviously cannot send real push notifications to real user devices in the field, so we have a "fake" push backend emulating Google, Apple, Amazon, Huawei and other push notification systems, which accurately emulate the real public APIs for those providers. So yeah -- Digital Twins for third party services is a great way to test, and being able to scale up end-to-end testing with LLM automation is a very interesting idea.

The developer productivity paradox: Why faster coding doesn’t mean faster software delivery

jm — 2025-11-24T12:40:59+00:00

The paradox is this simple gap: high individual confidence in AI speed, versus stubborn organizational metrics that just won’t budge:

- Perceived speed is high: Adoption is near-universal (90% usage reported), and confidence is overwhelming (over 80% believe AI has increased their productivity). AI is great at handling cognitive toil and boilerplate, which lets engineers generate bigger code batches and feel genuinely productive. - Systemic failure persists: The reality, confirmed by DORA in their 2025 report, is that the system often fails to carry or amplify these individual gains. The challenge is that AI models, as massive generative systems, inherently produce failures (mispredictions). As code volume increases, this constant misprediction rate impacts systemic stability. Interestingly, even leading providers of AI solutions like OpenAI and Anthropic continue to be challenged by the issue of hallucinations and mispredictions, as well as the risks generated by AI. Speaking at a university in India, Sam Altman recently said “I probably trust the answers that come out of ChatGPT the least of anybody on Earth”. Without strategies and tools for alleviating the issues AI code produces downstream — such as improved observability to understand where something is going wrong — the “much bigger engine” of AI may not actually speed up software delivery after all.

Scrappy

jm — 2025-06-18T09:44:50+00:00

"make little apps for you and your friends":

The apps we use are almost exclusively mass-market, sold on an app-store, made for thousands if not millions of users. Or they are enterprise apps that are custom-built for hundreds of thousands of dollars. But there isn’t really any equivalent of home-made software — apps made lovingly by you for your friends and family. Apps that aren’t polished or flashy, but are made to your preference and help you with your particular needs. [...] We ended up creating a research prototype that we call Scrappy — a tool for making scrappy apps for just you and your friends. First and foremost, we aim to contribute a vision of what home-made software could be like. We want to make this vision as concrete as we can, by sharing a working tool and examples of apps made in it. Scrappy, in its current state, is a prototype, not a robust tool, but we hope it paints the picture we carry in our heads — of software as something that can be creative, personal, expressive. Made by anyone, for themselves and their loved ones.

Very Hypercard-ish!

Weather Strip

jm — 2025-05-29T10:56:56+00:00

A very pretty weather forecast app, for iPhone, iPad and Mac

permacomputing

jm — 2025-05-06T09:47:34+00:00

Permacomputing is both a concept and a community of practice oriented around issues of resilience and regenerativity in computer and network technology inspired by permaculture. ପໄଓ☾☼✫ -☆:*´ There are huge environmental and societal issues in today's computing, and permacomputing specifically wants to challenge them in the same way as permaculture has challenged industrial agriculture. With that said, permacomputing is an anti-capitalist political project. It is driven by several strands of anarchism, decoloniality, intersectional feminism, post-marxism, degrowth, ecologism. Permacomputing is also a utopian ideal that needs a lot of rethinking, rebuilding and technical design work to put in practice. This is why a lot of material on this wiki is highly technical.

18F's shutdown page

jm — 2025-03-03T13:26:18+00:00

"We are dedicated to the American public and we're not done yet". legends!

For over 11 years, 18F has been proudly serving you to make government technology work better. We are non-partisan civil servants. 18F has worked on hundreds of projects, all designed to make government technology not just efficient but effective, and to save money for American taxpayers. However, all employees at 18F – a group that the Trump Administration GSA Technology Transformation Services Director called "the gold standard" of civic tech – were terminated today at midnight ET.

The Anti-Capitalist Software License

jm — 2025-02-27T17:23:16+00:00

Here it is in full:

ANTI-CAPITALIST SOFTWARE LICENSE (v 1.4)

This is anti-capitalist software, released for free use by individuals and organizations that do not operate by capitalist principles.

Permission is hereby granted, free of charge, to any person or organization (the "User") obtaining a copy of this software and associated documentation files (the "Software"), to use, copy, modify, merge, distribute, and/or sell copies of the Software, subject to the following conditions:

1. The above copyright notice and this permission notice shall be included in all copies or modified versions of the Software.

2. The User is one of the following:
a. An individual person, laboring for themselves
b. A non-profit organization
c. An educational institution
d. An organization that seeks shared profit for all of its members, and allows non-members to set the cost of their labor

3. If the User is an organization with owners, then all owners are workers and all workers are owners with equal equity and/or equal vote.

4. If the User is an organization, then the User is not law enforcement or military, or working for or under either.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT EXPRESS OR IMPLIED WARRANTY OF ANY KIND, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

This is fun because it would make esr's head explode.

Capture less than you create

jm — 2024-10-15T10:16:39+00:00

I've disagreed with David Heinemeier Hansson on plenty of occasions in the past, but this is one where I'm really happy to find myself in agreement. Matt Mullenwegg of WordPress went low, laying in digs about how DHH didn't profit from the success of Rails; DHH's response is perfect:

The moment you go down the path of gratitude grievances, you'll see ungrateful ghosts everywhere. People who owe you something, if they succeed. A ratio that's never quite right between what you've helped create and what you've managed to capture. If you let it, it'll haunt you forever. So don't! Don't let the success of others diminish your satisfaction with your own efforts. Unless you're literally Mark Zuckerberg, Elon Musk, or Jeff Bezos, there'll always be someone richer than you! The rewards I withdraw from open source flow from all the happy programmers who've been able to write Ruby to build these amazingly successful web businesses with Rails. That enjoyment only grows the more successful these business are! The more economic activity stems from Rails, the more programmers will be able to find work where they might write Ruby. Maybe I'd feel different if I was a starving open source artist holed up somewhere begrudging the wheels of capitalism. But fate has been more than kind enough to me in that regard. I want for very little, because I've been blessed sufficiently. That's a special kind of wealth: Enough. And that's also the open source spirit: To let a billion lemons go unsqueezed. To capture vanishingly less than you create. To marvel at a vast commons of software, offered with no strings attached, to any who might wish to build. Thou shall not lust after thy open source's users and their success.

Spot on.

Paying down tech debt

jm — 2024-09-12T15:05:44+00:00

by Gergely Orosz and Lou Franco:

Q: “I’d like to make a better case for paying down tech debt on my team. What are some proven approaches for this?” The tension in finding the right balance between shipping features and paying down accumulated tech debt is as old as software engineering. There’s no one answer on how best to reduce tech debt, and opinion is divided about whether zero tech debt is even a good thing to aim for. But approaches for doing it exist which work well for most teams. To tackle this eternal topic, I turned to industry veteran Lou Franco, who’s been in the software business for over 30 years as an engineer, EM, and executive. He’s also worked at four startups and the companies that later acquired them; most recently Atlassian as a Principal Engineer on the Trello iOS app.

Apparently Lou has a book on the topic imminent.

25 Years of Krita

jm — 2024-06-05T13:35:15+00:00

Amazingly, I hadn't made the connection between the "Krita" app that my kids were using to draw art on OSX, with the KDE project at the turn of the century. gg Krita team!

AWS told to pay $525M in cloud storage patent suit - The Register

jm — 2024-04-11T17:09:00+00:00

"It is notable that none of the Kove patents in the case were actually granted by the US Patent and Trademark Office before the launch of Amazon S3, the first AWS service, on March 14, 2006. However, Kove states in its complaint that applications relating to these patents were filed on July 8, 1998, perhaps implying that Amazon should have been aware of the filings before the launch of its cloud platform." Crappy software patenting strikes again.

Boeing's 737 Max Software Outsourced to $9-an-Hour Engineers

jm — 2024-03-13T12:55:17+00:00

Every line of this story is jaw-dropping. This sounds like it's going to go down in history as a colossal mistake:

It remains the mystery at the heart of Boeing Co.’s 737 Max crisis: how a company renowned for meticulous design made seemingly basic software mistakes leading to a pair of deadly crashes. Longtime Boeing engineers say the effort was complicated by a push to outsource work to lower-paid contractors. The Max softwareーplagued by issues that could keep the planes grounded months longer after U.S. regulators this week revealed a new flawーwas developed at a time Boeing was laying off experienced engineers and pressing suppliers to cut costs. Increasingly, the iconic American planemaker and its subcontractors have relied on temporary workers making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace ー notably India. [..] Rabin, the former software engineer, recalled one manager saying at an all-hands meeting that Boeing didn’t need senior engineers because its products were mature. “I was shocked that in a room full of a couple hundred mostly senior engineers we were being told that we weren’t needed,” said Rabin, who was laid off in 2015. [..] At a meeting with a chief 787 engineer in 2008, one staffer complained about sending drawings back to a team in Russia 18 times before they understood that the smoke detectors needed to be connected to the electrical system... [...] During the crashes of Lion Air and Ethiopian Airlines planes that killed 346 people, investigators suspect, the MCAS system pushed the planes into uncontrollable dives because of bad data from a single sensor. That design violated basic principles of redundancy for generations of Boeing engineers, and the company apparently never tested to see how the software would respond, Lemme said. “It was a stunning fail,” he said. “A lot of people should have thought of this problem – not one person – and asked about it.”

No More Stale Bots

jm — 2023-09-25T08:47:47+00:00

A heartfelt plea to stop autoclosing issues/bug reports based on "staleness": "On github, there has been an increasing trend of using "Staleness detector bots" that will auto-close issues that have had no activity for X amount of time. In concept, this may sound fine, but the effects this has, and how it poisons the core principles of Open Source, have been damaging and eroding projects for a long time, often unknowingly." 100% agree...

"The Fallacy of AI Functionality"

jm — 2023-06-07T09:17:54+00:00

I love this paper! I've been saying this for years:

Deployed AI systems often do not work. They can be constructed haphazardly, deployed indiscriminately, and promoted deceptively. However, despite this reality, scholars, the press, and policymakers pay too little attention to functionality. This leads to technical and policy solutions focused on “ethical” or value-aligned deployments, often skipping over the prior question of whether a given system functions, or provides any benefits at all. To describe the harms of various types of functionality failures, we analyze a set of case studies to create a taxonomy of known AI functionality issues. We then point to policy and organizational responses that are often overlooked and become more readily available once functionality is drawn into focus. We argue that functionality is a meaningful AI policy challenge, operating as a necessary first step towards protecting affected communities from algorithmic harm.

One mastodon user notes: "My favorite (sarcasm) example of this was police departments buying ML for identifying gunshots. The models were all trained for earthquakes, and the vendor basically repurposed earthquake detection as gunshot detection, made bank, and left departments with a flood of false positives."

MariaDB.com is dead, long live MariaDB.org

jm — 2023-04-07T08:50:26+00:00

Oof. Looks like the commercial company behind MariaDB is going south quickly:

Monty, the creator of MySQL and MariaDB founder, hasn’t been at a company meeting for over a year and a half. The relationship between Monty and the CEO, Michael Howard, is extremely rocky. At a company all-hands meeting Monty and Michael Howard were shouting at each other while up on stage in the auditorium in front of the entire staff. Monty made his position perfectly clear as he shouted his last words before he walked out: “You’re killing my fu&#@$! company!!!” Monty was subsequently voted off the board in July of 2022 solidifying the hostile takeover by Michael Howard. Buyer beware, Monty and his group of founders and database experts are no longer at the company.

At least the open-source product is still trustworthy, though.

Sumana Harihareswara: "Pinboard brittleness"

jm — 2022-11-29T09:35:53+00:00

Worrying thread -- I didn't realise Pinboard was at risk of atrophy. This blog is built on it!

Dan Luu on the "cold boot" scenario

jm — 2022-11-16T15:21:16+00:00

Thought-provoking Mastodon thread about full-scale disaster recovery for large-scale modern software platforms. Here's a gem:

When I was in Azure, I asked around about what the plan was if "the really big one" hit since deep expertise was nearly totally concentrated in Redmond and, at the time, Azure was guaranteed to have a global outage if a major earthquake incapacitated Redmond. Of course the plan was that there was no real plan and people expected that Azure would have a very extended global outage and an org that was on its way to becoming a $1T business unit would have its value basically wiped out.

Overview of "sustainable software" groups

jm — 2022-01-02T15:11:57+00:00

There are now a number of groups and foundations pushing a green agenda for developers. [...] The most obvious players who are specifically focussed on software and operations appear to be: The Green Software Foundation (a Linux Foundation started with Microsoft). [...] The Green Web Foundation (a not-for-profit group). Amazon, through their sustainability initiative. The EU’s Sustainable Digital Infrastructure Alliance (SDIA). OpenUK (a not-for-profit organisation).

This blog post gives a quick once-over for each one.

An oral history of Bank Python

jm — 2021-11-04T15:36:17+00:00

This is fascinating, and I particularly like this part:

Minerva is obviously heavily influenced by the technological path dependency of the financial sector, which is another way of saying: there is a lot of MS Excel. Any new software solution is going to be compared with MS Excel and if the result is unfavourable people will often just use continue to use Excel instead. Many, many technologists have taken one look at an existing workflow of spreadsheets, reacted with performative disgust, and proposed the trifecta of microservices, Kubernetes and something called a "service mesh". This kind of Big Enterprise technology however takes away that basic agency of those Excel users, who no longer understand the business process they run and now has to negotiate with ludicrous technology dweebs for each software change. The previous pliability of the spreadsheets has been completely lost. Using simple Python functions, in a source controlled system, is a better middle ground that the modern-day equivalent of J2EE. Financiers are able to learn Python, and while they may never be amazing at it they can contribute to a much higher level and even make their own changes and get them deployed.

Don't knock spreadsheets. If only they had a decent way to handle unit tests, they'd be the business. Update: sounds like this is based on J. P. Morgan, specifically: https://www.reddit.com/r/programming/comments/qmi5fm/comment/hja6hqg/?utm_source=reddit&utm_medium=web2x&context=3

A potted history of IONA Technologies' early years

jm — 2021-07-01T11:34:21+00:00

"19 March 1991: Three software developers. One desk. No chairs." -- I was there!

Tui plane in ‘serious incident’ due to software bug

jm — 2021-04-09T08:21:32+00:00

Holy cow this could have been pretty serious:

A software mistake caused a Tui flight to take off heavier than expected as female passengers using the title “Miss” were classified as children, an investigation has found. The departure from Birmingham airport to Majorca with 187 passengers on board was described as a “serious incident” by the Air Accidents Investigation Branch (AAIB). An update to the airline’s reservation system while its planes were grounded due to the coronavirus pandemic led to 38 passengers on the flight being allocated a child’s “standard weight” of 35kg as opposed to the adult figure of 69kg. This caused the load sheet – produced for the captain to calculate what inputs are needed for take-off – to state that the Boeing 737 was more than 1,200kg lighter than it actually was.

Iona Technologies' 30 year anniversary

jm — 2021-03-19T11:41:32+00:00

Iona was founded in 1991, as a campus company spin-off from Trinity College Dublin -- here's a potted history of the company, which I worked for as the first hired employee. I never knew it had at one stage been the biggest user of internet bandwidth in Ireland!

12 of the Biggest Spreadsheet Fails in History

jm — 2021-02-13T12:37:35+00:00

(via Yoz)

Amazon, Amex to Fund Software Developers in New GitHub Program - Bloomberg

jm — 2020-12-08T21:29:35+00:00

'Amazon.com Inc., American Express Co., Daimler AG and Stripe Inc. are among those joining a new GitHub program that will let companies directly fund open-source projects and software developers that are key to their businesses.' interesting

Patent case against GNOME resolved

jm — 2020-05-21T10:33:48+00:00

Great result for open source at large, too!

Today, on the 20th of May 2020, the GNOME Foundation, Rothschild Patent Imaging, and Leigh M. Rothschild are pleased to announce that the patent dispute between Rothschild Patent Imaging and GNOME has been settled. In this walk-away settlement, GNOME receives a release and covenant not to be sued for any patent held by Rothschild Patent Imaging. Further, both Rothschild Patent Imaging and Leigh Rothschild are granting a release and covenant to any software that is released under an existing Open Source Initiative approved license (and subsequent versions thereof), including for the entire Rothschild portfolio of patents, to the extent such software forms a material part of the infringement allegation.

An open letter to software engineers criticizing Neil Ferguson's epidemics simulation code

jm — 2020-05-18T13:56:40+00:00

the main message of this letter is something different: it’s about your role in this story. That’s of course a collective you, not you the individual reading this letter. It’s you, the software engineering community, that is responsible for tools like C++ that look as if they were designed for shooting yourself in the foot. It’s also you, the software engineering community, that has made no effort to warn the non-expert public of the dangers of these tools. Sure, you have been discussing these dangers internally, even a lot. But to outsiders, such as computational scientists looking for implementation tools for their models, these discussions are hard to find and hard to understand. There are lots of tutorials teaching C++ to novices, but I have yet to see a single one that starts with a clear warning about the dangers. You know, the kind of warning that every instruction manual for a microwave oven starts with: don’t use this to dry your dog after a bath. A clear message saying “Unless you are willing to train for many years to become a software engineer yourself, this tool is not for you.”

Google’s medical AI was super accurate in a lab. Real life was a different story. | MIT Technology Review

jm — 2020-04-28T15:55:43+00:00

When it worked well, the AI did speed things up. But it sometimes failed to give a result at all. Like most image recognition systems, the deep-learning model had been trained on high-quality scans; to ensure accuracy, it was designed to reject images that fell below a certain threshold of quality. With nurses scanning dozens of patients an hour and often taking the photos in poor lighting conditions, more than a fifth of the images were rejected. Patients whose images were kicked out of the system were told they would have to visit a specialist at another clinic on another day. If they found it hard to take time off work or did not have a car, this was obviously inconvenient. Nurses felt frustrated, especially when they believed the rejected scans showed no signs of disease and the follow-up appointments were unnecessary. They sometimes wasted time trying to retake or edit an image that the AI had rejected. Because the system had to upload images to the cloud for processing, poor internet connections in several clinics also caused delays. “Patients like the instant results, but the internet is slow and patients then complain,” said one nurse. “They’ve been waiting here since 6 a.m., and for the first two hours we could only screen 10 patients.” The Google Health team is now working with local medical staff to design new workflows. For example, nurses could be trained to use their own judgment in borderline cases. The model itself could also be tweaked to handle imperfect images better.

Treeware

jm — 2020-03-23T11:59:06+00:00

'a style of software distribution similar to Postcardware, distributed by the author on the condition that users buy the author a tree.'

Ironies of automation

jm — 2020-01-09T17:32:47+00:00

Wow, this is a great paper recommendation from Adrian Colyer - 'Ironies of automation', Bainbridge, Automatica, Vol. 19, No. 6, 1983.

In an automated system, two roles are left to humans: monitoring that the automated system is operating correctly, and taking over control if it isn’t. An operator that doesn’t routinely operate the system will have atrophied skills if ever called on to take over. Unfortunately, physical skills deteriorate when they are not used, particularly the refinements of gain and timing. This means that a formerly experienced operator who has been monitoring an automated process may now be an inexeperienced one. Not only are the operator’s skills declining, but the situations when the operator will be called upon are by their very nature the most demanding ones where something is deemed to be going wrong. Thus what we really need in such a situation is a more, not a lesser skilled operator! To generate successful strategies for unusual situtations, an operator also needs good understanding of the process under control, and the current state of the system. The former understanding develops most effectively through use and feedback (which the operator may no longer be getting the regular opportunity for), the latter takes some time to assimilate.

(via John Allspaw)

Algorithms interviews: theory vs. practice

jm — 2020-01-06T11:07:59+00:00

Good critique of the current practice of using algorithm questions during tech interviews from Dan Luu

At this point, we've gone through a few decades of programming interview fads, each one of which looks ridiculous in retrospect. Either we've finally found the real secret to interviewing effectively and have reasoned our way past whatever roadblocks were causing everybody in the past to use obviously bogus fad interview techniques, or we're in the middle of another fad, one which will seem equally ridiculous to people looking back a decade or two from now. Without knowing anything about the effectiveness of interviews, at a meta level, since the way people get interview techniques is the same (crib the high-level technique from the most prestigious company around), I think it would be pretty surprising if this wasn't a fad. I would be less surprised to discover that current techniques were not a fad if people were doing or referring to empirical research or had independently discovered what works.

Prof John Byrne: the man who turned Ireland into a tech world power

jm — 2019-12-20T13:03:58+00:00

TK Whitaker may be known as the man who made modern Ireland, but the highly respected civil servant wasn’t the only person who helped make the State what it is today. For those who wonder how Ireland came to excel both at luring the biggest and best tech companies to set up here and at producing a good few homegrown tech heroes, a great deal of credit must go to Prof John Byrne, the man who helped kickstart a revolution.

Treeware

jm — 2019-12-02T22:39:55+00:00

'You're free to use this package, but if it makes it to your production environment I would highly appreciate you buying the world a tree. It’s now common knowledge that one of the best tools to tackle the climate crisis and keep our temperatures from rising above 1.5C is to plant trees. If you contribute to my forest you’ll be creating employment for local families and restoring wildlife habitats.'

Using offset.earth.

An epic treatise on scheduling, bug tracking, and triage

jm — 2019-10-25T11:41:00+00:00

by apenwarr. Excellent stuff -- most of what we do in Swrve for scheduling is included here

Collapse OS

jm — 2019-10-21T13:23:23+00:00

'Bootstrap post-collapse technology' -- quite a pessimistic view on the future, but an interesting thought experiment I guess. 'An operating system designed to run on ad-hoc machines built from scavenged parts. Its development is going well and the main roadblocks are out of the way: it self-replicates on very, very low specs (for example, on a Sega Genesis which has 8K of RAM for its z80 processor).'

Margaret Hamilton interviewed by The Guardian

jm — 2019-07-19T09:50:17+00:00

good interview with the software engineering pioneer

Download Starburst Distribution of Presto

jm — 2019-06-18T11:35:35+00:00

Starburst's free distro of Presto; there are additional enterprise features which require a license key but the basic distro is OSS. Docs at https://docs.starburstdata.com/latest/index.html

research!rsc: Our Software Dependency Problem

jm — 2019-01-24T15:28:19+00:00

The kind of critical examination of specific dependencies that I outlined in this article is a significant amount of work and remains the exception rather than the rule. But I doubt there are any developers who actually make the effort to do this for every possible new dependency. I have only done a subset of them for a subset of my own dependencies. Most of the time the entirety of the decision is “let’s see what happens.” Too often, anything more than that seems like too much effort. But the Copay and Equifax attacks are clear warnings of real problems in the way we consume software dependencies today. We should not ignore the warnings. I offer three broad recommendations. * Recognize the problem. If nothing else, I hope this article has convinced you that there is a problem here worth addressing. We need many people to focus significant effort on solving it. * Establish best practices for today. We need to establish best practices for managing dependencies using what’s available today. This means working out processes that evaluate, reduce, and track risk, from the original adoption decision through to production use. In fact, just as some engineers specialize in testing, it may be that we need engineers who specialize in managing dependencies. * Develop better dependency technology for tomorrow. Dependency managers have essentially eliminated the cost of downloading and installing a dependency. Future development effort should focus on reducing the cost of the kind of evaluation and maintenance necessary to use a dependency. For example, package discovery sites might work to find more ways to allow developers to share their findings. Build tools should, at the least, make it easy to run a package’s own tests. More aggressively, build tools and package management systems could also work together to allow package authors to test new changes against all public clients of their APIs. Languages should also provide easy ways to isolate a suspect package.

Do not fall into Oracle's Java 11 trap

jm — 2018-09-26T15:29:06+00:00

The key part of the terms is as follows: You may not: use the Programs for any data processing or any commercial, production, or internal business purposes other than developing, testing, prototyping, and demonstrating your Application; The trap is as follows: Download Oracle JDK (because that is what you've always done, and it is what the web-search tells you); Use it in production (because you didn't realise the license changed); Get a nasty phone call from Oracle's license enforcement teams demanding lots of money In other words, Oracle can rely on inertia from Java developers to cause them to download the wrong (commercial) release of Java. Unless you read the text/warnings/legalese very carefully you might not even realise Oracle JDK is now commercial, and that you are therefore liable to pay Oracle for Java.

Software as Craft: software delivery and open source in a Cloud & Enterprise world

jm — 2018-09-09T21:09:03+00:00

Niall Murphy sends this on:

Microsoft is very pleased to welcome Maggie Pint and Dr. Nicole Forsgren to our new campus, to talk about open source and the deep connections between how software is written, and how successful it is. For those of you who are not aware, Maggie Pint is a software engineering lead in Azure’s Production Infrastructure Engineering (PIE) organization. Maggie’s team works on improving the engineering systems experience for Microsoft’s web-focused developers. She co-ordinates open source and inner source education and execution through Azure PIE. Outside of her day job, Maggie maintains the popular Moment.js JavaScript library, and is the JS Foundation’s delegate to TC39, the standards committee for JavaScript. She is passionate about dogs, coffee, the JavaScript language, and helping others live open source values in their day-to-day work. Dr. Nicole Forsgren is the co-founder and Chief Scientist of the DevOps Research and Assessment joint venture with Jez Humble and Gene Kim, also well-known leaders in the DevOps community. She is best known as a co-author of Accelerate: The Science of Lean Software and DevOps and lead investigator for the largest-scale DevOps studies undertaken to date. She is also member of the ACM Queue editorial board, a research affiliate for a number of universities, and earned her PhD in Management Information Systems from the University of Arizona. This event comprises two public technical talks, with an intended audience of a few hundred software and systems professionals, including technical managers and SREs.

Basho investor to pay up $20m in damages for campaign that put biz on 'greased slide to failure' • The Register

jm — 2018-07-10T08:36:22+00:00

This is disappointing. Basho was very promising.

An investment fund and its manager have been ordered to pay up $20.3m after "misinformation, threats and combative behaviour" helped put NoSQL database biz Basho on a "greased slide to failure". As reported by The Register, the once-promising biz, which developed the Riak distributed database, faded away last year amid severe criticisms of the way its major investor, Georgetown Capital Partners, operated. These centred around the control the investment firm and boss Chester Davenport gained over Basho, and how that power was used to block other funders and push out dissenting voices, with the hope of selling the company off fast.

A Taxonomy of Tech Debt | Riot Games Engineering

jm — 2018-04-12T08:38:39+00:00

Very sensible way to group/classify tech debt work -- we did something similar in Swrve internally at one point, but this is much more evolved

'A Look into 30 Years of Malware Development from a Software Metrics Perspective'

jm — 2018-01-27T23:19:03+00:00

'During the last decades, the problem of malicious and unwanted software (malware) has surged in numbers and sophistication. Malware plays a key role in most of today’s cyber attacks and has consolidated as a commodity in the underground economy. In this work, we analyze the evolution of malware since the early 1980s to date from a software engineering perspective. We analyze the source code of 151 malware samples and obtain measures of their size, code quality, and estimates of the development costs (effort, time, and number of people). Our results suggest an exponential increment of nearly one order of magnitude per decade in aspects such as size and estimated effort, with code quality metrics similar to those of regular software. Overall, this supports otherwise confirmed claims about the increasing complexity of malware and its production progressively becoming an industry.'

Steven Bellovin on Bitcoin

jm — 2018-01-01T23:45:31+00:00

When you engineer a system for deployment you build it to meet certain real-world goals. You may find that there are tradeoffs, and that you can't achieve all of your goals, but that's normal; as I've remarked, "engineering is the art of picking the right trade-off in an overconstrained environment". For any computer-based financial system, one crucial parameter is the transaction rate. For a system like Bitcoin, another goal had to be avoiding concentrations of power. And of course, there's transaction privacy. There are less obvious factors, too. These days, "mining" for Bitcoins requires a lot of computations, which translates directly into electrical power consumption. One estimate is that the Bitcoin network uses up more electricity than many countries. There's also the question of governance: who makes decisions about how the network should operate? It's not a question that naturally occurs to most scientists and engineers, but production systems need some path for change. In all of these, Bitcoin has failed. The failures weren't inevitable; there are solutions to these problems in the acdemic literature. But Bitcoin was deployed by enthusiasts who in essence let experimental code escape from a lab to the world, without thinking about the engineering issues—and now they're stuck with it. Perhaps another, better cryptocurrency can displace it, but it's always much harder to displace something that exists than to fill a vacuum.

The Impenetrable Program Transforming How Courts Treat DNA Evidence | WIRED

jm — 2017-11-30T11:45:46+00:00

'So the lab turned to TrueAllele, a program sold by Cybergenetics, a small company dedicated to helping law enforcement analyze DNA where regular lab tests fail. They do it with something called probabilistic genotyping, which uses complex mathematical formulas to examine the statistical likelihood that a certain genotype comes from one individual over another. It’s a type of DNA testing that’s becoming increasingly popular in courtrooms. ' [...] 'But now legal experts, along with Johnson’s advocates, are joining forces to argue to a California court that TrueAllele—the seemingly magic software that helped law enforcement analyze the evidence that tied Johnson to the crimes—should be forced to reveal the code that sent Johnson to prison. This code, they say, is necessary in order to properly evaluate the technology. In fact, they say, justice from an unknown algorithm is no justice at all.'

An empirical study on the correctness of formally verified distributed systems

jm — 2017-05-29T10:06:17+00:00

We must recognise that even formal verification can leave gaps and hidden assumptions that need to be teased out and tested, using the full battery of testing techniques at our disposal. Building distributed systems is hard. But knowing that shouldn’t make us shy away from trying to do the right thing, instead it should make us redouble our efforts in our quest for correctness.

Moom removed from sale due to patent violation claim | Hacker News

jm — 2017-05-15T10:28:43+00:00

Well this sucks. Some scumbag applied for a patent on tiling window management in 2008, and it's been granted. I use Moom every day :(

The World Is Getting Hacked. Why Don’t We Do More to Stop It? - The New York Times

jm — 2017-05-15T09:09:01+00:00

Zeynep Tufekci is (as usual!) on the money with this op-ed. I strongly agree with the following:

First, companies like Microsoft should discard the idea that they can abandon people using older software. The money they made from these customers hasn’t expired; neither has their responsibility to fix defects. Besides, Microsoft is sitting on a cash hoard estimated at more than $100 billion (the result of how little tax modern corporations pay and how profitable it is to sell a dominant operating system under monopolistic dynamics with no liability for defects). At a minimum, Microsoft clearly should have provided the critical update in March to all its users, not just those paying extra. Indeed, “pay extra money to us or we will withhold critical security updates” can be seen as its own form of ransomware. In its defense, Microsoft probably could point out that its operating systems have come a long way in security since Windows XP, and it has spent a lot of money updating old software, even above industry norms. However, industry norms are lousy to horrible, and it is reasonable to expect a company with a dominant market position, that made so much money selling software that runs critical infrastructure, to do more. Microsoft should spend more of that $100 billion to help institutions and users upgrade to newer software, especially those who run essential services on it. This has to be through a system that incentivizes institutions and people to upgrade to more secure systems and does not force choosing between privacy and security. Security updates should only update security, and everything else should be optional and unbundled.

More on this twitter thread: https://twitter.com/zeynep/status/863734133188681732

The power of role models

jm — 2017-02-28T15:40:29+00:00

At dinner I asked some of the women to speak to me about this, how astronomy became so (relatively) egalitarian. And one topic became clear: role models. Astronomy has a long history of women active in the field, going all the way back to Caroline Herschel in the early 19th century. Women have made huge contributions to the field. Dava Sobel just wrote a book about the women who laid the foundations for the discovery of the expansion of the universe. Just a couple of weeks ago, papers ran obituaries of Vera Rubin, the remarkable observational astronomer who discovered the evidence for dark matter. I could mention Jocelyn Bell, whose discovery of pulsars got her advisor a Nobel (sic). The most famous astronomer I met growing up was Helen Hogg, the (adopted) Canadian astronomer at David Dunlap Observatory outside Toronto, who also did a fair bit of what we now call outreach. The women at the meeting spoke of this, a history of women contributing, of role models to look up to, of proof that women can make major contributions to the field. What can computing learn from this? It seems we're doing it wrong. The best way to improve the representation of women in the field is not to recruit them, important though that is, but to promote them. To create role models. To push them into positions of influence.

'Software Engineering at Google'

jm — 2017-02-14T10:51:45+00:00

20 pages of Google's software dev practices, with emphasis on the build system (since it was written by the guy behind Blaze). Naturally, some don't make a whole lot of sense outside of Google, but still some good stuff here

Remarks at the SASE Panel On The Moral Economy of Tech

jm — 2016-10-04T15:35:35+00:00

Excellent talk. I love this analogy for ML applied to real-world data which affects people:

Treating the world as software promotes fantasies of control. And the best kind of control is control without responsibility. Our unique position as authors of software used by millions gives us power, but we don't accept that this should make us accountable. We're programmers—who else is going to write the software that runs the world? To put it plainly, we are surprised that people seem to get mad at us for trying to help. Fortunately we are smart people and have found a way out of this predicament. Instead of relying on algorithms, which we can be accused of manipulating for our benefit, we have turned to machine learning, an ingenious way of disclaiming responsibility for anything. Machine learning is like money laundering for bias. It's a clean, mathematical apparatus that gives the status quo the aura of logical inevitability. The numbers don't lie.

Particularly apposite today given Y Combinator's revelation that they use an AI bot to help 'sift admission applications', and don't know what criteria it's using: https://twitter.com/aprjoy/status/783032128653107200

“Racist algorithms” and learned helplessness

jm — 2016-04-07T15:39:02+00:00

Whenever I’ve had to talk about bias in algorithms, I’ve tried be careful to emphasize that it’s not that we shouldn’t use algorithms in search, recommendation and decision making. It’s that we often just don’t know how they’re making their decisions to present answers, make recommendations or arrive at conclusions, and it’s this lack of transparency that’s worrisome. Remember, algorithms aren’t just code. What’s also worrisome is the amplifier effect. Even if “all an algorithm is doing” is reflecting and transmitting biases inherent in society, it’s also amplifying and perpetuating them on a much larger scale than your friendly neighborhood racist. And that’s the bigger issue. [...] even if the algorithm isn’t creating bias, it’s creating a feedback loop that has powerful perception effects.

CharybdeFS: a new fault-injecting filesystem for software testing

jm — 2016-02-17T14:40:52+00:00

a FUSE-based filesystem from ScyllaDB to test filesystem-related failure scenarios. great idea

How open-source software developers helped end the Ebola epidemic in Sierra Leone

jm — 2016-01-04T12:38:40+00:00

Little known to the rest of the world, a team of open source software developers played a small but integral part in helping to stop the spread of Ebola in Sierra Leone, solving a payroll crisis that was hindering the fight against the disease. Emerson Tan from NetHope, a consortium of NGOs working in IT and development, told the tale at the Chaos Communications Congress in Hamburg, Germany. “These guys basically saved their country from complete collapse. I can’t overestimate how many lives they saved,” he said about his co-presenters, Salton Arthur Massally, Harold Valentine Mac-Saidu and Francis Banguara, who appeared over video link.

"Hidden Technical Debt in Machine-Learning Systems" [pdf]

jm — 2015-12-07T12:07:40+00:00

Another great paper about from Google, talking about the tradeoffs that must be considered in practice over the long term with running a complex ML system in production.

Structural and semantic deficiencies in the systemd architecture for real-world service management, a technical treatise

jm — 2015-11-02T11:38:40+00:00

Despite its overarching abstractions, it is semantically non-uniform and its complicated transaction and job scheduling heuristics ordered around a dependently networked object system create pathological failure cases with little debugging context that would otherwise not necessarily occur on systems with less layers of indirection. The use of bus APIs complicate communication with the service manager and lead to duplication of the object model for little gain. Further, the unit file options often carry implicit state or are not sufficiently expressive. There is an imbalance with regards to features of an eager service manager and that of a lazy loading service manager, having rusty edge cases of both with non-generic, manager-specific facilities. The approach to logging and the circularly dependent architecture seem to imply that lots of prior art has been ignored or understudied.

Twins denied driver’s permit because DMV can’t tell them apart

jm — 2015-10-28T17:05:27+00:00

"The computer can recognize faces, a feature that comes in handy if somebody’s is trying to get an illegal ID. It apparently is not programmed to detect twins." As Hilary Mason put it: "You do not want to be an edge case in this future we are building."

Taming Complexity with Reversibility

jm — 2015-07-28T19:28:48+00:00

This is a great post from Kent Beck, putting a lot of recent deployment/rollout patterns in a clear context -- that of supporting "reversibility":

Development servers. Each engineer has their own copy of the entire site. Engineers can make a change, see the consequences, and reverse the change in seconds without affecting anyone else. Code review. Engineers can propose a change, get feedback, and improve or abandon it in minutes or hours, all before affecting any people using Facebook. Internal usage. Engineers can make a change, get feedback from thousands of employees using the change, and roll it back in an hour. Staged rollout. We can begin deploying a change to a billion people and, if the metrics tank, take it back before problems affect most people using Facebook. Dynamic configuration. If an engineer has planned for it in the code, we can turn off an offending feature in production in seconds. Alternatively, we can dial features up and down in tiny increments (i.e. only 0.1% of people see the feature) to discover and avoid non-linear effects. Correlation. Our correlation tools let us easily see the unexpected consequences of features so we know to turn them off even when those consequences aren't obvious. IRC. We can roll out features potentially affecting our ability to communicate internally via Facebook because we have uncorrelated communication channels like IRC and phones. Right hand side units. We can add a little bit of functionality to the website and turn it on and off in seconds, all without interfering with people's primary interaction with NewsFeed. Shadow production. We can experiment with new services under real load, from a tiny trickle to the whole flood, without affecting production. Frequent pushes. Reversing some changes require a code change. On the website we never more than eight hours from the next schedule code push (minutes if a fix is urgent and you are willing to compensate Release Engineering). The time frame for code reversibility on the mobile applications is longer, but the downward trend is clear from six weeks to four to (currently) two. Data-informed decisions. (Thanks to Dave Cleal) Data-informed decisions are inherently reversible (with the exceptions noted below). "We expect this feature to affect this metric. If it doesn't, it's gone." Advance countries. We can roll a feature out to a whole country, generate accurate feedback, and roll it back without affecting most of the people using Facebook. Soft launches. When we roll out a feature or application with a minimum of fanfare it can be pulled back with a minimum of public attention. Double write/bulk migrate/double read. Even as fundamental a decision as storage format is reversible if we follow this format: start writing all new data to the new data store, migrate all the old data, then start reading from the new data store in parallel with the old.

We do a bunch of these in work, and the rest are on the to-do list. +1 to these!

Evidence-Based Software Engineering

jm — 2015-06-17T10:58:52+00:00

Objective: Our objective is to describe how software engineering might benefit from an evidence-based approach and to identify the potential difficulties associated with the approach. Method: We compared the organisation and technical infrastructure supporting evidence-based medicine (EBM) with the situation in software engineering. We considered the impact that factors peculiar to software engineering (i.e. the skill factor and the lifecycle factor) would have on our ability to practice evidence-based software engineering (EBSE). Results: EBSE promises a number of benefits by encouraging integration of research results with a view to supporting the needs of many different stakeholder groups. However, we do not currently have the infrastructure needed for widespread adoption of EBSE. The skill factor means software engineering experiments are vulnerable to subject and experimenter bias. The lifecycle factor means it is difficult to determine how technologies will behave once deployed. Conclusions: Software engineering would benefit from adopting what it can of the evidence approach provided that it deals with the specific problems that arise from the nature of software engineering.

(via Mark Dennehy)

'Microservice AntiPatterns'

jm — 2015-04-28T13:22:41+00:00

presentation from last week's Craft Conference in Budapest; Tammer Saleh of Pivotal with a few antipatterns observed in dealing with microservices.

StackShare

jm — 2015-04-26T20:52:18+00:00

'Discover and discuss the best dev tools and cloud infrastructure services' -- fun!

Stack Overflow Developer Survey 2015

jm — 2015-04-07T16:02:05+00:00

wow, 52.5% of developers prefer a dark IDE theme?!

The Pizza Party Where Everyone Got Fired

jm — 2015-02-12T23:49:22+00:00

The testers at [MAJOR PUBLISHER] had just finished wrapping up testing on a project we'll call "Biolands." And to congratulate them, the man in charge arranged a huge bowling/pizza party for the end of the week. Of course everyone is hyped for the event. So the day finally arrives and all the testers show up. They all start bowling and eating pizza. After a few hours of everyone enjoying themselves, the VP asks for everyone's attention. When he does manage to get the team to listen, he begins to thank them for their hard work and has the leads hand them their termination papers.

And many other horror stories from the worst software industry of all -- games.

A Quiet Defense of Patterns

jm — 2015-02-02T00:32:11+00:00

Marc Brooker: 'When it comes to building working software in the long term, the emotional pursuit of craft is not as important as the human pursuit of teamwork, or the intellectual pursuit of correctness. Patterns is one of the most powerful ideas we have. The critics may be right that it devalues the craft, but we would all do well to remember that the craft of software is a means, not an end.'

Tech’s Meritocracy Problem — Medium

jm — 2014-10-13T13:00:25+00:00

Meritocracy is a myth. And our belief in it is holding back the tech industry from getting better.

To "patch" software comes from a physical patch applied to paper tape

jm — 2014-10-11T22:28:31+00:00

hmason: TIL that the phrase software "patch" is from a physical patch applied to Mark 1 paper tape to modify the program.

It's amazing how a term like that can become so divorced from its original meaning so effectively. History!

on using JSON as a config file format

jm — 2014-09-17T09:15:16+00:00

Ben Hughes on twitter: "JSON is fine for config files, if you don't want to comment your config file. Which is a way of saying, it isn't fine for config files."

Software patents are crumbling, thanks to the Supreme Court

jm — 2014-09-15T15:48:41+00:00

Now a series of decisions from lower courts is starting to bring the ruling's practical consequences into focus. And the results have been ugly for fans of software patents. By my count there have been 11 court rulings on the patentability of software since the Supreme Court's decision — including six that were decided this month. Every single one of them has led to the patent being invalidated. This doesn't necessarily mean that all software patents are in danger — these are mostly patents that are particularly vulnerable to challenge under the new Alice precedent. But it does mean that the pendulum of patent law is now clearly swinging in an anti-patent direction. Every time a patent gets invalidated, it strengthens the bargaining position of every defendant facing a lawsuit from a patent troll.

Open Invention Network Symposium on Open Source Software and Patents in Context

jm — 2014-09-11T12:18:14+00:00

Dublin, 24th September 2014, hosted by Enterprise Ireland. Hosted by former Ubuntu counsel (via gcarr)

The Ramifications of Alice: A Conversation with Mark Lemley - IPWatchdog.com

jm — 2014-09-05T08:41:07+00:00

I think you need to review what is actually happening at the USPTO in terms of rejections and how the Federal Circuit is applying Alice to find software patent claims patent ineligible. We are not crying wolf. It is really, factually, truthfully happening.

On the face of it, this sounds like great news ;)