Pinboard (jm)
https://pinboard.in/u:jm/public/
recent bookmarks from jmAn oral history of Bank Python2021-11-04T15:36:17+00:00
https://calpaterson.com/bank-python.html
jmMinerva is obviously heavily influenced by the technological path dependency of the financial sector, which is another way of saying: there is a lot of MS Excel. Any new software solution is going to be compared with MS Excel and if the result is unfavourable people will often just use continue to use Excel instead. Many, many technologists have taken one look at an existing workflow of spreadsheets, reacted with performative disgust, and proposed the trifecta of microservices, Kubernetes and something called a "service mesh".
This kind of Big Enterprise technology however takes away that basic agency of those Excel users, who no longer understand the business process they run and now has to negotiate with ludicrous technology dweebs for each software change. The previous pliability of the spreadsheets has been completely lost. Using simple Python functions, in a source controlled system, is a better middle ground that the modern-day equivalent of J2EE. Financiers are able to learn Python, and while they may never be amazing at it they can contribute to a much higher level and even make their own changes and get them deployed.
Don't knock spreadsheets. If only they had a decent way to handle unit tests, they'd be the business.
Update: sounds like this is based on J. P. Morgan, specifically: https://www.reddit.com/r/programming/comments/qmi5fm/comment/hja6hqg/?utm_source=reddit&utm_medium=web2x&context=3]]>banking finance python coding excel spreadsheets software bank-pythonhttps://pinboard.in/https://pinboard.in/u:jm/b:b803af882769/nocodb2021-05-28T08:57:02+00:00
https://github.com/nocodb/nocodb
jmairtable database sql mysql nocodb spreadsheets ui webhttps://pinboard.in/https://pinboard.in/u:jm/b:61352fd4e436/12 of the Biggest Spreadsheet Fails in History2021-02-13T12:37:35+00:00
https://blogs.oracle.com/smb/10-of-the-costliest-spreadsheet-boo-boos-in-history
jmvia:yoz spreadsheets excel fail bugs softwarehttps://pinboard.in/https://pinboard.in/u:jm/b:807eed2e053b/Harvard-UC Boulder Portable Air Cleaner Calculator for Schools2020-08-04T23:12:39+00:00
https://docs.google.com/spreadsheets/d/1NEhk1IEdbEi_b3wa6gI_zNs8uBJjlSS-86d4b7bW098/edit#gid=0
jmair-cleaners filtration spreadsheets covid-19 schools kids air-quality airhttps://pinboard.in/https://pinboard.in/u:jm/b:26cb85dde2e2/Specification gaming examples in AI2018-11-09T10:54:04+00:00
https://docs.google.com/spreadsheets/u/1/d/e/2PACX-1vRPiprOaC3HsCf5Tuum8bRfzYUiKLRqJmbOoC-32JorNdfyTiRRsR7Ea5eWtvsWzuxo8bjOxCG84dAg/pubhtml
jm'Creatures bred for speed grow really tall and generate high velocities by falling over'
]]>ai funny humor spreadsheets machine-learning ml fitness-functionshttps://pinboard.in/https://pinboard.in/u:jm/b:533a6067a0e9/One in five genetics papers contains errors thanks to Microsoft Excel | Science | AAAS2018-08-22T22:24:45+00:00
https://www.sciencemag.org/news/2016/08/one-five-genetics-papers-contains-errors-thanks-microsoft-excel
jmscience microsoft excel spreadsheets autoformatting clippy fail papers geneticshttps://pinboard.in/https://pinboard.in/u:jm/b:143492bf59ec/_What We Know About Spreadsheet Errors_ [paper]2015-10-15T13:20:49+00:00
http://panko.shidler.hawaii.edu/SSR/Mypapers/whatknow.htm
jmAs we will see below, there has long been ample evidence that errors in spreadsheets are pandemic. Spreadsheets, even after careful development, contain errors in one percent or more of all formula cells. In large spreadsheets with thousands of formulas, there will be dozens of undetected errors. Even significant errors may go undetected because formal testing in spreadsheet development is rare and because even serious errors may not be apparent.
]]>business coding maths excel spreadsheets errors formulas error-ratehttps://pinboard.in/https://pinboard.in/u:jm/b:ed0cf69716fb/Snake-Oil Superfoods2015-05-28T13:37:39+00:00
http://www.informationisbeautiful.net/visualizations/snake-oil-superfoods-static-version/
jmgoogle snake-oil superfoods food dataviz bubble-race-chart graphics infographics google-docs spreadsheetshttps://pinboard.in/https://pinboard.in/u:jm/b:acbdbfae769e/A Virtual Machine in Excel2014-12-27T23:49:57+00:00
http://hackaday.com/2014/12/25/writing-a-virtual-machine-in-excel/
jmvms excel hacks spreadsheets codinghttps://pinboard.in/https://pinboard.in/u:jm/b:370a48bcea84/You probably shouldn’t use a spreadsheet for important work2013-04-25T14:49:04+00:00
http://lemire.me/blog/archives/2013/04/24/you-probably-shouldnt-use-a-spreadsheet-for-important-work/
jmThere are several critical problems with a tool like Excel that need to be widely known:
* Spreadsheets do not support testing. For anything that matters, you should validate and test your code automatically and systematically;
* Spreadsheets make code reviews impractical. To visually inspect the code, you need to click and each and every cell. In practice, this means that you cannot reasonably ask someone to read over your formulas to make sure that there is no mistake;
* Spreadsheets encourage redundancies. Spreadsheets encourage copy-and-paste. Though copying and pasting is sometimes the right tool, it also creates redundancies. These redundancies make it very difficult to update a spreadsheet: are you absolutely sure that you have changed the formula throughout?
Agreed on all three, particularly on the impossibility of testing. IMO, everyone who may be in a job where automation via spreadsheet is likely, needs training in SDE fundamentals: unit testing, the important of open source and open data for reproducibility, version control, and code review. We are all computer scientists now.]]>spreadsheets excel coding errors bugs testability unit-testing testing quality sde sde-fundamentals dryhttps://pinboard.in/https://pinboard.in/u:jm/b:94c2b0ce5080/Excel, untestability, and the reliability of quants2013-04-20T21:57:54+00:00
http://baselinescenario.com/2013/02/09/the-importance-of-excel/
jm
In his remarks on the final panel, Frank Partnoy mentioned something I missed when it came out a few weeks ago: the role of Microsoft Excel in the “London Whale” trading debacle. [..] To summarize: JPMorgan’s Chief Investment Office needed a new value-at-risk (VaR) model for the synthetic credit portfolio (the one that blew up) and assigned a quantitative whiz [...] to create it. The new model “operated through a series of Excel spreadsheets, which had to be completed manually, by a process of copying and pasting data from one spreadsheet to another.” The internal Model Review Group identified this problem as well as a few others, but approved the model, while saying that it should be automated and another significant flaw should be fixed. After the London Whale trade blew up, the Model Review Group discovered that the model had not been automated and found several other errors. Most spectacularly, “After subtracting the old rate from the new rate, the spreadsheet divided by their sum instead of their average, as the modeler had intended. This error likely had the effect of muting volatility by a factor of two and of lowering the VaR ...”
I write periodically about the perils of bad software in the business world in general and the financial industry in particular, by which I usually mean back-end enterprise software that is poorly designed, insufficiently tested, and dangerously error-prone. But this is something different. [...] While Excel the program is reasonably robust, the spreadsheets that people create with Excel are incredibly fragile. There is no way to trace where your data come from, there’s no audit trail (so you can overtype numbers and not know it), and there’s no easy way to test spreadsheets, for starters. The biggest problem is that anyone can create Excel spreadsheets -- badly. Because it’s so easy to use, the creation of even important spreadsheets is not restricted to people who understand programming and do it in a methodical, well-documented way.
This is why the JPMorgan VaR model is the rule, not the exception: manual data entry, manual copy-and-paste, and formula errors. This is another important reason why you should pause whenever you hear that banks’ quantitative experts are smarter than Einstein, or that sophisticated risk management technology can protect banks from blowing up. At the end of the day, it’s all software. While all software breaks occasionally, Excel spreadsheets break all the time. But they don’t tell you when they break: they just give you the wrong number.
]]>excel reliability software coding ides jpmorgan value-at-risk finance london-whale quants spreadsheets unit-tests testability testinghttps://pinboard.in/https://pinboard.in/u:jm/b:c2089e0606d0/Austerity policies founded on Excel typo2013-04-16T17:03:25+00:00
http://www.slate.com/blogs/moneybox/2013/04/16/reinhart_rogoff_coding_error_austerity_policies_founded_on_bad_coding.html
jmYou've probably heard that countries with a high debt:GDP ratio suffer from slow economic growth. The specific number 90 percent has been invoked frequently. That's all thanks to a study conducted by Carmen Reinhardt and Kenneth Rogoff for their book This Time It's Different. But the results have been difficult for other researchers to replicate. Now three scholars at the University of Massachusetts have done so in "Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff" and they find that the Reinhart/Rogoff result is based on opportunistic exclusion of Commonwealth data in the late-1940s, a debatable premise about how to weight the data, and most of all a sloppy Excel coding error.
Read Mike Konczal for the whole rundown, but I'll just focus on the spreadsheet part. At one point they set cell L51 equal to AVERAGE(L30:L44) when the correct procuedure was AVERAGE(L30:L49). By typing wrong, they accidentally left Denmark, Canada, Belgium, Austria, and Australia out of the average. When you run the math correctly "the average real GDP growth rate for countries carrying a public debt-to-GDP ratio of over 90 percent is actually 2.2 percent, not -0.1 percent."
]]>austerity politics excel coding errors bugs spreadsheets economics economyhttps://pinboard.in/https://pinboard.in/u:jm/b:b2ded17573ca/Arena.Xlsm is an RPG made by an accountant and played entirely in Microsoft Excel • News • PC • Eurogamer.net2013-04-02T11:09:56+00:00
http://www.eurogamer.net/articles/2013-03-25-arena-xlsm-is-an-rpg-made-by-an-accountant-and-played-entirely-in-microsoft-excel
jmWhat do you get if you take one accountant with "a fondness for spreadsheets, finance and business" and mix with "a life-long passion for video games"?
Well it's obvious isn't it? A turn-based RPG made and played entirely in Microsoft Excel.
(via Paul Moloney)
]]>via:oceanclub arena.xlsm excel spreadsheets games gaming rpghttps://pinboard.in/https://pinboard.in/u:jm/b:ee897b89cd6f/