Josh

I read this article about software development, which I knew about because I saw Prime reacting to it: notashelf.dev/posts/curse-of… For the most part I think it is fine: a relatively young programmer is doing the healthy work of introspecting on what he should really be doing. But there's one part of the article that I think is a deep mistake, and the author doesn't know it's so wrong because he has never experienced the alternative: "Software doesn’t stay solved. Every solution you write starts to rot the moment it exists. Not now, not later, but eventually. Libraries deprecate. APIs change. Performance regressions creep in. Your once-perfect tool breaks silently because libfoo.so is now libfoo.so.2. 2 I have had scripts silently fail because a website changed its HTML layout. I have had configuration formats break because of upstream version bumps. I have had Docker containers die because Alpine Linux rotated a mirror URL. In each case, the immediate emotional response was not just inconvenience but something that moreso resembles guilt." Yes, this is true in much of the programming world. But there is another world in which people build things that last much longer. I have done it many times. I shipped a binary for this game Braid in 2009 that you can still download and play on Steam 16 years later. If you are pretty young (like 35), you can run binaries on Windows that were compiled before you were even born, which is amazing given how hard they have been trying to f up Windows lately. On an emulator like MAME, you can play arcade games programmed in 1979. If today's software "technology" is so much better, why does it fall apart like tissue paper? The author is not wrong about the cited decay. But this decay is not inherent to the practice of software. It's due to choices made, usually foolishly, by the people designing the systems being interacted with. And, it's due to a lack of knowing better, non-exposure to the sector of programmers who are very concerned with their code lasting a long time, actually. The way you make code last a long time is you minimize dependencies that are likely to change and, to the extent you must take such dependencies, you minimize the contact surface between your program and those dependencies. The actual algorithms you program, the actual functioning machinery you build, is a mathematical object defined by the semantics of your programming language, and mathematical objects are eternal, they will last far longer than your human life. The goal then is to avoid introducing decay into the system. You must build an oasis of peace that is insulated from this constant bombardment of horrible decisions, and only hesitantly interface into the outside world. This means, for example: If you are shipping on iOS, you only reluctantly use any functions iOS gives you, because when you use them, Tim Apple will come along and break your program next year for arbitrary pointless reasons, because Tim Apple does not respect you or anyone you know. This means a program cannot last forever on iOS, because Tim Apple likes breaking your things and watching you submissively clean them up. But the core of your program, which could be 95% of the code, is fine, and you can deploy it elsewhere. This means you have to insulate from Linux userspace, because of all the jackass decision making that introduces constant incompatibilities while somehow never making the system better. Using a library dependency to do font rendering or sparse matrix math? That dependency gets checked into your source tree, a copy of exactly the version you use. Ten years later you can pull down that source and recompile, and it works, because your program is a mathematical object. If you want to upgrade to something newer that has bug fixes and so forth, you are free to do so, but you are also free not to do so, and your program still works. (And how many of these bug fixes do you really need? Your program worked correctly when you shipped it to the greatest extent you could measure, because you are a skillful software engineer who wants to ship things of a high quality). Everyone who got into programming for the joy of it knows, at some level, that the magic of programs is that they represent complexity that is replicable over time (and thus they exist outside of time). But the trashy programmer culture of the past 20 years stopped aspiring to this, and now has forgotten it is even possible. And so long as people have forgotten, decisions will continue to be made that make the problem worse. There are programmers who only write glue code, and who think that's what programming is; to these people what I have written above will not make sense. But the good news for that contingent is, they can always just stop writing glue code and start doing something else! If today's software "technology" is so good, why do you think it needs so much glue? Maybe there is a stylistic problem. So if you are looking for what to do in the world of software that can represent a lasting contribution, maybe this is food for thought. @NotAShelf @ThePrimeagen

Some people might read this and think, that's just not my world, I am stuck in this world where software breaks all the time and everything I build is disposable. Even if that is kind of the case for you, there is still good news, because this isn't an all-or-nothing problem. It's a dial that can be turned; you can turn that dial in a direction that reduces flailing and results in more-stable long-term progress. You don't have to remove all the dependencies, because every dependency you remove contributes to stability. Even getting rid of 1/3 of your dependencies can do amazing things. You can look at all the things you depend on and divide them into two categories: major and minor. Major dependencies are things that, realistically, you are never going to have your own version of. I am never going to make my own graphics API, so those count as major dependencies for me (DirectX12, Vulkan, Metal, etc). I am not going to write my own CPU-side font rasterization, so anything I choose to use there (FreeType, stb_truetype) goes in that category. With Major Dependencies, you limit your contact surface with them: You call only the functions you really need, and you do this only from the surface of your program -- you don't build data structures deep into your program that propagate the particular data structures or API decisions of any of these systems. A good API author will help you do this (stb_truetype), a bad API author will be trying as hard as they can to screw you up and force you to become tied to their system forever (anything from Microsoft or Apple). Understanding that many API authors are hostile can cause a big change of perspective here, and once you see it, the correct tactics become much more obvious. So, that's the major dependencies. Minor dependencies are things that are smaller, and that you want to use much more thoroughly throughout your program: for systems programmers this might be a data structure like an expanding array or hash table, for Web, maybe there are some string or file operations that you like to do. Minor dependencies can be eliminated and it's not even hard. You just do one at a time: hey, I need this data structure, I have been importing this other code to provide that functionality, I have suffered X, Y and Z problems because of this, how about if I just implement my own simple version of this one thing? People can get scared of implementing core stuff like this, because they look at the implementation they are using now, and it looks huge and complicated and hard to reproduce. But the thing to realize is most of this implementation is spam. It is mostly doing things for people who are not you, for reasons you don't necessarily agree with, chosen by a decision-making method that is deeply flawed. Your own implementation can be cleaner and smaller, and it can give you good feelings when you go look at it. You don't need all the functionality of the thing you are importing; you only need 8% of the functionality. Implementing that is easy. Once you do this a few times, you have your own stable body of code that you bring with you from project to project. It won't break unless you mess with it. You can keep improving it if you want, incrementally over time, but the cost of this is small because this code represents stable algorithms that don't change with fashion, so work on this is never forced. Every big company has their own internal version of this, but the problem in that scenario is that a big company is full of people who want different things, and have varying levels of decision-making skill, so these usually end up not so good. But when it's your own personal thing, it can in fact be very good, and help make you happy on a daily basis. And, your software will break much less often. Which is great. @NotAShelf @ThePrimeagen teapawt.net/Jonathan_Blow/status/1…