The Age of Software Artisans

99 pointsposted 10 months ago
by jairojair
(jairojair.com)

109 Comments

mustache_kimono

10 months ago

I'm going to be a little negative, so please bare with me.

The world needs artisanal programmers like it needs artisanal attorneys. See: https://www.mcsweeneys.net/articles/i-am-an-artisanal-attorn...

    How is an artisanal attorney different from any other attorney? Like other artisans, I pay close attention to my ingredients and process; I am intimately involved in all stages of creation. Other attorneys print their documents on paper they buy in mass-produced boxes, tens of thousands of sheets at a time, using ink that mechanically jets onto the page. I make my own paper by hand,...
This article is so similar to this satire I couldn't believe so many commenters were taking it so seriously.

If you are new to programming, this is a mostly ridiculous notion which plays to your vanity. While admiring your reflection, be careful not to fall, face first, in the pool.

Now, how do I know? Because this isn't the first time I've encountered this idea. Jonathan Blow, in his own way, has been making the same silly argument for years. As I've said before -- his frame of "programmers were real men once" makes each video/rant seem both one note and self aggrandizing.

For him, it seems to be a marketing exercise, but also an intellectual muddle, which with one breath tells you to romantically forget this is an engineering discipline, with economics and tradeoffs and constraints and compromises, and with the next, which actually helpfully(!), tells you about physical constraints of the machine. See: https://en.wikipedia.org/wiki/Data-oriented_design

The problem as I see it is that Blow and Muratori are half right about most things. They're almost always right about the particulars of programming (strict OOP patterns and "Clean Code" aren't great for performance). It's their frame that "civilization is in decline" (or "programmers were real men once" as here) that is wrong/catastrophic. Civilization simply has different priorities than game engine priorities (performance is a constraint, but not the primary constraint of most software).

Veserv

10 months ago

“Civilization in decline” is overstating the problem.

It is a common refrain that performance and security are poor because they are not priorities. The corollary being that the only reason things are slow and insecure is because we do not try to make them fast and secure due to incentives. If the industry were incentivized, then wham-bam secure and fast. Right now.

That is wrong. The incentives against fast and secure have been going for so long that the institutional knowledge, if it even existed previously, is not present. Programmers have spent their entire careers not knowing how to make things fast or secure. They do not know how to make things fast or secure, right now, even if the incentives changed. Making things slow is not a tradeoff or a choice as they do not even know (right now) how to do the alternative.

Yes, if the incentives changed then the industry as a whole could relearn how to do things over years and decades. The individuals who adapt and learn the new techniques that are now incentivized would thrive and come to dominate the industry. But, it is a relearning process, not a overnight thing.

This is analogous to the loss of manufacturing expertise in the US due to offshoring and then the process of onshoring. You must rebuild the knowledge and experience. Except it is even worse in this case because you can not even bring the offshore experts to re-teach you what you have forgotten. You have to learn from the scraps of forgotten lore and rediscover and reinvent what was lost.

This is more like the loss of knowledge involved in making the Saturn V rocket. Sure, we could make a new Moon rocket if the incentives aligned, hell, we could probably make a better one with modern technology, but we can not do that right now. We must relearn and retool to achieve that.

That is the problem they are seeing, whether they actually realize what is happening or not, and it is a serious problem if we need to make a sudden course correction to achieve these goals if they suddenly become incentivized (*cough* security *cough*). If we decide to change when we need it right now, then we are in for a rough few years to decades.

mustache_kimono

10 months ago

> It is a common refrain that performance and security are poor because they are not priorities. > We must relearn and retool to achieve that. That is the problem they are seeing, whether they actually realize what is happening or not.

I think I broadly agree.

I'd say -- I'm not sure we ever really knew how to be fast and secure. I think our problems re: memory safety and remote exploits, etc., are pretty relatively new.

I'd also say that this yearning for a forgotten yesterday is somewhat understandable, though mostly weird, a kind of tech revanchism.

My problem with Blow, et. al. is probably that they don't frame these as simply cultural differences or matters of priorities, whether or not, if the culture shifted, results could be achieved immediately or otherwise. Instead it is always a great big catastrophe.

Seriously -- I think he does it because it sells more games if the games are produced by some mercurial genius, by the Last Keeper of the Flame.

Veserv

10 months ago

Having seen some of their talks, I do not think they realize the underlying problem and understand its manifestation. They just see that their “fellows” churn out slow software without even understanding how to make things fast due to tradeoffs that society has made on their behalf that left their fellows with no knowledge of performance optimization. They feel a sense of wrongness that things are so different across the aisle, but can not put their finger on it precisely.

The incentives in their industry support performance. They look around confused at the rest of the world with different incentives. And they keep running into people who say: “We can make it fast if it is incentivized” and then do a amateurish job, despite seeming to be professionals, because they lack the institutional knowledge that is commonplace in the games industry. They are amateurs (in this field) due to long-term societal incentives despite having all the other trappings of competence. It is like being Chinese and seeing a regular grown adult being unable to use chopsticks; jarring and bizarre if you lack the context that they do not use chopsticks in their home country.

It also leads to what, from their perspective, are incongruous tradeoffs. They see a 10x speedup that would take a month to implement. The developers say that is a worthy tradeoff, but they do not do it, why? Well, they lack the institutional knowledge. To reinvent all of the knowledge and technologies that would make that 10x speedup take a month would take 5 years assuming they even recognized the opportunity. 5 years for a 10x speedup is not worth it, but 1 month is; it is the researched tech tree that makes the difference and allows worthy tradeoffs to be implemented efficiently. Losing tech and knowledge leads to losing the ability to even make those tradeoffs.

xyzzy123

10 months ago

I can think of 50 things which would make the project I'm on faster, better and more secure but the business is not particularly interested in them right now.

The business wants its stated requirements met (plus a short list of key non-functionals) while minimising cost. Performance is not even specified. I don't think there's any "lost knowledge" involved in this particular case, they just don't want to pay me to make it better than it needs to be.

Doing my job well means minimising the time (and by implication the attention and effort) I put in to the software.

This seems quite directly opposed to the artisanal ethos.

soco

10 months ago

An artisan was historically a guy working for himself in a small market shop, carving at a chair or sewing leather shoes. The masons tasked to build the castle walls were not "artisans" so there we could see the same focus on efficiency - although a bit of beauty had still its place here and there. My point? Today we don't have those artisan shops, just expectations from regular masons to carve gargoyles at every wall corner.

FrustratedMonky

10 months ago

But aren't there artisanal attorney's?

There are some in specialized areas, perhaps even inside a specialized area there are a few firms that are known to be at the top, and find creative solutions to problems. They aren't all Law-and-Order.

Same with accountants. Sure, it's all number crunching. But then why do some become the top business leaders, the CEO's. Maybe because in that field, being a top artisan and able to play with financial instruments, allows you to build a business. (and I'm playing devils advocate for the small percentage of accountants/business that are actually trying to build something, not just playing with money to bilk people)

mustache_kimono

10 months ago

> But aren't there artisanal attorney's?

Not in the way there are artisanal bread-makers. See the linked article.

> There are some in specialized areas

Let's presume the following definition:

    artisanal: (of a product, especially food or drink) made in a traditional or non-mechanized way.
I am saying -- there is no attorney marketing their unique skill with the Smith Corona Galaxie or with a quill. Instead, you probably want them to write a brief for you and don't particularly care how they physically do it.

FrustratedMonky

10 months ago

I guess by this argument, writes/authors also cannot be 'artisanal'?

This seems like we are instead discussing a narrow definition of 'artisanal'.

Where I was taking more broad definition of someone at the top of their 'craft', an expert with deep knowledge of a specialized area.

mustache_kimono

10 months ago

> This seems like we are instead discussing a narrow definition of 'artisanal'.

Perhaps see the article. This is the definition which is most like the definition it uses.

FrustratedMonky

10 months ago

"Perhaps see the article"

Are you joking, it was 2 paragraphs. And did not offer any definition.

You are reading into it you're own internal monologue, and thinking the article stated more than it did.

So if we go by the article using example of 'making a table', you could say writers/authors/accountants would NOT be artisans. Since they are not physically crafting something.

or

If it is more about "enjoy the process, ... , improve yourself, learn new things", then it is very much more general and includes many fields.

mustache_kimono

10 months ago

> You are reading into it you're own internal monologue, and thinking the article stated more than it did.

No, I'm reading critically. Just because an article does not readily offer up a definition, it does not mean it isn't working with one.

However, I'll try to be intellectually charitable, here, and say it's plausible a reader could feel the article unhelpfully conflates many definitions of "artisan". Unfortunately, all such definitions don't make much sense in the context of programming. What's wrong is not the concluding sentiment of "enjoy the process, ... , improve yourself, learn new things", it's that "artisan", even in its many definitions, doesn't touch upon those noble sentiments. Attorneys and programmers simply aren't artisans, even if they enjoy their process, and improving themselves, and learning new things, and your insistence to the contrary is self-aggrandizement.

That isn't to say I would limit "artisanal" to the physical crafts. I might extend it to any endeavor where the process is part of our use of the product. Like -- I may know this or that writer drinks 20 year old Scotch, and smokes only three cigars a day, while he types at exactly 45 words per minute on his decades old Smith Corona. I can understand being romantically engaged by that process.

However, while this kind of process might be important to our enjoyment of that literature, is is almost entirely unimportant to our use of software, because the artifact is still what is most important, by a mile. I couldn't care less whether a game designer copied a method from Boost or wrote it from scratch him/herself, just as, I couldn't care less whether my attorney saw the face of God after smelling the dust mites in the Westlaw Reporters at the law library. The romance of the "artisan" is almost entirely incoherent in this context.

It's why "I am an Artisanal Attorney (because I make my own paper and write briefs with a quill)" is so funny. Do me a favor and try to write your own. "I am an Artisanal Programmer (because I do all my Objective C programming on a NeXTcube)..."

FrustratedMonky

10 months ago

You have again, only offered examples of things that 'are not' artisan, and offered no definition or real argument against the sentiment of the article. No argument for what is artisan, and why it does not apply.

I'd say you have instead backed yourself into a corner where nothing is artisan, that the term can't be applied to any field at all.

If it is purely "the artifact is still what is most important, by a mile", then of course there are many that would include iPhones, an algorithm, a mathematical proof,... etc...

If an artisan can only be judged by their products, then any trash that is popularly agreed upon as 'artisanal' can make the producers an 'artisan'. It is all 'in the eye of the beholder'.

Thanks for the intellectual charity. I'll buy a coffee with it for what it is worth.

Edit: To stretch this further, if 'artisanal' using a 'traditional' or 'non-mechanist' methods, then since programming is now many decades old, I could say ::

"Well son, I'm programming in VBScript, like my father did, and his father before him. In the traditional manner of a dying generation. I don't dig these new fangled editors with their 'code completion', leave VSCode to the kids. I do things the old way and it works and my customers love it".

mustache_kimono

10 months ago

> You have again, only offered examples of things that 'are not' artisan,

That's simply not true. I said:

>> That isn't to say I would limit "artisanal" to the physical crafts. I might extend it to any endeavor where the process is part of our use of the product.

And went on to explain how a writer might be an "artisan".

> and offered no definition or real argument against the sentiment of the article.

That's because I do agree that some of the sentiments of the article are good, but I also disagree that muddling such good sentiments with the wrong word, "artisan", is good. I said "enjoy the process, ... , improve yourself, learn new things" isn't "artisan" because there is no definition of "artisan" which encompasses those things. An Oxford don might also enjoy the process, improve him/herself, and learn new things. He/she may be in a what we now consider a "skilled" profession (though significantly not a "trade"), but he/she never was and is not now an "artisan". See: https://www.oed.com/search/dictionary/?scope=Entries&q=artis... and https://www.oed.com/search/dictionary/?scope=Entries&q=artis...

> No argument for what is artisan, and why it does not apply.

Again, no, I don't need to make an argument for what definition applies when I've ceded the ground that any one definition is better than another.

> If an artisan can only be judged by their products, then any trash that is popularly agreed upon as 'artisanal' can make the producers an 'artisan'.

I'm pretty sure you've completely misunderstood me here. Please reread my comment (or see the double quoted section above). My argument was precisely the opposite. Programmers, and attorneys are NOT artisans because what matters most is their product, whereas bakers and cobblers MIGHT BE artisans because their process MAY contribute to our use and enjoyment of the product.

> Edit: To stretch this further, if 'artisanal' using a 'traditional' or 'non-mechanist' methods, then since programming is now many decades old, I could say ::

Yes! Exactly. This is why this notion is ridiculous.

FrustratedMonky

10 months ago

I think we have stumbled into the age old debate over what is 'art'. And there is no answer.

I might like to think of programmers as 'craftsman' that can reach some height of achievement/skill to be deemed as 'artisans' in their 'craft'. Maybe like a Dijkstra, or even a John Carmack/Doom.

But to your argument I think. There must be some 'physical' product. So there are some programmers or even accountants that have reached some heights of understanding their craft but their product does not count because it is too ephemeral. There is a necessary quality of a 'repeatable' physical process and forming a physical product to count as 'artisanal'.

A lot of programmers are crap or mid. Just like the woodworker, or chef, that could be great, but most of them are mid or crap.

I'm sure there are a lot of people that shop at Hobby Lobby, that thing of themselves as artisans. But are mostly not.

So you need some combination of a process and also some general agreement of the admirers of the product.

I just fall on the other side of this gray shady line, that programmers and writers, do generally produce something, a product, and they practice to become experts at it. They also have a process that contributes to the product.

Even if like 99% of Chefs, 99% of programmers are crap, with no process and crappy products. They both are producing products, and the best ones have some process.

"Perhaps it is undeserved, because you don't seem to be reading what I wrote."

On this point, I think there is also some bias. You think your points are clear, but frequently people think they are clear because they have their own internal perspective that makes what they write seem clear to themselves. But are not really quite getting it out onto paper. If your points aren't landing, maybe at some point it is not the readers fault for not grasping them. I really have tried to follow yours, and don't think we are even disagreeing that much.

If I was to be intellectually charitable, I think this is probably a case of a very contentious subject that has been argued for a few thousand years, and we jumped into an internet argument in the middle, with very different backgrounds, that can't be resolved.

What is art? Programming isn't the only time this argument has happened when a new medium/technology came into existence. Is photography art? Many think not, many think it is. Can you have an artisanal photographer? Is a portrait painter an artist, but not a portrait photographer? What about metal working, welding, you would say yes? Or Poets, you would say no?

meiraleal

10 months ago

> this is an engineering discipline

oh, well. I just checked all 7 old and new wonders of the world and all of them needed engineers. I don't think being an engineering discipline limits software in any way related to what can be achieved with it.

mustache_kimono

10 months ago

> I don't think being an engineering discipline limits software in any way related to what can be achieved with it.

I don't either. I think programming can be art, etc.

My issue here is that I don't think favoring artisanal methods are part of the appeal of software. We are much more interested in what software does.

For instance, it might be useful to know x86_64 vector assembly. It might make your program run faster. It might even be necessary for you to know it for your program to work correctly. What I am saying is -- if a higher level construct exists, like C, C++, or Rust, which can produce an equivalent program, but you choose to write your entire program in x86_64 assembly, because that is/was the artisanal way, that's mostly unimportant/irrelevant to the user, except as an oddity.

meiraleal

10 months ago

Got it. I don't think writing in x86_64 is any more artisanal than Python or JavaScript. It's actually quite easy to make the distinction. Assembled software - using multiple libraries in place of crafting (almost) every piece of it - is the opposite of artisanal software. You can have that with assembly, C/C++, Java, or JavaScript.

As an example, I'm developing an IDE from the ground up. The current conventional approach would be to fork VSCode, a bloated, slow, over-engineered, ultra-complex Electron app reliant on Microsoft's standards.

Alternatively, I can code it artisanally, which will likely result in something worse in many ways and break in unexpected ways. But for my specific use case, it provides a much better user experience than taking the assembly-line approach to software development. The chances of this project failing are much higher using this customized approach than if I used the assembly-line method, but it doesn't matter to me - coding it this way is already so much more fun.

mustache_kimono

10 months ago

> Assembled software - using multiple libraries in place of crafting (almost) every piece of it - is the opposite of artisanal software.

> The chances of this project failing are much higher using this customized approach than if I used the assembly-line method, but it doesn't matter to me - coding it this way is already so much more fun.

I am sympathetic to this way of thinking about "artisanal". I guess for me it has so much baggage re: food, I don't think of it this way. I'd perhaps call that/your way the "DIY way", because you're right -- it is sometimes appealing and advantageous, even if I'm not certain that's what the article intended.

meiraleal

10 months ago

> even if I'm not certain that's what the article intended.

I have the impression that was exactly the intention of the article:

  > In the coming years, we’ll see more and more people making software like someone building a table in their backyard or garage, they’ll enjoy the process and add their own personal touch.
  > If you love coding the way we do it now, keep at it! enjoy every moment, improve yourself, learn new things, keep coding!

mustache_kimono

10 months ago

> > I don't think writing in x86_64 is any more artisanal than Python or JavaScript. > I have the impression that was exactly the intention of the article:

Okay, but you seem to be ignoring the first half of the article. You know the part that tells us it is a great big joke:

"Real Programmers wrote in machine code. Not FORTRAN. Not RATFOR. Not, even, assembly language. Machine Code. Raw, unadorned, inscrutable hexadecimal numbers. Directly."

meiraleal

10 months ago

True, I think I really skipped that quote. I disagree with this part and think it doesn't link very well with the conclusion or with software artisans. Not sure if I mixed it up or the author :)

wavemode

10 months ago

Sure but demand for world wonders is low. Demand for basic cookie-cutter houses is very high.

An engineer's job isn't to build something perfect or beautiful, it's to build something that's just good enough for its intended use case. Lots of software engineers forget that.

meiraleal

10 months ago

> An engineer's job isn't to build something perfect or beautiful, it's to build something that's just good enough for its intended use case.

Why? Who is responsible for making it look good? fast? resilient? The architect? In software development the engineer can be the architect, the decorator, the user. AI will unleash the age of artisan software - which already exists, so many great free software developers came before us but now it will spread like a "plague". It has the potential to disrupt SaaS as we know it today.

FredPret

10 months ago

Software is somewhat unique because the design, execution, and mass distribution can all be one step. So you really can be a software artisan, though the world needs the mass production kind of software too.

mustache_kimono

10 months ago

> So you really can be a software artisan, though the world needs the mass production kind of software too.

I think this is true only if "artisan" means whatever you want it to mean. Here, you seem to think it means "made by one person." But, here, my definition, and the definition of the article, is "making software using the old methods."

   artisanal: (of a product, especially food or drink) made in a traditional or non-mechanized way.

FredPret

10 months ago

Ah, I guess you’re right.

To me, “artisanal” means one / very few experts carefully crafted a custom product to their own exacting standards, driven by their internal need for perfection rather than guided by a spec sheet / quality level specified externally.

There are definitely hackers who do this.

musicale

10 months ago

> I'm going to be a little negative, so please bare with me.

took off my shoes, does that count?

vijucat

10 months ago

Artisan is not a bad metaphor. Take furniture. There's a place for IKEA, and there's a place for expensive, hand-crafted Scandinavian furniture.

https://www.scandinaviastandard.com/this-is-why-that-sofa-is...

didgetmaster

10 months ago

I'm not sure building furnace is a good metaphor for crafting good software.

The nature of software is that every piece of it can be mass produced (copied and distributed) once the first piece is finished. All the 'artisan' effort goes into making that first copy.

ivancho

10 months ago

That's a pretty narrow view of software too. The nature of building software systems is not always towards being able to mass-produce them. What would be the point of copying and distributing 10K copies of my script that runs migrations for one legacy database in a very specific way?

sudobash1

10 months ago

I have written a lot of software for myself or some close associates that is really only for our purposes (or sometimes just for our amusement). Certainly I could trivially distribute the software to whomever I wished, but it wouldn't be useful, appreciate it, or perhaps even run outside of the environment I wrote it for.

For instance I have some lovingly crafted bash scripts, that feel somewhat artisan. They keep my home server humming along, applying updates, making regular backups, and reporting status. The scrips are very tailored to my use and would not easily integrate elsewhere.

I could have used off the shelf software for much of this (ala "IKEA"), I'm sure, but I am a tinkerer, hobbyist, and, if I flatter myself, an artisan.

user

10 months ago

[deleted]

andsoitis

10 months ago

> there's a place for expensive, hand-crafted Scandinavian furniture.

And then there’s Italian craftsmanship https://artemest.com/

ivancho

10 months ago

So we'll have expensive artisanal software for rich people? Self-assembly takes a whole new meaning for the rest of us.

jairojair

10 months ago

you get it! I think the exactly same way.

herval

10 months ago

where in the analogy does social software fit?

andai

10 months ago

Are you referring to that article about software that serves a small local community?

I can't remember the name of the author but there was an interesting article about this, I think from some college or university in New York, talking about some examples of software that was set up at specific locations on the campus for specific purposes, and explaining how they wouldn't have worked as an online global thing.

Edit: Here it is!

https://web.archive.org/web/20040411012949/http://www.shirky...

user

10 months ago

[deleted]

interstice

10 months ago

My 10,000 hours are well behind me and I build web apps a bit like this. Previous jobs are like a shed full of parts that will get 60% of the way there, and the rest is making a couple of custom parts and a lot of fettling. Sometimes I build a table from scratch just to remind myself I can.

boricj

10 months ago

Artisans can make objects out of raw material, but they can also take apart objects to reclaim raw material. Software engineering tends to only ever do the former, because traditional toolchains are a one-way street from source code to assembly code to object files to programs.

That article rings very differently to me because of the tooling I've developed. With the ability to break apart programs into object files and reusing them to make new programs, I can do the latter. In a sense, using both pristine source code and second-hand binary code to make programs is as artisanal as software development can get.

QuadmasterXLII

10 months ago

That’s awesome. As software gently rots, I suspect this kind of radical maintenance will only get more relevant.

bschmidt1

10 months ago

> break apart programs into object files and reusing them

So, libraries?

boricj

10 months ago

No, it's actually ripping out bits from an executable and turning them into relocatable object files. The technical term I've come up with for this is delinking, although the decompilation community calls it binary splitting.

Putting it another way, you can make libraries out of a program with this technique.

WJW

10 months ago

Anyone with the patience to read through an existing github repo can do this? If there are valuable bits to be salvaged it's easy enough to copy-paste them into a new library if you so choose.

boricj

10 months ago

Well, it's one of those things where you know the rules so well that you can break them, but my Ghidra extension is indeed on GitHub: https://github.com/boricj/ghidra-delinker-extension

Unless you meant copying source code from GitHub, which isn't an option if the program is a 1998 PlayStation video game and you don't have the source code for it.

nradov

10 months ago

Yikes. I can't imagine the circumstances where this would be a good option for software that's going to be used beyond a quick one-shot job.

boricj

10 months ago

It's very useful if you don't have access to the original source code.

You can do things like decompiling a program piece by piece like the Ship of Theseus. The linker will mend both original and rewritten parts together into a working executable at each step.

If you change the functionality of the rewritten parts, you have a modding framework that's far more malleable than what traditional binary patching allows.

As for quick one-shot jobs, I've created an assert extractor for a PlayStation video game by stuffing its archive code inside a Linux MIPS executable and calling into it, without figuring out the proprietary archive format or how the code inside the delinked object file actually works.

fallingsquirrel

10 months ago

Is that meaningfully different from reverse engineering? You can't use individual functions without knowing the data structures they operate on, and info about data structures is usually not present in the final binary*.

* (excluding languages that compile to an IL like C# of course, but decompiling C# is trivial)

boricj

10 months ago

> You can't use individual functions without knowing the data structures they operate on, and info about data structures is usually not present in the final binary.

It turns out you can. Linkers do not care about types or data structures, all they do is lay out sections (a fancy name for arrays of bytes) inside a virtual address space and fix up relocations.

I've written case studies on my blog where I've successfully carved out whole pieces out of a program and reuse them without reverse-engineering them. I've even made a native port of a ~100 KiB Linux a.out x86 proprietary program from 1995 to Windows and all I had to do was write 50 lines of C to thunk between the glibc 1.xx-flavored object file and the MinGW runtime.

One user of my tooling managed to delink a 7 MiB executable for a 2009 commercial Windows video game in six weeks (including the time required to fix all the bugs in my alpha-quality COFF exporter and x86 analyzer), leaving out the C standard library. They then relinked it at a different base address and the new executable works so well it's functionally indistinguishable from the original one. They didn't reverse-engineer the thousands of functions or the hundreds of kilobytes of data inside that program to do so.

This is complete heresy according to conventional computer sciences, which is why you can't apply it here. I'd be happy to talk at length about it, but without literature on the topic I'm forced to explain this from basic principles every time and Hacker News isn't the place to write a whole thesis.

karpierz

10 months ago

The question on my mind: how do you figure out what the functions do without reverse engineering?

If I were to guess, you're saying that you reverse engineer the API boundary without reverse engineering the implementation. But then figuring out what the API contact is without documentation seems intractable for most API boundaries.

boricj

10 months ago

For context, my tooling is a Ghidra extension, so there's all the usual Ghidra stuff that applies here.

Indeed, it depends what the API boundary is for the selection to be exported.

If it's the whole program without some well-known libraries (like the C runtime library for a statically linked executable or the Psy-Q SDK for a PlayStation video game), then the API boundary is trivial in the sense that it's a documented one. The hard part is actually figuring where those libraries are so that you can cut them out of your selection while exporting.

If it's a particular subset internal to the program then it's trickier because you don't have that (but if you know you want to export it, then you must already know something about it). Traditional reverse-engineering techniques do apply here, it's just that you only care about the ABI boundary instead of the whole implementation, so it's usually less work to figure out.

However, if you get it wrong then the toolchain will usually not detect it and you'll have some very exotic undefined behavior on your hands when you try to run your Frankensteinized program. Troubleshooting these issues can be very tricky because my tooling doesn't generate debugging symbols, so the debugging experience is atrocious.

I've always managed to muddle through so far, but one really nasty case did take me a couple of weekends to track down (don't cut across a variable when you're exporting because you'll truncate it, which can lead among other things to corrupting whatever was laid out next in memory at runtime when the original delinked code tries to write to it).

jairojair

10 months ago

Hi folks!

I really didn't think many people would see this post, thank you for the comments and discussion.

The central idea is that technology is cyclical, new tools will emerge and the way we work will change.

However, in my view, using tools doesn't make you less or more of a craftsman; creating something different from mass production does.

In a world where everything is very similar, websites are copies of each other, every new startup has the same color palette, creating something with small flaws, but different, is a distinguishing factor.

Perfection lies in imperfection.

Keep coding!

jagged-chisel

10 months ago

I have attempted, in vain, to bring this kind of thinking into the companies and teams that I have worked with. They don’t want it.

And in interviews, no one wants a Software Artisan. The cynic in me says that it’s because people want the job security of the typical bullshit position and don’t want anyone upsetting that.

tkiolp4

10 months ago

Companies don’t need software artisans. They need replaceable people who can do the job until they run out of investor’s money.

ChrisMarshallNY

10 months ago

> replaceable people

Until that mindset changes, the disasters will keep coming.

Achieving true "replaceable people" is a major task. I have seen it done (properly), and suspect most folks on this board would be aghast at the compromises that need to be made. I didn't like it.

I managed a team of extremely experienced C++ engineers that were really hard to replace, so I did my best to keep them on board.

As to "artisan"? I don't really care what I'm called. I like coding, and I like writing really good UX.

luckylion

10 months ago

Why do you think the mindset that everyone should be replaceable is wrong / leads to disasters?

I agree completely that it's not trivial and comes with its own trade-offs and side-effects, but when you achieve a good chunk of it, it brings a lot of flexibility, stability and peace of mind for everyone involved.

You can put more people on some task because it's not something only one person can do. You won't nose-dive when somebody leaves. And developers know that they can go on vacation and don't need to worry about what's going to happen to the rest of the team (I know that not everyone thinks like that, but in my experience enough people do).

ChrisMarshallNY

10 months ago

It's really a long topic, but the basic TL;DR, is that people are not interchangeable, unless you reduce the task to the lowest common denominator, and that means that we squander the talent of the most skilled, in order to cater to the least skilled. Basically, tell good coders to write bad code, because bad coders need to understand it.

We get "meh" results.

The issue is that everyone seems to be fine with "meh."

That's a shame.

The alternative is hire only good coders, and do what it takes to keep them around, and that's a really unpopular stance.

sfn42

10 months ago

I'd say good code is easier to understand.

Things get at least as complex as they need to be. Some things are just hard, but most things are fairly simple. The difference between a good programmer and a bad one is that the latter will make the simple things look complicated and hard to understand.

ChrisMarshallNY

10 months ago

> I'd say good code is easier to understand.

In my experience, this is highly subjective.

One of my employees was probably the best coder I’ve ever known. His code was well-structured, well-documented, clever, efficient, bug-free, and a total bitch to comprehend. He would regularly produce miraculous results. We worked together for almost 27 years.

I guarantee that a lot of folks here would call him a “bad coder,” because they would have difficulty easily understanding his code.

I find that when people talk about “good,” they emphasize traits that they, themselves, have, and disparage traits that they don’t have.

In the aggregate, results matter, and “results” are more than just getting today’s goals accomplished. It would be nice if that were the only concern.

For example, I ran a shop, where we maintained a huge legacy codebase of C++ image processing code. We had to maintain it, ourselves, as well as train others to work on it. This meant good documentation, good code structure, and good foundational substrate.

Introducing new coders to our codebase meant immersive training. Japan regularly sent folks over for months, to learn our code. These were top talent. Just about every one of our Japanese engineers had at least a Master’s degree in Engineering. It was a marquee corporation, and employed top-shelf engineers.

That’s quite different from hiring bootcamp-trained JS programmers, and expecting them to maintain Swift code. I know many people that insist that “any good engineer” can learn any language in a couple of weeks.

I’ve been at this game for a while, and have worked in quite a few languages. I’m smarter than the average bear, and have made fearful messes, because I didn’t understand the tools completely.

I have found that I can learn about 75% of a language quite quickly.

That other 25%, though, could take years, and languages seldom stay static, so I’m constantly catching up on the latest stuff. It also includes style, which is often an artifact of understanding how to use the language properly.

Also, learning a language is only part of it, as the APIs and SDKs are even more important. Learning those can be difficult, and I have often had to throw away years of work, when an SDK is swapped out. I’ve been writing Apple software since 1986, so I’ve had to do that a number of times. My first Apple code was in procedural Pascal and 68000 ASM. Adobe Photoshop is probably one of the oldest codebases still in use, and it started off as object Pascal, using MacApp 0.9 beta.

Tom Knoll was definitely top-shelf, and I was tasked with adapting some of his code for use in a different context, back in the 1990s. It wasn’t easy, but I did it, and learned to be a better coder, as a result.

Over the years, I have changed my opinions and points of view; frequently by screwing up, and also, by taking the time to understand code -and not just code, but also the reasoning-, of talented predecessors.

I remember once, hearing someone say “That ‘cruft’ you’re complaining about, is what I call ‘bug fixes.’”

I love coding, because I’m constantly challenged and humbled. I stand on the shoulders of giants, and am grateful for their example.

lioeters

10 months ago

> Tom Knoll

I recognize this name from the screenshot of Photoshop version 1.

https://www.webdesignmuseum.org/uploaded/web-design-history/...

Thomas Knoll: Complete Biography, History, and Inventions - https://history-computer.com/people/thomas-knoll-complete-bi...

Was he an "artisan of software"? I imagine back then most programmers were artisans, but maybe that's because I've only heard of the exceptional ones who made a mark in history with innovations that require an artisanal approach and not by mere "replaceable people".

(That term makes me throw up a bit, it's an insult to our humanity. I understand businesses and factories need replaceable people, but nobody wants to be one. Besides, if they're so replaceable, "AI" and robots will replace them in the near future.)

ChrisMarshallNY

10 months ago

He’s the original author of Photoshop, along with his brother John, who used to run the ILM Effects team.

I saw Photoshop before Adobe brought it. It was running on a Mac II, in a hospital, in Ann Arbor. Back then, it was still his thesis project.

I have no idea if he was an “artisan.” We never really thought of ourselves in such terms. Software was much more of a fun game, but the introduction of huge piles of money changed everything.

WJW

10 months ago

Not all companies float on PE money.

Swizec

10 months ago

The problem with hiring artisans is that most companies need a Honda Civic not an Enzo Ferrari.

If they hire an artisan it’s just going to be frustrating for everyone involved. The business will pay more than the value received and wonder wtf and the artisan will be bored out of their mind looking for fun projects to do.

zrobotics

10 months ago

One thing where that analogy breaks down is that a Honda civic is more highly engineered and a more refined product than a Ferrari. The Ferrari makes all sorts of sacrifices for performance and styling, sacrifices such as manufacturability, cost, and reliability. We tend to forget or underestimate the sheer work that Honda engineers spend making a car that is fuel efficient, crashworthy, long-term reliable, and affordable. A civic certainly isn't as sexy as a Ferrari, but stepping back and thinking about the sheer amount of human effort that went into making such a good machine at such a reasonable (compared to a Ferrari) price is astounding.

I would argue that software would be improved if it was approached in the same engineering fashion as a civic or corolla, what we have now is the engineering effort of a Ferrari with all the warts that entails but released to the mass-market. Ferrari can't invest the engineering effort to release as polished of a product as Honda or Toyota, but that's fine since it's a niche vehicle and the buyers accept the tradeoffs since nobody uses one as a daily commuter anyway. But a small team of software developers can release an under-designed product to market and that software is available nearly instantly worldwide with little limits. Making 1M physical items is far more difficult than having 1M users of a program.

ahoka

10 months ago

This is a false dichotomy.

slowhadoken

10 months ago

McDonalds isn’t looking for a chef.

jagged-chisel

10 months ago

Actually, you make a great point: these companies attempt to convince employees that they’re not McDonald’s, when that’s exactly what they are.

And I shouldn’t be falling for it myself.

slowhadoken

10 months ago

I’ve avoided working directly for the tech industry. I was contacted by Google and Amazon without ever having reaching out to them but that was right before the mass layoffs. Beside high level proprietary projects I’d rather work for myself doing whatever I want.

at_a_remove

10 months ago

"Artisan" is covering a great deal of ground.

At a previous job, probably fifteen years ago, we were replacing one bit of niche software (the company was gone, it was unsupported, and it had some very antiquated methods of ingestion, like faxes) with another, mostly similar bit of software. Continuity was desired.

Different philosophies went into structuring the two databases, so it was like lifting a vast set of webs made by social spiders from one tree and placing it on another, just for the data import. I interviewed various people who were in different roles as to how they they used the old system. I checked the databases to see what they did not tell me about how they actually used the system. I scoured over the incoming data feed sources and made robust ingestion processes, validating and correcting and logging. Knowing the vagaries of software vendors, I wrote bits to detect changes made by vendors to the database structure, or at least the portions I cared about, and whine when those occurred. I grokked how the several different roles people had in interactions with the system led to differing needs and desires, incorporating as much of how things "used to work."

The eventual juggernaut which resulted was something of a force of nature, a compilation of my best thoughts on my best days, outperforming me. It would tell you if a new type of data appeared which had not been announced, whom to ask about this data, and where to put it. It functioned for a decade of my oversight, occasionally murmuring that it wanted this or that, but without changes to the code.

That was the artisinal bit to me, that it was carved to fit some very exacting spaces, such that you hardly knew it was present. Yes, it took extra time. Yeah, it was just me. But the fit was tight and the finish was smooth.

(Naturally, some bright spark got a touch of the New Broom Syndrome and blew it and all of the documentation away after I left, I am told that there are continual complaints about the system now. Sadly, with even minimal prompting, the various tiny details arose from memory, prompted by the SMS messages.)

mlinhares

10 months ago

I might just be at a very different environment but the usual basic software discussions (class design, database modeling, architecture) are mostly irrelevant as there’s always some premade thing I can use to do the job and most of my time is either making sure requirements are clear of that the teams are aligned on their deliveries.

I think 10 years ago I cared a lot about this whole software artisan thing, nowadays I’m happy when stuff gets to prod on time and doesn’t cause large scale incidents. I wish I could have these problems and spend time discussing what the best architecture is or what class design is the best

jumploops

10 months ago

Everyone is saying that LLMs will kill programmers.

The author here has the right idea, and the correct historical context to back it up.

Generative AI will 10x (at least) the number of people that program computers, but that programming will look different than it does today.

Gone are the semicolons and curly braces we’re so used to, just like BEQ and JMP before them.

Truly gone? No, the next Rollercoaster Tycoon has yet to be built :)

jairojair

10 months ago

Good point! A car built 10 years ago is very different from a car built today, but it's still a car, right?

Some people like old cars, others prefer Teslas, and that's perfectly fine.

The world will change, and that's not a bad thing.

jauntywundrkind

10 months ago

I sure hope we can get to a happier software harmony, of enjoying code and systems.

One of the recurring trends thats cropped up is so called barefoot developers, folks just cobbling together some kind of task not for big industrial software with millions or even dozens of users, but just because it makes their lives or a small group of people's lives better.

Maggie Appleton recently followed up with a post saying that these are the folks ML/LLMs might best be able to help. To get them their stuff. https://news.ycombinator.com/item?id=40633029 https://maggieappleton.com/home-cooked-software

But I also have this feeling, that almost all the webdev we do is artisanal handcraft as is. We hand author endpoint after endpoint for our objects even though there's this high degree of similarity across endpoints; authorize/validate the request, do the thing in the db, return it to the user.

Part of me is sick as shit of just how artisanal software is; it stinks like rot that we keep cranking out more piles of code for each entity & as we intricate/complicate/embellish each entity/resource. We so rarely have broader high level systems where we've escaped from hand crafting web server middle tier glorified-translators of a very basic 3-tirr client-server-db architecture.

I just want companies to pay me to tell them how much less they could do, if they let me PoC their system in something ordered & higher order, such as Hasura. Like CORBA, the UML world also has great scorn as unmaintainable, but again it's like, those folks felt on to something amazing & mighty & we have immense unsurvivor anti-bias, we are pat & confident that this tower of babel built too high & all future towers too will fall & that the effort is folly.

But man, the chaos & lines of code we have been creating as an industry is just so unnecessary & so out of control.

eastbound

10 months ago

I agree obviously. But each time I look at my code, it’s subtly different from one entity to the next, each one with its intricacies.

Yes I’ve coded permission systems a hundred times. But here it’s wired to the app we integrate with, and here it’s wired with groups. Here it’s by document, here it’s by space. A thousand hoops, never the same exact logic. Like a thousand humans, never the same identical one. We humans can see the same parts and easily designate “a human”, but this pattern recognition ignores a lot of important details. And those details are what makes IT accurate and reproducible, which has been the trait we’ve liked about it.

Since 1970, have we misled ourselves in believing we were repeating the same patterns, while in fact, every program really was different?

lttlrck

10 months ago

At what layer of abstraction are you no longer an artisan? Or doesn't it matter?

It's a nice "label" but it's a bit murky what it means the farther away from machine code (the raw materials) you get. Are you an artisan if you use an IDE?

I do enjoy the sentiment however.

skydhash

10 months ago

I think it all depends on amount of care you put in the human aspect of the software. Does it solve a specific need for a person you can name, or the stories are all about imaginary characters and generic persona. Crafting requires empathy and the realization that this will be used by and for people, and you want to make it easy for them (even if people means you as one person). And the promise to make it better the next time you're working on it or something similar.

j7ake

10 months ago

Artisan products are usually tailor made for one person.

So an artisan program would involve many hardcoded variables relevant for a specific user, and hard links that connects files for a specific users computer.

Basically one that has as little abstraction as possible. The end product would look like a script.

ok_dad

10 months ago

Is woodworking valid with power tools or is the tool doing the art? What about a CNC in the workshop? I think it’s all valid because the art is in the planning and resulting furniture. Same thing for coding. Even an LLM needs human creative input.

moffkalast

10 months ago

It depends purely on the level of snobbery you want to strive for.

slowhadoken

10 months ago

It’s not snobbery, it’s knowledge and craft. You depend on the transfer of that knowledge from one generation to the next.

fragmede

10 months ago

Except we don't, because technology advances and techniques get lost because they're no longer useful. I have no idea how to make a buggy whip or drive a horse to market, because we don't do that anymore. I'm sure someone can find a manufacturer that survived the downturn but a lot of that knowledge has simply been lost to the sands of time.

Fortunately we have YouTube University and (for now) the Internet Archive, where hopefully current techniques won't be lost, but we'll just have to see how the future goes.

The privilege to use old timey techniques and then sell furniture on the basis that no power tools were used in the making of it is practically the definition of snobby, as if using power tools to make furniture lead to bad energy being imbued in the table that makes it unsuitable for being used as a table. You don't have to buy any furniture if you don't want it, especially if it's poorly made, but power tools don't automatically mean it's poorly made. You can make really shite furniture with hand tools just as much as with power tools.

slowhadoken

10 months ago

Knowledge transfer is a serious thing in important companies. Technology advances but it’s based on the same logical and mathematical principals. Techniques aren’t lost, they’re warehoused. I’ve rewritten advanced data structures that were purposed in the 1960’s, successfully applied in the 1990’s, and they’re still relevant and cutting edge today.

ok_dad

10 months ago

I hate saying that anything is no longer useful, but the commenter above has a point about how tech will get lost as it becomes outdated and useless. Your example, for instance, shows that those data structures were useful, because they were warehoused and saved in some way. You don't see people using a lot of magnetic core memory these days, even in space. Using a handsaw is still useful today, even for non-snobby furniture, because there are some cuts that a power tool cannot replicate accurately. A hand plane is still valuable in the workshop for many things. One thing you see less and less of today is a hand drill, however, because they are objectively less good at drilling accurate holes than a power drill of some type. I wouldn't call the users of hand drills snobs, because some people prefer hand tools for many reasons, such as being quieter, but they are definitely not using a hand drill to gain an advantage over power tools.

slowhadoken

10 months ago

I get that hardware changes but some of the most impressive projects I’ve seen involve the programmer understanding the hardware of the day. There are young programmers right now writing to the PlayStation 1. And again the principals of space and time complexity don’t change. Dijkstra wrote his shortest path finding algorithm in 1956 and it’s still useful today. But the idea of irrelevance still continues to throw fear into good programmers.

ok_dad

10 months ago

Well, I guess I was coming from a position that hobbies can be as irrelevant as you want and use whatever age of tech you want, whereas if you're doing this for a living you generally want to use the tech of the day. Sometimes this tech is old stuff that survives and thrives to this day, especially with CS concepts that won't really age (like data structures for the most part), however there is some stuff that just is not efficient enough to use for making money, sadly, since that's mostly about efficiency these days and not usually about doing things perfectly right, or even to any decent quality standard, sadly.

I, personally, use old and/or shitty equipment and techniques when I do woodworking, for example, like using chisels to make mortise and tenons rather than a fancy plunge router, or just skipping those and using one of those fancy Festool machines that do the "fancy biscuits".

When I program, I mostly use Python these days (don't judge me!) and I tend not to use third party extensions or libraries, so I guess you could say I use the "old school" technique of using the standard library for Python for everything, where if I were trying to make money I would be using `poetry add` for things like requests, flask, etc.

I think irrelevant and out of date tech can be used for making money, but it's a bit harder that way. I personally like using the "old ways" quite often in my personal and professional lives, and not trying to optimize my personal or professional life as much as most do, but I can see that I'm not the norm there, and most people just memory-hole old tech and move on to the next house of cards.

slowhadoken

10 months ago

You know what an artisan is though. There are varying degrees of artisans and craftsmen. Just extend that metaphor to programming. Layers of abstraction aren’t infinite. I’d say metal on metal is a master craftsman in terms of programming.

foobarbecue

10 months ago

I would like to thank this website for reminding me to drink water.

slowhadoken

10 months ago

That’s what I’ve been working towards, even despite recent trends.

boredtofears

10 months ago

Ive always thought that software is a lot more of a creative expression than we give it credit for. As much as we try to think of it as an engineering discipline, code exists primarily to communicate to other humans before machine. There’s a lot of humanity in writing and reading code and therefore a lot of ways to observe art in it.

cglan

10 months ago

It definitely fulfills the same itch as painting or photography for me weirdly. But that goes away the more “corporate” and tedious it is. But luckily my job gives me a lot of leeway. It can be and largely is a creative field I’d say. There’s room for traditional and formal engineering in it, but I don’t think that’s the majority

lifeisstillgood

10 months ago

Agreed (in fact my profile claims I do artisanal software :-)

But I go further - software is a form of literacy, and everyone should learn to code, just as at some point we thought “hey laws are written down, novels are great, policy manuals and international letters help, let’s teach the peasents to read” and all of a sudden we have working class people going to university and discovering things like Relativity and Covid vaccines.

Software is literacy and we will all benefit when we can run / examine our society through it.

Book to follow

avaldez_

10 months ago

> But I go further - software is a form of literacy, and everyone should learn to code,

Why though? I've read a lot of people saying _everyone should know how to code_ but why? I don't see the same trend for example with physics, maths or music, which are arguably a form of literacy too. As cool as it may seem on paper, I can imagine everyone being able to write papers on topological quantum field theory, studying homotopy type theory as a hobby and shipping react to-do apps while playing Beethoven's Hammerklavier sonata.

lifeisstillgood

10 months ago

But why not? Not so long ago we did not teach the poor / women / slaves to read because similar reasons .

One day we hope there will be sufficient wealth created in society, and distributed fairly so that toil is eliminated and all humans can choose to work - at which point what the hell is everyone going to do other than sit around and play guitar