rando103747202
4 days ago
LFS takes the wrong approach in my opinion. I did it once manually over a couple of days. I suppose I hoped enduring the pain of waiting for the packages to compile would somehow make me smarter. It didn’t. I didn’t learn much. It was like following a baking recipe. I know I need to add eggs and sugar in the ratio it says but I’d be damned if I could tell you why that specific ratio works.
It’s just too much to take in at once. You start by building a cross compilation toolchain but… why? And you add the bootloader dead last, even though it’s the first thing you see when you boot a Linux machine?
I reckon a better approach would be to build it step by step, only adding new things once you’ve discovered a need for them. First you get the bootloader working, then you get it to actually load the initramfs and kernel, etc… you could even create your own extremely simple initramfs first before adding dracut, etc…
This gets you to a system you can boot and play with asap. And then you can slowly start to discover why you actually need all these things through exploration and experimentation.
degaart
4 days ago
I'd like to counter your comment.
LFS helped me a lot. It taught me how to compile packages manually, and how GNU/Linux components fit together.
Armed with that knowledge, I could do things others could not do:
- Make applications that bundle/automatically download their dependencies, compile them with custom flags (static linking)
- Diagnose build errors buried in deep dependencies when using vcpkg
- Compile programs for esotheric systems: e.g: Intel MacOS X 10.5, an obsolete system noone else on earth is using except the company I worked at
- Create application targetting obsolete systems but using the latest version of its dependencies
- Compile recent programs on esotheric systems, by bootstraping a newer gcc on the newer system, patching as necessary
- Create my own stripped down distro, tuned to my specifications
- Use that stripped distro as a docker container
- Run recent linux on obsolete hardware no longer supported by mainstream distros
- Reformat a cloud dedicated server without physical access by first bootstraping a minimal distro on a ramdisk, unmounting the main drive, partitioning that drive, and installing the new distro on that drive. Reboot and pray.
- Build a cross-compiler for targetting a freestanding environment
- Build latest KDE/XFCE for debian stable. No more "but debian ships with outdated software!!11"
- Build applications on a distro they were not designed for (Zimbra OSE on debian)
pjmlp
4 days ago
Learning how to compile packages manually it was a common skill long before Linux came to be, and how GNU/Linux fits together could also easily be learnt with Slackware or Yggdrasil.
Intermernet
4 days ago
As someone who started using Linux in 1996 (Slackware, of course), I learnt the hard way about compiling packages. I remember trying to compile the kernel, and doing "make config" and following hundreds of options, and not knowing that I then needed to do "make" or "make zimage" or "make install" after any of the config stuff. Obvious, dumb, rookie errors, but still understandable for a teenager coming from Windows. Doing LFS really taught me a lot about dependencies and complexity, as well as understanding of the actual boot process. All of this helped me greatly in later years.
abhiyerra
3 days ago
Your experience sounds exactly like mine. I downloaded Linux 2.6 which took 2 days to compile. Then when I installed it on my Slackware system completely hosed it. But learned so much…
fuzztester
4 days ago
>Learning how to compile packages manually it was a common skill long before Linux came to be
Right. I used to compile many open source packages from source on many flavors of Unix back in the day, for years. Perl, Python, MySQL, many others. Miscellaneous utilities. Esoteric languages. Even Oracle, once, on a Unix server, right at a large manufacturing company site. Even building Unix kernels from source was common on some Unix versions. And not just for a regular install. Many times, (re)building Unix kernels from source, was the only way to get some software packages or hardware devices or peripherals, like printers or terminal controllers or even hard disk controllers, to work on the system. Typically you would edit some config files or settings files manually, change values of params, or in some cases use a TUI to do that, and then rebuild via a makefile. You had to save the old kernel in case the new one didn't work, and if so, lather, rinse, repeat, till everything did work.
And that practice was nothing special. A lot if people did it routinely. Most, in fact, I'd say. Talking about devs and sysadmins here, not end users, although some of the latter probably did it too.
Many apps, tools and utilities came as source code, often in a .tar or .tar.gz file. You would untar or gunzip it into a new directory, and do your stuff, running makefiles, other shell commands, setting env. vars., etc.
user
3 days ago
immmmmm
4 days ago
I started with Slackware 1.0 on i386.
I certainly do not miss compiling X11!
mobilio
3 days ago
For me also helps me to cross compile Nvidia kernel drivers for other version of kernel than running one. Plus compile stripped-down version of X.
All this for running on embedded Linux with 32MB flash drive running from read-only volume powering two kiosk displays.
motbus3
3 days ago
I agree with you. Maybe the person from the first comment read first few pages only which basically give some context.
shevy-java
4 days ago
None of that requires LFS; I compiled KDE via scripts many years ago. Why would LFS be needed for that? That makes no sense. Debian being slow is their problem, other distributions aren't snail-like like that. The default compiler used in LFS is also hardly a real cross-compiler, it is more a bare-minimum compiler for C/C++. This annoys me because e. g. I need fortran for R, so a bare-minimum compiler isn't as useful as a more feature-complete GCC. And GCC also causes some issues, e. g. like my problem with options.cc.
I think you missed the main argument though - LFS isn't quite as good as it could be. That is a correct statement that was made.
smitty1e
4 days ago
"Requires"/"needed".
LFS is about the muscle memory; the documentation; the cohesiveness.
I always recommend it to people and consider it foundational to my understanding of How Stuff Works.
Whether that amounts to a hill of beans in the AI world remains to be seen.
charcircuit
4 days ago
It is more how stuff is compiled than how stuff works.
Fundamental things like how processes are scheduled are not explained by LFS.
gchamonlive
4 days ago
"works" is a loaded term. LFS is how the rubber meets the road, you want how the rubber is made, for that you need something like https://www.baeldung.com/linux/processes-guide or https://linux-kernel-labs.github.io/refs/heads/master/lectur...
newswasboring
3 days ago
And if you want to get to at least the bottom most visible turtle you go with something like NAND to Tetris https://www.nand2tetris.org/
charcircuit
3 days ago
>LFS is how the rubber meets the road
Again I disagree. Way too much time is spent building software than actual practical OS development.
mid-kid
3 days ago
LFS is not OS development. You build a distribution by assembling all the components. If you want to develop an OS from scratch, perhaps you're looking for osdev.org.
charcircuit
3 days ago
What is the point of making a distro, if not to make an OS? It's the same thing.
gchamonlive
3 days ago
One supports the other, they aren't laid in opposition, but they are different, well-defined disciplines
smitty1e
3 days ago
OK: "How *nix tools functioned together in command-line days".
wazdra
4 days ago
> It was like following a baking recipe. I know I need to add eggs and sugar in the ratio it says but I’d be damned if I could tell you why that specific ratio works.
I know a french cookbook[0] which contains not only recipes for dishes, but also for some of the intermediary ingredients, like stocks, that can now be bought at the supermarket. I feel that I’ve learned a lot by following those recipes, be it simply in the structure of french cuisine. And to me, LFS is a cookbook in a similar sense.
[0]: https://www.ferrandi-paris.com/fr/le-grand-cours-de-cuisine-... in French, sorry!
fransje26
3 days ago
Tangential, but how good/useful/approachable is the book you cited as example?
How would it compare, in your experience, to other (modern) cooking books with a similar scope?
binarin
4 days ago
I've learned the most from LFS long time ago, when I had no internet. I had a mismatching set of Slackware and redhat source CDs, and LFS itself didn't match the versions on CDs too. Since then I can build anything and anywhere, even if it's not cooperating :)
tosti
4 days ago
> And you add the bootloader dead last, even though it’s the first thing you see when you boot a Linux machine?
That actually makes perfect sense. If you did the bootloader first, how are you going to build it? How do you know what to boot from which filesystem on what device? By the time you've got your root filesystem complete, it's easy to tell.
If you do things step by step incrementally, you keep having to go further back in the process. Starting over repeatedly gets tedious, boring and frustrating. And it'll take ages to finish the complete OS.
notme43
4 days ago
Agreed, it's near the end of the Gentoo installation as well. If you forget to install GRUB, just chroot back into it and continue.
If I installed the bootloader first, I'd have to remember to run dracut after the system was installed. Or I install it last, create the init at the same time and make a habit of it.
aragilar
4 days ago
Debian and Arch are the same, grub at the end.
mrheosuper
4 days ago
You build it on your non-lfs computer. What is the other option?
And you dont need to boot anything, just a busy loop after a log "Hello World" is enough, if you feel fancy, maybe make a simple shell ?
tosti
4 days ago
You could do that, but if that were to happen to me I'd say the installer sucks and my boss should hire someone else ;)
shevy-java
4 days ago
Except that if you fail to install the bootloader - which happened to me many times - then you do not have a working system. So really the prior argument makes sense.
IMO the guy is right - LFS could be way better. But it has this current structure and does not deviate from it. That's the bad part.
> If you do things step by step incrementally, you keep having to go further back in the process. Starting over repeatedly gets tedious, boring and frustrating. And it'll take ages to finish the complete OS.
Pray tell how that is necessary for the bootloader? The chain used in LFS isn't the only one that makes sense or is required. IMO it would indeed be better to have a system you boot into as quickly as possible, on real hardware, then continue from there.
tosti
4 days ago
If you're going to run the OS, you have to make sure it compiles and runs the bootloader and its installer. The toolchain could be different enough for that to fail. And you do boot into a reasonably small OS and continue from there. That's what BLFS is all about.
cogman10
3 days ago
TBH, though, the only lesson you learn from BLFS is that value of package managers.
If you like the idea of everything being purpose built for your computer, then after you do LFS you should switch to gentoo.
necovek
4 days ago
As pointed out already, LFS is a cookbook, and you learn most by having an idea about how you want to tune your Linux system.
So at points you want to dive deeper, you do — and go off the beaten path.
It requires a desire not just to understand how bits and pieces fit together, but how you can make them work even better for yourself and what is and is not customizable?
Arguably, even if building it today is easier — back when I did it in 2000, you sometimes struggled to find where the canonical source code of the package is, not to mention compilation errors, linking errors, etc. Only then did you get to configuring the init RC system, choosing between them (no systemd yet), etc.
But a requirement is — IMO — that you are strongly opinionated about how a system should serve you, and look for opportunities to tune it.
zsoltkacsandi
4 days ago
> LFS takes the wrong approach in my opinion.
Shameless plug: I started a series of blog posts exactly because this. https://serversfor.dev/linux-inside-out/
Unfortunately due to some cervical spine issues and a surgery I wasn't able to finish it, but I am planning to pick it up and continue.
viralpoetry
4 days ago
Thanks for posting this, I hope you will continue
sixtyj
4 days ago
LFS is known but it is really more a cookbook than explanatory principles.
Principles can be studied from “Operating Systems: Three Easy Pieces“ book - https://pages.cs.wisc.edu/~remzi/OSTEP/
Rob Love wrote the book “Linux Kernel Development”. https://rlove.org/
And then continue with reading LFS.
dark-star
4 days ago
I used CLFS (Cross-Linux From Scratch) once to bootstrap Linux on my SGI Indy, and I learned quite a lot from this. That was back in 2008 or so, so way before AI and extensive online guides for almost everything. It taught me how cross compilers work, what dependencies exist in the GNU toolchain, how the SGI boots, and much more.
Too bad Cross-LFS is no longer a thing. Then again, with all these microcontrollers and ARM and RISC-V systems today, cross-compiling is no longer as magic (and tricky) as it was back then...
jan_m_savage
4 days ago
This should be said more often, and in many other contexts as well.
Most educational resources are what- and how-based, only the best ones are primarily why-based, what Wirth once wrote in the preface to some Pascal book (I paraphrase) 'discussions are motivated, so that the learner gets coverage and understanding, instead of memorization'.
IslandRebel
4 days ago
You need to actually read the book and not just copy and paste commands. Many of the questions you have are answered in the book.
numlock86
4 days ago
> I did it once manually over a couple of days. I suppose I hoped enduring the pain of waiting for the packages to compile would somehow make me smarter. It didn’t. I didn’t learn much.
Exactly my experience. Back in the day when all I knew was Windows and a couple of BSDs I wanted to get into Linux, because it seemed like it was going to become very relevant in the next decades. After years of slacking I thought LFS would probably be the ideal choice. Turned out it wasn't, at least for me. I ended up "learning Linux" with distros like Gentoo, Arch and later even Ubuntu, and by just replacing my production systems with Linux, which were still running on OpenBSD mostly. I have revisited LFS a couple of years ago and honestly didn't take much from it. I get the appeal, but it's not for everyone.
zvmaz
3 days ago
Exactly my experience. I tried LFS several times, but I didn't learn much. I learned more with Arch Linux, but I truly appreciate the work that was put in LFS, marvelous for the right type of learner.
sylware
4 days ago
The step by step is the right approach... but you'll discover that many 'components' are _INSANE_ and that includes their SDK.
Bare LFS is not enough (I run my own): you need a kind of userland (above glibc) multi-version system which some kind of "atomic-ish" switching (always have a stable SSH running in case something goes wrong, better than unplugging the system disk and fix it on another computer).
For linux, same thing, with a kind of flip-flop-ing for updates/fix/etc.
The insight that gives you will probably scare you: the current "open source" stack is an abomination (the worst is the SDKs I think).
That's why we need _LEAN_ open source, and that includes the SDKs.
tosti
4 days ago
If you think that's an abomination, try corporate billed-by-the-hour enterprise software where nothing ever changes unless it's approved by several layers of management.
I promised myself not to work for any of those anymore :)
sylware
3 days ago
Well, in this very case, it is more complicated than that: they have to protect their software against planned obsolescence and developer tantrum (it would be the same for any "in production software").
Modifications and changes must be weighted very carefully. For instance "removing" (including in the SDK) is usually a less worse modifications than the others and it decreases the global technical size and complexity/number of layers of the stack/SDK (which is top priority in "security"), namely most of the time but not everytime a good thing.
Because, on most complex software (open source or not), a "believed" benign modification can be disastrous. It is even worse if close to bare metal involving complex hardware (even with massive QA[testing]).
tosti
2 days ago
Free software doesn't typically have an SDK. A compiler is just one of the available programs. So is a text editor, a shell and script interpreters, some incarnation of make, etc. There's nothing really special about the compiler and you can write perfectly fine programs without ever touching it.
That said, port trees usually have devel-* sections. Does that count?
sylware
2 days ago
Ohohoh! The complexity and cost of the various "open source SDKs" is on stellar scale. I build my own distro, and I usually remove SDKs to switch to linear orthogonal brutal build shell scripts... and I am going to warn you, you are going to see a lot of ugly close to insane.
You'll see also a good amount of more than questionable code generators. And when you will have a look at some _GNU_ makefiles of big corner-stone projects, you will question the sanity of their authors (the "makefile" itself is often, but not all the time, very questionable too nowdays). Everything was with a good intent, turned sour and painful on the long run.
On our modern computers, we should have "one-compilation-units" for most, if not all binary products (exes or shared libs) anyways. "Same language" binary static linking in the open source world should not be a thing anymore (have a look at those 'header-only' projects).
The invisible backdoor injectors (namely compilers) are a problem, for many reasons that most know here (barrier to implementation of real-life alternatives, [open source] developer lock-in, etc). The more complex is the syntax of a computer language, the worst it is (on c++ syntax complexity scale, or with a heavy/intrusive required runtime like many "new" languages do shove down your throat).
If you don't see any issue, get cproc/qbe C compiler (and the binutils gas assembler), then you compile linux (with a brutal and linear shell script, let's say for a real life x86_64 desktop) and we'll talk again after, ok? And once you succeed, we'll see how it holds in 7-10 years (the time ISO takes to break in a non pertinent way many things).
tosti
a day ago
I reckon what you call SDK is what I'd call the build tools. C++ projects tend to use CMake and that's what I dread most. There have been projects where I brute forced my way on the command line towards succesfully compiling a binary. It's rare, but I kinda know the pain.
If I'd want to roll my own distro again, I'll have a look at an old dir with lots of scripts and patches where I tried doing just that. (I bet almost none of it works with current tarball versions.)
I think I'll retire with NetBSD. Keep up the good work though, someone ought to highlight the insanity of build tools building build tools.
sylware
18 hours ago
In the SDK, I do include the "compiler": the computer language syntax complexity is critical here since it is a barrier of entry of any new "real-life" alternative compiler. NetBSD won't protect you here.
For this major reason, computer languages with a "simple" syntax and "alternative" real-life compilers should be favored (or you know it is reasonable to build one from scratch). The current _LESS WORSE_ alternative is C and similar. But this is far from a silver bullet. I think our only way out, namely getting rid of all this toxicity is assembly without abuse of any macro-preprocessor, that's why worldwide non-IP locked ISA standard like RISC-V are so much important.
tosti
17 hours ago
NetBSD can be compiled from another OS and a compiler can be compiled to shell script. You'd take a minimal compiler (eg lcc), compile it to shell script, run it on the target, have it compile itself, older gcc and work your way up to one that builds the rest of the OS.
If you really want to start over from scratch, some form of assembly is indeed required :) And the first assembler would have to be assembled by hand. I'm not sure if riscv is useful for that, I'd try finding the oldest m68k still serviceable. And the sooner you get to C, the better. Starting from scratch is one thing, doing it all in asm takes more than a lifetime. And from C to various interpreters, of course. Most programs can be written in a scripting language and have fewer bugs than if they were written in more verbose languages.
sylware
10 hours ago
I did not get everything you said, sorry.
But where I do agree: I presented the "assembly saving the world from big tech toxicity" would come with very high level scripting languages (probably a mess... but less toxic than what we have today), but with their interpreters/JIT "compilers" written is such assembly without abuse of any macro pre-processor.
I gave some thoughts about porting Bellard quickjs to pure hand written/tool assisted RISC-V assembly.
tosti
5 hours ago
I'd favor more incremental steps. The sooner you get the specific CPU instructions out of the way the better. Otherwise your liberation party might be short lived.
What I described is an example of build bootstrapping. There's a website about it, https://www.bootstrappable.org/
myaccountonhn
4 days ago
Setting up Oasis <https://github.com/oasislinux/oasis> taught me more than LFS personally. Everything being statically compiled makes it less magic.
shevy-java
4 days ago
To some extent I agree with you. I'd like to decouple some parts, e. g. the tutorial part is great but it should ideally be distinct from "just go at it" 1:1. I also fail to compile various things right now, which is frustrating; two or three years ago all went well. The whole LFS build up is too brittle. In part this is due to external complexity, but LFS needs to re-think what it does. It's also sad that they are now systemd-only, which further adds intrinsic uselessness. I don't need anything systemd offers; all my boot up logic is stored in yaml files and autogenerated to target formats via ruby as-is.
blablabla123
4 days ago
> I did it once manually over a couple of days.
Sounds like a speedrun of the tutorial. How did you approach packaging then and what system did you use to compile everything?
I did the tutorial in the late 2000s I think and it took me 4-6 months if I remember correctly. At the time I wasn't completely new to Linux but setting up things like the boot loader was quite tricky for me.
For me the main motivation was getting a deeper understanding, being able to fine tune and customize my system. Of course I didn't go blindly through the tutorial but only did steps that I understood. (In case something goes wrong) Also I started to use (x)stow for everything.
shevy-java
4 days ago
It is possible to do it over a few days really, even with not too much knowledge.
Requiring four months seems as if it was a side project, which is of course totally fine, but for any main task it really should not take more than a few days, even on a semi-slow computer (well, depends on how slow).
blablabla123
3 days ago
Well as mentioned it was in the 2000s. Compiling KDE alone took several days. Also there was no automation/script, so you really had to go to ftp.gnu.org and look in which folder the coreutils package is ;) I realize there's now a list of packages with checksums
IslandRebel
3 days ago
I did it in a VM took me 20 hours. Most of that time was waiting for things to compile and fixing mistakes I made because I didn't read the instructions properly.
r3trohack3r
4 days ago
I liked KISS for this
It was like automated LFS from git
FreeBSD’s Makefile system is also in this ballpark
nullify88
4 days ago
I learned a lot doing Gentoo stage 1. I was also following the guide like a recipe, but at the time I hadn't been exposed to the underlying machinery that is often automated, abstracted away, or done for me already. It was also a good test on how well I could follow instructions and later on apply my own customisations. Like lego.
HarlequinHair
3 days ago
Except you can write bootloader only if you have something to boot.
Bootloaders as we know them came after OSes, because the need of a bootloader is to boot something. Following your logic of building a piece only if needed, a bootloader should come after.
I like the recipe analogy and I actually think adding things only if needed will help cutting unuseful features and possibly to have different implementations for solutions you need.
No point in reinvent the whole thing, tho. Following your analogy, if all cakes need egg, there must be a reason even if we don't know it.
robotresearcher
3 days ago
You probably want to use a working bootloader to see if your ‘hello world’ OS-fragment works at all. No one is writing much OS without running it, surely.
HarlequinHair
3 days ago
"Linux from scratch" involves Linux, not Libreboot, not Grub (just to mention instances of BIOS/UEFI and bootloader).
We already have means to test the new OS.
There is real point in writing a new one until the OS is complete, is there?
robotresearcher
3 days ago
If you’re satisfied using a third party bootloader, and one exists, there’s no point writing one ever.
convolvatron
3 days ago
in the bad olden times you definitely started to write your operating system for your new computer from the first instruction executed upwards. it wasn't at all common to have a comprehensive virtualization/simulation environment.
HarlequinHair
3 days ago
I can't understand what is the point you are trying to make
convolvatron
3 days ago
until very recently you couldn't even do OS development without a boot loader, so asserting that that would be the last thing you would write is just nonsensical
HarlequinHair
3 days ago
Let me rephrase.
We already have a bootloader.
"Linux from scratch" involves Linux, not Libreboot, not Grub (just to mention instances of BIOS/UEFI and bootloader).
We already have means to test the new OS.
What I meant with my previous post is "newer" bootloader (which are actually pretty old, like grub), came after OSes passed a certain complexity.
Previously we technically had bootloaders, but they were interactive, mostly.
Am I misremembering?
convolvatron
3 days ago
oh. I'm talking about authoring OS and boot loaders from an absolute blank slate. like starting from the world where you toggled them in from switches, and later by burning proms
kaashmonee
4 days ago
I second this a thousand million times. But with AI, i can't help but admit i find myself indulging in building things backwards, now that i can try a couple different approaches simultaneously for it.
ngriffiths
3 days ago
> only adding new things once you’ve discovered a need for them.
Agree! imo this is the guiding principle that makes great tutorials great.
wolfi1
4 days ago
well, if you happen to be in a 'rm -Rf /' situation (which with agentic AI seems to be more often the case) to know how to operate in a chroot environment could be helpful
gigatexal
4 days ago
Compiling the packages doesn’t teach much it’s all the glue in between. No? It’s the startup scripts and the configs and creating a kernel and all the things.
It’s on my list of things to do when I have some free time
self_awareness
4 days ago
You know, you could also just read about each step to learn more. I mean if you feel you're following a baking recipe, you can just... read about it more. And then have LFS as a simple way to experiment if you want to change anything or not. Also, technically you don't need to have a bootloader installed on your main hard drive if you already have it on another disk.
I mean, eh... last time I checked, this was Hacker news. Yesterday, there was an Ask HN topic on how to enable dark mode on HN. Is this the new hacker culture? Complaining that the resources merely point in some direction instead of specifying details?