> tend to use FreeBSD because in many ways it is simpler and less surprising, especially when accounting for the passage of time. ifconfig is still ifconfig, and it works great. rc.d is all I need for my own stuff.

That sounds very appealing to me. I have to keep a small number of servers running, but its not my main focus and I would like to spend as little time on it as possible.

I have started using Alpine Linux for servers (not for my desktop, yet) because it is light and simple. Maybe BSD will be the next step.

I completely agree! If you're looking to deeply understand how Linux works under the hood, I highly recommend trying out Linux From Scratch. It gave me invaluable insight into the system, especially when I first explored it 20 years ago. Building everything from the ground up—without relying on prepackaged distros or libc—was a game changer.

Check it out: https://www.linuxfromscratch.org/

I really really enjoyed using FreeBSD. It isn't a distro but a full OS that you can and easily fully rebuild from source. The people are great, the tech is good.

The CON is coming up, https://freebsdfoundation.org/news-and-events/event-calendar...

Fall 2024 FreeBSD Summit November 7-8, 2024 San Jose, CA

Yeah, you know, a lot of us talk about FreeBSD being great because you aren't surprised by design choices—it mostly just makes sense.

But I don't think we talk enough about the joy of not being surprised by updates. I'm about to do an upgrade from 13.2 to 14.1 this weekend and I am very confident that I won't have to worry about configuring a new audio subsystem, changing all my startup scripts to work with a new service manager paradigm, or half my programs suddenly being slow because they're some new quasi-container package format.

What sort of projects did you do to get into network stack and driver development on BSD?

I also still use FreeBSD on my NAS. But after many years, the desktop experience was pretty sad and made me switch to Windows + Linux for my hardware tinkering. On one side, the lack of manpower shows in many places, unfortunately. I'm talking modern WiFi, GPU support, or power-save mechanisms. On the other side, many Open Source projects only support Linux and getting their projects to compile + run on FreeBSD was a pain, too.

I mean, in addition to what kev009 mentioned, FreeBSD has so many great things to offer: For example, a full-featured "ifconfig" instead of ip + ethtool + iwconfig. Or consistent file-system snapshots since like forever on UFS (and ZFS, of course). I never understood how people in a commercial setup could run filesystem-level backups on a machine without that, like on Linux with ext4. It's just asking for trouble.

So, I'm happy to see this thread about FreeBSD here! Maybe we can make the Open Source scene a bit more diverse again with regards to operating systems…

I see the reason for dual stack but I would rather focus my efforts into securing one OS. If you buy into that, why not 3 or 4 different OSes?

> it will probably expand your knowledge

It really just fragments my knowledge to be honest.

Say "I gotta get things done".

Get me to a terminal. You've got Mac OS command line flags, GNU, BSD. Great.

Then it's some kind of asinine config to interact with some piece of software, all to achieve "generally the same thing", just a different way/flavor.

I really don't see the benefits.

> As someone who admins a lot of btrfs, it seems very unlikely that this was unrecoverable.

As someone who used it all day every day in my day job for 4 years, I find it 100% believable.

I am not saying you're wrong: I'm saying, experiences differ widely, and your patterns of use are not be universal.

It's the single most unreliable untrustworthy filesystem I've used in the 21st century.

FWIW, this wasn't always the case. I recall that BTRFS reliability was much different, say, 10–15 years ago. The post touched those ancient times as well, so that isn't that much of a stretch.

Around that time, SLES made btrfs their default filesystem. It caused so many problems for users that they reversed that decision almost immediately.

I was pleased with my home lab btrfs, had a 12TB raid1, and the PSU rail connected to the backplane sometimes would go down under load. Many scary errors but never lost anything. Took me 2 months to debug and replace the PSU

If btrfs knows the data is intact, shouldn't btrfs recover automatically?

Why do people use btrfs and similar filesystems for production use? They are by no means dumpster fires. But the internet is littered with stories of "X happened, then I realized Y & that I wasn't getting my data back"

No, after 20 years of use and comparative testing of similar setups. Frankly, I have always placed little importance on benchmarks as I consider them extremely specific. I am interested in real-world use cases.

The goal of the talk and the article is not to urge people to migrate all their setups, but simply to share my experience and the results achieved. To encourage the use of BSDs for their own purposes as well. It’s not to say that they are the best solution; there is no universal solution to all problems, but having a range of choices can only be positive.

The problem with the IT crowd is they think they are ahead of the optimization curve, and since everyone does, nobody is.

> So what does all the IT optimization bring? Just more wealth for the owners [...] It is time people in IT got to understand this

I understand it alright, but I'm trapped. Closer to 50 than to 40, I've got a family to run. I could be interested in another profession, but our daily lives & savings would tank if I stopped working, for learning another profession. Also, there's no other profession that I could realistically learn that would let me take nearly the same amount of money home every month. If someone lives alone, they could adjust their standard of living (-> downwards, of course); how do you do that for a family?

Furthermore, there is no switchover between "soulless software job for $$$" and "inspiring software job for $". There are only soulless jobs, only the $ varies. Work sucks absolutely everywhere; the only variable is compensation -- at best we can get one that "sucks less".

When I was a teenager, I could have never dreamt that programming would devolve into such a cruel daily grind for me. Mid-life crisis does change how we look at things, doesn't it. We want more meaning to our work (society has extremely decoupled livelihood from meaning), but there's just no way out. Responsibilities, real or imaginary, keep us trapped. I'd love to reboot my professional life, but the risks are extreme.

FWIW, I still appreciate interesting tasks at work; diving into the details lets me forget, at least for a while, how meaningless it all is.

The current generation of Americans are absolutely rich enough to just get food and have shelter. The ones that struggle are the ones that want to live in popular cities precisely because of the available improvements beyond shelter and food.

The houses of the 50s were shit tier and spread around the entire US. You can go buy them today for cheap in the 98% of locations people don’t want to live in.

> I would work for myself, following my own philosophy.

Sounds like he understood it just fine. He owns the means of production.

FreeBSD has pretty good Java support? Sure if there is a new LTS release it takes a couple of months before it is ported to FreeBSD but that's about it?

Thanks for this! I have been using LXC/LXD for a long time and never knew about systemd-nspawn. Time to go learn something new!

I'm interested too. I'm using XFS only, and have for many years. On my own boxes, but my company also uses XFS for all the data on customer computers. We did extensive testing many years back, and XFS was the only filesystem at the time which gave a linear, constantly very high performance when writing and reading huge amounts of data (real-time data, dips in performance is a 100% no-no), and also not degrading when having huge numbers of files. We've never had a customer lose data due to XFS problems, and at this point I can't imagine how much data that would be, except that it's astronomical.

When that's said, we had routine XFS losses on SGI boxes. That was a very well known scenario: Write constantly to a one-page text file, say, every few seconds, then power cycle the machine. The file would be empty afterwards. This doesn't happen on Linux, I vaguely recall discussing this with someone some years ago (maybe here on HN) and something was changed at some point, maybe when SGI migrated XFS to Linux, or shortly after.

It hard to know the timeline with his data loss, but I am assuming it was a long time ago.

XFS is originally from SGI Irix and was designed to run on higher end hardware. SGI donated it to Linux in 1999 and it carried a lot of its assumptions over.

For example on SGI boxes you had "hardware raid" with cache, which essentially is a sort of embedded computer with it's own memory. That cache had a battery backup so that if the machine had a crash or sudden power loss the hardware raid would live on long enough to finish its writes. SGI had tight control over the type of hardware you could use and it was usually good quality stuff.

In the land of commodity PC-based servers this isn't often how it worked. Instead you just had regular IDE or SATA hard drives. And those drives lied.

On cheap hardware the firmware would report back it had finished writes when in fact it didn't because it made it seem faster in benchmarks. And consumers/enterprise types looking to save money with Linux mostly bought whatever is the cheapest and fastest looking on benchmarks.

So that if there was a hardware failure or sudden power loss there would could be several megs of writes that were still in flight when the file system thought they were safely written to disk.

That meant there was a distinct chance of dataloss when it came to using Linux and XFS early on.

I experienced problems like that in early 2000s era Linux XFS.

This was always a big benefit to sticking with Ext4. It is kinda dumb luck that Ext4 is as fast as it is when it came to hosting databases, but the real reason to use it is because it had a lot of robust recovery tools. It was designed from the ground up with the assumption that you were using the cheapest crappiest hardware you can buy (Personal PCs).

However modern XFS is a completely different beast. It has been rewritten extensively and improved massively over what was originally ported over from SGI.

It is different enough that a guy's experience with it from 2005 or 2010 isn't really meaningful.

I have zero real technical knowledge on file systems except as a end-user, but from what I understand FreeBSD uses UFS that uses a "WAL" or "write ahead log".. where it records writes it is going to do before it does it. I think this is a simpler but more robust solution then the sort of journalling that XFS or Ext4 uses. The trade off is lower performance.

As far as ZFS vs Btrfs... I really like to avoid Btrfs as much as possible. A number of distros use it by default (OpenSUSE, Fedora, etc), but I just format everything as a single partition as Ext4 or XFS on personal stuff. I use it on my personal file server, but it really simple setup with UPS. I don't use ZFS, but I strongly suspect that btrfs simply failed to rise to its level.

One of the reasons Linux persists despite not having something up to the level of ZFS is that most of ZFS features are redundant to larger enterprise customers.

They typically use expensive SAN or more advanced NAS that has proprietary storage solutions that provide ZFS-like features long before ZFS was a thing. So throwing something as complicated as ZFS on top of that really provides no benefit.

Or they use one of Linux clustered file system solutions, of which there is a wide selection.

I had one a few years back where we ran out of inodes on a Jenkins machine on CentOS 7 and it crashed and couldn’t remount the filesystem. I had to restore a backup which was time consuming on a 4TB volume with crazy amounts of files.

Used it since the late 90s on IRIX, think there were a few issues early on with the endian swap, but no issues for the best part of twenty years for me!

Indeed, most of our public-facing services are hosted on OpenBSD, and all of our routers and firewalls run it. We started managing everything with Ansible to make it easier to ignore what the host OS is for a deployment, and that has worked well both in moving things to [Open,Net]BSD for experiments, and also standing up tests on various Linux distros just to make sure we're not running into a BSD vs. GNU issue, or even a "problem only on this specific distro" issue.

#1 reason we chose Ansible over other tools was support for the BSDs.

It's what I use at home and at work.

I've helped build two top 10 service provider networks (10s of Tbps). One on FreeBSD, and one on Linux with Kubernetes.

I don't really see Kubernetes as being a game changer. The biggest pro, it makes it easier to onboard both development and operations personnel having a quasi-standard for how a lot of things like scheduling and application networking work.

But it also seems to come with a magnitude of accidental and ornamental complexity. I would imply the same about microservices versus, say, figuring out your repository, language, and deployment pipelines to provide a smooth developer and operator experience. Too much of this industry is fashion and navel gazing instead of thinking about the core problems and standing behind a methodology that works for the business. Unless google moves its own infrastructure to Kubernetes, then maybe there's something to be had that couldn't reasonably be done otherwise :)

Same here, otherwise I'd be considering the BSDs.

Can't you just run k8s on bsd? You might have to build and maintain your own release of it, but I'm sure someone has done it already.

They might be really good for specific tasks, but someone else will also need to maintain the setups, which will make things harder when most of the folks in the job market have experience with the various Linux distros, but finding BSD experience might be a tad more difficult.

I am personally on board with using the various BSDs when it makes sense (though maybe just pick FreeBSD and stick with it, as opposed to fragmenting the install base, the same way how I've settled on Ubuntu LTS wherever possible; it's not ideal but it works), except the thing is that most job ads and such call for Linux experience in particular, same with tooling like Kubernetes and OCI/Docker containers and such. Ergo, that's where most of my time goes, I want to remain employable and also produce solutions that will be understandable to most people, instead of getting my DMs pinged whenever someone is confused by what they're seeing.

https://chimera-linux.org/ is kind of that melding, at least so far as one can go with a vanilla kernel.

> Who thought withering device names was a good idea, so that /dev no longer reflects the state of attached hardware?

Sorry, could you clarify what this means? I'm not super familiar with freebsd and don't understand what withering means here.

There is more than one Linux distribution that's designed to work without systemd.

Not an abuse at all. I've a contract with those clients, and I can move the VMs, change the services, etc. freely as long as it doesn't cost more than the amount we've previously set.

Otherwise, I'd never dare to do something like that.

> Abusing the trust of your customers is an absolute no-no in my book.

How do people on the Internet come to such random conclusions when there is no way you could have known the full terms of the contract between the author and their client?

Abusing trust is a bit strong, customers pay for a service and beyond a certain level of abstraction these obscure technical details (from their perspective) are not their concern. They're paying to have that abstracted.

> Abusing the trust of your customers

Yes. I also always let my customers sign off when I change the libraries I use. Completely sane approach.

How is it an abuse? As long as the customer continues to receive the service they paid for, who cares?

The major providers such as GCP, AWS etc share very few details about their underlying infrastructure with their customers. They change all sorts of things all the time.

I wouldn't call it abuse of trust but it's a bad idea to do a migration or any operation that can fail and cause downtime without warning the clients. Come Monday and no servers are online, what do you say, "oops, I tried to change something that didn't work"? that is fine only if they knew there was a migration over the weekend. On my end this situation would fireable offense or close to it.

What!? Changing implementation details is not “abusing trust”. Where would you even draw the line with this attitude!? Should I be informing my customers whenever I update the version of left-pad I have installed!?

I still remember discovering FidoNet sometime in the 90s.

It was a time where sending regular mail to different countries could take weeks, and cross country phonecalls would cost between $2 and $20 per minute, and here was FidoNet that promised to allow communications across the globe with only 1-4 days delay and basically for free.

My 15-18 year old self was instantly sold. I spent countless hours reading through the "forums" on there. So much knowledge just at the tip of my fingers.

Of course some time later it was more or less replaced (for me) by email, usenet and IRC, but the memory still remains.

Ah, pair. Reminds me of this gem:

http://www.archub.org/ycoutage.txt

I used them for shared personal website hosting until the Obama era when they were mostly/all FreeBSD. I moved off around the time AWS came about.

It doesn’t, though? Gcc doesn’t even ship oob, the project itself uses clang, and almost all packages are also compiled with the system c/c++ toolchain.

I'd recommend using FreeBSD as your first BSD. It has a more recent version of ZFS integrated in the kernel than NetBSD. OpenBSD does not have ZFS support; it's a direction they chose not to take for security and simplicity reasons.

TrueNAS

Ok, done. Thanks!

Please don't do this here. Not that we don't appreciate a good old-fashioned flame, but that the long-term costs outweigh the benefits, and we want this site to survive long-term.

I've a contract with those clients, and I can move the VMs, change the services, etc. freely as long as it doesn't cost more than the amount we've previously set.

Otherwise, I'd never dare to do something like that.

And I'm not so crazy as to do such an operation without the appropriate tests and foundations. Of course, when I started, I had all the conditions to be able to do it, and I had already conducted all possible tests. :-)

The client is paying for the VM. The underlying system is an abstraction. As long as service agreements weren’t interrupted I don’t see the problem. It sounds shady to say “without telling them,” because saying so implies they should have. I do a lot of optimizations for my customers without telling them, it’s not usually worth mentioning. I assume what they intended to convey was that this change caused no interruption of service so there was no need to contact or warn the customer.

he gave himself a pat on the back