Oh wow, this is truely sad news.

I only recently went down the homelab/selfhosted path and the majority of my containers were setup using tteck's scripts.

Careful. I too thought about this, but docker containers have the following benefits over the LXC scripts:

- Updates and automatic upgrades between major versions.

- The developer who wrote the software created the container (most of the time), this means its a supported environment. Also, as they have the insight into the application and future upgrades the environment has been setup correctly for each version.

If you want to achieve your goal, I'd suggest an LXC with your favourite Linux distro + docker + app container(s) for each app you have. It gives you the same thing, but with the benefits above.

It is also worth mentioning that Proxmox uses ZFS making snapshotting quick and Proxmox also has a very good backup system.

If you want to treat your self-hosted applications as "sheep" (1) , then terraform k8s etc. is a better bet.

But if you are happy to manually restore from a backup or snapshot when something goes wrong, or automatically have your LXC container shifted to different hardware if you have a cluster, then Proxmox is for you. The reality is that in a home setup you will spend about as much or less time maintaining your "pets" than than you would your "farm".

(1) I write this from New Zealand

Proxmox is more about the management of the hosts and resources on them, including the live migration of VMs between hosts, support for some types of HA and failover.

You can likely manage the configuration of the VMs themselves through terraform or similar in combination with Proxmox if that's your desire.

proxmox is using KVM for virtualization and Linux Containers (LXC) for the containers. I agree that something like terraform and/or ansible would make more sense for an IAC (infrastructure as code) deployment. Most of the people I talk to that use proxmox for a homelab prefer to do things manually and don't bother with any IAC implementation.

For work I'm a firm believer in reproducible environments and IAC. We actually a combination of vagrant, libvirt, and KVM to spin up local clusters for quick testing and development. It works out pretty well, but in my homelab I don't have anything complicated enough to bother setting up terraform/ansible for. Although I imagine if my server crashed I probably wouldn't think that way anymore.

You can split the difference with the Proxmox provider for Terraform[1]. The workflow would be:

- provision VMs with Terraform - configure/maintain your VM with something like Ansible

The provider also allows your to schedule LXC if you'd like to target that instead.

[1]: https://github.com/Telmate/terraform-provider-proxmox

No it isn't basic and common (it is for me but perhaps not for you and certainly not for most people)

OK, so you want to virtualise a router and firewall. That's fine. I have deployed roughly 200 pfSense firewall/routers as VMs and physical boxes and OPNSense is similar, so I can probably help.

At a minimum you will need two physical interfaces (one will actually do but you will need to know what you are doing!). You need "WAN" and "LAN". OPNSense is still FreeBSD based, I think, so it will not run in a L[inux]XC container for obvious reasons.

Your last paragraph seems rather confused. I don't know what you mean by "glue scripts". VMs communicate via networks

I suggest you try a few experiments to get to grips with virtualisation properly and then move on from there. If you swing by the Proxmox forums with specific issues we'll try to help out but in the end you need to dive in full on ... or not.

I run proxmox and have set up VLANs.

The router port to the proxmox machine is set up for tagged packets that isolate incoming/outgoing traffic.

After that my VMs and Containers are easily set up to "live" on one or more networks.

For me the firewall rules on the router determine what traffic can be relayed between vlans through the router.

I'm pretty sure you could set up opnsense running in a container or vm to do the same thing, selectively passing traffic from one vlan to another.

There is no OPNSense script I think historically in part because any misconfig could expose the Promox instance to the world. It is easy enough for advanced users to spin up a VM with the ISO. There has been a request for a OPNSense script made recently.

I agree with OMV. It certainly can be used as is, but not usually how people want to use it. A note was added to the script a few days ago.

> I don't see any glue scripts to get VMs talking to each other

There is a Tailscale script which technically helps them talk to each other (over Tailscale) :)

The scripts are designed to setup self contained LCX containers. We are trying to avoid building our own k8s.

I use Proxmox with an OPNSense VM and have multiple NICs - one is dedicated to the fibre ONT. I also use an external wifi mesh. I have a couple of other vms (unRaid hosting Dockers with sata card passthrough for legacy reasons and a vm for Home Assistant OS) and lots of other LXCs. It works superbly.

> I don't see any glue scripts to get VMs talking to each other

I'm confused by what you mean here? Don't they just use the network like any other computer?

I haven't had to do any special configuration to get my VMs to talk to each other.

HAOS as a VM on proxmox works well.

I used some of tteck's helper scripts to set up mqtt and zigbee2mqtt LXC containers with a passthrough of the USB zigbee device.

The scripts for both these projects work very well. I would recommend Home Assistant HAOS in a VM over a LXC or docker.

It's certainly a different model of deployment. I like it, though it does have its warts.

However there is a (community) TF module...? https://registry.terraform.io/providers/Telmate/proxmox/late... (I have no experience with it as I typically reach for Ansible).

Also, easy-to-install ZFS makes it hard for me to cajol myself into trying something else. And if I want k8s for play time I can always spin up (a/some) VM(s).

I've pretty thoroughly drunk the NixOS Kool-aid.

For awhile I ran Docker Swarm with a bunch of SBCs, then k8s, then just a big server running Ubuntu + Cockpit, then Proxmox, until I have finally settled on NixOS.

NixOS has decent container support if necessary, but I've found that its declarative nature means I almost never bother with containers. "Uninstalling" something is generally as simple as "remove it from the config file, rebuild", and it's not hard to do cgroupey stuff if you need to manage memory and the like.

Not to mention that I think NixOS's nginx DSL is wonderful. It's so nice being able to have my proxy configs (along with LetsEncrypt) managed directly (and correctly) by the config environment instead of me writing my own scripts and the like.

(I'm not sure if there are any distributed NixOS things, because I could totally see something neat being built on Flakes)

My homelab has never been simpler and I've never been happier with it.

I've used this[1] Terraform provider together with the Talos[2] distribution for deploying a Kubernetes cluster. I agree that the APIs available with Proxmox are not fully featured, but it more than suits my needs.

I'm running a four node cluster on salvaged SFF machines backing up lvm snapshots to home brewed TruNAS storage and it all makes me happy.

----

[1] https://github.com/Telmate/terraform-provider-proxmox

[2] https://factory.talos.dev/

Proxmox doesn’t preclude you from having k8s. You can create VM(s) in Proxmox and then install k8s on them, then run your app workloads in k8s.

You do have to treat Proxmox VMs like “pets, not cattle” since they are more difficult to automate, but that’s the same story as if you were managing your k8s host on bare metal too. The benefit with Proxmox-hosted VMs though is that you can use Proxmox for whole-VM backups and migrations, so you can have the best of both proxmox and k8s!

Different technologies are for different approaches and applications.

It's relatively trivial to use the pve command line utility to create or modify vms in proxmox.

Still, the originating reason of this post is due to a large number of useful scripts to help make things more manageable and maintainable, and the founder of it having to step away, and there being gratitude for their help to make things much more manageable.

But you can just chuck Kubernetes nodes on Proxmox? I have my nodes running on XCP-ng. The beauty of running a hypervisor is maximum flexibility. I can try out different distros etc, either for k8s nodes or otherwise. I run my router on there (opnsense). I can play with stuff like nix and guix and could even install Windows if for some reason I wanted to.

Have you tried incus?

https://linuxcontainers.org/incus/

I have a single VM on my proxmox server that I spin up all my docker containers in... This is the simplest thing I could think of in terms of config. I haven't had to wire containers together, though, so maybe I've found the sweet spot for my deployment needs.

Clustering, migration, high availability, backups, Ceph integration, virtual networks as of recent, can do containers as well as VMs to name a few off-hand. The web interface is optional, too. You should check out their webpage for more.

KVM is just kernel side of things, it's not full vmm by itself, you always need some userspace application too. Firecracker, Qemu, Cloud-Hypervisor are some vmms built on top of kvm.

While qemu is common way of using kvm, but running qemu directly is quite annoying. So you have stuff like libvirt and proxmox as wrappers around qemu.

Proxmox runs on top of Debian. The Debian is part of the install but I think you could install the packages separately on an existing Debian install. You could even install cockpit if you want to.

I wanted to delegate management of my raid array to higher level tools since it died on me seemingly for nothing (I was able to recover all the drives but none of the files).

I tried TrueNAS but it's very rigid. Proxmox seems to give you more control over what's installed on the server but it's also quite locked down. Don't remember exactly what was it that pushed me off Proxmox. I think it was that I needed to manage some VMs over LXD API and others over proxmox and I couldn't mix and match, I had to choose one without extra hacks.

Yes, installing Proxmox is akin to installing ESXi.

Usually, proxmox is the base OS (meaing replaces whatever Linux/Windows/FreeBSD/etc that there was before). It is possible to run Proxmox inside KVM, but that isn't the usual choice.

Monitoring is there in Proxmox

At the risk of leaking info to any toes I step on w.r.t my home environment,

- home assistant - Network Video Recorder - Jellyfin - network management such as ubiquiti or omada etc - vault warden/1pass/other secrets servers - tailscale or wire guard server - build server/k8s test environment - private artifactory or mirror (especially useful if you're using the same distro on a bunch of devices but don't want to overload the actual mirror+improves download times) - torrents (someone's gotta seed Wikipedia) - onsite backups - bastion into your home network (see also: wire guard) - some people even use it for their router

You could also take a look at the tteck scripts, there's a bunch of cool stuff in there

I have Proxmox on my older (11th gen) Intel NUC - it runs my Unifi network controller for my WiFi APs, an Unbound DNS cache (because my router doesn't support DNS over TLS or DNS over HTTPS), a NAS (using ZFS on some hard drives in a USB 3.2 Gen 2, 6-drive disk chassis - the ZFS managed directly by Proxmox and then a Debian container just doing Samba).

That's all for now but I've just installed Home Assistant but haven't set that up yet. I also intend to try out Jellyfin as a media server and Frigate as a video recorder when I get some cheap cameras.

I started with a home lab a few months back. Its basically an old miniPC running Proxmox virtualization and LCX containers. Mostly it helps me learn various technology I have not much experience with in my work. I run about 20+ services on the home lab. Some of them include:

Open WebUI which can connect to the OpenAI api or a local Ollama LLM. You can also connect various tools to the LLM like a calculator or web search to augment them. The AI has helped me learn how to configure and debug stuff. Like I got step-ca to setup a local certificate authority and give certificates to my various internal services. I played around with configuring Caddy and Nginx along with ACME to the the step-ca. The LLM was even helping be debug my config files.

I'm also using Hoarder for bookmarking and it can use AI to automatically tag your bookmarks. It can even backup the webpages.

I've been using Mealie to clip and save online recipes.

I'm running Uptime Kuma to check if my computers and services are up and if they are down, I'll get a notification.

Also just started out few months ago:

- home assistant (just a few currently, more soon) - paperless:absolutely awesome document management system - immich: image management with automatic synchro of my mobile taken images (ML features) - tailscale - StirlingPDF: simple tools for all things PDF

- Home Assistant, Mosquitto, Scrypted, Frigate, rtl_433

- Postgres, Maria, Influx, Grafana

- Plex, Arr suite, Transmission

- Ollama, OpenWebUI

- Web change tracker

- Teslamate

Not sure if we're seeing the same site but this site is better than 90% of the web when accessed via mobile for me. No page number when navigating forward/back through scripts, true, but I doubt you want to go through more than 1 or 2 pages without searching first. Search worked for me and turned up what I wanted on the first hit. Descriptions being 3 lines is pretty much the best it can do with the limited screen space; to improve this I feel you would need dedicated summarized content for mobile, which is a price that most platforms don't want to pay. Your description made me think that this was in the bottom 1% of mobile sites but honestly it's above average.