One thing I'm struggling with at the moment which I may as well ask here.
Many of my projects use pnpm, which ideally uses hard links from a shared store of dependencies.
One of my goals is to reduce the disk overhead of VM based isolation, so I'd like a shared pnpm store across projects (and the same for maven, etc as well)
With https://virtio-fs.gitlab.io/ mounts, each mount inside the VM is a separate FS, and I can't hard link across them. I've tried playing around with ideas like bind mounting the directory structure from the host in a folder to have a single virtio-fs mount inside the VM - but the bind mounts still can't be hard linked across it seems.
I'm basically wondering if I'm missing something or just trying to do something that isn't possible.
All the host folders are on the same FS - but I don't want to organise the host system around the VM mounting needs, and I also don't want to mount extraneous directories into the VM (eg: simple solution could be just mount ~/projects but kinda self defeating)
(Fedora workstation host)
Use a host fs that supports CoW.
add me as well. I learned about incus from HN ~a month ago, and was poking an idea of having a "development base image".
The advantage compared to containers, is that you can hot-plug devices to the running instance: mounts, secrets, etc.
incus also supports --vm and container mode with minimal difference in cli commands.
https://linuxcontainers.org/incus/
my main bottleneck was exactly in usability: how to make it least effort to spin up per use case and keep the set of operating scripts small - i didn't want to build an app for this, just shortcuts to use existing app.
> Mines based on libvirt and uses cloud-init for creating the images, with shared golden images across projects.
This sounds good. Cloud-init is still worth the time, I think (as I just said in another comment)
Add me to the list, although I use podman containers (https://github.com/evertheylen/probox). I find that the hard part is not restricting the container/VM, but to make it "easy" to do things in the container that require something protected. How will you commit? Where does your agent run, and can it access github? How will you deploy? How will you SSH to a server?
Happy to hear what other people are thinking. It seems like putting an intercepting HTTP proxy is getting popular in other projects.
It feels like a fundamental problem. The more you restrict the agents, the less useful they become. The less you restrict them, the more dangerous they become.
It makes sense, but a good system where you can say "give this VM access to repo X, branch Y, and server Z for the next 4 hours" without leaking any secrets into the VM would be a major improvement already.
Human coders and sys admins routinely make catastrophic mistakes, too, so it's not like we're facing a novel problem.
On the git access part, I don't let my agent VMs have write access to GitHub currently. I do give them dedicated users on a self hosted gitea instance, and basically manage the blast radius through branch protection etc.
I think at a high level I'm trying to treat them as junior/intern employees where I give them enough permission to do their role, but not enough rope to hang themselves.
I see. But then I have to debug some failing CI pipeline and the agent is quite good at going through the logs, so I give in and `gh auth login` anyway...
I'm pretty sure you don't need to give it full access to accomplish that. It's possible to create scoped PAT etc. alternatively use a separate account. Ultimately your risk tolerance to decide
You can have a separate GitHub account for your agent (GitHub ToS actually allows that since April) and that account can fork your repo and open PRs. This way it cannot do any real damage but can still push.
See this user for example:
https://github.com/filippo-agent
> GitHub ToS actually allows that since April
Do you have more background on this? My team actually ran into some issues with GH when we tried this earlier this year (felt like it was around that same time).
https://docs.github.com/en/site-policy/github-terms/github-t...
3. Account Requirements
We have a few simple rules for Accounts on GitHub's Service.
You must be a human to create an Account. Accounts registered by "bots" or other automated methods are not permitted. We do permit machine accounts:
A machine account is an Account set up by an individual human who accepts the Terms on behalf of the Account, provides a valid email address, and is responsible for its actions. A machine account is used exclusively for performing automated tasks. Multiple users may direct the actions of a machine account, but the owner of the Account is ultimately responsible for the machine's actions. You may maintain no more than one free machine account in addition to your free Personal Account.
One person or legal entity may maintain no more than one free Account (if you choose to control a machine account as well, that's fine, but it can only be used for running a machine).
Also been working on something similar, with qemu/kvm; the idea is to sandbox agents and limit resource usage per project. Its not production-ready yet, but the repo is public - https://github.com/blackshieldpt/catchpen
Add me to the list lmao. It seems like this is pretty popular among folks
Ha, add me to the list. I built one with libvirt and cloud-init too, mostly focused on network containment (private/host networks are blocked) and VM hardening. There's also a check command that runs nmap from inside the VM to verify nothing private is reachable.
https://github.com/ertug/kvm-sandbox
Heh, yeah also very similar. I don't strip devices like you are - I actually want the memory balloon functionality for example.
My working threat model for my projects is less about "malicious agent trys to escape VM" and more "well meaning agent got confused and screwed up" - the added protection of a VM is desirable, and it's useful for stuff like nested virtualization (agent working on improving the VM orchestration), makes containers inside the sandbox straightforward
I hadn't seen the stuff about filtering spoofed packets so I'll probably add that.