Disclaimer: I'm a maintainer of the project

Pixie [1] is a similar project and offers the self hosted model you are looking for.

We also support 11 application protocols [2] with TLS handshake tracing and MQTT support coming soon (encrypted traffic tracing has been supported for a long time).

[1] https://px.dev

[2] https://docs.px.dev/reference/datatables/

re self-hosting: yep! Use the -devtools flag to get a stripped down version of Subtrace running entirely locally:

    subtrace run -devtools=/subtrace -- python3 -m http.server

Thanks!

re the Wireshark analogy: the reason I used that was because: (1) Subtrace operates at roughly the same level in the operating system stack, (2) has similar capabilities, (3) has an overlap in use-cases, and (4) has been the most effective at communicating what Subtrace is in my experience so far. I can see why the analogy is not a perfect 1:1 mapping (obligatory xkcd: https://xkcd.com/624), but naming things is hard and taglines are just names in idea space :)

I can already use Wireshark itself with docker containers.

IMO the real UX problem is that there’s nothing in between “Wireshark” and “DevTools.”

What would be cool is a program that renders the real DevTools UI from a .pcap file (or stdin).

There’s a lot of use cases where I can capture all the traffic with Wireshark but I just want a basic DevTools interface.

For now, yes :)

Since we operate at the TCP level, we can actually handle pretty much any protocol. I have an implementation of a postgres handler in my git stash that intercepts and shows the SQL queries executed + the resulting rows alongside the HTTP request that triggered it (I still need to do some robustness and correctness testing before it's ready to merge). With a handful of other protocols like MySQL, Mongo, Redis, Kafka, or even FTP lol, I think Subtrace can cover most practical dev workloads.

Btw Subtrace can already record .pcap files today since it's just a simple TCP stream proxy, but raw network packet captures are mostly only useful when you're implementing new protocols, which 99% of the people using Docker containers today aren't doing. It's also a solved problem because you can just run `apt-get install tcpdump` inside the container.

Automatic tracing for app-level protocols that is easy to setup, works everywhere, lightweight for prod, fast to search, and can show the data in a clean interface is still insanely difficult today. That's the problem Subtrace is trying to solve.

Yes, but we've managed to do it automatically without any library/language specific hooks! It's probably one of my favourite things in Subtrace :)

We generate an ephemeral TLS root CA certificate and inject it into the system store. The generated certificate is entirely in-memory and never leaves the machine. To make this work without root privileges, we intercept the open(2) syscall to see if it's /etc/ssl/certs/ca-certificates.crt (or equivalent). If so, we append the ephemeral root CA to the list of actual CA certificates; if not, we let the kernel handle the file open like usual. This way, none of the other programs are affected, so only the program you start with `subtrace run` sees and trusts the ephemeral root CA.

After we get the program to trust the ephemeral root CA, we can proxy outgoing TLS connections through Subtrace transparently but also read the cleartext bytes.

All of this is fully automatic, of course.

You can use Subtrace entirely locally without talking to subtrace.dev using the -devtools flag. For example:

    subtrace run -devtools=/subtrace -- python3 -m http.server

We have fully on-prem installations for enterprises, of course (e.g. if you're in a highly regulated industry like healthcare).

One of my litmus tests for software is "can I use it on a flight?", so you'll be pleased to know Subtrace works great locally without ever talking to subtrace.dev!

    subtrace run -devtools=/subtrace -- python3 -m http.server

You can tag each request with arbitrary key-value maps and search over these later. For example, if you add a `x-subtrace-tags: user=foo, project=bar` header on the response, you can apply a `tags.user == "foo"` filter in the dashboard to see all requests across your entire infra from that user and only that user. Each request is pre-populated by default with tags like hostname, pod name, AWS/GCP location, etc.

It's like Honeycomb's wide events but even better because: (1) you can see whole request including the payload alongside the event fields, and (2) it's fully automatic and requires no code changes out of the box (you can incrementally add these tags when you find a need for each one instead of the huge upfront cost from instrumenting the hell out of your codebase).

Can you share more about how you are doing this?

IIRC, Server-Timing headers are Firefox-only and they are not available via fetch or XHR.

standards ftw!

What would you like to see?

One of things we're thinking about is automatic method/function call tracing. Something like attaching the entire stack trace of calls done to handle the API request. Ideally using the same UI so that you can see the headers/payload that was sent and the function-level stack trace right next to each other. None of the OpenTelemetry verbosity, all of the observability!

More than 0.01 ms, less than 0.10 ms. Noticeable when you're a high frequency trading firm, probably not otherwise :)

In the hot path, Subtrace is just a dumb proxy that copies bytes. All of the processing + indexing happens offline in a different machine (in Clickhouse).

Flat pricing with unlimited requests hosted in the cloud (https://subtrace.dev/pricing).

We also have an on-prem version of Subtrace for enterprises. It runs in their own AWS account without ever talking to subtrace.dev so that companies in regulated industries like healthcare can use Subtrace.

Stratoshark was discussed last month https://news.ycombinator.com/item?id=42793777

Stratoshark: https://wiki.wireshark.org/Stratoshark :

> Stratoshark captures and analyzes system calls and logs using libsinsp and libscap, and can share capture files with the Sysdig command line tool and Falco

Not in the very short term, but we've talked about it internally and we might do it in the future!

If you'd like to use Subtrace on Windows, it would be super helpful to understand your use-case deeply so that we build the right things in the right order. Please reach out to me at adtac@subtrace.dev, I'd love to chat!

Disclaimer: I'm a maintainer of the project

Pixie (https://px.dev) can be installed in under 5 mins and gives this level of visibility across all applications. No need to change your application (wrap in `subtrace run`) to get instant visibility.

We also support 11 application protocols (https://docs.px.dev/reference/datatables/) with TLS handshake tracing and MQTT support coming soon (encrypted traffic tracing has been supported for a long time).

Subtrace already works great on Kubernetes (https://docs.subtrace.dev/kubernetes)! Add a single line to your image's Dockerfile and that's it.

I'm working on an even simpler way where you can just `kubectl apply` a DaemonSet or a Helm chart to get automatic tracing for all pods in your cluster instantly without any code-level changes. If anyone is interested in beta testing this, email me at adtac@subtrace.dev, I'd love to understand your usecase!

Might be an overkill to use it in this capacity but SUSE NeuVector (https://github.com/neuvector/neuvector) can build a connectivity graph between pods and allows to perform packet capture to PCAP. It works at the virtual switch layer and can capture all traffic.

They can, but for open-source startups I recommend they do a Show HN first, for the following reasons:

(1) If the Show HN succeeds, then they can save their Launch HN card to play later; if it doesn't succeed, they can go ahead and do a Launch HN soon afterwards.

(2) Show HN is more about the project/tech, especially when it's open-source. Launch HN is more about the startup/business.

(3) Nowadays many startups offer an open-source (or open core) version for free, then make money with a hosted version. For these, it's a good fit to first do a Show HN about the open-source offering, and then later do a Launch HN about the cloud product.