btown
3 hours ago
> Flexibility doesn't mean you can query arbitrarily deep payload structures at production scale without cost. Indexing in EventSourcingDB is primarily on metadata (like type, subject, timestamps). If you filter on arbitrary JSON payload fields, you may trigger a full scan. That's by design and by honesty: we don't hide the trade-offs. That's also why we say use EventQL wisely. It's perfect for ad hoc analysis, debugging, or one-off data extraction. If a query is needed repeatedly or must run at scale, build a projection – projections are precomputed and optimized.
As they note, representing derived products of event stores as metadata-only uncached queries/views will only get you so far.
If you want something that bridges exploratory analysis to production-grade, consistent materialized views, I really like the approach that Materialize (materialize.com) and Feldera (feldera.com - see https://news.ycombinator.com/item?id=41685689 and specifically https://news.ycombinator.com/item?id=41687690) take. They're mathematically sound ways of representing an entire hierarchy of derived materialized views in pure SQL, and having them update in real time... much like Excel would, if you change an input cell numerous sheets of dependencies away. There are some incredible papers here - DBSP and Differential Dataflow are the keywords.
I do think that both Feldera and Materialize have focused a lot on analytics use cases, rather than on what implications they could have for event sourcing in high-reliability operational situations. The unifying key, of course, is that the moment you have the ability to declaratively specify a network of downstream projections, you can hot-swap them, version-control them, and time-travel along your code-versioning-time-dimension and data-time-dimension independently - which can be incredible.
But I think there's a ton of tooling left to be built, and culture to change around event sourcing being an enterprisey way of building things, before we start to see the future appear here. But that future will be amazing.