dejobaan
an hour ago
Not a data lake expert here, but I think there's a parallel with telemetry/logging for video games. The author talks about mass "ingestion without purpose," which is what I've sometimes flubbed in the past—during playtesting, we'd collect piles of data about where the players won/lost/moved/whatever, and there was so darned much of it that we'd just toss our hands up and not do anything with it.
The article suggests (sensibly!) that the right thing to do is to actually go in and design what's being logged with purpose (i.e., everything should answer a known question). But I don't always know what I'm looking for when I'm testing a thing. At the risk of using LLMs as a hammer that hammers everything, I wonder if they'd help with that problem?
I've pointed Codex at Posthog logs, for instance, and it's been able to say, "yeah, well, look at this pile of residential proxies hitting your site." I wonder if it'd similarly be able to say, "hey, I notice a pattern where everyone who picks the Cleric class stops playing after level 3"?