hackernews client

Yadda 3.0.0: BDD in the Age of AI Agents

49 pointsposted 8 hours ago

24 Comments

wesselbindt

3 hours ago

So, I get the idea of having a layer of abstraction between your tests and the system under test. You decouple _what_ is being done from _how_ it is done, which leads to robust, easily maintained, and even more re-usable test suites. I've seen it in action, it's a great idea, I've reaped the benefits myself. Great stuff.

What I've never really understood is why some people choose natural language for the abstraction. It just seems like such an odd and expensive choice. Most of the layers of abstraction we add in our codebase are written in the programming language of choice itself, they're functions, classes, what have you. But for this one specific flavor of abstraction, folks reach for natural language.

The one potential benefit I could think of for this over a more standard DSL is that non-technical colleagues would be able to read or even write these specs. But in practice I've only seen this pan out once in my life, and that was with a more traditional DSL rather than natural language.

Is there a hidden benefit I'm overlooking?

righthand

2 hours ago

Because the intent has always been that Product delivers the BDD statements not Engineering, but Product people have (for the most part in my experience) refused to deliver what BDD flows need to be tested. That’s why most of the time the test suites aren’t as resilient because the engineer is guessing at which flows to test.

In that context it makes more sense that Llms are a push to replace Product people not Engineering people.

jesol

5 hours ago

I also think BDD is a really powerful tool for UI acceptance tests! I've now worked on adding it to two AI codebases (one engineered, another vibecoded), and so far it has been mostly positive. Having non-engineers be able to see what effects their code changes have in the UI is nice; and I find having high level API and UI tests is forcing the first layer of cleanup (standardize access to DB through models/services, and a single API SDK).

I am also really interested in standardizing the REST interface with typespec, and the database schema with DBML. I think creating these closed-loops with high-level specs is one of the best ways to make an AI project ownable by a team.

stevefan1999

4 hours ago

Yep, I've been experimenting Playwright with CucumberJS and Deno for the last 3 months and it's super effective

skybrian

3 hours ago

How are you doing that? Is the Playwright usage a hidden detail or do images appear in the spec?

lifeisstillgood

5 hours ago

The example of “transcibe the initial project discussion, turn that into BDD and that makes the rails to guide the AI is … either brilliant and insightful or a snake eating it’s own tail.

I think it has to be worth a try though…

jaggederest

3 hours ago

I've been doing that for almost a year and I feel like I'm taking crazy pills when I talk about it, because it works so well.

I transcribe most product meetings, turn them into a transcript, run that through an accuracy pass with a bunch of context (mostly vocabulary definitions etc), and use that as a starting doc for generating issues and tests. There are apps that do it fully end to end now I think.

lifeisstillgood

3 hours ago

Any hints for running pilot projects (at large, bureaucratic org)

jaggederest

2 hours ago

Yeah, basically I would just start transcribing everything, if you aren't already, and get a bunch of context. It's best to start from the very beginning but the reality of enterprise is that nothing is truly greenfield.

Maybe start with a project kickoff or brainstorm, some kind of planning meeting, and select one task out of it to start. I always start with just the meeting transcript as a context, and expand context from there as needed with things like internal standards docs, libraries, references from other projects, etc, and only as little as is necessary. That generates the project plan, and then I feed that into claude code or whatever.

I think the biggest thing I've done recently is start to be extremely critical about the output, basically using promptfoo to try to figure out what the right level of context and instructions are, not just for external facing e.g. chatbots or whatever, but for my own internal development process, planning, everything. The more you can pin down and calibrate the better - it's easy to run a given prompt for a month, so spending a day (or even a week!) out of the month to refine your prompts beats just grinding ad hoc.

Edit: another thing I do is generate tests bdd-style separately from the code that they're testing (for e2e and high level at least), that derives out of the project plan too.

krautsauer

7 hours ago

"Behaviour Driven Development"

I was expecting something on Binary Decision Diagrams. SAOD is worst with TLAs.

bossyTeacher

5 hours ago

This is probably HN's tech background speaking. In the enterprise dev world, everyone knows what BDD means just like every dev out there knows what OOP means.

WalterGR

7 hours ago

What is SAOD? Do you mean SOAD?

contradictioned

6 hours ago

Severe Acronym Overload Disorder.

Funny enough, Google's overview box also suggests soad: "SAOD (or SOAD) most commonly refers to the rock band System of a Down"

WalterGR

6 hours ago

Thanks. (That’s what it suggested for me as well. Clarifying that I meant in the context of software, it suggested a misspelling of SOAD - Service-Oriented Analysis and Design.)

scresswell

8 hours ago

Yadda 3.0.0 is out. The release modernises the JavaScript BDD library, but more interestingly, it was largely built by Claude Code and points to why executable specifications may become even more valuable in an agentic development world.

ur-whale

6 hours ago

One suggestion: my bet is that close to no one knows what the acronym BDD stands for. The first comment on this post is in fact exactly about that.

I'd consider explaining what BBD is at the start of the front page of your project.

davepeck

5 hours ago

It may be an age thing. 15-ish years ago, TDD, BDD, Red/Green Testing, and a whole bunch of others were hot topics. They’re still useful approaches to know about, even if some (like, IMO, BDD) didn’t really stand the test of time.