It's insane to me that the "fix" for AI errors is adding more "PLEASE PLEASE DO BETTER" to the prompt

If it makes you feel any better, the Markdown part is optional (and has no semantics). Somehow it feels about right that the Markdown file can actually just be a YAML file with the wrong extension.

(Actually, to be more specific, a YAML file with no directives, explicitly-signalled start-of-document-content, and followed by a second null document. I will note that frontmatter syntax is not specified; the non-normative Appendix B is the only place that suggests it means prefix and suffix --- lines. And no, frontmatter is not part of Markdown, or CommonMark, and is in fact incompatible with both. And it’s invalid YAML too, the end-of-frontmatter line should be ... to indicate end of document without starting a new document.)

it's looking like llms are interpreters, and markdown plus english text is the language of choice to run non deterministic programs on it

QUALITY.md feels similar to CONSTITUTION.md

Looks like unless something better comes up, we'll be stuck with it for a while.

I find markdown useful for repo-specific conventions, especially skills.

I thought the same about Yaml and Kubernetes/Helm…

It already refuses to read the AGENT.md/CLAUDE.md files, what's the point of giving it even more markdown it won't read until you yell at it.

it is until we define real consistent deterministic gates and protocols. It really is a symptom of the lack of concerted effort. Everyone has a personal preference on how to shove the context and most of them are just "here's some good text I've found to work in my context"

THe problem is that humans often don't know - this is as much about encouraging getting the humans aligned as the agents. Completely agree agents really aren't stakeholders. Fine point. I'll update description to clarify ... thank you!

Not OP, but it seems to me the idea is that stakeholders can collaborate and come to consensus on the contents of QUALITY.md.

No magic. QUALITY.md describes what is unique and valueble to your proejct context that model providers won't have insight into.

OP here. You're spot on. Trade-offs matter. The trade-offs are implied by the selection of what quality factors/attributes are selected and their requirements. A statement like "performance can take a huge hit at the cost of being dead easy to read and reason about" can sit right there in the QUALITY.md as a comment or in the markdown body.

No commercial offering associated with this effort. But a lot of potential for others to incorporate the QUALITY.md standard into products that assess/evaluate quality at varying levels of a loop stack.

The agent skill/CLI that's provided generate a quality evaluation report with recommendations for handoff (ideal for loop engineering) is just one example of how the QUALITY.md file can be used.

It's easy to imagine a SaaS that does the same that provides better eval, reporting, and integration capabilities.

I'd really like feedback on the standard/specification. In short, it defines a quality model of quality factors/characteristics (which you can define as security, reliability, etc), requirements (how those qualities are assessed), a customizable rating scale, and "areas" to have different attributes/requirements for different areas of your project (e.g. frontend/backend, tests, specs, etc). That's basically it - and it follows a consistent pattern of how quality models have been practiced for decades. They are simple and powerful, but - until AI - kind of a pain/toil to get started with. Simple but not easy - until now. QUALITY.md + AI makes it easy. However, you still have to put in the work/care/attention to what goes into your QUALITY.md so you can get maximum leverage from it.

So is every proposal to standardize new things, and eventually the cream rises to the top, even though some people are perfectly happy sticking with milk.