The specs for things like microformats were mostly fine. The issue is that search engines mostly obfuscate their behavior and microformats fizzled out in the places that could have benefited from them. This was mainly due to relatively low adoption; not necessarily abuse. This was the kind of thing that techies obsessed about that most other people ignored.
So, search engines couldn't rely on sites properly using microformats on most sites and had to resort to other things to pull out structured information. Browsers removed what little support they had for microformats. And then blogging became more of a niche thing as people moved to social networks and things like substack, tumblr, and similar sites.
Mostly, you can still use microformats. It doesn't hurt anything and there might be some things out there that benefit from having a little bit of extra structure. But otherwise there's just very little point to it (beyond some niche uses); and there never really was. It's the classic chicken egg problem where it would be great if everybody used it but because most people didn't it wasn't all that great. You only get network effects if you get a critical mass of adoption. That just never happened with this stuff. Most developers treated it as a low value, optional thing, that was maybe nice to have but not worth obsessing over.
These days there are related things that are still widely used (e.g. opengraph and similar meta attributes). Also there are new semantic html tags that help tell apart main articles from navigation, footers, etc. that are fairly widely used. That helps things like screen readers, rendering previews of shared links, etc. And it also helps for SEO. And SEO is a great incentive for people to adopt stuff.
XFN and FOAF were indeed mostly relevant in the pre-social network era and became quickly irrelevant. The point with social networks is that they are walled gardens. Standardization was not something that the owners of those actively pursued because they are simply not interested in interoperability. Exclusive ownership of the relationship of the user is the whole point for exploiters of social networks like meta, tik tok, x/twitter, linkedin, etc.
> It was a major clash between the idealistic vision of building global interconnected communities and the financial reality of companies wanting to optimize everything for their own profits.
I think its even simpler than that. HTML et al was a nice idea but its largely ignored in the pursuit of making web pages seem like apps.
Berners Lee had this idea of documents like pages in a book, nothing like the full-fledged applications in the browser we get today, yet that legacy Berners Lee cruft still remains whilst people find ever more inventive ways to overcome it