Hi, author here

If you haven’t already, try https://r.jina.ai for cleaning up web pages for agents. It’s pretty good, until you hit JS heavy pages.

I spin up tiny headless chrome into LLM pipeline for this case, give it a try at Unrender

Please share anything you have on how to make it better