Ask HN: How does one scrape a website?

2 pointsposted 6 hours ago
by terry_hc

Item id: 46465371

1 Comments

reliefcrew

6 hours ago

Just ask AI how to mirror w/ wget. But, beware that if the site relies on javascript, wget may not be enough. In that case you'll need to program some kind of headless browsing. Didn't the internet archive (archive.org) take care of everything for you already though?