Retr0id
4 hours ago
Can any brave soul wade through the LLM prose to provide a human-readable summary?
simonw
4 hours ago
"I uploaded 4.5 billion of those videos to Hugging Face" = I scraped the metadata (title, view count, etc) for 4.5 billion TikTok videos using the same API as their Android app. I posted the data to HF as Parquet. The video content itself is not included. I'll sell you my Go scraping code.
arcfour
4 hours ago
Man, I could use $699. If this gets even a single sale, then maybe I need to try having less decency...
(I guess that is sort of a roundabout summary...)
echoangle
4 hours ago
Would you say that this is immoral? Since it’s data that’s public anyways, I don’t see why putting it in a table and selling it is a bad thing.
kenniskrag
2 hours ago
Because a lot of things is public online and can't be copied and sold e.g. due to copyright, patents and trademark. Also if you access a website you are bound to a ToS contract and this is a breach of that contract.
echoangle
2 hours ago
Breaching a contract isn’t a crime though, it’s a civil matter.
breezybottom
4 hours ago
What is "public" data? It admits to violating the ToS.
arcfour
4 hours ago
Something posted with the understanding that it could potentially be viewed by anyone on the Internet with minimal/no restriction.
ToS are just what you follow if you don't want to get banned off of the site. If you don't care about that, then you can go hog wild, though you're being a bit of a jerk/not playing nice obviously.
breezybottom
4 hours ago
Any company could potentially be hacked, so by that standard all data is public. And no, the ToS is legally binding on the company as well.
echoangle
3 hours ago
You don’t see a difference between accessing data by hacking a service (which is illegal) and downloading/scraping data that’s not protected? The TOS are a contract and only a civil thing afaik.
arcfour
4 hours ago
That's a blatant strawman - that's not a reasonable position at all. A reasonable person does not expect their private medical records to be accessible to you or me just because they are stored in an EHR system.
They might be surprised that you or I looked at their TikTok video when we aren't the intended audience, but they still posted it publicly, with the understanding that it would be made freely available to others.
breezybottom
4 hours ago
It's definitely not a strawman. Perhaps you meant that it's a false equivalency, but I don't think that's true either.
With how common data hacks are, why wouldn't a reasonable person expect their medical records to leak? I received at least two such breach notices just last year.
arcfour
3 hours ago
Obviously I meant that any data a person expects to be public, like a public post on a social media site... I was not referring to data exposed in data breaches, which is a different subject entirely...
If that was not obvious to you then I apologize; though it really should have been, since you are encouraged to interact with others in good faith on HN.
And if you expect your medical records to be public...then what is the point of this discussion?
breezybottom
3 hours ago
The user only gave TikTok permission to use the videos according to the ToS. And considering the majority of TikTok users are children, I think it's hard to justify morally, even if you could make a legal case.
arcfour
3 hours ago
A "data breach" refers to unauthorized access to nonpublic or protected data. Scraping content that is publicly viewable without logging in - or even with logging in, since an account is effectively disposable - is not a "data breach" (as far as any typical usage of the term goes).
In hiQ Labs v. LinkedIn, the 9th Circuit (US) ruled that scraping publicly accessible data does not violate the CFAA's "without authorization" clause (hiQ was bulk scraping public LinkedIn profile data - in violation of LinkedIn's ToS). The Supreme Court later specifically narrowed the CFAA in Van Buren v. United States saying "exceeds authorized access" applies to accessing areas of a system you aren't entitled to enter at all, not misusing access you legitimately have (regardless of ToS violations).
Other CFAA cases have ruled similarly - being legitimately granted access (i.e. signing up for an account, or browsing publicly without logging in, since the site is intended to be available to the public) and then misusing it is not "hacking".
So in the U.S., it's not a computer crime ("hacking"/"breach") to scrape data, and nobody uses the term "data breach" to refer to scraping publicly available data on a public site, except for apparently you.
breezybottom
3 hours ago
You still haven't checked the ToS:
"use automated scripts to collect information from or otherwise interact with the Services" — this covers the entire scraping operation.
"make unauthorised copies, modify, adapt, translate, reverse engineer, disassemble, decompile or create any derivative works of the Services... or determine or attempt to determine any source code" — e.g. reverse-engineering the X-Argus/X-Gorgon/X-Ladon signing scheme.
"interfere with or attempt to interfere with the proper working of the Services... or bypass any measures we may use to prevent or restrict access to the Services" — TLS-fingerprint spoofing, the empty-200 soft block, and the proxy IP rotation to get around rate limiting.
"use or attempt to use another's account, service or system without authorisation from TikTok, or create a false identity on the Services" — this covers the forged device registrations (fake Android handset + carrier profiles)
"use the Services, without our express written consent, for any commercial or unauthorized purpose" — the website is monetizing the dataset and selling the code itself.
edoceo
2 hours ago
Still not a hack or a breach. Still "public" data. Still a TOS violation. Your up-thread claim that potential hack is the same as public is still incorrect.
echoangle
2 hours ago
And who cares about ToS? That’s a purely civil thing, surely hacking (which is a crime) is worse than breaching a contract?
echoangle
3 hours ago
Is it a data breach if I right click images in google search and save them to my disk?
breezybottom
3 hours ago
A data breach could leak into search engine results, sure.
echoangle
2 hours ago
I wasn’t asking if a data breach could be in the search results, I asked if saving search engine results in itself would be a data breach. Apparently you think that scraping publicly accessible data from tik tok is a data breach in itself, so does this apply in other cases too?
echoangle
3 hours ago
I could get the same data in principle by downloading the app and accessing the post. There’s no hacking or circumventing access controls. That’s what I meant by public data.
arcfour
4 hours ago
No, I don't care at all, but I recognize that my morals might be lower than others here. To me it's just public data...whatever.
My criticism was basically - this is trying to sell an AI slop project for $699 a pop - I could get this out of a few Claude Code sessions if I had the storage and network bandwidth to run such a scraper. The value proposition is questionable when the writing shows that the entire project was AI generated, and clearly Claude understands the way the TikTok Android app internal API works quite well...
arvid-lind
4 hours ago
The AI giants got where they are by pushing similar limits and setting aside moral (and legal) issues to be settled later in court, so the idea seems to fit the general zeitgeist we're living in. Not really criticism, just an observation.
405error
4 hours ago
I cannot verify whether it is technically correct, but it's about how to defeat Tiktok's bot filters to scrape it.