"I uploaded 4.5 billion of those videos to Hugging Face" = I scraped the metadata (title, view count, etc) for 4.5 billion TikTok videos using the same API as their Android app. I posted the data to HF as Parquet. The video content itself is not included. I'll sell you my Go scraping code.
Something posted with the understanding that it could potentially be viewed by anyone on the Internet with minimal/no restriction.
ToS are just what you follow if you don't want to get banned off of the site. If you don't care about that, then you can go hog wild, though you're being a bit of a jerk/not playing nice obviously.
That's a blatant strawman - that's not a reasonable position at all. A reasonable person does not expect their private medical records to be accessible to you or me just because they are stored in an EHR system.
They might be surprised that you or I looked at their TikTok video when we aren't the intended audience, but they still posted it publicly, with the understanding that it would be made freely available to others.
It's definitely not a strawman. Perhaps you meant that it's a false equivalency, but I don't think that's true either.
With how common data hacks are, why wouldn't a reasonable person expect their medical records to leak? I received at least two such breach notices just last year.
No, I don't care at all, but I recognize that my morals might be lower than others here. To me it's just public data...whatever.
My criticism was basically - this is trying to sell an AI slop project for $699 a pop - I could get this out of a few Claude Code sessions if I had the storage and network bandwidth to run such a scraper. The value proposition is questionable when the writing shows that the entire project was AI generated, and clearly Claude understands the way the TikTok Android app internal API works quite well...
The AI giants got where they are by pushing similar limits and setting aside moral (and legal) issues to be settled later in court, so the idea seems to fit the general zeitgeist we're living in. Not really criticism, just an observation.
> Everything described here is a private Go repository. One-time payment, permanent access, complete source.
> $699 one time · lifetime access
Not open-source apparently.
And I cant find the reddit post but I think I read that videos/assets are not actually pre-downloaded, they have to be requested through Tiktok API using the provided code. So if Tiktok patches, the code will need updates too.
I think the post is essentially a decent technical-explainer (value adding and interesting) in exchange for effectively a little product placement (selling either just the code or code already running on a server at additional cost)
The AI keeps mentioning how a HTTP 200 can silently pollute your dataset. Why not just check contents of body? Usually APIs follow strict JSON contract for successful queries, alert or throw an error when that changes.
It's probably AI coded and hallucinated many things. That drumming up the importance of a minor thing is a real tell. Another hallucination - it hasn't found any video APIs (despite statements that it has and uploaded it). It has video metadata.
It's that mix of dense, impressive sounding jargon, but even scanning across it raises glaring problems. Like, if you have 4.5 billion videos on HF, and it's 289GB, it's about 60 bytes per video. Checking the column fields as well, there doesn't seem to be any video files*.
More directly, there simply aren't any video files uploaded. It's just parquet files, which contain no video columns (I'm not even sure if it supports it).
Seems the expectation is the purchaser uses the info/parameters in the 289gb file to decide which videos to download, then uses the /v1/video/info endpoint to download individual videos.
>Is it legal? It is against TikTok's terms of service. It is sold for research and educational use.
Oh, ok. Otherwise, very detailed deconstruction to scrape their API. Lots of layers of registration and creating a request that looks like it is valid client.
(I guess that is sort of a roundabout summary...)
ToS are just what you follow if you don't want to get banned off of the site. If you don't care about that, then you can go hog wild, though you're being a bit of a jerk/not playing nice obviously.
They might be surprised that you or I looked at their TikTok video when we aren't the intended audience, but they still posted it publicly, with the understanding that it would be made freely available to others.
With how common data hacks are, why wouldn't a reasonable person expect their medical records to leak? I received at least two such breach notices just last year.
My criticism was basically - this is trying to sell an AI slop project for $699 a pop - I could get this out of a few Claude Code sessions if I had the storage and network bandwidth to run such a scraper. The value proposition is questionable when the writing shows that the entire project was AI generated, and clearly Claude understands the way the TikTok Android app internal API works quite well...
Not open-source apparently.
And I cant find the reddit post but I think I read that videos/assets are not actually pre-downloaded, they have to be requested through Tiktok API using the provided code. So if Tiktok patches, the code will need updates too.
But I think this is the 289GB data (free): https://huggingface.co/datasets/kuben-developer/tiktok-video...
Very LLMish language!
Stable Diffusion was enabled by LAION, for example. That was metadata about images and URLs to those images, but not the actual image files.
Oh, ok. Otherwise, very detailed deconstruction to scrape their API. Lots of layers of registration and creating a request that looks like it is valid client.
https://shikigami.dev - more details here