Metadata for 2.6 million Pornhub videos spanning 320k playlists

NSFWdataset

I scraped metadata for 2.6M pornhub videos based on the 320k most recently updated playlists as of mid-July 2019. In total, the data is 350MB in compressed json form, separated into playlist, video, and matching/cross-referencing files. They're directly downloadable from these links:

https://datahub.io/racydata/final/r/dfp.json.gz (14MB)

https://datahub.io/racydata/final/r/dfv.json.gz (120MB)

https://datahub.io/racydata/final/r/dfm.json.gz (210MB)

​

And here's an example jupyter notebook that uses the matching data (40 million pairs of (videoid,playlistid)) to make a sparse matrix with dimensions of (number of unique videos x number of playlists) and reduce the dimensionality with SVD. Then you can get "recommendations" for playlists/videos similar to a particular playlist/video based on the distance in this reduced dimensional space.

https://gist.github.com/racydata/92ae85ea47da7c2d0bf50442bb0e83ea

The notebook also shows what columns you can play with in those three files.

Posted Jul 29, 2019, 4:54 PM UTC

27311
moazim1993•

for “research purposes”

65
oneonetwooneonetwo•

There's something surreal about that notebook. It's a perfectly normal dataframe that you'd use at work every day until you see what's in "title"

15
EyedMoon•

And there goes a shameful upvote

But great to see bigscrapping projects like that and not only reposts of articles all the time

30
Sxi139•

you trying to get a job with Mindgeek or some shit?

17
mdni007•

I wonder what the office environment is like when you have posters of naked women all around you

1
MaximRouiller•

Coworker of mine already went there. There's really not from what I understood.

They host meetups and other events but unless you know? You'd never know which industry they're in.

8
Sxi139•

I really don't think they have posters of naked women in their office environment. Just because they are the parent company of porn studios doesn't mean they'd have naked photos laying about.

2
throwawaytimmay•

Great effort!

I see getting actual recommendations still runs O(N) which can be fixed with this: https://github.com/spotify/annoy. It's a pretty neat library to speed nearest-neighbour searches up, and this is a great use-case to show why waiting for recommendations can suck.

9
jiejenn•

I hope your post doesn't blow up 😁

5

Quick, how many playlists have Ryan Creamer?!

2
bunny113•

Saved for later use

1
Mechamod2•

Is the duration in seconds?

1