]> code.communitydata.science - cdsc_reddit.git/log
projects / cdsc_reddit.git / log
? search:
summary | shortlog | log | commit | commitdiff | tree
first ⋅ prev ⋅ next
cdsc_reddit.git
3 years agoscript for getting submissions dumps from pushshift.
commit | commitdiff | tree
Nate E TeBlunthuis [Fri, 3 Jul 2020 00:40:17 +0000 (17:40 -0700)]
script for getting submissions dumps from pushshift.

3 years agoExtract variables from pushshift comment to parquet
commit | commitdiff | tree
Nate E TeBlunthuis [Thu, 2 Jul 2020 21:06:36 +0000 (14:06 -0700)]
Extract variables from pushshift comment to parquet

A spark script

Building parquet tables from pushshift reddit dumps.

Community Data Science Collective || Want to submit a patch?