code.communitydata.science - cdsc

]> code.communitydata.science - cdsc_reddit.git/search

2020-08-12	Nate E TeBlunthuis	Update submissions to parse using the backfill queue.	commit \| commitdiff \| tree
2020-08-11	Nate E TeBlunthuis	bugfix in checking submission shas	commit \| commitdiff \| tree
2020-08-10	Nate E TeBlunthuis	Use multiword expressions in tf.	commit \| commitdiff \| tree
2020-08-10	Nate E TeBlunthuis	Finish generating multiword expressions.	commit \| commitdiff \| tree
2020-08-09	Nate E TeBlunthuis	Bugfix	commit \| commitdiff \| tree
2020-08-09	Nate E TeBlunthuis	Use groupby - joins instead of windows	commit \| commitdiff \| tree
2020-08-04	Nate E TeBlunthuis	renamte tf_comments part 2.	commit \| commitdiff \| tree
2020-08-04	Nate E TeBlunthuis	rename tf_reddit_comments.py step1.	commit \| commitdiff \| tree
2020-08-04	Nate E TeBlunthuis	Improve tokenization following data. Generate author...	commit \| commitdiff \| tree
2020-08-04	Nate E TeBlunthuis	improve tokenizer.	commit \| commitdiff \| tree
2020-08-04	Nate E TeBlunthuis	TF reddit comments.	commit \| commitdiff \| tree
2020-08-04	Nate E TeBlunthuis	code to sort tf	commit \| commitdiff \| tree
2020-07-10	Nate E TeBlunthuis	remove is_submitter field from submissions which doesn...	commit \| commitdiff \| tree
2020-07-08	Nate E TeBlunthuis	Bugfixes in scripts.	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	clean up comments in streaming example.	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	update .gitignore	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	update examples with working streaming	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	Build comments dataset similarly to submissions and...	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	update .gitignore	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	Script for example of streaming pyarrow.	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	Script to demonstrate reading parquet.	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	Check the shas when we download dumps	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	Script to run both parts of submissions_2_parquet.sh	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	Cache before sorting so we don't extract twice.	commit \| commitdiff \| tree
2020-07-07	Nate E TeBlunthuis	Move the spark part of submissions_2_parquet to a separate...	commit \| commitdiff \| tree
2020-07-06	Nate E TeBlunthuis	Fix whitespace at top of file.	commit \| commitdiff \| tree
2020-07-06	Nate E TeBlunthuis	Secondary sort for the by_author dataset should be...	commit \| commitdiff \| tree
2020-07-06	Nate E TeBlunthuis	Create a second dataset sorted by author.	commit \| commitdiff \| tree
2020-07-06	Nate E TeBlunthuis	Create parquet datasets of reddit submissions from...	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	Rename spark script to reflect that it is for comments.	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	update .gitignore	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	bugfix in retrieving old data and rename file.	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	Script for checking shas for submissions.	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	Bugfix: use timestamp types	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	update the reddit comment dumps	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	Don't clobber old dumps so that we can just download...	commit \| commitdiff \| tree
2020-07-03	Nate E TeBlunthuis	script for getting submissions dumps from pushshift.	commit \| commitdiff \| tree
2020-07-02	Nate E TeBlunthuis	Extract variables from pushshift comment to parquet	commit \| commitdiff \| tree

Building parquet tables from pushshift reddit dumps.

RSS Atom

Community Data Science Collective || Want to submit a patch?