From: Benjamin Mako Hill <mako@atdot.cc>
Date: Sat, 28 Sep 2024 19:38:06 +0000 (-0700)
Subject: Merge remote-tracking branch 'flask_repo/main' into flask
X-Git-Url: https://code.communitydata.science/coldcallbot-discord.git/commitdiff_plain/f1f29cdbe2a9c5580edd871342259a886b47e8ae?hp=aee2c90d1e0b7ac35acae6416ef7e08e2a6183d0

Merge remote-tracking branch 'flask_repo/main' into flask
---

diff --git a/.gitignore b/.gitignore
index b04e720..9b00fb8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,3 @@
-config.py
-__pycache__/
+*~
+__pycache__
 *.pyc
diff --git a/README b/README
new file mode 100644
index 0000000..733a2e8
--- /dev/null
+++ b/README
@@ -0,0 +1,154 @@
+Setting up the Discord Bot
+======================================
+
+I run the Discord bot from my laptop. It requires the discord Python
+module available in PyPi and installable like:
+
+    $ pip3 install discord
+
+Setting up the Bot
+=====================================
+
+The documentation for the `discord` python package
+(https://discordpy.readthedocs.io/en/latest/discord.html) does a good
+job explaining how to set up a Discord bot with your server. Follow
+the steps there, with one important exception:
+
+1. On the "Bot" tab in the discord application configuration page you
+need to enable both "Privileged Gateway Intents."  This allows the bot
+to see who is present and active in the channel.
+
+Finally, you need to copy your bot'ss Token (also found on the "Bot" tab)
+into coldcallbot.py. Pass it as the argument to `ccb.run()`.
+
+
+
+Using the Cold Call Bot
+======================================
+
+1. All students must have the role "Student" in Discord. If they do
+   not have the roll, they will not be called upon.
+
+2. The "classroom" is the "Classroom Voice" channel. This is currently
+   hard coded.
+
+3. The bot has only one command: "$next" which calls a person and
+   records this information in the logs. You can run this command in
+   any channel that the bot has access to (e.g., #bot-commands) but I
+   do it a public channel called "#classroom-questions" so that
+   students can watch it operate.
+
+
+Daily Process
+======================================
+
+You need to start the bot from the laptop each day. I do that by:
+
+  $ ./coldcallbot.py
+
+The bot will run in the terminal, print out data as it works including
+detailed weights as it goes, and it will record data into files in the
+/data/ directory.
+
+
+After class, you will have two new files created that will be named
+like this (with today's date):
+
+    attendance-2020-10-05.tsv
+    call_list-2020-10-05.tsv
+
+Each day, you need to open up "call_list-YYYY-MM-DD.tsv" and edit the
+final two columns. The first columns `answered` means that the person
+responded and answered the question (i.e., they were present in the
+room but away from their computer and unresponsive). This is almost
+always TRUE but would be FALSE if the student were missing.
+
+The final column `assessment` is GOOD, FAIR, or BAD in my rubric. I've
+detailed what that means on this page:
+
+https://wiki.communitydata.science/User:Benjamin_Mako_Hill/Assessment#Rubric_for_case_discussion_answers
+
+I take notes on student answers on paper during class (typically I
+only note down non "GOOD" answers) and then add these to the sheet
+immediately after class.
+
+I keep my entire data directory in git and I'd recommend that you do
+too.
+
+I don't expect that these will necessary work without
+modification. It's a good idea to go line-by-line through these to
+make sure they are doing what *you* want and that you agree with the
+assessment logic built into this.
+
+Assessment and Tracking
+======================================
+
+These scripts rely on a file in this repository called
+`data/student_information.csv` which I have set to be downloaded
+automatically from a Google form using a 1-line wget command.
+
+For reference, that file has the following column labels (this is the
+full header, in order):
+
+    Timestamp
+    Your UW student number
+    Name you'd like to go by in class
+    Your Wikipedia username
+    Your username on the class Discord server
+    Preferred pronouns
+    Anything else you'd like me to know?
+
+The scripts in this directory are meant to be run or sourced *from*
+the data directory. As in:
+
+    $ cd ../data
+    $ R --no-save < ../assessment_and_tracking/track_participation.R
+
+There are three files in that directory:
+
+track_enrolled.R:
+
+    This file keeps track of who is in Discord, who is enrolled for
+    the class, etc. This helps me remove people from the
+    student_informaiton.csv spreadsheet who are have dropped the
+    class, deal with users who change their Discord name, and other
+    things that the scripts can't deal with automatically.
+
+    This all need to be dealt with manually, one way or
+    another. Sometimes by modifying the script, sometimes by modifying
+    the files in the data/ directory.
+
+    This requires an additional file called
+    `myuw-COM_482_A_autumn_2020_students.csv` which is just the saved
+    CSV from https://my.uw.edu which includes the full class list. I
+    download this one manually.
+
+track_participation.R:
+
+    This file generates histograms and other basic information about
+    the distribution of participation and absences. I've typically run
+    this weekly after a few weeks of the class and share these images
+    with students at least once or twice in the quarter.
+
+    This file is also sourced by compute_final_case_grades.R.
+
+compute_final_case_grades.R:
+
+    You can find a narrative summary of my assessment process here:
+
+    https://wiki.communitydata.science/User:Benjamin_Mako_Hill/Assessment#Overall_case_discussion_grade
+
+    This also requires the registration file (something like
+    `myuw-COM_482_A_autumn_2020_students.csv`) which is described
+    above.
+
+    To run this script, you will need to create the following subdirectories:
+
+    data/case_grades
+    data/case_grades/student_reports
+
+
+One final note: A bunch of things in these scripts assumes a UW 4.0
+grade scale. I don't think it should be hard to map these onto some
+other scale, but that's an exercise I'll leave up to those that want
+to do this.
diff --git a/assessment_and_tracking/compute_final_case_grades.R b/assessment_and_tracking/compute_final_case_grades.R
new file mode 100644
index 0000000..b26270b
--- /dev/null
+++ b/assessment_and_tracking/compute_final_case_grades.R
@@ -0,0 +1,72 @@
+## load in the data
+#################################
+
+myuw <- read.csv("myuw-COMMLD_570_A_spring_2021_students.csv", stringsAsFactors=FALSE)
+
+## class-level variables
+question.grades <- c("GOOD"=100, "FAIR"=100-(50/3.3), "WEAK"=100-(50/(3.3)*2))
+
+source("../assessment_and_tracking/track_participation.R")
+setwd("case_grades")
+
+rownames(d) <- d$unique.name
+
+## show the distribution of assessments
+table(call.list$assessment)
+prop.table(table(call.list$assessment))
+table(call.list$answered)
+prop.table(table(call.list$answered))
+
+total.questions.asked <- nrow(call.list)
+
+## generate grades
+##########################################################
+
+d$part.grade <- NA
+
+## print the median number of questions for (a) everybody and (b)
+## people that have been present 75% of the time
+median(d$num.calls)
+
+questions.cutoff <- median(d$num.calls)
+
+## helper function to generate average grade minus number of missing
+gen.part.grade <- function (x.unique.name) {
+    q.scores <- question.grades[call.list$assessment[call.list$unique.name == x.unique.name]]
+    base.score <- mean(q.scores, na.rm=TRUE)
+
+    ## number of missing days
+    # missing.days <- nrow(missing.in.class[missing.in.class$unique.name == x.unique.name,])
+
+    ## return the final score
+    data.frame(unique.name=x.unique.name,
+               part.grade=(base.score))
+}
+
+
+tmp <- do.call("rbind", lapply(d$unique.name, gen.part.grade))
+
+d[as.character(tmp$unique.name), "part.grade"] <- tmp$part.grade
+
+## generate the baseline participation grades as per the process above
+
+## map part grades back to 4.0 letter scale and points
+d$part.4point <-round((d$part.grade / (50/3.3)) - 2.6, 2)
+
+d[sort.list(d$part.4point),]
+
+
+## writing out data
+d.print <- merge(d, myuw[,c("StudentNo", "FirstName", "LastName", "UWNetID")],
+           by.x="student.num", by.y="StudentNo")
+write.csv(d.print, file="final_participation_grades.csv")
+
+## library(rmarkdown)
+
+## for (x.unique.name in d$unique.name) {
+##     render(input="../../assessment_and_tracking/student_report_template.Rmd",
+##            output_format="html_document",
+##            output_file=paste("../data/case_grades/student_reports/",
+##                              d.print$UWNetID[d.print$unique.name == x.unique.name],
+##                              sep=""))
+## }
diff --git a/assessment_and_tracking/student_report_template.Rmd b/assessment_and_tracking/student_report_template.Rmd
new file mode 100644
index 0000000..a0b2145
--- /dev/null
+++ b/assessment_and_tracking/student_report_template.Rmd
@@ -0,0 +1,25 @@
+**Student Name:** `r paste(d.print[d.print$discord.name == x.discord.name, c("FirstName", "LastName")])`
+
+**Discord Name:** `r d.print[d.print$discord.name == x.discord.name, c("discord.name")]`
+
+**Participation grade:** `r d.print$part.4point[d.print$discord.name == x.discord.name]`
+
+**Questions asked:** `r d.print[d$discord.name == x.discord.name, "prev.questions"]`
+
+**Days Absent:** `r d.print[d.print$discord.name == x.discord.name, "days.absent"]` / `r case.sessions`
+
+**List of questions:**
+
+```{r echo=FALSE}
+call.list[call.list$discord.name == x.discord.name,]
+```
+
+**Luckiness:** `r d.print[d.print$discord.name == x.discord.name, "prop.asked.quant"]`
+
+If you a student has a luckiness over 50% that means that they were helped by the weighting of the system and/or got lucky. We did not penalize *any* students with a luckiness under 50% for absences.
+
+
+
+
+
+
diff --git a/assessment_and_tracking/track_enrolled.R b/assessment_and_tracking/track_enrolled.R
new file mode 100644
index 0000000..f0d0fcb
--- /dev/null
+++ b/assessment_and_tracking/track_enrolled.R
@@ -0,0 +1,23 @@
+myuw <- read.csv("myuw-COMMLD_570_A_spring_2021_students.csv")
+gs <- read.delim("student_information.tsv")
+
+## these are students who dropped the class (should be empty)
+gs[!gs$Your.UW.student.number %in% myuw$StudentNo,]
+
+## these are students who are in the class but didn't reply to the form
+myuw[!myuw$StudentNo %in% gs$Your.UW.student.number,]
+
+## read all the folks who have been called and see who is missing from
+## the google sheet
+
+## call.list <- unlist(lapply(list.files(".", pattern="^attendance-.*tsv$"), function (x) {
+##    d <- read.delim(x)
+##    strsplit(d[[2]], ",")
+## })
+## )
+## present <- unique(call.list)
+## present[!present %in% gs[["Your.username.on.the.class.Discord.server"]]]
+
+## and never attended class..
+## gs[["Your.username.on.the.class.Discord.server"]][!gs[["Your.username.on.the.class.Discord.server"]] %in% present]
+
diff --git a/assessment_and_tracking/track_participation.R b/assessment_and_tracking/track_participation.R
new file mode 100644
index 0000000..28b8a4e
--- /dev/null
+++ b/assessment_and_tracking/track_participation.R
@@ -0,0 +1,25 @@
+setwd("~/online_communities/coldcallbot/data/")
+
+library(ggplot2)
+library(data.table)
+
+gs <- read.delim("student_information.tsv")
+d <- gs[,c(2,4)]
+colnames(d) <- c("student.num", "unique.name")
+
+call.list <- do.call("rbind", lapply(list.files(".", pattern="^call_list-.*tsv$"), function (x) {read.delim(x, stringsAsFactors=FALSE)[,1:4]}))
+
+colnames(call.list) <- gsub("_", ".", colnames(call.list))
+
+table(call.list$unique_name[call.list$answered])
+
+## drop calls where the person wasn't present
+call.list.full <- call.list
+call.list[!call.list$answered,]
+call.list <- call.list[call.list$answered,]
+
+call.counts <- data.frame(table(call.list$unique.name))
+colnames(call.counts) <- c("unique.name", "num.calls")
+
+d <- merge(d, call.counts, all.x=TRUE, all.y=TRUE, by="unique.name"); d
+
diff --git a/coldcall.py b/coldcall.py
new file mode 100644
index 0000000..1ba96a5
--- /dev/null
+++ b/coldcall.py
@@ -0,0 +1,117 @@
+#!/usr/bin/env python3
+
+from collections import defaultdict
+from datetime import datetime
+from random import choices
+from os import listdir
+from csv import DictReader
+
+import os.path
+import re
+
+class ColdCall():
+    def __init__ (self, record_attendance=True):
+        self.today = str(datetime.date(datetime.now()))
+        # how much less likely should it be that a student is called upon?
+        self.weight = 2
+        self.record_attendance = record_attendance
+
+        # filenames
+        self.__fn_studentinfo = "data/student_information.tsv"
+        self.__fn_daily_calllist = f"data/call_list-{self.today}.tsv"
+        self.__fn_daily_attendance = f"data/attendance-{self.today}.tsv"
+
+        self.preferred_names = self.__get_preferred_names()
+        
+    def __load_prev_questions(self):
+        previous_questions = defaultdict(int)
+
+        for fn in listdir("./data/"):
+            if re.match("call_list-\d{4}-\d{2}-\d{2}.tsv", fn):
+                with open(f"./data/{fn}", 'r') as f:
+                    for row in DictReader(f, delimiter="\t"):
+                        if not row["answered"] == "FALSE":
+                            previous_questions[row["unique_name"]] += 1
+
+        return previous_questions
+
+    def __get_preferred_names(self):
+        # translate the unique name into the preferred students name,
+        # if possible, otherwise return the unique name
+
+        preferred_names = {}
+        with open(self.__fn_studentinfo, 'r') as f:
+            for row in DictReader(f, delimiter="\t"):
+                preferred_names[row["Your username on the class Teams server"]] = row["Name you'd like to go by in class"]
+
+        return(preferred_names)
+        
+    def __get_preferred_name(self, selected_student):
+        if selected_student in self.preferred_names:
+            return self.preferred_names[selected_student]
+        else:
+            return None
+
+    def __select_student_from_list (self, students_present):
+        prev_questions = self.__load_prev_questions()
+        
+        # created a weighted list by starting out with everybody 1
+        weights = {s : 1 for s in students_present}
+        
+        for s in students_present:
+            for i in range(0, prev_questions[s]):
+                # reduce the weight by a factor of 1/weight each time the student has been called upon
+                weights[s] = weights[s] / self.weight
+
+        # choose one student from the weighted list
+        # print(weights) # DEBUG LINE
+        return choices(list(weights.keys()), weights=list(weights.values()), k=1)[0]
+
+    def __record_attendance(self, students_present):
+        # if it's the first one of the day, write it out
+        if not os.path.exists(self.__fn_daily_attendance):
+            with open(self.__fn_daily_attendance, "w") as f:
+                print("\t".join(["timestamp", "attendance_list"]), file=f)
+
+        # open for appending the student
+        with open(self.__fn_daily_attendance, "a") as f:
+            print("\t".join([str(datetime.now()),
+                             ",".join(students_present)]),
+                  file=f)
+
+    def __record_coldcall(self, selected_student):
+        # if it's the first one of the day, write it out
+        if not os.path.exists(self.__fn_daily_calllist):
+            with open(self.__fn_daily_calllist, "w") as f:
+                print("\t".join(["unique_name", "timestamp", "answered", "assessment"]), file=f)
+
+        # open for appending the student
+        with open(self.__fn_daily_calllist, "a") as f:
+            print("\t".join([selected_student, str(datetime.now()),
+                             "MISSING", "MISSING"]), file=f)
+
+    def coldcall(self, students_present):
+        selected_student = self.__select_student_from_list(students_present)
+
+        # record the called-upon student in the right place
+        if self.record_attendance:
+            self.__record_attendance(students_present)
+        self.__record_coldcall(selected_student)
+
+        preferred_name = self.__get_preferred_name(selected_student)
+        if preferred_name:
+            coldcall_message = f"{preferred_name} (@{selected_student}), you're up!"
+        else:
+            coldcall_message = f"@{selected_student}, you're up!"
+        return coldcall_message
+
+# cc = ColdCall()
+
+# test_student_list = ["jordan", "Kristen Larrick", "Madison Heisterman", "Maria.Au20", "Laura (Alia) Levi", "Leona Aklipi", "anne", "emmaaitelli", "ashleylee", "allie_partridge", "Tiana_Cole", "Hamin", "Ella Qu", "Shizuka", "Ben Baird", "Kim Do", "Isaacm24", "Sam Bell", "Courtneylg"]
+# print(cc.coldcall(test_student_list))
+
+# test_student_list = ["jordan", "Kristen Larrick", "Mako"]
+# print(cc.coldcall(test_student_list))
+
+# test_student_list = ["jordan", "Kristen Larrick"]
+# print(cc.coldcall(test_student_list))
diff --git a/coldcallbot-manual.py b/coldcallbot-manual.py
new file mode 100755
index 0000000..a4268ea
--- /dev/null
+++ b/coldcallbot-manual.py
@@ -0,0 +1,15 @@
+#!/usr/bin/env python3
+
+from coldcall import ColdCall
+import re
+
+## create the coldcall object
+cc = ColdCall(record_attendance=False)
+
+student_list = cc.preferred_names
+
+# print out 100 students
+
+for i in range(100):
+    print(f"{i + 1}. {cc.coldcall(student_list)} [ ] [ ]\n")
+
diff --git a/coldcallbot.py b/coldcallbot.py
new file mode 100755
index 0000000..392028a
--- /dev/null
+++ b/coldcallbot.py
@@ -0,0 +1,43 @@
+#!/usr/bin/env python3
+
+from coldcall import ColdCall
+import re
+import discord
+
+## create the coldcall object
+cc = ColdCall()
+
+class ColdCallBot (discord.Client):
+    async def on_ready(self):
+        print(f'Logged on as {self.user}! Ready for class!')
+
+    async def on_message(self, message):
+        if message.author == self.user:
+            return
+
+        if message.content.startswith('$next'):
+            classroom = discord.utils.get(message.guild.voice_channels, name='Classroom Voice')
+            
+            present_students = []
+            for member in classroom.members:
+                if 'Students' in [r.name for r in member.roles]:
+                    present_students.append(re.sub(r'^(.*)\#.*$', r'\1', member.name))
+
+            # print who is online
+            print(f'currently online: {",".join(present_students)}')
+
+            if len(present_students) < 1:
+                msg_text = "I don't see any students currently in the Classroom Voice channel!"
+            else:
+                msg_text = cc.coldcall(present_students)
+                
+            await message.channel.send(msg_text)
+
+# this is necessary to get information about who is online
+intents = discord.Intents.default()
+intents.members = True
+intents.presences = True
+
+ccb = ColdCallBot(intents=intents)
+ccb.run('CHANGEME')
+
diff --git a/data/.gitignore b/data/.gitignore
new file mode 100644
index 0000000..9efa8d5
--- /dev/null
+++ b/data/.gitignore
@@ -0,0 +1,3 @@
+questions_absence_histogram_combined.png
+questions_absenses_boxplots.png
+case_grades/
diff --git a/data/README b/data/README
new file mode 100644
index 0000000..6a67aed
--- /dev/null
+++ b/data/README
@@ -0,0 +1 @@
+This directory should contain data created by the repository in real time.
diff --git a/data/download_data.sh b/data/download_data.sh
new file mode 100755
index 0000000..8687509
--- /dev/null
+++ b/data/download_data.sh
@@ -0,0 +1,4 @@
+#!/bin/bash
+
+wget 'https://docs.google.com/spreadsheets/d/FIXME/export?gid=FIXME&format=tsv' -O 'student_information.tsv'
+
diff --git a/app.py b/flask_app/app.py
similarity index 100%
rename from app.py
rename to flask_app/app.py
diff --git a/static/main.css b/flask_app/static/main.css
similarity index 100%
rename from static/main.css
rename to flask_app/static/main.css
diff --git a/static/process_button.js b/flask_app/static/process_button.js
similarity index 100%
rename from static/process_button.js
rename to flask_app/static/process_button.js
diff --git a/templates/cold_caller.html b/flask_app/templates/cold_caller.html
similarity index 100%
rename from templates/cold_caller.html
rename to flask_app/templates/cold_caller.html
diff --git a/templates/group_maker.html b/flask_app/templates/group_maker.html
similarity index 100%
rename from templates/group_maker.html
rename to flask_app/templates/group_maker.html
diff --git a/templates/shuffler.html b/flask_app/templates/shuffler.html
similarity index 100%
rename from templates/shuffler.html
rename to flask_app/templates/shuffler.html
diff --git a/flask_app/test.csv b/flask_app/test.csv
new file mode 100644
index 0000000..dfe743c
--- /dev/null
+++ b/flask_app/test.csv
@@ -0,0 +1,71 @@
+name,date,answered,assessment
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,
+owen,2022-01-01,,
+dad,2022-01-01,,
+dad,2022-01-01,,