colnames(tbl.tmp) <- c("Statistics", "")
kable(tbl.tmp)
-
```
The program was similar in size to the last 2-3 years in terms of the number of submissions. OpenSym is a small but mature and stable venue for research on open collaboration. This year was also similar, although slightly more competitive, in terms of the conference acceptance rate (`r round(num.papers.accepted / (nrow(submissions) - 2)*100)`%—it had been slightly above 50% in previous years).
# Scores and Reviews
-As in previous years, review was single blind in that reviewers' identities are hidden but authors identities are not. Each papers received between 3 and 4 reviews plus a metareview by the Associate Chair assigned to the paper. Almost all papers received 3 reviews but ACs were encouraged to call in a 4th reviewer at any point in the process. In addition to the text of the reviews, we used a -3 to +3 scoring system where papers that are seen as borderline will be scored as 0. Reviewers scored papers using half-point increments.
+As in previous years, review was single blind in that reviewers' identities are hidden but authors identities are not. Each papers received between 3 and 4 reviews plus a metareview by the Associate Chair assigned to the paper. Almost all papers received 3 reviews but ACs were encouraged to call in a 4th reviewer at any point in the process. In addition to the text of the reviews, we used a -3 to +3 scoring system where papers that are seen as borderline will be scored as 0. Reviewers scored papers using full-point increments.
```{r, echo=FALSE}
## generate the score graphs
```
-The figure above shows scores for each paper submitted. The vertical grey lines reflect the distribution of scores where the minimum and maximum scores for each paper are the ends of the lines. The colored dots show the arithmetic mean for each score (unweighted by reviewer confidence). Colors show whether the papers were accepted, rejected, or presented as a poster. It's important to keep in mind that two papers were *submitted* as posters. Although Associate Chairs made the final decisions on a case-by-case basis, every paper that had an average score of less than 0 (the horizontal orange line) was rejected and most (but not all) papers with positive average scores were accepted. We ultimately accepted `r num.papers.accepted` papers (`r paste(round(num.papers.accepted / (nrow(submissions) - 2)*100), "%", sep="")`) of those submitted.
+The figure above shows scores for each paper submitted. The vertical grey lines reflect the distribution of scores where the minimum and maximum scores for each paper are the ends of the lines. The colored dots show the arithmetic mean for each score (unweighted by reviewer confidence). Colors show whether the papers were accepted, rejected, or presented as a poster. It's important to keep in mind that two papers were *submitted* as posters.
+
+Although Associate Chairs made the final decisions on a case-by-case basis, every paper that had an average score of less than 0 (the horizontal orange line) was rejected and most (but not all) papers with positive average scores were accepted. Although a positive average score seemed to be a required publication, negative individual scores weren't necessary showstoppers and we accepted `r table(unique(na.omit(scores.after$sub.id[scores.after$score < 0])) %in% submissions$sub.id[submissions$result == "ACCEPT" & submissions$type == "full paper"])["TRUE"]` papers with at least one negative score. We ultimately accepted `r num.papers.accepted` papers (`r paste(round(num.papers.accepted / (nrow(submissions) - 2)*100), "%", sep="")`) of those submitted.
# Rebuttals
projects / opensym2017_postmortem.git / blobdiff