From: Benjamin Mako Hill Date: Mon, 15 Jan 2018 03:43:05 +0000 (-0800) Subject: fix typos pointed out by kaylea X-Git-Url: https://code.communitydata.science/opensym2017_postmortem.git/commitdiff_plain/7cbdc92ed5fad3438f80af61b41719c2640df160?ds=inline;hp=81a31bb2835c4270d9303334c83c5c3e530c3a70 fix typos pointed out by kaylea --- diff --git a/opensym2017_postmortem.Rmd b/opensym2017_postmortem.Rmd index 62db158..b3296f0 100644 --- a/opensym2017_postmortem.Rmd +++ b/opensym2017_postmortem.Rmd @@ -91,7 +91,7 @@ In OpenSym 2017, we made several changes to the way the conference has been run: * **We allowed authors to write rebuttals before reviews were finalized.** Reviewers and ACs were allowed to modify their reviews and decisions based on rebuttals. * To assist in assigning papers to ACs and to reviewers, **we made extensive use of bidding**. This means we had to recruit the pool of reviewers before papers were submitted. -Although each of these things have been tried in other conferences, all were new to +Although each of these things have been tried in other conferences, all were new to OpenSym. # Overview @@ -157,7 +157,7 @@ ggplot(data=topics) + aes(x=topic, fill=decision) + theme(legend.position="bottom", legend.direction = "horizontal") ``` -The figure above shows a breakdown of papers in terms of these categories as well as indicators of how many papers in each group were accepted. Research on FLOSS and Wikimedia/Wikipedia continue to make up a sizable chunk of OpenSym's submissions and publications. That said, these now make up a minority of total submissions. Although Wikipedia and Wikimedia research made up a smaller proportion of submission pool, it was accepted at a higher rate. Also notable is the fact that 2017 saw an uptick in the number of papers on open innovation. I suspect this was due, at least in part, to work by the General Chair [Lorraine Morgan's](https://www.nuigalway.ie/our-research/people/lorrainemorgan/) involvement (she specializes in that area). Somewhat surprisingly to me, we had a number of submission about Bitcoin and blockchains. These are natural areas of growth for OpenSym but have never been a big part of work in our community in the past. +The figure above shows a breakdown of papers in terms of these categories as well as indicators of how many papers in each group were accepted. Research on FLOSS and Wikimedia/Wikipedia continue to make up a sizable chunk of OpenSym's submissions and publications. That said, these now make up a minority of total submissions. Although Wikipedia and Wikimedia research made up a smaller proportion of the submission pool, it was accepted at a higher rate. Also notable is the fact that 2017 saw an uptick in the number of papers on open innovation. I suspect this was due, at least in part, to work by the General Chair [Lorraine Morgan's](https://www.nuigalway.ie/our-research/people/lorrainemorgan/) involvement (she specializes in that area). Somewhat surprisingly to me, we had a number of submission about Bitcoin and blockchains. These are natural areas of growth for OpenSym but have never been a big part of work in our community in the past. # Scores and Reviews @@ -181,7 +181,7 @@ The figure above shows scores for each paper submitted. The vertical grey lines # Rebuttals -This was the first time that OpenSym used a rebuttal or author response and were thrilled with how it went. Although they were entire optional, almost everybody used it! Authors of `r length(reviews[reviews$label == "Author response","sub.id"])` of our `r nrow(submissions)` submissions (`r round(length(reviews[reviews$label == "Author response","sub.id"]) / nrow(submissions)*100)`%!) submitted rebuttals. +This was the first time that OpenSym used a rebuttal or author response and we are thrilled with how it went. Although they were entire optional, almost everybody used it! Authors of `r length(reviews[reviews$label == "Author response","sub.id"])` of our `r nrow(submissions)` submissions (`r round(length(reviews[reviews$label == "Author response","sub.id"]) / nrow(submissions)*100)`%!) submitted rebuttals. ```{r, echo=FALSE} # histogram of changes @@ -222,7 +222,7 @@ Although, I won't post any analysis or graphs, bidding worked well. With only tw Given a reviewer pool whose diversity of expertise matches that in your pool of authors, bidding works fantastically. *But everybody needs to bid*. The only problems with reviewers we had were with people that had failed to bid. It might be reviewers who don't bid are less committed to the conference, more overextended, more likely to drop things in general, etc. It might also be that reviewers who fail to bid get good matches become less interested, willing, or able to do their reviews well and on time. -Having used bidding twice as chair or track-chair, my sense is that bidding is a fantastic thing to incorporate into any conference review process. The major limitations are that you need to build a PC before the conference (rather than finding the perfect reviewers for specific papers) and you have to find ways to incentive or communicate the importance of getting your PC members to bid. +Having used bidding twice as chair or track-chair, my sense is that bidding is a fantastic thing to incorporate into any conference review process. The major limitations are that you need to build a PC before the conference (rather than finding the perfect reviewers for specific papers) and you have to find ways to incentivize or communicate the importance of getting your PC members to bid. # Conclusions