typos for ch5 and more elaborate tidyverse example code w more verbose

diff --git a/os_exercises/ch5_exercises_solutions.html b/os_exercises/ch5_exercises_solutions.html
index ef2049493bf9d7a70462edff819c9f6bc43c1c15..a30a2721f2b888b1e821f0663139b6c1fc9ee48a 100644 (file)
--- a/os_exercises/ch5_exercises_solutions.html
+++ b/os_exercises/ch5_exercises_solutions.html
@@ -1569,15 +1569,15 @@ SE = \sqrt{\frac{p(1-p)}{n}}\\
 \end{array}\]</span></p></li>
 </ol>
 </div>
 \end{array}\]</span></p></li>
 </ol>
 </div>

-<div id="twitter-users-and-news-part-i" class="section level1">
-<h1>5.8 Twitter users and news, Part I</h1>
+<div id="twitter-users-news-i" class="section level1">
+<h1>5.8 Twitter users &amp; news I</h1>

 <p>The general formula for a confidence interval is <span class="math inline">\(point~estimate~±~z^*\times~SE\)</span>. Where <span class="math inline">\(z^*\)</span> corresponds to the z-score for the desired value of <span class="math inline">\(\alpha\)</span>.</p>
 <p>To estimate the interval from the data described in the question, identify the three different values. The point estimate is 45%, <span class="math inline">\(z^* = 2.58\)</span> for a 99% confidence level (that’s the number of standard deviations around the mean that ensure that 99% of a Z-score distribution is included), and <span class="math inline">\(SE = 2.4\%\)</span>. With this we can plug and chug:</p>
 <p><span class="math display">\[52\% ± 2.58 \times 2.4\%\]</span> And that yields: <span class="math display">\[95\% CI = (45.8\%, 58.2\%)\]</span></p>
 <p>Which means that from this data we are 99% confident that between 45.8% and 58.2% U.S. adult Twitter users get some news through the site.</p>
 </div>
 <p>The general formula for a confidence interval is <span class="math inline">\(point~estimate~±~z^*\times~SE\)</span>. Where <span class="math inline">\(z^*\)</span> corresponds to the z-score for the desired value of <span class="math inline">\(\alpha\)</span>.</p>
 <p>To estimate the interval from the data described in the question, identify the three different values. The point estimate is 45%, <span class="math inline">\(z^* = 2.58\)</span> for a 99% confidence level (that’s the number of standard deviations around the mean that ensure that 99% of a Z-score distribution is included), and <span class="math inline">\(SE = 2.4\%\)</span>. With this we can plug and chug:</p>
 <p><span class="math display">\[52\% ± 2.58 \times 2.4\%\]</span> And that yields: <span class="math display">\[95\% CI = (45.8\%, 58.2\%)\]</span></p>
 <p>Which means that from this data we are 99% confident that between 45.8% and 58.2% U.S. adult Twitter users get some news through the site.</p>
 </div>

-<div id="twitter-users-and-news-part-ii" class="section level1">
-<h1>5.10 Twitter users and news, Part II</h1>
+<div id="twitter-users-news-ii" class="section level1">
+<h1>5.10 Twitter users &amp; news II</h1>

 <ol style="list-style-type: lower-alpha">
 <li><p>False. See the answer to exercise 5.8 above. With <span class="math inline">\(\alpha = 0.01\)</span>, we can consult the 99% confidence interval. It includes 50% but also goes lower. A null hypothesis of <span class="math inline">\(p=0.50\)</span> would not be rejected at this level.</p></li>
 <li><p>False. The standard error of the sample proportion does not contain any information about the proportion of the population included in the sample. It estimates the variability of the sample proportion.</p></li>
 <ol style="list-style-type: lower-alpha">
 <li><p>False. See the answer to exercise 5.8 above. With <span class="math inline">\(\alpha = 0.01\)</span>, we can consult the 99% confidence interval. It includes 50% but also goes lower. A null hypothesis of <span class="math inline">\(p=0.50\)</span> would not be rejected at this level.</p></li>
 <li><p>False. The standard error of the sample proportion does not contain any information about the proportion of the population included in the sample. It estimates the variability of the sample proportion.</p></li>

diff --git a/os_exercises/ch5_exercises_solutions.rmd b/os_exercises/ch5_exercises_solutions.rmd
index 5965223686a3a28694f80bcd30cae31921ebb949..334cb9a4cdf8a148e75e817994e46d17f220d72a 100644 (file)
--- a/os_exercises/ch5_exercises_solutions.rmd
+++ b/os_exercises/ch5_exercises_solutions.rmd
@@ -57,7 +57,7 @@ SE = \sqrt{\frac{p(1-p)}{n}}\\
 \phantom{SE} = 0.0177
 \end{array}$$
 
 \phantom{SE} = 0.0177
 \end{array}$$
 

-# 5.8 Twitter users and news, Part I
+# 5.8 Twitter users & news I

 
 The general formula for a confidence interval is $point~estimate~±~z^*\times~SE$. Where $z^*$ corresponds to the z-score for the desired value of $\alpha$.
 
 
 The general formula for a confidence interval is $point~estimate~±~z^*\times~SE$. Where $z^*$ corresponds to the z-score for the desired value of $\alpha$.
 


@@ -70,7 +70,7 @@ $$95\% CI = (45.8\%, 58.2\%)$$
 
 Which means that from this data we are 99% confident that between 45.8% and 58.2% U.S. adult Twitter users get some news through the site.
 
 
 Which means that from this data we are 99% confident that between 45.8% and 58.2% U.S. adult Twitter users get some news through the site.
 

-# 5.10  Twitter users and news, Part II
+# 5.10  Twitter users & news II

 
 (a) False. See the answer to exercise 5.8 above. With $\alpha = 0.01$, we can consult the 99% confidence interval. It includes 50% but also goes lower. A null hypothesis of $p=0.50$ would not be rejected at this level.
 
 
 (a) False. See the answer to exercise 5.8 above. With $\alpha = 0.01$, we can consult the 99% confidence interval. It includes 50% but also goes lower. A null hypothesis of $p=0.50$ would not be rejected at this level.
 

diff --git a/psets/pset3-worked_solution.html b/psets/pset3-worked_solution.html
index 0d9a58933afb6340c72c70e6f47e8b48798f77bf..20d0eae0c0bc8325c02365bf6b305b7a87a10c64 100644 (file)
--- a/psets/pset3-worked_solution.html
+++ b/psets/pset3-worked_solution.html
@@ -1786,18 +1786,73 @@ round(
 ##   other                   0.00 0.00
 ##   white                   0.64 0.51</code></pre>
 <p>Here the noteworthy comparisons again arise within the black and hispanic categories. Both groups account for a substantially larger proportion of stops resulting in searches vs. those that do not result in searches.</p>
 ##   other                   0.00 0.00
 ##   white                   0.64 0.51</code></pre>
 <p>Here the noteworthy comparisons again arise within the black and hispanic categories. Both groups account for a substantially larger proportion of stops resulting in searches vs. those that do not result in searches.</p>

-<p>For the sake of completeness/comparison, here’s a way to do similar cross-tabulations in single a chunk of tidyverse code. I include conditional proportions of all stops to facilitate comparison with some of the tables I created earlier as well:</p>
+<p>For the sake of completeness/comparison, here’s a way to do similar cross-tabulations in chunks of tidyverse code. This first bit summarizes the number of stops and proportion of total stops accounted for within each of the categories of <code>subject_race</code>.</p>

 <pre class="r"><code>ilstops %&gt;%
   group_by(subject_race) %&gt;%
   filter(!is.na(subject_race)) %&gt;%
 <pre class="r"><code>ilstops %&gt;%
   group_by(subject_race) %&gt;%
   filter(!is.na(subject_race)) %&gt;%

+  summarize(
+    n_stops = n(),
+    prop_total_stops = round(n() / nrow(ilstops), digits = 3),
+  )</code></pre>
+<pre><code>## # A tibble: 5 x 3
+##   subject_race           n_stops prop_total_stops
+##   &lt;fct&gt;                    &lt;int&gt;            &lt;dbl&gt;
+## 1 asian/pacific islander    4053            0.032
+## 2 black                    25627            0.202
+## 3 hispanic                 16940            0.133
+## 4 other                      335            0.003
+## 5 white                    80105            0.63</code></pre>
+<p>In that block I first make a call to <code>group_by()</code> to tell R that I want it to run subsequent commands on the data “grouped” within the categories of <code>subject_race</code>. Then I pipe the grouped data to <code>summarize()</code>, which I use to calculate the number of stops within each group (in this data that’s just the number of observations within each group) as well as the proportion of total stops within each group.</p>
+<p>What about counting up the number and proportion of searches within each group? One way to think about that is as another call to <code>summarize()</code> (since, after all, I want to calculate the summary information for searches within the same groups). Within the Tidyverse approach to things, this kind of summarizing within groups and within another variable (<code>search_conducted</code> in this case) can be accomplished with the <code>across()</code> function.</p>
+<p>In general, the <code>across()</code> function seems to usually be made within a call to another verb like <code>summarize()</code> or <code>mutate()</code>. The syntax for <code>across()</code> is similar to these others. It requires two things: (1) at least one variable to summarize across (<code>search_conducted</code> here) and (2) the outputs I want.</p>
+<p>In this particular case, I’ll use it to calculate the within group sums of <code>search_conducted</code>. Notice that I also filter out the missing values from <code>search_conducted</code> before I call <code>summarize</code> here.</p>
+<pre class="r"><code>ilstops %&gt;%
+  group_by(subject_race) %&gt;%
+  filter(!is.na(subject_race), !is.na(search_conducted)) %&gt;%
+  summarize(
+    across(search_conducted, sum)
+  )</code></pre>
+<pre><code>## # A tibble: 5 x 2
+##   subject_race           search_conducted
+##   &lt;fct&gt;                             &lt;int&gt;
+## 1 asian/pacific islander               68
+## 2 black                              1806
+## 3 hispanic                           1049
+## 4 other                                14
+## 5 white                              3010</code></pre>
+<p>If I want <code>across()</code> to calculate more than one summary, I need to provide it a list of things (in a <code>name = value</code> format sort of similar to <code>summarize()</code> or <code>mutate()</code>).</p>
+<pre class="r"><code>ilstops %&gt;%
+  group_by(subject_race) %&gt;%
+  filter(!is.na(subject_race) &amp; !is.na(search_conducted)) %&gt;%
+  summarize(
+    across(
+      search_conducted,
+      list(
+        sum = sum,
+        over_n_stops = mean
+      )
+    )
+  )</code></pre>
+<pre><code>## # A tibble: 5 x 3
+##   subject_race           search_conducted_sum search_conducted_over_n_stops
+##   &lt;fct&gt;                                 &lt;int&gt;                         &lt;dbl&gt;
+## 1 asian/pacific islander                   68                        0.0168
+## 2 black                                  1806                        0.0707
+## 3 hispanic                               1049                        0.0620
+## 4 other                                    14                        0.0419
+## 5 white                                  3010                        0.0376</code></pre>
+<p>I can clean this up a bit by using two functions to the output in descending order by one of the columns. I do this with a nested call to two functions <code>arrange()</code> and <code>desc()</code>. I can also insert my earlier summary statistics for the number and proportions of stops by group back into the table.</p>
+<pre class="r"><code>ilstops %&gt;%
+  group_by(subject_race) %&gt;%
+  filter(!is.na(subject_race) &amp; !is.na(search_conducted)) %&gt;%

   summarize(
     n_stops = n(),
     prop_total_stops = round(n() / nrow(ilstops), digits = 3),
     across(
       search_conducted,
       list(
   summarize(
     n_stops = n(),
     prop_total_stops = round(n() / nrow(ilstops), digits = 3),
     across(
       search_conducted,
       list(

-        sum = ~ sum(.x, na.rm = TRUE),
-        over_n_stops = ~ round(mean(.x, na.rm = TRUE), digits = 3)
+        sum = sum,
+        over_n_stops = mean

       )
     )
   ) %&gt;%
       )
     )
   ) %&gt;%


@@ -1805,12 +1860,11 @@ round(
 <pre><code>## # A tibble: 5 x 5
 ##   subject_race    n_stops prop_total_stops search_conducted… search_conducted_o…
 ##   &lt;fct&gt;             &lt;int&gt;            &lt;dbl&gt;             &lt;int&gt;               &lt;dbl&gt;
 <pre><code>## # A tibble: 5 x 5
 ##   subject_race    n_stops prop_total_stops search_conducted… search_conducted_o…
 ##   &lt;fct&gt;             &lt;int&gt;            &lt;dbl&gt;             &lt;int&gt;               &lt;dbl&gt;

-## 1 white             80105            0.63               3010               0.038
-## 2 black             25627            0.202              1806               0.071
-## 3 hispanic          16940            0.133              1049               0.062
-## 4 asian/pacific …    4053            0.032                68               0.017
-## 5 other               335            0.003                14               0.042</code></pre>
-<p>Notice the use of <code>across()</code> within a call to <code>summarize()</code> provides one way to calculate conditional summariy info. I also use a nested call to <code>arrange()</code> and <code>desc()</code> at the end to sort my results in descending order by one of the columns.</p>
+## 1 white             80043            0.63               3010              0.0376
+## 2 black             25548            0.201              1806              0.0707
+## 3 hispanic          16914            0.133              1049              0.0620
+## 4 asian/pacific …    4049            0.032                68              0.0168
+## 5 other               334            0.003                14              0.0419</code></pre>

 </div>
 <div id="searches-by-date" class="section level3">
 <h3>Searches by <code>date</code></h3>
 </div>
 <div id="searches-by-date" class="section level3">
 <h3>Searches by <code>date</code></h3>

diff --git a/psets/pset3-worked_solution.pdf b/psets/pset3-worked_solution.pdf
index ebe5c8a458434262cc9789644b30fb7da16c46e3..c9d3bcdfa28cc6abbe6a6c42cc489c216bd37304 100644 (file)
Binary files a/psets/pset3-worked_solution.pdf and b/psets/pset3-worked_solution.pdf differ

diff --git a/psets/pset3-worked_solution.rmd b/psets/pset3-worked_solution.rmd
index e4b6d32985e421c6b90e7d5890953ffc4dddd8ce..f99c666c989f45757448c22cfd9f6bd358087000 100644 (file)
--- a/psets/pset3-worked_solution.rmd
+++ b/psets/pset3-worked_solution.rmd
@@ -176,26 +176,65 @@ round(
 ```
 Here the noteworthy comparisons again arise within the black and hispanic categories. Both groups account for a substantially larger proportion of stops resulting in searches vs. those that do not result in searches. 
 
 ```
 Here the noteworthy comparisons again arise within the black and hispanic categories. Both groups account for a substantially larger proportion of stops resulting in searches vs. those that do not result in searches. 
 

-For the sake of completeness/comparison, here's a way to do similar cross-tabulations in single a chunk of tidyverse code. I include conditional proportions of all stops to facilitate comparison with some of the tables I created earlier as well:  
-```{r tidyverse crosstabs}
+For the sake of completeness/comparison, here's a way to do similar cross-tabulations in chunks of tidyverse code. This first bit summarizes the number of stops and proportion of total stops accounted for within each of the categories of `subject_race`.
+```{r tidyverse stops by subject_race}

 ilstops %>%
   group_by(subject_race) %>%
   filter(!is.na(subject_race)) %>%
   summarize(
     n_stops = n(),
     prop_total_stops = round(n() / nrow(ilstops), digits=3),
 ilstops %>%
   group_by(subject_race) %>%
   filter(!is.na(subject_race)) %>%
   summarize(
     n_stops = n(),
     prop_total_stops = round(n() / nrow(ilstops), digits=3),

+    )
+```
+In that block I first make a call to `group_by()` to tell R that I want it to run subsequent commands on the data "grouped" within the categories of `subject_race`. Then I pipe the grouped data to `summarize()`, which I use to calculate the number of stops within each group (in this data that's just the number of observations within each group) as well as the proportion of total stops within each group.  
+
+What about counting up the number and proportion of searches within each group? One way to think about that is as another call to `summarize()` (since, after all, I want to calculate the summary information for searches within the same groups). Within the Tidyverse approach to things, this kind of summarizing within groups and within another variable (`search_conducted` in this case) can be accomplished with the `across()` function. 
+
+In general, the `across()` function seems to usually be made within a call to another verb like `summarize()` or `mutate()`. The syntax for `across()` is similar to these others. It requires two things: (1) at least one variable to summarize across (`search_conducted` here) and (2) the outputs I want.
+
+In this particular case, I'll use it to calculate the within group sums of `search_conducted`. Notice that I also filter out the missing values from `search_conducted` before I call `summarize` here.
+```{r }
+ilstops %>%
+  group_by(subject_race) %>%
+  filter(!is.na(subject_race), !is.na(search_conducted)) %>%
+  summarize(
+    across(search_conducted, sum)
+    )
+```
+If I want `across()` to calculate more than one summary, I need to provide it a list of things (in a `name = value` format sort of similar to `summarize()` or `mutate()`).  
+
+```{r}
+ilstops %>%
+  group_by(subject_race) %>%
+  filter(!is.na(subject_race) & !is.na(search_conducted)) %>%
+  summarize(

     across(
       search_conducted,
       list(
     across(
       search_conducted,
       list(

-        sum = ~ sum(.x, na.rm=TRUE),
-        over_n_stops = ~ round(mean(.x, na.rm=TRUE), digits=3)  
+        sum = sum,
+        over_n_stops = mean

         )
       )
         )
       )

-    ) %>%
-      arrange(desc(n_stops))
+    )

 ```
 ```

+I can clean this up a bit by using two functions to the output in descending order by one of the columns. I do this with a nested call to two functions `arrange()` and `desc()`. I can also insert my earlier summary statistics for the number and proportions of stops by group back into the table.

 
 

-Notice the use of `across()` within a call to `summarize()` provides one way to calculate conditional summariy info. I also use a nested call to `arrange()` and `desc()` at the end to sort my results in descending order by one of the columns.  
+```{r}
+ilstops %>%
+  group_by(subject_race) %>%
+  filter(!is.na(subject_race) & !is.na(search_conducted)) %>%
+  summarize(
+    n_stops = n(),
+    prop_total_stops = round(n() / nrow(ilstops), digits=3),
+    across(
+      search_conducted,
+      list(
+        sum = sum,
+        over_n_stops = mean)  
+        )
+      ) %>%
+      arrange(desc(n_stops))
+```

 
 ### Searches by `date`
 
 
 ### Searches by `date`