Okay, thatâs a lovely univariate time series plot. Now letâs make this more sophisticated and interesting by incorporating more data, more dimensions, and more variables. In order to do that, I want to start with a little detour into data structures. Try to stay with meâthis turns out to be super important for working more efficiently with tools like ggplot as well as learning to manage more complex statistical analysis strategies (that we wonât really cover in the course, but so be it).
+
Long versus wide data (and why long data is often helpful)
+
So now you want to plot a multivariate time series (e.g., the same plot for more than one state and/or for more than one measure). As always, you have a number of options, but the most effective way to achieve this with ggplot involves learning to work with âlongerâ data.
+
Thus far, we have worked mostly with âwideâ format data where (nearly) every row corresponds to a single unit/observation and every column corresponds to a distinct variable (for which we usually have no more than one value attributed to any unit/observation). This often results in wider format data that is great for many things. However, it turns out that longer format data can be super helpful for a number of purposes. Producing richer, multidimensional ggplot visualizations is one of them.
Consider the format of my tidied dataframe that I used for plotting:
il_weekly_cases
-
## # A tibble: 38 x 3
+## # A tibble: 39 x 3
## weekdate new_cases date
## <fct> <dbl> <date>
## 1 2020-01-20 1 2020-01-20
@@ -1733,11 +1744,11 @@ plot4
## 8 2020-03-09 87 2020-03-09
## 9 2020-03-16 953 2020-03-16
## 10 2020-03-23 3568 2020-03-23
-## # ⦠with 28 more rows
-
This dataframe is in a âwideâ format. Each row is a week and each column is a variable unique to that week.
-
Our original dataframe was a bit âlongerâ:
+## # ⦠with 29 more rows
+
This dataframe is in a pretty âlongâ format. Each row is a week and each column is a variable unique to that week (okay, I could consolidate my weekdate and date columns into just one, but thatâs not really the point here. The idea is that thereâs minimal redundant information in the rows and in the columns).
+
Our original dataframe was also pretty âlongâ:
d
-
## # A tibble: 12,004 x 5
+## # A tibble: 12,334 x 5
## date state fips cases deaths
## <date> <fct> <chr> <dbl> <dbl>
## 1 2020-01-21 Washington 53 1 0
@@ -1750,8 +1761,10 @@ plot4
## 8 2020-01-25 Washington 53 1 0
## 9 2020-01-26 Arizona 04 1 0
## 10 2020-01-26 California 06 2 0
-## # ⦠with 11,994 more rows
-
We see multiple observations per state (I think I would say the units or rows correspond to âstate-datesâ or something like that). Itâs not completely âlongâ however, because we also have multiple columns corresponding to the two variables of interest: cases and deaths. The point I want to make is that there are a number of ways we can make this data âlonger.â For the purposes of producing a multi-state plot like the one above, the most important of these is going to involve dropping the step where I filtered by state=="Illinois" and replacing by a group_by step before I create my weekdate variable. Iâm also going to go ahead and drop the date and fips variables because theyâre just getting in my way at this point. Iâll start there
+## # ⦠with 12,324 more rows
+
Here we have multiple observations per state (I think I would say the units or rows correspond to âstate-datesâ or something like that). Itâs not as âlongâ as possible, though, because we also have multiple columns corresponding to the two variables of interest: cases and deaths.
+
For the purposes of producing a multi-state and multivariate set of plots, the most important thing I want to do is consolidate my dataset into a format where I have the following columns: date (collapsed into weeks), state, variable (which will either have a value of new cases or new deaths), and a column for value that will hold the corresponding state-week count for the variable in each row. If that doesnât make sense, donât worry, weâll get there soon enough.
+
Doing this involves a different approach to tidying up my data. Iâll start by dropping the step where I filtered by state=="Illinois" and replacing it with a group_by step before I create my weekdate variable. Iâm also going to go ahead and drop the date and fips variables because theyâre just getting in my way.
weekly <- d %>%
group_by(state) %>%
mutate(
@@ -1759,7 +1772,7 @@ plot4
) %>%
select(state, cases, deaths, weekdate)
weekly
-
## # A tibble: 12,004 x 4
+## # A tibble: 12,334 x 4
## # Groups: state [55]
## state cases deaths weekdate
## <fct> <dbl> <dbl> <fct>
@@ -1773,17 +1786,19 @@ weekly
## 8 Washington 1 0 2020-01-20
## 9 Arizona 1 0 2020-01-20
## 10 California 2 0 2020-01-20
-## # ⦠with 11,994 more rows
-
Iâm getting somewhere with this, I promise. One of the principles of âtidyâ data is to make it so that every variable has a column, every observation has a row, and every value has a cell. Right now, Iâve got multiple observations for each state-week spread across multiple rows. Remember that my cases and deaths variables are actually cumulative counts, so I really only need to store the maximum value for each state-week in order to calculate the new cases per state-week. Letâs see what to do about that:
+## # ⦠with 12,324 more rows
+
Now Iâve got multiple observations for each state-week spread across multiple rows (because my rows were structured around a more granular measure of time). My next move is to collapse these into a single observation for each state-week. Remember that my cases and deaths variables are still cumulative counts, so as I do this aggregation by week I will only need to store the maximum value for each state-week in order to calculate the number of new cases per state-week.
tidy_weekly <- weekly %>%
group_by(state, weekdate) %>%
summarize(
cum_cases = max(cases, na.rm = T),
cum_deaths = max(deaths, na.rm = T)
)
-
tidy_weekly$weekdate <- as.Date(as.character(tidy_weekly$weekdate))
-
-tidy_weekly <- tidy_weekly %>%
+Notice that the call to group_by groups by multiple variables. The order here matters! If I reversed it to read group_by(weekdate, state) the results would be very different. With the correct ordering, I have things bundled up into state-week sub-groups and then I move on to calculate the maximum value of cumulative cases within each bundle.
+Next, I can fix up my weekdate variable again so that it is a Date object.
+tidy_weekly$weekdate <- as.Date(as.character(tidy_weekly$weekdate))
+This will allow me to do some sorting within my state-week bundles to ensure things are in the proper order before I convert my weekly cumulative case count into weekly new case counts.
+tidy_weekly <- tidy_weekly %>%
group_by(state) %>%
arrange(-desc(weekdate)) %>%
mutate(
@@ -1792,7 +1807,7 @@ tidy_weekly <- tidy_weekly %>%
)
tidy_weekly
-## # A tibble: 1,780 x 6
+## # A tibble: 1,835 x 6
## # Groups: state [55]
## state weekdate cum_cases cum_deaths new_cases new_deaths
## <fct> <date> <dbl> <dbl> <dbl> <dbl>
@@ -1806,8 +1821,9 @@ tidy_weekly
## 8 Massachusetts 2020-01-27 1 0 1 0
## 9 Washington 2020-01-27 1 0 0 0
## 10 Arizona 2020-02-03 1 0 0 0
-## # ⦠with 1,770 more rows
-This is headed in the right direction. For some purposes, though, itâs still not quite âlongâ enough For starters, I can drop the cumulative cases and deaths columns. The other thing I can do is âpivotâ the data to organize the new_cases and new_deaths measures a little differently. To manage this, Iâll use the pivot_longer() function (part of the tidyr package from the tidyverse). I will also go ahead and coerce my weekdate into a Date object again:
+## # ⦠with 1,825 more rows
+
Weâre much closer to our goal now!
+
I can go ahead and drop the cumulative cases and deaths columns with a call to select in my next step. Then the big next (and nearly final) step is to âpivotâ the data to organize the new_cases and new_deaths measures in the way I described above. To manage this, Iâll use the pivot_longer() function (part of the tidyr package from the tidyverse):
long_weekly <- tidy_weekly %>%
select(state, weekdate, new_cases, new_deaths) %>%
pivot_longer(
@@ -1817,7 +1833,7 @@ tidy_weekly
)
long_weekly
-
## # A tibble: 3,560 x 4
+## # A tibble: 3,670 x 4
## # Groups: state [55]
## state weekdate variable value
## <fct> <date> <chr> <dbl>
@@ -1831,38 +1847,39 @@ long_weekly
## 8 Washington 2020-01-20 new_deaths 0
## 9 Arizona 2020-01-27 new_cases 0
## 10 Arizona 2020-01-27 new_deaths 0
-## # ⦠with 3,550 more rows
-
Can you see what that did? I now have two rows of data for every state-week. One that contains a value for new_cases and one that contains a value for new_deaths. Both of those variables have been âpivotedâ into a single variable column.
-
Before we move forward Iâm going to clean up the values of variable.
+## # ⦠with 3,660 more rows
+
Can you see what that did? I now have two rows of data for every state-week. One row contains a value for new_cases and one contains a value for new_deaths. Both of those variables have been âpivotedâ into a single variable column and their corresponding values recorded in another new column. Note that this makes our dataframe a little longer even though it does not technically reduce the âwidthâ of this particular dataset (because weâve taken two columns and pivoted them to createâ¦two different columns). However, consider that we could accommodate as many additional numerical variables and values as we might like in this manner and you can start to see how this pivoting step could result in much longer data (the length becomes a function of the number of units in your dataset and the variables you include in your pivoting step).
+
Before we move forward Iâm also going to clean up the values of variable. This turns out to be helpful later on when weâre plotting, but makes more sense to implement here before I start creating any plot layers.
long_weekly <- long_weekly %>%
mutate(
variable = recode(variable, new_cases = "new cases", new_deaths = "new deaths")
)
-
Okay, prepared with my tidy_weekly and my long_weekly tibbles, Iâm now ready to generate some more interesting multidimensional plots. Letâs start with the same sort of time series of new cases we made for Illinois before so we can see how to replicate that with this new data structure:
+
Okay, prepared with my long_weekly tibble, Iâm now ready to generate some more interesting and multidimensional plots. Letâs start with the same univariate time series of new cases we made for Illinois before so we can see how to replicate that figure with this new data structure:
long_weekly %>%
filter(
state == "Illinois" & variable == "new cases"
) %>%
ggplot(aes(weekdate, value)) +
geom_line()
-
-
Now we can easily plot Illinois cases against deaths from the same tibble:
+
+
With our âlongerâ data format, we can plot Illinois cases against deaths from the same tibble by incorporating a color=variable argument :
long_weekly %>%
filter(state == "Illinois") %>%
ggplot(aes(weekdate, value, color = variable)) +
geom_line()
-
-
That plot isnât so great because the death counts are dwarfed by the case counts. Thank goodness!
-
Now letâs compare Illinois case counts against some its neighbors in the upper midwest:
+
+
Unfortunately, that plot isnât so great because the death counts are dwarfed by the case counts (thank goodness!).
+
Now letâs compare Illinois case counts against some the neighboring states in the upper midwest:
upper_midwest <- c("Illinois", "Michigan", "Wisconsin", "Iowa", "Minnesota")
long_weekly %>%
filter(state %in% upper_midwest & variable == "new cases") %>%
ggplot(aes(weekdate, value, color = state)) +
geom_line()
-
-
Now thatâs getting a bit more interesting.
-
What about finding some way to also incorporate the death counts? Well, ggplot has another layer option called âfacetsâ that can help produce multiple plots and present them alongside each other (or in a grid). Hereâs an example that creates a faceted âgridâ (really just a side-by-side comparison) of case counts and deaths for the same five states.
+
+
Notice that I use the %in% operator to filter for the values of the state vector that are âinâ the upper_midwest vector (see help(%in%) for more).
+
Also notice that we now have ourselves a multivariate time series!
+
So now how about finding some way to also incorporate those death counts? If I just add them to this same plot weâll run into the same issue we did with the Illinois data because the death counts look tiny plotted on the same scale as the case counts. A good solution in such a situation is to create a second plot for weekly deaths that we can display together with this weekly cases plot that uses a differently scaled y-axis. The ggplot way to do this involves another type of layer called âfacets.â Hereâs an example that creates a faceted âgridâ (noy much of a grid since there are only two variables or categories weâre using to do the faceting) of weekly case counts and deaths for the same five states.
midwest_plot <- long_weekly %>%
filter(state %in% upper_midwest) %>%
ggplot(aes(weekdate, value, color = state)) +
@@ -1870,10 +1887,12 @@ long_weekly %>%
facet_grid(rows = vars(variable), scales = "free_y")
midwest_plot
-
-
Now we can clean up some of the other elements we worked on with the original plot (axes, title, etc.). Iâll bake that into a single chunk below.
+
+
Nice! Now we can clean up some of the other elements we worked on with the original plot (axes, title, etc.). Iâll bake that into a single chunk below.
midwest_plot + scale_x_date(date_labels = "%b", date_breaks = "1 month", date_minor_breaks = "1 week") + scale_y_continuous(label = comma) + labs(x = "Week (in 2020)", y = "", title = "COVID-19 cases in the Upper Midwest") + theme_light()
-
+
+
Thatâs it! Mission accomplished. Weâve got ourselves a nice concise visualization of weekly COVID-19 cases and deaths across five upper midwest states over nearly 8 months of the pandemic.
+
