-
-
-
-
-
-
-
-
-
Setup
-
library(tidyverse)
-library(tictoc)
-library(broom)
-library(patchwork)
-
-
-
Objectives
-
-- How to use
purrr to automate workflow in a cleaner, faster, and more extendable way
-
-
-
-
Copy-and-paste programming
-
-Copy-and-paste programming, sometimes referred to as just pasting, is the production of highly repetitive computer programming code, as produced by copy and paste operations. It is primarily a pejorative term; those who use the term are often implying a lack of programming competence. It may also be the result of technology limitations (e.g., an insufficiently expressive development environment) as subroutines or libraries would normally be used instead. However, there are occasions when copy-and-paste programming is considered acceptable or necessary, such as for boilerplate, loop unrolling (when not supported automatically by the compiler), or certain programming idioms, and it is supported by some source code editors in the form of snippets. - Wikipedia
-
-
-
# Data
-df <- tibble("a" = -99,
- "b" = -99,
- "c" = -99,
- "d" = -99)
-
# Copy and paste
-df$a[df$a == -99] <- NA
-df$b[df$b == -99] <- NA
-df$c[df$c == -99] <- NA
-df$d[df$d == -99] <- NA
-
-- Challenge 1. Explain why this solution is not very efficient. (e.g., If
df$a[df$a == -99] <- NA has an error, how are you going to fix it?) A solution is not scalable if it’s not automatable and, thus, scalable.
-
-
-
-
Using a function
-
-Let’s recall what’s function in R: input + computation + output
If you write a function, you gain efficiency because you don’t need to copy and paste the computation part.
-
` function(input){
-
computation
-
return(output)
-
} `
-
# Function
-fix_missing <- function(x) {
- x[x == -99] <- NA
- # This is better
- return(x)
-}
-
-# Apply function to each column (vector)
-df$a <- fix_missing(df$a)
-df$b <- fix_missing(df$b)
-df$c <- fix_missing(df$c)
-df$d <- fix_missing(df$d)
-
-Challenge 2 Why using function is more efficient than 100% copying and pasting? Can you think about a way we can automate the process?
Many options for automation in R: for loop, apply family, etc.
Here’s a tidy solution comes from purrr package.
The power and joy of one-liner.
-
purrr::map_df(df, fix_missing)
-
-
-
-
map() is a higher-order function that applies a given function to each element of a list/vector.
-
-
- Input: Takes a vector/list.
-
-- Computation: Calls the function once for each element of the vector
-
-- Output: Returns in a list or whatever data format you prefer (e.g., `_df helper: dataframe`)
-
Challenge 3 If you run the code below, what’s going to be the data type of the output?
-
map_chr(df, fix_missing)
-
## a b c d
-## NA NA NA NA
-
-
# Built-in data
-data("airquality")
-
tic()
-
-out1 <- vector("double", ncol(airquality)) # Placeholder
-
-for (i in seq_along(airquality)) { # Sequence variable
-
- out1[[i]] <- mean(airquality[[i]], na.rm = TRUE) # Assign a computation result to each element
-
-}
-
-toc()
-
## 0.006 sec elapsed
-
tic()
-
-out1 <- airquality %>% map_dbl(mean, na.rm = TRUE)
-
-toc()
-
## 0.033 sec elapsed
-
-In short, map() is more readable, faster, and easily extensive with other data science tasks (e.g., wrangling, modeling, and visualization) using %>%.
Final point: Why not base R apply family?
-
Short answer: purrr::map() is simpler to write. For instance,
-
map_dbl(x, mean, na.rm = TRUE) = vapply(x, mean, na.rm = TRUE, FUN.VALUE = double(1))
-
Additional tips
-
Performance testing (profiling) is an important part of programming. tictoc() measures the time that needs to take to run a target function for once. If you want a more robust measure of timing as well as information on memory (speed and space both matter for performance testing), consider using the bench package that is designed for high precising timing of R expressions.
-
map_mark <- bench::mark(
-
- out1 <- airquality %>% map_dbl(mean, na.rm = TRUE)
-
- )
-
-map_mark
-
## # A tibble: 1 × 6
-## expression min median `itr/sec`
-## <bch:expr> <bch:tm> <bch:tm> <dbl>
-## 1 out1 <- airquality %>% map_dbl(mean, na.rm = TRUE) 62.1µs 78.1µs 11966.
-## # … with 2 more variables: mem_alloc <bch:byt>, gc/sec <dbl>
-
-
-
Applications
-
-- Many models
-
-
-- One popular application of
map() is to run regression models (or whatever model you want to run) on list-columns. No more copying and pasting for running many regression models on subgroups!
-
-
# Have you ever tried this?
-lm_A <- lm(y ~ x, subset(data, subgroup == "group_A"))
-lm_B <- lm(y ~ x, subset(data, subgroup == "group_B"))
-lm_C <- lm(y ~ x, subset(data, subgroup == "group_C"))
-lm_D <- lm(y ~ x, subset(data, subgroup == "group_D"))
-lm_E <- lm(y ~ x, subset(data, subgroup == "group_E"))
-
-- For more information on this technique, read the Many Models subchapter of the R for Data Science.
-
-
# Function
-lm_model <- function(df) {
- lm(Temp ~ Ozone, data = df)
-}
-
# Map
-models <- airquality %>%
- # Determines group variable
- group_by(Month) %>%
- nest() %>% # Create list-columns
- mutate(ols = map(data, lm_model)) # Map
-
# Add tidying
-tidy_lm_model <- purrr::compose( # compose multiple functions
- broom::tidy, # convert lm objects into tidy tibbles
- lm_model
-)
-
-tidied_models <- airquality %>%
- group_by(Month) %>%
- nest() %>% # Create list-columns
- mutate(ols = map(data, tidy_lm_model))
-
-tidied_models$ols[1]
-
## [[1]]
-## # A tibble: 2 × 5
-## term estimate std.error statistic p.value
-## <chr> <dbl> <dbl> <dbl> <dbl>
-## 1 (Intercept) 62.9 1.61 39.2 2.88e-23
-## 2 Ozone 0.163 0.0500 3.26 3.31e- 3
-
-- Simulations
-
-
A good friend of map() function is rerun() function. This comibination is really useful for simulations. Consider the following example.
-
-
set.seed(1234)
-
-small_n <- 100 ; k <- 1000 ; mu <- 500 ; sigma <- 20
-
-y_list <- rep(list(NA), k)
-
-for (i in seq(k)) {
-
- y_list[[i]] <- rnorm(small_n, mu, sigma)
-
-}
-
-y_means <- unlist(lapply(y_list, mean))
-
-qplot(y_means) +
- geom_vline(xintercept = 500, linetype = "dotted", color = "red")
-
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
-
-
-
small_n <- 100 ; k <- 1000
-
-y_tidy <- rerun(k, rnorm(small_n, mu, sigma))
-
-y_means_tidy <- map_dbl(y_tidy, mean)
-
-# Visualize
-(qplot(y_means) +
- geom_vline(xintercept = 500, linetype = "dotted", color = "red")) +
-(qplot(y_means_tidy) +
- geom_vline(xintercept = 500, linetype = "dotted", color = "red"))
-
## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
-## `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
-
-
-
-
-
-
-
-
-
. Please do so before attending the worskshop as it takes a while (especially, if you do it for the first time).
This work is licensed under a Creative Commons Attribution 4.0 International License.
-
-