docs: some vignette tweaks

Author: maelle

vignettes/prudence.Rmd

@@ -1,8 +1,8 @@
 ---
-title: "Memory protection: Prudence"
+title: "Memory protection: controlling automatic materialization"
 output: rmarkdown::html_vignette
 vignette: >
-  %\VignetteIndexEntry{10 Memory protection: Prudence}
+  %\VignetteIndexEntry{10 Memory protection: controlling automatic materialization}
   %\VignetteEngine{knitr::rmarkdown}
   %\VignetteEncoding{UTF-8}
 ---
@@ -30,22 +30,19 @@ knitr::opts_chunk$set(
 Sys.setenv(DUCKPLYR_FALLBACK_COLLECT = 0)
 ```
 
-Unlike traditional data frames, duckplyr defers computation until absolutely necessary, allowing DuckDB to optimize execution.
-This article explains how to control the materialization of data while maintaining a seamless dplyr-like experience.
-
-
+This article explains how to control the materialization of data to maintain a seamless dplyr-like experience as well as to protect memory.
 
 ```{r attach}
 library(conflicted)
 library(dplyr)
 conflict_prefer("filter", "dplyr")
 ```
 
-## Introduction
+## dplyr drop-in replacement: eager data frames
 
-Data frames backed by duckplyr, with class `"duckplyr_df"`, behave as regular data frames in almost all respects.
+Data frames backed by duckplyr, with class `"duckplyr_df"`, behave as regular data frames in almost all respects from a user's perspective.
 In particular, direct column access like `df$x`, or retrieving the number of rows with `nrow()`, works identically.
-Conceptually, duckplyr frames are "eager": from a user's perspective, they behave like regular data frames.
+Therefore, conceptually, duckplyr frames are "eager".
 
 ```{r}
 df <-
@@ -60,11 +57,14 @@ df$y
 nrow(df)
 ```
 
-Under the hood, two key differences provide improved performance and usability:
-lazy materialization and prudence.
+Under the hood though, two key differences provide improved performance and usability:
+
+- **lazy materialization**: Unlike traditional data frames, duckplyr defers computation until absolutely necessary, i.e. lazily, allowing DuckDB to optimize execution.
+- **prudence**: Automatic materialization is controllable, as automatic materialization of large data might otherwise inadvertently lead to memory problems.
+
 The term "prudence" is introduced here to set a clear distinction from the concept of "laziness", and because "control of automatic materialization" is a mouthful.
 
-## Eager and lazy computation
+## DuckDB optimization: lazy evaluation
 
 For a duckplyr frame that is the result of a dplyr operation, accessing column data or retrieving the number of rows will trigger a computation that is carried out by DuckDB, not dplyr.
 In this sense, duckplyr frames are also "lazy": the computation is deferred until the last possible moment, allowing DuckDB to optimize the whole pipeline.
@@ -109,10 +109,11 @@ The result becomes available when accessed:
 system.time(mean_arr_delay_ewr$mean_arr_delay[[1]])
 ```
 
-### Comparison
+### Comparison with similar tools
 
-The functionality is similar to lazy tables in dbplyr and lazy frames in dtplyr.
+The functionality is similar to lazy tables in [dbplyr](https://dbplyr.tidyverse.org/) and lazy frames in [dtplyr](https://dtplyr.tidyverse.org/).
 However, the behavior is different: at the time of writing, the internal structure of a lazy table or frame is different from a data frame, and columns cannot be accessed directly.
+Users need to explicitly `collect()` the data, the data frame is not "eager" at all.
 
 |             | **Eager** 😃 | **Lazy** 😴 |
 |-------------|:------------:|:-----------:|
@@ -121,7 +122,7 @@ However, the behavior is different: at the time of writing, the internal structu
 | **dtplyr**  |              |      ✅     |
 | **duckplyr**|      ✅      |      ✅     |
 
-In contrast, with dplyr, each intermediate step and also the final result is a proper data frame, and computed right away, forfeiting the opportunity for optimization:
+In contrast, with [dplyr](https://dplyr.tidyverse.org/), each intermediate step and also the final result is a proper data frame, and computed right away, forfeiting the opportunity for optimization:
 
 ```{r}
 system.time(
@@ -139,31 +140,65 @@ system.time(
 
 See also the [duckplyr: dplyr Powered by DuckDB](https://duckdb.org/2024/04/02/duckplyr.html) blog post for more information.
 
-## Prudence
+## Memory protection: control of automatic materialization with `prudence`
 
 Being both "eager" and "lazy" at the same time introduces a challenge:
-it is too easy to accidentally trigger computation,
-which is prohibitive if an intermediate result is too large.
-Prudence is a setting for duckplyr frames that limits the size of the data that is materialized automatically.
+**it is too easy to accidentally trigger computation**,
+which is prohibitive if an intermediate result is too large to fit into memory.
 
-### Concept
+Fortunately, duckplyr frames have a setting called `prudence` that limits the size of the data that is materialized automatically,
+and that the user can choose based on the data size.
+
+### When to automatically materialize?
 
 Three levels of prudence are available:
 
-- _Lavish_: materialize the data right away, as in the first example.
-- _Frugal_: throw an error when attempting to access the data.
-- _Thrifty_: materialize the data if it is small, otherwise throw an error.
+- __lavish__: _always_ automatically materialize, as in the first example.
+- __frugal__: _never_ automatically materialize, throw an error when attempting to access the data.
+- __thrifty__: automatically materialize the data _if it is small_, otherwise throw an error.
 
 For lavish duckplyr frames, as in the two previous examples, the underlying DuckDB computation is carried out upon the first request.
 Once the results are computed, they are cached and subsequent requests are fast.
 This is a good choice for small to medium-sized data, where DuckDB can provide a nice speedup but materializing the data is affordable at any stage.
 This is the default for `duckdb_tibble()` and `as_duckdb_tibble()`.
 
 For frugal duckplyr frames, accessing a column or requesting the number of rows triggers an error.
-This is a good choice for large data sets where the cost of materializing the data may be prohibitive due to size or computation time, and the user wants to control when the computation is carried out.
+This is a good choice for large data sets where the cost of materializing the data may be prohibitive due to size or computation time, and the user wants to control when the computation is carried out and how (to memory, or to a file).
 Results can be materialized explicitly with `collect()` and other functions.
 
-Thrifty duckplyr frames are a compromise between lavish and frugal, discussed further below.
+Thrifty duckplyr frames are a compromise between lavish and frugal, discussed below.
+
+### Thrift
+
+Thrifty is a compromise between frugal and lavish.
+Materialization is allowed for data up to a certain size, measured in cells (values) and rows in the resulting data frame.
+
+```{r}
+nrow(flights)
+flights_partial <-
+  flights |>
+  duckplyr::as_duckdb_tibble(prudence = "thrifty")
+```
+
+With this setting, the data is materialized only if the result has fewer than 1,000,000 cells (rows multiplied by columns).
+
+```{r error = TRUE}
+flights_partial |>
+  select(origin, dest, dep_delay, arr_delay) |>
+  nrow()
+```
+
+The original input is too large to be materialized, so the operation fails.
+On the other hand, the result after aggregation is small enough to be materialized:
+
+```{r}
+flights_partial |>
+  count(origin) |>
+  nrow()
+```
+
+Thrifty is a good choice for data sets where the cost of materializing the data is prohibitive only for large results.
+This is the default for the ingestion functions like `read_parquet_duckdb()`.
 
 
 ### Example
@@ -198,7 +233,7 @@ flights_frugal[[1]]
 ```
 
 
-### Enforcing DuckDB operation
+### Side effect: Enforcing DuckDB operation
 
 For operations not supported by duckplyr, the original dplyr implementation is used as a fallback.
 As the original dplyr implementation accesses columns directly, the data must be materialized before a fallback can be executed.
@@ -224,7 +259,7 @@ flights_frugal |>
 By using operations supported by duckplyr and avoiding fallbacks as much as possible, your pipelines will be executed by DuckDB in an optimized way.
 
 
-### From frugal to lavish
+### Conversion between prudence levels
 
 A frugal duckplyr frame can be converted to a lavish one with `as_duckdb_tibble(prudence = "lavish")`.
 The `collect.duckplyr_df()` method triggers computation and converts to a plain tibble.
@@ -252,43 +287,20 @@ flights_frugal |>
   class()
 ```
 
-### Comparison
+### Comparison with similar tools
 
 Frugal duckplyr frames behave like lazy tables in dbplyr and lazy frames in dtplyr: the computation only starts when you *explicitly* request it with `collect.duckplyr_df()` or through other means.
 However, frugal duckplyr frames can be converted to lavish ones at any time, and vice versa.
 In dtplyr and dbplyr, there are no lavish frames: collection always needs to be explicit.
 
+## Conclusion
 
-## Thrift
-
-Thrifty is a compromise between frugal and lavish.
-Materialization is allowed for data up to a certain size, measured in cells (values) and rows in the resulting data frame.
-
-```{r}
-nrow(flights)
-flights_partial <-
-  flights |>
-  duckplyr::as_duckdb_tibble(prudence = "thrifty")
-```
-
-With this setting, the data is materialized only if the result has fewer than 1,000,000 cells (rows multiplied by columns).
-
-```{r error = TRUE}
-flights_partial |>
-  select(origin, dest, dep_delay, arr_delay) |>
-  nrow()
-```
-
-The original input is too large to be materialized, so the operation fails.
-On the other hand, the result after aggregation is small enough to be materialized:
+The duckplyr package provides 
 
-```{r}
-flights_partial |>
-  count(origin) |>
-  nrow()
-```
+- a drop-in replacement for duckplyr, which necessitates "eager" data frames that automatically materialize like in dplyr, 
+- optimization by DuckDB, which means lazy evaluation where the data is materialized at the latest possible stage.
 
-Thrifty is a good choice for data sets where the cost of materializing the data is prohibitive only for large results.
-This is the default for the ingestion functions like `read_parquet_duckdb()`.
+Automatic materialization can be dangerous for memory with large data, so duckplyr provides a setting called `prudence` that controls automatic materialization: 
+is the data automatically materialized _always_ ("lavish" frames), _never_ ("frugal" frames) or _up to a certain size_ ("thrifty" frames).
 
 See `vignette("large")` for more details on working with large data sets, `vignette("fallback")` for fallbacks to dplyr, and `vignette("limits")` for the operations supported by duckplyr.