edit the read-cases episode

Karim-Mane · Karim-Mane · commit 9e90e3f2422c · 2025-06-24T07:29:25.000Z
diff --git a/episodes/read-cases.Rmd b/episodes/read-cases.Rmd
@@ -9,8 +9,8 @@ editor_options:
 :::::::::::::::::::::::::::::::::::::: questions 
 
 - Where do you usually store your outbreak data?
-- How many different data formats can you use for analysis? 
-- Can you import data from databases and health APIs? 
+- What data formats do you use for analysis? 
+- Can you import data from databases and Health Information Systems (HIS) through their APIs? 
 ::::::::::::::::::::::::::::::::::::::::::::::::
 
 ::::::::::::::::::::::::::::::::::::: objectives
@@ -30,9 +30,9 @@ This episode requires you to be familiar with:
 
 ## Introduction
 
-The initial step in outbreak analysis typically involves importing the target dataset into the `R` environment from either a local source (like a file on your computer) or external source (like a database). Outbreak data can be stored in diverse formats, relational database management systems (RDBMS), or health information systems (HIS), such as [REDCap](https://www.project-redcap.org/) and [DHIS2](https://dhis2.org/), which provide application program interfaces (APIs) to the database systems so verified users can easily add and access data entries. The latter option is particularly well-suited for collecting and storing large-scal institutional health data. This episode will elucidate the process of reading cases from these sources.
+The initial step in outbreak analysis typically involves importing the target dataset into the `R` environment from either a local source (like a file on your computer) or external source (like a database). Outbreak data can be stored in diverse formats, relational database management systems (RDBMS), or health information systems (HIS), such as [REDCap](https://www.project-redcap.org/) and [DHIS2](https://dhis2.org/), which provide application program interfaces (APIs) to the system's database  so verified users can easily add and fetch data entries. The latter option is particularly well-suited for collecting and storing large-scale institutional health data. This episode will elucidate the process of reading cases from these sources.
 
-Let's start by loading the package `{rio}` to read data and the package `{here}` to easily find a file path within your RStudio project. We'll use the pipe `%>%` to easily connect some of their functions, including functions from the data formatting package `{dplyr}`. We'll therefore call the tidyverse package, which includes both the pipe and `{dplyr}`:
+Let's start by loading the `{rio}` package to read data and the `{here}` package to easily find a file path within your RStudio project. We'll use the pipe `%>%` operator from the `{magrittr}` package to easily connect some of their functions, including functions from the data formatting package `{dplyr}`. We'll therefore call the `{tidyverse}` package, which includes both `{magrittr}` and `{dplyr}`.
 
 ```{r,eval=TRUE,message=FALSE,warning=FALSE}
 # Load packages
@@ -43,13 +43,17 @@ library(here) # for easy file referencing
 
 ::::::::::::::::::: checklist
 
-### The double-colon
+### The double-colon (`::`) operator
 
-The double-colon `::` in R lets you call a specific function from a package without loading the entire package into the current environment. 
+The `::` in R lets you access functions or objects from a specific package without attaching the entire package to the search path. It offers several important
+advantages including the followings:
 
-For example, `dplyr::filter(data, condition)` uses `filter()` from the `{dplyr}` package, without having to use `library(dplyr)` at the start of a script.
+* Telling explicitly which package a function comes from, reducing ambiguity and potential conflicts when several packages have functions with the same name.
+* Allowing to call a function from a package without loading the whole package
+with library().
 
-This help us remember package functions and avoid namespace conflicts (i.e. when two different packages include functions with the same name, so R does not know which to use).
+For example, the command `dplyr::filter(data, condition)` means we are calling
+the `filter()` function from the `{dplyr}` package.
 
 :::::::::::::::::::
 
@@ -66,15 +70,15 @@ This help us remember package functions and avoid namespace conflicts (i.e. when
 
 ## Reading from files 
 
-Several packages are available for importing outbreak data stored in individual files into `R`. These include [rio](https://gesistsa.github.io/rio/), [readr](https://readr.tidyverse.org/) from the `tidyverse`, [io](https://bitbucket.org/djhshih/io/src/master/), [ImportExport](https://cran.r-project.org/web/packages/ImportExport/index.html), and [data.table](https://rdatatable.gitlab.io/data.table/). Together, these packages offer methods to read single or multiple files in a wide range of formats.
+Several packages are available for importing outbreak data stored in individual files into `R`. These include [{rio}](https://gesistsa.github.io/rio/), [{readr}](https://readr.tidyverse.org/) from the `{tidyverse}`, [{io}](https://bitbucket.org/djhshih/io/src/master/), [{ImportExport}](https://cran.r-project.org/web/packages/ImportExport/index.html), [{data.table}](https://rdatatable.gitlab.io/data.table/), and similar functions from the {base} R package. Together, these packages offer methods to read single or multiple files in a wide range of formats.
 
-The below example shows how to import a `csv` file into `R` environment using `{rio}` package. We use the `{here}` package to tell R to look for the file in the `data/` folder of your project, and `as_tibble()` to convert into a tidier format for subsequent analysis in R.
+The below example shows how to import a `csv` file into `R` environment using `{rio}` package. We use the `{here}` package to tell R to look for the file in the `data/` folder of your project, and `as_tibble()` to convert it into a tidier format for subsequent analysis in R.
 
 ```{r,eval=FALSE,echo=TRUE}
 # read data
-# e.g., the path to our file is data/raw-data/ebola_cases_2.csv then:
+# e.g., if the path to our file is "data/raw-data/ebola_cases_2.csv" then:
 ebola_confirmed <- rio::import(
-  here::here("data", "ebola_cases_2.csv")
+  here::here("data", "raw-data", "ebola_cases_2.csv")
 ) %>%
   dplyr::as_tibble() # for a simple data frame output
 
@@ -84,7 +88,6 @@ ebola_confirmed
 
 
 ```{r,eval=TRUE, echo=FALSE, message=FALSE}
-# internal for DBI::dbWriteTable()
 # read data
 ebola_confirmed <- rio::import(
   file.path("data", "ebola_cases_2.csv")
@@ -95,18 +98,19 @@ ebola_confirmed <- rio::import(
 ebola_confirmed
 ```
 
-Similarly, you can import files of other formats such as `tsv`, `xlsx`, ... etc.
+Similarly, you can import files of other formats such as `tsv`, `xlsx`, ... etc
+using the same function.
 
 :::::::::::::::::::: checklist
 
 ### Why should we use the {here} package?
 
-The `{here}` package is designed to simplify file referencing in R projects by providing a reliable way to construct file paths relative to the project root. The main reason to use it is **Cross-Environment Compatibility**.
+The `{here}` package is designed to simplify file referencing in R projects by providing a reliable way to construct file paths relative to the project root. The main reason to use it is to ensure **Cross-Environment Compatibility**.
 
 It works across different operating systems (Windows, Mac, Linux) without needing to adjust file paths. 
 
 - On Windows, paths are written using backslashes ( `\` ) as the separator between folder names: `"data\raw-data\file.csv"` 
-- On Unix based operating system such as macOS or Linux the forward slash ( `/` ) is used as the path separator: `"data/raw-data/file.csv"`
+- On Unix based operating system such as macOS or Linux the forward slash ( `/` ) is used as separator between folder names: `"data/raw-data/file.csv"`
 
 The `{here}` package is ideal for adding one more layer of reproducibility to your work. If you are interested in reproducibility, we invite you to [read this tutorial to increase the openess, sustainability, and reproducibility of your epidemic analysis with R](https://epiverse-trace.github.io/research-compendium/)
 
@@ -122,14 +126,14 @@ Can you read data from a compressed file in `R`? Download this [zip file](https:
 ::::::::::::::::: hint
 
 You can check the [full list of supported file formats](https://gesistsa.github.io/rio/#supported-file-formats) 
-in the `{rio}` package on the package website. To expand {rio} to the full range of support for import and export formats run:
+from the `{rio}` package website. To expand {rio} to its current unsupported file formats, run:
 
 
 ```{r, eval=FALSE}
 rio::install_formats()
 ```
 
-You can use this template to read the file: 
+You can use this example to read the file: 
 
 `rio::import(here::here("some", "where", "downto", "path", "file_name.zip"))`
 
@@ -147,29 +151,29 @@ rio::import(here::here("data", "Marburg.zip"))
 
 ## Reading from databases
 
-The [DBI](https://dbi.r-dbi.org/) package serves as a versatile interface for interacting with database management systems (DBMS) across different back-ends or servers. It offers a uniform method for accessing and retrieving data from various database systems.
+The [{DBI}](https://dbi.r-dbi.org/) package serves as a versatile interface for interacting with relational database management systems (DBMS) across different back-ends or servers. It offers a uniform method for accessing and retrieving data from various database systems.
 
 ::::::::::::: discussion
 
-### When to read directly from a database?
+### Advantages of reading data directly from a database?
 
-We can use database interface packages to optimize memory usage. If we process the database with "queries" (e.g., select, filter, summarise) before extraction, we can reduce the memory load in our RStudio session. Conversely, conducting all data manipulation outside the database management system by loading the full dataset into R can use up much more computer memory (i.e. RAM) than is feasible on a local machine, which can lead RStudio to slow down or even freeze.
+We can use database interface packages to optimize the amount of memory used during our R session. If we query the database with data filtration requests (e.g., select, filter, summarise) before extraction, we can reduce the memory load in our RStudio session. Conversely, conducting all data manipulation outside the database management system by loading the full dataset into R can use up much more computer memory (i.e. RAM) than is feasible on a local machine, which can lead RStudio to slow down or even freeze.
 
-External relational database management systems (RDBMS) also have the advantage that multiple users can access, store and analyse parts of the dataset simultaneously, without having to transfer individual files, which would make it very difficult to track which version is up-to-date.
+Relational database management systems (RDBMS) also have the advantage that multiple users can access, store and analyse parts of the dataset simultaneously, without having to transfer individual files, which would make it very difficult to track which version is up-to-date.
 
 :::::::::::::
 
-The following code chunk demonstrates in four steps how to create a temporary SQLite database in memory, store the `ebola_confirmed` as a table on it, and subsequently read it:
+The following code chunk demonstrates in four steps how to create a SQLite database in memory, store the `ebola_confirmed` as a table on it, and subsequently read it.
 
-### 1. Connect with a database
+### 1. Connection to the a database
 
-First, we establish a connection to an SQLite database created on our machine and stored in its local memory with `DBI::dbConnect()`. 
+We first need to establish a connection to an SQLite database created on our machine and stored in its local memory with `DBI::dbConnect()`. 
 
 ```{r,warning=FALSE,message=FALSE}
 library(DBI)
 library(RSQLite)
 
-# Create a temporary SQLite database in memory
+# Create a SQLite database in memory
 db_connection <- DBI::dbConnect(
   drv = RSQLite::SQLite(),
   dbname = ":memory:"
@@ -180,7 +184,7 @@ db_connection <- DBI::dbConnect(
 
 A real-life connection to an external SQLite database would look like this:
 
-```r
+```{r}
 # in real-life
 db_connection <- DBI::dbConnect(
   RSQLite::SQLite(), 
@@ -192,12 +196,12 @@ db_connection <- DBI::dbConnect(
 
 :::::::::::::::::
 
-### 2. Write a local data frame as a table in a database
+### 2. Create a table in the database from a data frame
 
-Then, we can write the `ebola_confirmed` into a table named `cases` within the database using the `DBI::dbWriteTable()` function.
+After establishing the connection with the database, we can now write out the `ebola_confirmed` data frame into a table named `cases` within the database using the `DBI::dbWriteTable()` function.
 
 ```{r,warning=FALSE,message=FALSE}
-# Store the 'ebola_confirmed' dataframe as a table named 'cases'
+# Store the 'ebola_confirmed' data frame as a table named 'cases'
 # in the SQLite database
 DBI::dbWriteTable(
   conn = db_connection,
@@ -206,7 +210,7 @@ DBI::dbWriteTable(
 )
 ```
 
-In a database framework, you can have more than one table. Each table can belong to a specific `entity` (e.g., patients, care units, jobs). All tables will be related by a common ID or `primary key`.
+In a relational database framework, you can have more than one table. Each table can belong to a specific `entity` (e.g., patients, care units, jobs). All tables will be related by a common ID or `primary key`.
 
 ### 3. Read data from a table in a database
 
@@ -220,17 +224,17 @@ In a database framework, you can have more than one table. Each table can belong
 <!-- ) -->
 <!-- ``` -->
 
-Subsequently, we reads the data from the `cases` table using `dplyr::tbl()`.
+We can reads the data from the `cases` table using `dplyr::tbl()`.
 
 ```{r}
 # Read one table from the database
 mytable_db <- dplyr::tbl(src = db_connection, "cases")
 ```
 
-If we apply `{dplyr}` verbs to this database SQLite table, these verbs will be translated to SQL queries.
+If we apply `{dplyr}` verbs to this table of a SQLite database, these verbs will be translated to an SQL queries.
 
 ```{r}
-# Show the SQL queries translated
+# Show the translated SQL queries 
 mytable_db %>%
   dplyr::filter(confirm > 50) %>%
   dplyr::arrange(desc(confirm)) %>%
@@ -249,7 +253,7 @@ extracted_data <- mytable_db %>%
   dplyr::collect()
 ```
 
-The `extracted_data` object represents the extracted, ideally after specifying queries that reduces its size.
+The `extracted_data` object represents the extracted data, ideally after applying the specified queries that reduces its size.
 
 ```{r,warning=FALSE,message=FALSE}
 # View the extracted_data
@@ -258,11 +262,11 @@ extracted_data
 
 :::::::::::::::::::::: callout
 
-### Run SQL queries in R using dbplyr
+### Run SQL queries in R using {dbplyr}
 
-Practice how to make relational database SQL queries using multiple `{dplyr}` verbs like `dplyr::left_join()` among tables before pulling down data to your local session with `dplyr::collect()`! 
+Practice how to make relational database SQL queries using multiple `{dplyr}` verbs like `dplyr::left_join()` among tables before pulling out data to your local session with `dplyr::collect()`! 
 
-You can also review the `{dbplyr}` R package. But for a step-by-step tutorial about SQL, we recommend you this [tutorial about data management with SQL for Ecologist](https://datacarpentry.org/sql-ecology-lesson/). You will find close to `{dplyr}`!
+You can also review the `{dbplyr}` R package. But for a step-by-step tutorial about SQL, we recommend you this [tutorial about data management with SQL for Ecologist](https://datacarpentry.org/sql-ecology-lesson/).
 
 ::::::::::::::::::::::
 
@@ -278,10 +282,11 @@ DBI::dbDisconnect(conn = db_connection)
 
 ## Reading from HIS APIs
 
-Health related data are also increasingly stored in specialized HIS APIs like **Fingertips**, **GoData**, **REDCap**, and **DHIS2**. In such case one can resort to [readepi](https://epiverse-trace.github.io/readepi/) package, which enables reading  data from HIS-APIs.  
+Health related data are also increasingly stored in specialized HIS APIs like **Fingertips**, **GoData**, **REDCap**, and **DHIS2**. In such case one can resort to the [{readepi}](https://epiverse-trace.github.io/readepi/) package, which enables reading data from HIS APIs.  
 -[TBC]
 
 ::::::::::::::::::::::::::::::::::::: keypoints 
 - Use `{rio}, {io}, {readr}` and `{ImportExport}` to read data from individual files.
+- Use {DBI}, {dplyr}, and {dbplyr} to import data from RDBMS
 - Use `{readepi}` to read data form HIS APIs and RDBMS.
 ::::::::::::::::::::::::::::::::::::::::::::::::
