06-1: How to write R codes, manage Projects, work with RStudio

Tips to make the most of the lecture notes

Transcript

Two navigation tricks before we start. The three stacked lines in the bottom left corner open a table of contents, so you can jump straight to a section. And pressing the letter o gives you a panel view of every slide at once. This deck works as a reference more than most of the others, because it is full of folder layouts and keyboard shortcuts you will want to come back to when you are actually setting up a project rather than sitting in class. So it is worth knowing how to find a specific slide quickly, because you will be doing exactly that in a few weeks.

  • Click on the three horizontally stacked lines at the bottom left corner of the slide, then you will see table of contents, and you can jump to the section you want

  • Hit letter “o” on your keyboard and you will have a panel view of all the slides

Transcript

The code boxes in this chapter are static examples. You can use the copy button in the top right corner to put the code on your clipboard and paste it into RStudio, but you cannot edit or run code within these slides. In RStudio, you can run part of an example by highlighting it and pressing Command Enter, or Control Enter on Windows. The hint of blue is how you recognize one of these static code boxes, and the copy control is the icon that looks like two sheets of paper stacked together. Once you paste the example into RStudio, make any changes there before you run it.

  • The boxes with a hint of blue contain static code examples.
  • You cannot edit or execute these examples within the slides.
  • To run only part of an example, first copy it into RStudio, highlight the part you want, and press Command + Enter for Mac (Ctrl + Enter for Windows).
  • Click the icon with two sheets of paper stacked on top of each other at the top right corner of a code block to copy the example, then paste it into RStudio.
  • Make any edits to the examples in RStudio, not in the slides.

Learning objectives

Transcript

Three objectives, and this chapter is different in character from the others. There is very little code to run. What we are covering is how to arrange your work: where files go, how code is structured inside a file, and a set of RStudio habits that make everything else faster. It can feel like housekeeping compared with learning dplyr or ggplot. It is not. Most of the pain in a research project comes from disorganization rather than from difficulty, and the difference between a project you can pick up after three months and one you cannot is almost entirely what this chapter covers.

  • Learn how to organize your project: folder, codes, data files, etc
  • Learn how to organize R codes
  • Learn how to use various RStudio tips for efficient programming

Reproducibility

Reproducibility

Transcript

Two words that get used interchangeably and should not be. Read both callouts carefully, because the distinction turns on what stays the same. Reproducible means somebody uses your materials and your procedure and gets your results. Replicable means somebody applies your procedure to different materials and reaches the same conclusion. So reproducibility is about whether your work can be checked, and replicability is about whether your finding is real. Both matter, but they are different questions, and this lecture is only about the first. If you cannot reproduce your own results in six months, nothing downstream of that is worth much.

First of all, you may have heard of “reproducibility” and “replicability.” While they sometimes are used interchangeably, they mean different things. Here are commonly used definitions of the two terms (Cacioppo et al. 2015).


Definition: Reproducibility

A research study is reproducible if anybody (including the author of the study) can generate exactly the same results by using the same materials (e.g., data) and procedures used in the study.


Definition: Replicability

A research study is replicable if other teams reach the same conclusion by applying the same procedure to the different materials (e.g., data).

This lecture focuses only on reproducibility and does not deal with replicability.

Transcript

Here is the minimum bar, and it is lower than people expect. Every action is documented well enough that somebody else can follow it. That is all. Read the paragraph underneath, because it corrects a common misunderstanding. Reproducibility does not require that everything be automated. If you delete rows in Excel by hand, that is still reproducible, provided you write down that you did it and you keep the original file. Discouraged, certainly, but still reproducible. The thing that actually breaks reproducibility is the hidden action, the step you took and did not record, because nobody can repeat what nobody knows about.

Minimum Requirement

Every single action taken during the entire research process is documented in a way that anybody can follow to implement the same actions (no hidden actions) to produce exactly the same results.


Note that this does not necessarily mean every single action needs to be computer-programmed and automated. Even if you manually delete rows of data on Excel (highly discouraged), this does not make your research non-reproducible as long as this action is recorded and the original data (before deletion of the rows) are provided because anybody can implement this action.

Transcript

Meeting the minimum makes a project reproducible. It does not make it good. If reproducing your work takes somebody three weeks of puzzling, you have technically satisfied the requirement and helped no one. The three characteristics on screen are what separate the two. An organized folder so files can be found, automation through annotated programs so the workflow can be run rather than reconstructed, and documentation covering the data, the methods, and the steps. Notice that all three are about somebody else’s time. And be clear who that somebody usually is: you, six months from now, having forgotten everything.

A project can be considered reproducible if the minimum requirements are met. However, it is still low-quality if reproducing it requires excessive time or resources. A high-quality reproducible project, on the other hand, demonstrates the following characteristics:


Organized Project Folder

A well-structured folder system makes it easy to locate files and navigate the project.


Streamlined Automation

Workflows are automated through clearly annotated programs, making replication straightforward and ensuring transparency in the process.


Comprehensive Documentation

Thorough documentation—including details on data, methods, and reproduction steps—saves time and enhances clarity for anyone reproducing the work.

Transcript

Who actually benefits, and the letters in brackets are the point. Scientific integrity benefits the community. Educational value benefits your team and the community. But look at the last three. Repeatability, transferability and reducing errors all carry Y and M, meaning you and your team. That is the argument worth internalising. Reproducibility is often presented as a duty you owe the profession, which makes it sound like unpaid overhead. Most of the payoff is actually yours. You are the one who will need to rerun this with one variable changed, hand it to a coauthor, or find the error before a referee does. Scientific integrity means the community can inspect the path from evidence to result. Educational value means teammates and other researchers can learn from that path instead of seeing only the final answer. Repeatability lets you regenerate the work after an input or specification changes, transferability lets someone else on the team take it over, and a transparent sequence makes mistakes easier to locate. Read Y, M, and S as a quick map of who receives each payoff, not as part of the benefit’s name.

The main beneficiaries of reproducible research include:

  • You (Y)
  • Members of your team (M)
  • The scientific community (S)


Here are some of the benefits of high-quality reproducible research with their beneficiaries:

  • Scientific Integrity (S)
  • Educational Value (MS)
  • Repeatability (YM)
  • Transferability (YM)
  • Reducing Errors (YM)
Transcript

A checklist to run against your own work, and I would genuinely use it. Six items: organized folders, documented data, everything programmed rather than done by hand, code that is annotated and organized, an instruction saying what to run and in what order, and no junk files lying around. That fifth one is the most commonly missed. People produce beautiful code and never write down which script runs first, which leaves the next person guessing. And the sixth matters more than it sounds, because a folder full of abandoned experiments makes it impossible to tell what is actually part of the project. Notice that this checklist asks for a higher-quality workflow than the minimum definition on the earlier tab. A recorded manual Excel edit can still meet the minimum for reproducibility, but programming data processing, analysis, figures, and tables makes the action exact, repeatable, and much easier to audit. Data documentation should tell the next person what the variables and source files mean, while the folder system should also give code, manuscripts, and supporting articles an obvious home.

How to organize your project

How to organize your project

Transcript

Why one dedicated folder per project. Two reasons on screen, and the first is the one that bites. If you work across several projects in one session, you will eventually have two objects with the same or similar name and use the wrong one. That produces results which are wrong but not obviously wrong, which is the worst kind of error. The second reason is memory, carrying around objects irrelevant to what you are doing. The deeper point is that a project should be a self-contained thing you can open, work in and close, with nothing leaking in from elsewhere.

You should have a single dedicated folder for a research project. This will avoid

  • confusions between objects of the same or similar name (accidentally using the one you do not intend to use)

  • wasting memory by holding objects on the global environment that are completely irrelevant to your working project

Transcript

Now the mechanics, in three steps in the nested tabs. Work through them, but understand what you are creating. An RStudio project is really just a folder with a dot Rproj file in it. What that file buys you is that opening it starts a fresh R session whose working directory is that folder. So your paths become relative to the project rather than to wherever your machine happens to be pointing, which means the whole thing can be moved, zipped, or handed to somebody else and still work. That is the entire value, and it is a large one. In Step 1, use the project control at the top right of RStudio and follow New Project, New Directory, then New Project, as the screenshot shows. In Step 2, enter the new directory’s name, choose the parent directory where it should be created, and click Create Project. Step 3 shows the result: RStudio takes you into a new session for that project, and the new folder contains a file such as test dot Rproj. That file stores project information, but you do not edit it. You normally open it to return to the project.

We can initiate an RStudio project with a dedicated folder from within RStudio.

At the top right corner of RStudio, navigate through:


Project (None)

-> New Project…

-> New Directory

-> New Project

  • type in a directory name
  • select the directory in which the project folder (directory) is going to be created
  • hit the Create Project button

You will be automatically taken to a new R session inside the newly-created RStudio project.

In this folder you just created, you have a single file named .Rproj (here, it is test.Rproj). It holds information about this project, but you do not have to touch it.

Transcript

A recommended layout, and the note at the top matters: modify it as you see fit. There is nothing sacred about these particular names. What is worth copying is the principle. Code is separated from data, and within data, raw is separated from processed. Within code, data preparation is separated from analysis. Results and writing get their own places. The reason to split raw from processed is the one that will save you one day. Raw data and other source materials must be preserved because rerunning your scripts cannot regenerate them. Processed data and generated results should be rebuildable by rerunning your scripts. The Literature folder gives journal articles and other reference documents a predictable home. Results holds generated regression output, figures, and tables, while Writing holds the qmd, Word, or LaTeX files that turn those results into a paper or report. The folder-tree image on the right shows the same hierarchy as the list on the left, with DataPrep and Analysis nested under Code, and Raw and Processed nested under Data.

Here is a recommended folder organization. You should modify/add folders as you see fit.

  • Code: all the codes go in here
    • DataPrep:
    • Analysis:
  • Data
    • Raw: place raw datasets here
    • Processed: save intermediate datasets here
  • Literature: journal articles and other relevant documents
  • Results: results (regression, figures, tables)
  • Writing: qmd, WORD, Latex files

Files

Transcript

A recommendation you may find surprising: write your analysis code in a qmd file rather than a plain R script. The reasons are on screen. Commenting is far easier, because prose between chunks is just prose rather than a wall of hash marks. Headers give you real structure. And the two practical ones at the bottom are what win people over. Run All Chunks Above rescues you when R crashes partway through, without rerunning everything by hand. And the navigator at the bottom of the source pane lets you jump between sections, which matters enormously once a file gets long. You can open Run All Chunks Above from the triangle beside the Run button. On a Mac, option-shift-command-P is the keyboard route shown here for running the preceding R Markdown code. The screenshot points to the navigator in the lower corner of the source pane; because it reads your section and subsection headers, meaningful headers give you both organization on the page and a usable menu for moving through the file.

Recommendation

Use a qmd file instead of an R file whenever you write codes

  • It is much easier to make comments in a qmd file than an R file

  • You can better organize your codes with markdown section headers (e.g., #, ##)

  • R crashed at a certain chunk and had to restart R and then run all the R codes up to the problematic chunk? Select Run All Chunks Above by clicking the triangle to the right of the Run button, or hit option + shift + command + P to run the previous Rmd code.

  • Easily move between sections and subsection using the navigator at the bottom lower corner of the source pane

Transcript

A recommended structure for the inside of a file, and it is worth adopting as a habit. Start with an objective statement saying what this file does, what it takes in and what it produces. Then setup: working directory, packages, functions. Then the actions, in order. The objective statement is the part people skip and the part that pays. When you come back in six months, the first thing you need to know is what this file is for and what it depends on. And read the note at the bottom: keep it current, because inputs and outputs drift as a project evolves. In the setup section, set a working directory only if it is necessary, then load the packages and functions the actions depend on. The Action 1 through Action n outline is intentionally generic: replace it with headers that describe the real sequence in this particular file. The image on the right lets you see what this outline looks like in an actual qmd source file, so the written objective, setup, and action sections remain visible alongside the code they organize.

Recommended Structure

  • Objective statement
    • state objectives
    • input: state input files and datasets
    • output: state output files and datasets
  • Setup
    • set the working directory (if necessary)
    • load packages
    • load functions
  • Actions
    • Action 1
    • Action 2
    • .
    • .
    • .
    • Action n

Note

Dynamically edit the “Objective statement” as its objectives, input, and output are likely to change.

Transcript

Style, in two nested tabs. The honest position is in the first sentence: you can write R however you like, and R will not care. The argument for a style guide is entirely about other people reading your code, and again, that includes you later. The two guides linked here are the common ones, with tidyverse the more widely used in this ecosystem. Then the second tab covers styler, which automates part of it. Do click through the linked examples on naming, spacing and piping rather than just noting that guides exist. Those four pages are most of what you actually need. The long-lines example is also part of that short practical list, because consistent line breaks keep code readable without horizontal hunting. On the styler tab, highlight the lines you want to change, open Addins, and choose Style selection. The package helps with part of the tidyverse style, so you still need judgment about names and organization. The callout warns that this addin has no keyboard shortcut by default. If you want one, go through Tools, Modify Keyboard Shortcuts, search for Style selection, and assign your own. Do not treat command-shift-A on a Mac, or control-shift-A on Windows, as the styler shortcut. That invokes RStudio’s built-in Reformat Code command, works without styler, and does not apply the tidyverse style guide.

You can write R codes however you would like. But, your code may get more readable to you and others who might read your codes by following a style guideline that is accepted by many R users. There are several popular styles of formatting R codes:

Examples

Here are some examples of the tidyverse style:

The styler package can help you partially follow the tidyverse coding style.

Highlight the lines you want to restyle, then click Addins in the menu at the top and select Style selection.

Want a keyboard shortcut?

Addins have no shortcut out of the box. To give Style selection one, go to Tools \rightarrow Modify Keyboard Shortcuts…, search for “Style selection”, and assign a key of your choosing.

Do not confuse this with cmd + shift + A (ctrl + shift + A on Windows). That is RStudio’s own “Reformat Code” command. It is built in, it works without styler, and it does not follow the tidyverse style guide.

Transcript

Four sets of rules, and if you remember only one thing, make it the first. Raw data goes in Data slash Raw, and you never overwrite it. You read from it, you never write to it. Everything else in your project can be regenerated by rerunning code. Raw data cannot, and if you overwrite it you have destroyed the one irreplaceable thing you had. The rest follows the same logic. Processing code in Code slash DataPrep writing to Data slash Processed, analysis code in Code slash Analysis writing to Results, and writing that refers to Results rather than keeping its own copies. Rule 1 also says the designated raw-data folder contains raw datasets and nothing else, which makes the read-only boundary unambiguous. Rule 2 keeps transformation and merging programs apart from the intermediate R objects or datasets they create, so do not mix code and data in one folder. Rule 3 sends regression tables, figures, and other tables to Results. Rule 4 puts qmd and Word writing files in Writing, then links them to the figures and tables in Results. That avoids duplicate output files quietly drifting apart.

Rules 1

  • place all the raw datasets (nothing else) in a designated folder inside the Data/Raw folder
  • do not ever override them, you only read them and keep them intact

Rules 2

  • write R codes to process (transform, merge, etc) the raw data and save all the R codes inside the Code/DataPrep folder
  • save intermediate R objects or datasets in the Data/Processed folder
  • do not mix codes and datasets in a single folder

Rules 3

  • write R codes to do analysis and save them in Code/Analysis
  • save the results/outputs (regression tables, figures, tables) in the Results folder

Rules 4

  • if you are using qmd to write a journal article or report, put them in the Writing folder (same goes for WORD)
  • refer to figures and tables in the Results folder to integrate them in the output document
Transcript

Two recommendations about naming, and the second is the useful one. Name files for what they do, and put numbers in front to encode the order they should be run in. Look at the example: zero one dash one download weather data, zero one dash two download boundary data, and so on, with data preparation numbered zero one and analysis numbered zero two. That numbering is doing real work. It is the instruction on how to reproduce your project, embedded in the filenames themselves, so anybody sorting the folder alphabetically sees the correct order to run things in. The rest of the first sequence makes the dependency explicit: zero one dash three summarizes the data, and zero one dash four merges it. Only then does the zero-two analysis sequence begin, first with regression analysis and then with generation of figures and tables. The descriptive words tell you each file’s purpose, while the two-part numeric prefix tells you both its broad phase and its position within that phase.

Recommendation

  • Name files so that you know what purposes they serve for you
  • Place numbers as prefix to indicate the order in which they should be run

Example

  • Data Collection and Preparation (in Code/DataPrep)
    • 01-1-download-weather-data.R
    • 01-2-download-political-boundary-data.R
    • 01-3-summarize-data.R
    • 01-4-merge-data.R
  • Data Analysis and Results Preparation (in Code/Analysis)
    • 02-1-regression-analysis.R
    • 02-2-gen-figures-tables.R

Example project

Transcript

Rather than take my word for any of this, go and look at one. The link is to a sample project on GitHub built to be reproducible. Clone it and open it, and read it the way somebody reproducing it would. Start at the top level, find the instruction telling you what to run, then follow the numbered files. Ask yourself honestly whether you could rerun the whole thing without needing to ask the author anything. That is the real test of everything in this chapter, and it is a far better way to judge the advice than reading a checklist about it.

Let’s take a look at an example project that is designed to be reproducible. First, Clone this repository. We will then look at how the project is organized and developed.

RStudio tips

Code snippets

Transcript

Snippets are text expansion inside RStudio. You type a short trigger and select the matching snippet from the completion list, or press shift tab to insert it immediately, and RStudio replaces it with something longer. That is the whole idea, and it is worth five minutes of setup, because the things you type most often in R are awkward to type. The pipe and the assignment arrow both need several keystrokes and a reach. Look at the syntax block, and then read the callout, because it is the thing that catches everybody. Every line you want printed must be indented with an actual tab character. Spaces will not do, and the snippet then silently fails to work. In that syntax block, the keyword snippet begins the definition, the combination of letters after it is the trigger you type, and the next line is the longer text RStudio inserts. The indentation is structural, not decorative: it tells RStudio that the line belongs to that snippet definition.

Code snippets map a short sequence of letters and symbols to a longer, more complicated one.


Syntax

snippet (combination of letters to invoke)
    (what you want to print)

Important

You need a literal tab before (what you want to print)

Transcript

Two examples in the nested tabs, and they are the two worth setting up first. Type p i and select the snippet from the completion list, or press shift tab to get the pipe. Do the same with a s to get the assignment arrow. Both are things you type dozens of times an hour, and both currently cost you an awkward key combination. Short triggers like these often collide with object or function names that RStudio is also trying to suggest, so make sure you select the snippet when you use the completion list. That is worth knowing before you decide a snippet is broken.


snippet pi
    %>%


Once you add this, type “pi”, then select the snippet from the completion list or press shift + tab to have %>% printed.


snippet as
    <-


Once you add this, type “as”, then select the snippet from the completion list or press shift + tab to have <- printed.

Transcript

Here is where the snippets actually live. Tools, then Global Options, then Code, then Edit Snippets, and choose the R tab. That path is worth remembering, because you will want to add more as you go. Then do the exercise in the callout, right now if you have RStudio open, because snippets are one of those things that sound marginal until you have used them for a day. Add the assignment one, type a s, then select the snippet from completion or press shift tab, and watch the arrow appear. And remember the literal tab indentation rule from the previous tab, since that is why most first attempts do nothing at all.

Follow Tools \rightarrow Global Options \rightarrow Code \rightarrow Edit Snippets , select R tab, and add snippets.


Try yourself

  • Place the following
snippet as
    <-
  • type “as” inside an R code chunk, then select the snippet from completion or hit shift+tab
Transcript

Something that confuses people the first time. A qmd file has two contexts. Inside an R chunk you are in R, and outside one you are in Markdown. Snippets are stored per context, so a snippet defined on the R tab only fires inside chunks, and a Markdown one only fires outside them. Look at the example in the second nested tab. A snippet that creates a whole R chunk necessarily belongs in the Markdown list, because you are outside a chunk at the moment you want to make one. And note it is invoked with shift tab there, rather than plain tab. The definition names its trigger r-m-c. Its two tab-indented output lines insert the opening fence for an R code chunk and the matching closing fence, leaving you a ready-made empty chunk. Put this definition on the Markdown tab, type r-m-c outside a chunk, and press shift-tab. The final instruction asks you to try the same trigger inside an R chunk and confirm that it does not expand there. That failed expansion is the demonstration of context-specificity, not an error in the snippet.

Suppose you are working on a Quarto document.

  • You are in the R context if your cursor is in an R code chunk
  • You are in the Markdown context if your cursor is outside of an R code chunk

Snippets defined in the R (Markdown) tab only work in the R (Markdown) context.

This snippet will let you create an R code chunk with typing “rmc”. Place it in the Markdown tab of the snippets list and hit shift+tab to invoke it.

snippet rmc
    `r ''````{r }
    ```

Confirm that this snippet does not work in the R environment.

Transcript

This is what makes snippets genuinely powerful rather than just shorter to type. The dollar sign with a number and a label marks a placeholder. Expand this one and your cursor lands in the chunk title; press tab and it jumps to the chunk contents. So a snippet is not simply fixed text, it is a small fill-in-the-blanks form that walks you through the parts you have to supply, in order. Once you see that, the useful snippets to write become obvious. Any structure you type repeatedly where only two or three pieces change each time is a candidate. In the code on screen, dollar-one with the chunk-title label sits inside the opening R chunk fence, while dollar-two with the chunk-content label sits in the body. The numbers define the cursor order, and the labels give you visible prompts for what to type at each stop. After you replace the first prompt, tab advances to the second one.

snippet rmc
    `r ''````{r ${1:chunk_title}}
      ${2:chunk_content}
    ```

$ is used as a special character to denote where the cursor should jump (by hitting tab) after completing each section of a snippet.

Transcript

Two more to finish, both ggplot skeletons, and they show the placeholder idea doing real work. Type g l and select the snippet from completion, or press shift tab, and you get a line plot skeleton with your cursor waiting on the dataset name, then the y variable, then x. Do the same with g d and you get a density plot. This is where snippets pay off most, because plotting code is highly repetitive in shape and entirely variable in content. Take these as templates rather than as the final word. The right snippets for you are whatever you catch yourself typing over and over this semester. In the g-l definition, ggplot’s data argument is placeholder one, so you choose the dataset first. The plus sign adds geom-line, and aes maps placeholder two to y and placeholder three to x. In the g-d definition, the data argument is again the first stop, then geom-density needs an x mapping as the second stop. The different trigger, geometry, and placeholder count reflect the plot you are building, while the repeated ggplot structure is exactly the typing the snippet saves.

ggplot (line plot)

snippet gl
    ggplot(data = ${1:dataset}) +
    geom_line(aes(y = ${2:y}, x = ${3:x}))


ggplot (density plot)

snippet gd
    ggplot(data = ${1:dataset}) +
    geom_density(aes(x = ${2:x}))

Resources

Transcript

Three places to go next. Efficient R Programming is a full book and much broader than this lecture, worth dipping into rather than reading end to end. The tidyverse style guide is the reference for the styling material we covered, and it is short. And the styler GitHub page is where to look when you want to configure the automatic reformatting rather than accept its defaults. None of these are required reading. But the habits in this chapter are ones you build slowly, and having somewhere to look when a specific question comes up beats trying to absorb it all now.