Introduction to tabular data

  • We will be working with data from the Portal Project.
  • Long-term experimental study of small mammals in Arizona.


  • Main way that reusable code is shared in R
  • Combination of code, data, and documentation
  • R has a rich ecosystem of packages for data manipulation & analysis
  • We’ve already installed the packages for this demo
  • Even if we’ve installed a package it is automatically available to do analysis with
  • This because different packages may have functions with the same names
  • So don’t want to have to worry about all of the packages we’ve installed every time we right a piece of code
  • Load all of the functions in the package:

Loading and viewing the dataset

  • Lets download the surveys.csv and load into R using read.csv().

Basic dplyr

  • Modern data manipulation library for R


  • Select a subset of columns.
  • They can occur in any order.

Do Shrub Volume Data Basics 1-2.


  • Add new columns with calculated values using mutate()
  • If we look at surveys now will it contain the new column?
  • Open surveys
  • All of these commands produce new values, data frames in this case
  • To store them for later use we need to assign them to a variable
  • Or we could overwrite the existing variable if we don’t need it

Do Shrub Volume Data Basics 3.


  • We can sort the data in the table using arrange
  • To sort the surveys table by by weight
  • We can reverse the order of the sort by “wrapping” weight in another function, desc for “descending
  • We can also sort by multiple columns, so if we wanted to sort first by plot_id and then by date

Do Shrub Volume Data Basics 4.


  • Use filter() to get only the rows that meet certain criteria.
  • Combine the data frame to be filtered with a series of conditional statements.
  • Column, condition, value
  • To filter the data frame to only keep the data on species DS
    • Type the name of the function, filter
    • Parentheses
    • The name of the data frame we want to filter, surveys
    • The column the want to filter on, species_id
    • The condition, which is == for “is equal to”
    • And then the value, "DS"
    • DS here is a string, not a variable or a column name, so we enclose it in quotation marks
  • Like with vectors we can have a condition that is “not equal to” using “!=”
  • So if we wanted the data for all species except “DS
  • We can also filter on multiple conditions at once
  • In computing we combine conditions in two ways “and” & “or”
  • “and” means that all of the conditions must be true
  • Do this in dplyr using additional comma separate arguments
  • So, to get the data on species “DS” for the year 1995:
  • Alternatively we can use the & symbol, which stands for “and”
  • This approach is mostly useful for building more complex conditions

  • “or” means that one or more of the conditions must be true

  • Do this using |

  • To get data on all of the Dipodomys species

Do Shrub Volume Data Basics 5-7.

Filtering null values

  • One of the common tasks we use filter for is removing null values from data
  • Based on what we learned before it’s natural to think that we do this by using the condition weight != NA
  • Why didn’t that work?
  • Null values like NA are special
  • We don’t want to accidentally say that two “missing” things are the same
    • We don’t know if they are
  • So use special commands
  • is.na() checks if the value is NA
  • So if we wanted all of the data where the weigh is NA
  • We’ll learn more about why this works in the same way as the other conditional statements when we study conditionals in detail later in the course

  • To remove null values we combine this with ! for “not”

  • So !is.na(weight) is conceptually the same as “weight != NA”
  • It is common to combine a null filter with other conditions using “and”
  • For example we might want all of the data on a species that contains weights

Do Shrub Volume Data Basics 8.