3 Loading and selecting data
This chapter introduces the first essential skills of data wrangling: loading data into R and choosing the rows and columns needed for an analysis. We will learn how functions and packages work, how to read CSV and Excel files, how to give objects meaningful names and how to use the dplyr package to select and filter data.
- Open Positron or – if you already have Positron open – start a new R session by clicking the Restart R (⟳) button in the Console panel. This makes sure no code you ran previously will interfere with the work you do in this chapter.
- Make sure you are working in the crime mapping project workspace you created in Section 1.5. In the top-right corner of the Positron window, you should see a folder icon and the words
crime_mapping. If not, click the File menu, then Open Folder … and choose thecrime_mappingfolder.
3.1 Introduction
From Chapter 3 to Chapter 7, we will learn all the skills needed to make a good crime map in R. We will learn to:
- load data from a variety of sources, select columns and filter rows (this chapter);
- create new columns in datasets, change existing columns and summarise groups of rows (Chapter 4);
- convert data to objects that can be used for mapping and use those objects to make a map (Chapter 5);
- estimate the density of crime in different places and show that on a map (Chapter 6); and
- add context to a map using titles, legends, scale bars and north arrows (Chapter 7).
A major step in using any data to make decisions or draw conclusions is data wrangling: the process of transforming data from the format in which we originally have it to the format needed to analyse and present it to our audience.
In this chapter, we will learn how to:
- download CSV and Excel files into a workspace;
- load and inspect tabular data;
- give R objects meaningful names; and
- select columns and filter rows.
The work in this chapter follows four main stages:
Download data
into the data/raw folder
Load data
from a file into an R object
Select columns
to keep the variables we need
Filter rows
to keep the observations we need
To get started, create the first of two scripts used in this chapter:
- Click the File menu in Positron, then click New File ….
- Type
chapter_03a.Rin the box marked Select File Type or Enter File Name…, then press ReturnReturn. - Save the file in the
Rfolder inside yourcrime_mappingworkspace.
As we learned in Chapter 2, permanent code needed to repeat the analysis belongs in this script. Temporary code that helps you inspect the data or develop the permanent code belongs in the R Console. Code chunks in this chapter are labelled to show where each piece of code belongs.
3.1.1 Functions
In this chapter we will learn how to wrangle data in R using functions – specialised pieces of code that do something to the data we give them. The code to use a function (sometimes called calling the function) has two parts: the function name followed by a pair of parentheses, inside which are zero or more arguments separated by commas. Arguments are a way of providing input that a function works on, or to fine-tune the way the function works (we will see many examples of this later). Remember that you can identify a function in R because the name will always have parentheses after it.
One basic R function is sqrt(), which calculates the square root of a number. The sqrt() function has only one argument: the number that we want to find the square root of. If we typed Code 3.1 into the R Console, R would show us the square root of 2.
When you run code in R, by default R prints the output of your code – in this case, just the number 1.414214 (for now, you can ignore the number [1] in square brackets).
3.1.2 Packages
R contains thousands of different functions that do different things. A few functions are contained in the default installation of R that you have already installed (this is sometimes referred to as base R). But most functions are contained in packages, which are extensions to base R. Most packages focus on a particular type of data analysis, so that there are packages devoted to time-series analysis, testing whether events are clustered in particular places, network analysis and thousands of other tasks. Packages are often developed by experts in the field, and are typically updated to introduce new features.
To use a function from a package we need to do two things: install that package on our computer, and then load the package into our R session. You may see code that does these two steps separately, using the install.packages() function to install a package and the library() function to load it. However, in this book we use the p_load() function from the pacman package to do both steps in one line of code. p_load() checks whether a package is already installed on your computer, and if it is not, installs it automatically and then loads it. Do not add install.packages() to your scripts. Re-installing packages every time a script runs is unnecessary, slows the script down and can make it harder for someone else to run your code on their computer.
In this chapter we will use functions from the here, httr2, readxl and tidyverse packages, so add this code to chapter_03a.R:
Now run this code by clicking anywhere on that line of code then pressing on your keyboard.
pacman::?
The pacman package is an R package that is used to manage other R packages. The p_load() function from that package is used to load packages. But we learned above that we must load a package before we can use it, which means we cannot use functions from the pacman package until we have loaded the pacman package. To get around that, instead of just using the function name p_load(), we can instead specify which package p_load() is from by prefixing the function name with the relevant package name, separated by ::.
Many packages are focused on specialist tasks and so are only used occasionally, but a few packages are likely to be useful in almost all the code we write. Fortunately, packages can themselves load other packages, and all the main packages we need are loaded by the tidyverse package. That is why you will often see pacman::p_load(tidyverse) at the top of R code in subsequent chapters – that short line of code loads several packages containing hundreds of functions that we can use in data analysis.
What is the purpose of data wrangling?
In general, what is a function in R?
3.2 Loading data
Before we can do anything with any data, we have to load it into R. In this course we will read tabular data in comma-separated values (CSV) and Excel formats, as well as spatial data in different formats (because there are lots of ways to store spatial data). We will learn how to read CSV and Excel data now, but leave loading spatial data until later.
Tabular data (sometimes known as rectangular data) describes data formats with multiple columns where every column has the same number of rows. For example, crime data might have columns for the type of crime, date and address at which the crime occurred.
| type | date | address |
|---|---|---|
| homicide | 13 Oct 2025 | 274 Main St |
| non-residential burglary | 23 May 2026 | 541 Station Rd |
| personal robbery | 25 Sep 2026 | 10 North Av |
Almost all the data we will use in this course will be in this rectangular format, and most of the functions we will use expect data to be rectangular.
3.2.1 Loading CSV data
Data stored in CSV format is easy to load with the read_csv() function from the readr package. readr is one of the packages loaded by the tidyverse package, so all we need to do to use this package is include the code pacman::p_load(tidyverse) on the first line of our R script. We will use comments (lines of code beginning with #) to help explain the code as we go.
During this course, much of the data we use will come from the crimemappingdata website. We will first download each original file to the data/raw folder and then load that local copy into R. Keeping a copy of the original data means that our analysis does not depend on repeatedly accessing a website, and that we can always return to the unchanged source file.
Add this code to the chapter_03a.R file and run it.
<httr2_response>
GET https://mpjashby.github.io/crimemappingdata/san_francisco_robbery.csv
Status: 200 OK
Content-Type: text/csv
Body: On disk '/Users/mattashby/Documents/Crime Mapping Book/data/raw/san_francisco_robbery.csv' (65580 bytes)The request() and req_perform() functions from the httr2 (pronounced ‘hit-r two’) package download the file, following the same pattern used in Section 2.3.2. The here() function from the here package helps you specify where on your computer you want to store or retrieve a file inside the crime_mapping folder we created in Section 1.5. In this case, here() specifies that the file should be stored in a file called san_francisco_robbery.csv inside the raw folder that is itself inside the data folder.
Now we’ve downloaded the data, we need to load it into R. Add this code to the script file and run it:
Rows: 951 Columns: 5
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr (1): offense_type
dbl (3): uid, longitude, latitude
dttm (1): date_time
ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.read_csv() mean?
By default, the read_csv() function prints a message when it loads data to summarise the format of each data column. In the case of the san_fran_rob dataset, read_csv() tells us that:
- there is one column called
offense_typethat contains character (chr) values, - there are three columns called
uid,longitudeandlatitudecontaining numeric (dbl) values, and - there is one column called
date_timethat contains values stored as dates and times (dttm).
There are some other possible types of data, but we will learn about these later on. The numeric values are referred to as dbl values because they are stored in a format that can handle numbers that are not whole numbers (e.g. 123.456). This format for storing numbers is called the double-precision floating-point format, which is often known as the double format for short. Most numbers in R are stored in double format, so you can think of the format code dbl as meaning ‘numeric’. You don’t need to remember that ‘double’ is short for double-precision floating-point format, just that it represents a number.
This code loads the data using the read_csv() function and stores the result in an object named san_fran_rob. Objects are places where we can store data. To create an object and store our data in it, we use the assignment operator <- (a less-than sign followed by a dash). Rather than typing a less-than sign and a dash every time you need to assign a value to an object, you can instead use the keyboard shortcut to type an assignment operator.
3.2.2 Naming objects
Object names should briefly describe what an object contains. For example, san_fran_rob is more informative than a generic name such as data. Meaningful names make a script easier to understand and reduce the chance that you will accidentally use the wrong object. Using the wrong object is a common cause of errors in code, and those errors can be surprisingly hard to track down and fix, so it’s best to avoid them by giving objects meaningful names in the first place.
Another way to make object names easier to work with is to only use names containing lowercase letters, numbers and underscores. Object names that follow these rules are said to be written in snake_case. Giving all your objects names in snake_case means you never have to remember whether you used upper or lower-case letters in an object name, and makes it easier to read object names containing multiple words. This is particularly important because R distinguishes between upper- and lower-case letters, so crime_data, Crime_Data and CRIME_DATA would be three different objects. Using lower-case snake case consistently means that you do not have to remember which variation you chose.
This rule applies even when one of the words in an object name normally uses upper-case letters, such as a name or an abbreviation. For example, you should call an object containing data on homicides in Abu Dhabi something like abu_dhabi_homicides, not Abu_Dhabi_Homicides or AbuDhabiHomicides. Similarly, if you had a dataset of offences of grievous bodily harm (usually abbreviated to GBH), you should call the object gbh_data, not GBH_data or GBHData.
These conventions also apply to column names. Data obtained from elsewhere will not always follow them, but we can use the janitor::clean_names() function to convert column names to snake case. We will use that function in Chapter 5.
Which is the best name for an object containing data on homicides in Abu Dhabi?
When choosing object names, it is important to remember that if you assign a value (such as the number 1 or the result of the function read_csv()) to an object name, R will overwrite any existing value of that object name. We can see this in a simple example:
If we were to run this code, the object one_to_ten would not actually hold the numbers from one to ten, but instead the value 1.414214 (the square root of two). There is no way to undo assignment of a value to an object, so once you have run the code one_to_ten <- sqrt(2) it is not possible to recover any previous value that was assigned to the object one_to_ten. For that reason, it is generally a bad idea to reuse object names to store different values in the same script.
Objects come in several different types, with tabular data typically being stored as a data frame. The read_csv() function produces a modern variation on the data frame called a tibble. Tibbles behave like data frames most of the time, but are designed to work conveniently with tidyverse functions. We will use tibbles throughout this course.
If the data are loaded successfully, R will list the columns in the data and the type of variable (numeric, date etc.) stored in each column.
To see the first few rows of data currently stored in an object, we can use the head() function. Remember that we run functions such as head() in the R Console, not in the script file, because it is temporary code (see Section 2.2). Add this code to the R Console and run it by pressing :
# A tibble: 6 × 5
uid offense_type date_time longitude latitude
<dbl> <chr> <dttm> <dbl> <dbl>
1 24103841 personal robbery 2019-01-01 19:50:00 -122. 37.8
2 24103948 personal robbery 2019-01-02 08:00:00 -122. 37.8
3 24104162 personal robbery 2019-01-03 00:30:00 -122. 37.8
4 24104203 personal robbery 2019-01-03 03:13:00 -122. 37.8
5 24104237 personal robbery 2019-01-03 09:30:00 -122. 37.7
6 24104238 personal robbery 2019-01-03 09:30:00 -122. 37.73.2.3 Loading Excel data
Loading data from Microsoft Excel files is very similar to loading CSV data, with a few important differences. Functions to load Excel data are contained in the readxl package.
Create a second R script by repeating the Positron steps you used earlier. Name the file chapter_03b.R and save it in the R folder. Add Code 3.7, then place the cursor on each statement and press to run it.
<httr2_response>
GET https://mpjashby.github.io/crimemappingdata/aggravated_assaults.xlsx
Status: 200 OK
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
Body: On disk '/Users/mattashby/Documents/Crime Mapping Book/data/raw/aggravated_assaults.xlsx' (383278 bytes)Now we have downloaded our data, we can load it into R. Excel files can contain multiple sheets, so we need to specify which sheet we would like to load into a tibble. We can use the excel_sheets() function in the R Console to get a list of sheets in an Excel file:
[1] "Austin" "Fort Worth" "Seattle" We can now load the sheet containing data for Austin and view the first few rows of the resulting object:
and, as usual, we can use the head() function to look at the data:
# A tibble: 6 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-01-01 00:00:00 -97.7 30.3 residence residence
2 2019-01-01 00:00:00 -97.8 30.2 residence residence
3 2019-01-01 00:01:00 -97.7 30.3 <NA> <NA>
4 2019-01-01 00:15:00 -97.7 30.3 <NA> <NA>
5 2019-01-01 00:27:00 -97.8 30.2 residence residence
6 2019-01-01 00:30:00 -97.7 30.3 <NA> <NA> Now we have learned how to load our data into an object, we can use other R functions to work with that data in many different ways.
Different types of data are loaded into R with different functions, e.g. CSV files are loaded with the read_csv() function from the readr package and Microsoft Excel files are loaded with the read_excel() function from the readxl package. Appendix A has a list of which function to use to load each type of file.
Learn more about importing data in the current edition of the free online book R for Data Science.
Excel data can often be messy and the readxl package contains various other functions that can be used to deal with this. You can learn more about how to handle messy Excel data in this online tutorial.
Answer the following questions to check your understanding of what we’ve learned so far in this chapter. If you get a question wrong, you can keep trying until you get the right answer.
What R package contains the function read_csv() to read CSV data?
What R code prints the first few rows of the tibble called san_fran_rob?
If we create an object using the code number_ten <- 10 and then run the code number_ten <- sqrt(2), what value will the object number_ten now have?
3.3 Selecting columns
In this section we will learn how to reduce the size of our data by selecting only the columns we need and discarding the rest. This can be particularly useful if we are working with a very-large dataset, or if we want to produce a table containing only some columns.
We can use the select() function from the dplyr package (one of the packages that is loaded automatically when we call the pacman::p_load(tidyverse) function) to select columns.
If we wanted to select just the date and location_type columns from the agg_assault_data we loaded in Section 3.2.3, we can use this code:
# A tibble: 8,696 × 2
date location_type
<dttm> <chr>
1 2019-01-01 00:00:00 residence
2 2019-01-01 00:00:00 residence
3 2019-01-01 00:01:00 <NA>
4 2019-01-01 00:15:00 <NA>
5 2019-01-01 00:27:00 residence
6 2019-01-01 00:30:00 <NA>
7 2019-01-01 00:51:00 <NA>
8 2019-01-01 01:00:00 residence
9 2019-01-01 01:00:00 <NA>
10 2019-01-01 01:12:00 residence
# ℹ 8,686 more rows
In Section 3.1.1, we mentioned that the code needed to run (or call) a function in R has two parts: the function name followed by a pair of parentheses, inside which are zero or more arguments separated by commas. The arguments in the select() function (and many other functions in the dplyr package) work in a slightly different way to many other functions. Here, the first argument is the name of the data object that we want to select from. All the remaining arguments (here, date and location_type) are the names of the columns we want to select from the data.
We can select as many columns as we want, by just adding the names of the columns separated by commas. The columns in our new dataset will appear in the order in which we specify them in the select() function.
We can also use select() to rename columns at the same time as selecting them. For example, to select the columns date and location_type while also renaming location_type to be called type, we can use:
# A tibble: 8,696 × 2
date type
<dttm> <chr>
1 2019-01-01 00:00:00 residence
2 2019-01-01 00:00:00 residence
3 2019-01-01 00:01:00 <NA>
4 2019-01-01 00:15:00 <NA>
5 2019-01-01 00:27:00 residence
6 2019-01-01 00:30:00 <NA>
7 2019-01-01 00:51:00 <NA>
8 2019-01-01 01:00:00 residence
9 2019-01-01 01:00:00 <NA>
10 2019-01-01 01:12:00 residence
# ℹ 8,686 more rowsselect() removes any columns that we don’t explicitly choose to keep. If we want to rename a column while keeping all the existing columns in the data, we can instead use the rename() function (also from the dplyr package):
# A tibble: 8,696 × 5
date longitude latitude type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-01-01 00:00:00 -97.7 30.3 residence residence
2 2019-01-01 00:00:00 -97.8 30.2 residence residence
3 2019-01-01 00:01:00 -97.7 30.3 <NA> <NA>
4 2019-01-01 00:15:00 -97.7 30.3 <NA> <NA>
5 2019-01-01 00:27:00 -97.8 30.2 residence residence
6 2019-01-01 00:30:00 -97.7 30.3 <NA> <NA>
7 2019-01-01 00:51:00 -97.7 30.3 <NA> <NA>
8 2019-01-01 01:00:00 -97.7 30.4 residence residence
9 2019-01-01 01:00:00 -97.7 30.3 <NA> <NA>
10 2019-01-01 01:12:00 -97.7 30.3 residence residence
# ℹ 8,686 more rowsRemember that functions in R generally do not change existing objects, but instead produce (or return) new ones. This means if we want to store the result of this function so we can use it later, we have to assign the value returned by the function to a new object (or overwrite the existing object, but this is a bad idea):
# A tibble: 6 × 2
lon lat
<dbl> <dbl>
1 -97.7 30.3
2 -97.8 30.2
3 -97.7 30.3
4 -97.7 30.3
5 -97.8 30.2
6 -97.7 30.3You can learn more about selecting, filtering and arranging data using the functions in the dplyr package by reading this Introduction to dplyr tutorial.
Which dplyr function allows you to change the name of columns while keeping all the columns in the original data?
Which dplyr function allows you to choose only some columns in the original data?
3.4 Filtering rows
Often in crime mapping we will only be interested in part of a particular dataset. In the same way that we can select particular columns in our data, we can filter particular rows using the filter() function from the dplyr package.
If we were only interested in offences in the agg_assault_data dataset that occurred in residences, we could use filter():
# A tibble: 4,385 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-01-01 00:00:00 -97.7 30.3 residence residence
2 2019-01-01 00:00:00 -97.8 30.2 residence residence
3 2019-01-01 00:27:00 -97.8 30.2 residence residence
4 2019-01-01 01:00:00 -97.7 30.4 residence residence
5 2019-01-01 01:12:00 -97.7 30.3 residence residence
6 2019-01-01 01:20:00 -97.7 30.4 residence residence
7 2019-01-01 01:50:00 -97.7 30.3 residence residence
8 2019-01-01 02:21:00 -97.8 30.2 residence residence
9 2019-01-01 02:26:00 -97.8 30.2 residence residence
10 2019-01-01 02:35:00 -97.7 30.4 residence residence
# ℹ 4,375 more rowsNote that:
- the column name
location_typeis not surrounded by quotes (because it represents a column in the data) but the column value"residence"is (because it represents the literal character value “residence”), and - the
==(equal to) operator is used, since a single equals sign=has another meaning in R.
We can filter using the values of more than one column simultaneously. To filter offences in which the location_category is ‘retail’ and the location_type is ‘convenience store’:
# A tibble: 90 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-01-06 16:58:00 -97.8 30.5 convenience store retail
2 2019-01-08 17:56:00 -97.7 30.3 convenience store retail
3 2019-01-10 01:25:00 -97.7 30.4 convenience store retail
4 2019-01-12 00:49:00 -97.6 30.4 convenience store retail
5 2019-01-12 05:53:00 -97.7 30.4 convenience store retail
6 2019-01-12 17:01:00 -97.7 30.4 convenience store retail
7 2019-01-12 19:12:00 -97.8 30.3 convenience store retail
8 2019-01-12 19:14:00 -97.7 30.4 convenience store retail
9 2019-01-13 04:10:00 -97.8 30.2 convenience store retail
10 2019-01-22 22:01:00 -97.7 30.3 convenience store retail
# ℹ 80 more rowsfilter() returns rows that meet all the criteria you specify
When you run Code 3.16, the result will contain only those rows in the original data for which the location_category column has the value ‘retail’ and the location_type column has the value ‘convenience store’.
As well as filtering using the == (equals) operator, we can filter using the greater-than (>), less-than (<), greater-than-or-equal-to (>=) and less-than-or-equal-to (<=) operators. For example, we can choose offences that occurred in residences on or after 1 July 2019:
# A tibble: 2,286 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-07-01 02:40:00 -97.8 30.4 residence residence
2 2019-07-01 07:26:00 -97.7 30.3 residence residence
3 2019-07-01 08:25:00 -97.8 30.2 residence residence
4 2019-07-01 09:39:00 -97.7 30.3 residence residence
5 2019-07-01 09:40:00 -97.7 30.4 residence residence
6 2019-07-01 16:24:00 -97.8 30.1 residence residence
7 2019-07-01 17:30:00 -97.8 30.2 residence residence
8 2019-07-01 17:41:00 -97.8 30.2 residence residence
9 2019-07-01 18:03:00 -97.8 30.2 residence residence
10 2019-07-01 18:16:00 -97.8 30.1 residence residence
# ℹ 2,276 more rowsymd() function do?
The ymd() function from the lubridate package (which we will find out more about below) converts a date stored as a text value to a value that R understands represents a calendar date. The function is called ymd() because it processes dates that are stored in year-month-date format. We will learn more about working with dates in Chapter 15.
Sometimes we will want to filter rows that are one thing or another. We can do this with the | (or) operator. For example, we can filter offences that occurred either in leisure facilities or shopping malls on or after 1 July 2019:
# A tibble: 10 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-07-01 09:00:00 -97.7 30.3 entertainment leisure
2 2019-07-15 14:06:00 -97.8 30.3 mall retail
3 2019-08-06 23:20:00 -97.7 30.4 mall retail
4 2019-08-18 13:31:00 -97.7 30.3 mall retail
5 2019-08-25 15:35:00 -97.8 30.3 mall retail
6 2019-09-06 15:58:00 -97.7 30.4 mall retail
7 2019-10-05 16:47:00 -97.7 30.2 mall retail
8 2019-11-01 22:23:00 -97.7 30.3 entertainment leisure
9 2019-11-18 16:29:00 -97.7 30.4 mall retail
10 2019-11-27 14:14:00 -97.8 30.3 mall retail If we want to filter offences that have any one of several different values in the same column, we can use the %in% (in) operator. To filter offences that occurred in either streets or publicly accessible open spaces:
# A tibble: 165 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-01-07 21:37:00 -97.8 30.5 green space open space
2 2019-01-08 14:02:00 -97.7 30.3 green space open space
3 2019-01-10 01:59:00 -97.6 30.3 green space open space
4 2019-01-11 17:00:00 -97.7 30.3 green space open space
5 2019-01-12 15:50:00 -97.8 30.2 green space open space
6 2019-01-12 16:00:00 -97.8 30.4 green space open space
7 2019-01-15 07:20:00 -97.7 30.2 green space open space
8 2019-01-17 00:00:00 -97.7 30.4 green space open space
9 2019-01-22 22:00:00 -97.7 30.2 green space open space
10 2019-01-29 01:40:00 -97.7 30.2 green space open space
# ℹ 155 more rowsThe code c("open space", "street") produces what is referred to in R as a vector (sometimes referred to as an atomic vector, especially in error messages). A vector is a one-dimensional sequence of values of the same type (i.e. all numbers, all character strings etc.). For example, a vector might hold several strings of text (as in the vector c("open space", "street")) or a series of numbers such as c(1, 2, 3). There is lots we could learn about vectors, but for now it’s only necessary to know that we can create vectors with the c() or combine function.
If we wanted to re-use a vector of values several times in our code, it might make sense to store the vector as an object. For example:
# A tibble: 165 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-01-07 21:37:00 -97.8 30.5 green space open space
2 2019-01-08 14:02:00 -97.7 30.3 green space open space
3 2019-01-10 01:59:00 -97.6 30.3 green space open space
4 2019-01-11 17:00:00 -97.7 30.3 green space open space
5 2019-01-12 15:50:00 -97.8 30.2 green space open space
6 2019-01-12 16:00:00 -97.8 30.4 green space open space
7 2019-01-15 07:20:00 -97.7 30.2 green space open space
8 2019-01-17 00:00:00 -97.7 30.4 green space open space
9 2019-01-22 22:00:00 -97.7 30.2 green space open space
10 2019-01-29 01:40:00 -97.7 30.2 green space open space
# ℹ 155 more rowsFinally, you can filter based on the output of any R function that returns TRUE or FALSE. For example, missing values are represented in R as NA. We can test whether a value is missing using the is.na() function. If we wanted to remove rows from our data that had missing location types, we would filter for those rows that are not NA. We can do this by combining the is.na() function with the ! (not) operator:
# A tibble: 5,344 × 5
date longitude latitude location_type location_category
<dttm> <dbl> <dbl> <chr> <chr>
1 2019-01-01 00:00:00 -97.7 30.3 residence residence
2 2019-01-01 00:00:00 -97.8 30.2 residence residence
3 2019-01-01 00:27:00 -97.8 30.2 residence residence
4 2019-01-01 01:00:00 -97.7 30.4 residence residence
5 2019-01-01 01:12:00 -97.7 30.3 residence residence
6 2019-01-01 01:20:00 -97.7 30.4 residence residence
7 2019-01-01 01:35:00 -97.7 30.3 hotel hotel
8 2019-01-01 01:50:00 -97.7 30.3 residence residence
9 2019-01-01 02:21:00 -97.8 30.2 residence residence
10 2019-01-01 02:26:00 -97.8 30.2 residence residence
# ℹ 5,334 more rowsThis might be slightly confusing, but what R is doing in this case is:
- identifying whether each row in the
agg_assault_datadataset has a missing value in thelocation_typecolumn (using theis.na()function), returning eitherTRUE(if the value is missing) orFALSE(if the value is not missing) for each row, then - reversing the
TRUEandFALSEvalues (using the!operator), so that rows with missing values are nowFALSEand rows without missing values are nowTRUE, and finally - returning only those rows for which the result of step 2 is
TRUE(i.e. those rows that do not have missing values in thelocation_typecolumn).
The overall effect of this code is therefore to remove any rows that have missing values in the location_type column.
We will see lots more examples of how to use filter() in future chapters.
What is a vector (sometimes known as an atomic vector) in R?
Which offences (rows) will be returned by the code filter(agg_assault_data, location_type %in% c("restaurant", "mall"))?
What does the <= operator mean?
3.5 Complete scripts
Before finishing the chapter, check that chapter_03a.R contains all the permanent code needed to download and load the San Francisco robbery data. Compare your script with the model answer after checking it yourself.
chapter_03a.R
# Load packages
pacman::p_load(here, httr2, readxl, tidyverse)
# Download the data from a URL and store it in a local file
request(
"https://mpjashby.github.io/crimemappingdata/san_francisco_robbery.csv"
) |>
req_perform(path = here("data", "raw", "san_francisco_robbery.csv"))
# Load the local file into R
san_fran_rob <- read_csv(here("data", "raw", "san_francisco_robbery.csv"))Now check that chapter_03b.R contains all the permanent code needed to download and load the Austin aggravated-assault data.
chapter_03b.R
# Load packages
pacman::p_load(here, httr2, readxl, tidyverse)
# Download the data from a URL and store it in a local file
request(
"https://mpjashby.github.io/crimemappingdata/aggravated_assaults.xlsx"
) |>
req_perform(path = here("data", "raw", "aggravated_assaults.xlsx"))
# Load the Austin data from the Excel workbook
agg_assault_data <- read_excel(
here("data", "raw", "aggravated_assaults.xlsx"),
sheet = "Austin"
)Save both scripts by pressing .
3.6 In summary
In this chapter we practised how to:
- download original CSV and Excel files into
data/raw; - load and inspect tabular data;
- distinguish functions, arguments, packages, objects and tibbles;
- choose meaningful snake-case object names;
- select or rename columns with
select()andrename(); and - choose rows that meet particular conditions with
filter().
Keep chapter_03a.R and chapter_03b.R in the R folder, together with the original downloaded files in data/raw. In Chapter 4 we will continue working with these datasets to create new columns, produce summaries and save processed data.
- What is data wrangling, and why is it usually needed before analysis?
- What is the difference between a function, an argument and a package?
- Why should R objects have meaningful names written in snake case?
- How do
select()andfilter()reduce a dataset in different ways?