Strings

Although not all of us are linguists or text analysts, R functions for operating with text strings are still useful. They will come in handy when you need to match records in the data or select a portion of a textual record (for example, only the first name and not the surname). This section covers the basics of these operations.

Matching patterns with regular expressions

Regexps are a very terse language that allow you to describe patterns in strings. They take a little while to get your head around, but once you understand them, you'll find them extremely useful.

To learn regular expressions, we'll use str_view() and str_view_all(). These functions take a character vector and a regular expression, and show you how they match. We'll start with very simple regular expressions and then gradually get more and more complicated. Once you've mastered pattern matching, you'll learn how to apply those ideas with various stringr functions.


Basic matches

The simplest patterns match exact strings:

x <- c("apple", "banana", "pear")
str_view(x, "an")
  • apple
  • banana
  • pear

The next step up in complexity is ., which matches any character (except a newline):

str_view(x, ".a.")
  • apple
  • banana
  • pear

But if "." matches any character, how do you match the character "."? You need to use an "escape" to tell the regular expression you want to match it exactly, not use its special behaviour. Like strings, regexps use the backslash, \, to escape special behaviour. So to match an ., you need the regexp \.. Unfortunately this creates a problem. We use strings to represent regular expressions, and \ is also used as an escape symbol in strings. So to create the regular expression \. we need the string "\\.".

# To create the regular expression, we need \\
dot <- "\\."

# But the expression itself only contains one:
writeLines(dot)
#> \.

# And this tells R to look for an explicit .
str_view(c("abc", "a.c", "bef"), "a\\.c")
  • abc
  • a.c
  • bef

If \ is used as an escape character in regular expressions, how do you match a literal \? Well you need to escape it, creating the regular expression \\. To create that regular expression, you need to use a string, which also needs to escape \. That means to match a literal \ you need to write "\\\\" - you need four backslashes to match one!

x <- "a\\b"
writeLines(x)
#> a\b

str_view(x, "\\\\")
  • a\b

In this book, I'll write regular expression as \. and strings that represent the regular expression as "\\.".


Anchors

By default, regular expressions will match any part of a string. It's often useful to anchor the regular expression so that it matches from the start or end of the string. You can use:

  • ^ to match the start of the string.
  • $ to match the end of the string. 
x <- c("apple", "banana", "pear")
str_view(x, "^a")
  • apple
  • banana
  • pear
str_view(x, "a$")
  • apple
  • banana
  • pear

To remember which is which, try this mnemonic which I learned from Evan Misshula: if you begin with power (^), you end up with money ($).

To force a regular expression to only match a complete string, anchor it with both ^ and $:

x <- c("apple pie", "apple", "apple cake")
str_view(x, "apple")
  • apple pie
  • apple
  • apple cake
str_view(x, "^apple$")
  • apple pie
  • apple
  • apple cake

You can also match the boundary between words with \b. I don't often use this in R, but I will sometimes use it when I'm doing a search in RStudio when I want to find the name of a function that's a component of other functions. For example, I'll search for \bsum\b to avoid matching summarise, summary, rowsum and so on.


Character classes and alternatives

There are a number of special patterns that match more than one character. You've already seen ., which matches any character apart from a newline. There are four other useful tools:

  • \d: matches any digit.
  • \s: matches any whitespace (e.g. space, tab, newline).
  • [abc]: matches a, b, or c.
  • [^abc]: matches anything except a, b, or c.

Remember, to create a regular expression containing \d or \s, you'll need to escape the \ for the string, so you'll type "\\d" or "\\s".

A character class containing a single character is a nice alternative to backslash escapes when you want to include a single metacharacter in a regex. Many people find this more readable.

# Look for a literal character that normally has special meaning in a regex
str_view(c("abc", "a.c", "a*c", "a c"), "a[.]c")
  • abc
  • a.c
  • a*c
  • a c
str_view(c("abc", "a.c", "a*c", "a c"), ".[*]c")
  • abc
  • a.c
  • a*c
  • a c
str_view(c("abc", "a.c", "a*c", "a c"), "a[ ]")
  • abc
  • a.c
  • a*c
  • a c

This works for most (but not all) regex metacharacters: $ . | ? * + ( ) [ {. Unfortunately, a few characters have special meaning even inside a character class and must be handled with backslash escapes: ] \ ^ and -.

You can use alternation to pick between one or more alternative patterns. For example, abc|d..f will match either ‘"abc"', or "deaf". Note that the precedence for | is low, so that abc|xyz matches abc or xyz not abcyz or abxyz. Like with mathematical expressions, if precedence ever gets confusing, use parentheses to make it clear what you want:

str_view(c("grey", "gray"), "gr(e|a)y")
  • grey
  • gray


Repetition

The next step up in power involves controlling how many times a pattern matches:

  • ?: 0 or 1
  • +: 1 or more
  • *: 0 or more 
x <- "1888 is the longest year in Roman numerals: MDCCCLXXXVIII"
str_view(x, "CC?")
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII
str_view(x, "CC+")
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII
str_view(x, 'C[LX]+')
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII

Note that the precedence of these operators is high, so you can write: colou?r to match either American or British spellings. That means most uses will need parentheses, like bana(na)+.

You can also specify the number of matches precisely:

  • {n}: exactly n
  • {n,}: n or more
  • {,m}: at most m
  • {n,m}: between n and m 
str_view(x, "C{2}")
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII
str_view(x, "C{2,}")
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII
str_view(x, "C{2,3}")
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII

By default these matches are "greedy": they will match the longest string possible. You can make them "lazy", matching the shortest string possible by putting a ? after them. This is an advanced feature of regular expressions, but it's useful to know that it exists:

str_view(x, 'C{2,3}?')
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII
str_view(x, 'C[LX]+?')
  • 1888 is the longest year in Roman numerals: MDCCCLXXXVIII


Grouping and backreferences

Earlier, you learned about parentheses as a way to disambiguate complex expressions. Parentheses also create a numbered capturing group (number 1, 2 etc.). A capturing group stores the part of the string matched by the part of the regular expression inside the parentheses. You can refer to the same text as previously matched by a capturing group with backreferences, like \1, \2 etc. For example, the following regular expression finds all fruits that have a repeated pair of letters.

str_view(fruit, "(..)\\1", match = TRUE)
  • banana
  • coconut
  • cucumber
  • jujube
  • papaya
  • salal berry

(Shortly, you'll also see how they're useful in conjunction with str_match().)