Manipulate dates easily with {lubridate}
R
This blog post is an excerpt of my ebook Modern R with the tidyverse that you can read for
free here. This is taken from Chapter 5, which presents
the {tidyverse} packages and how to use them to compute descriptive statistics and manipulate data.
In the text below, I scrape a table from Wikipedia, which shows when African countries gained
independence from other countries. Then, using {lubridate} functions I show you how you can
answers questions such as Which countries gained independence before 1960?.
Set-up: scraping some data from Wikipedia
{lubridate} is yet another tidyverse package, that makes dealing with dates or duration data
(and intervals) as painless as possible. I do not use every function contained in the package
daily, and as such will only focus on some of the functions. However, if you have to deal with
dates often, you might want to explore the package thoroughly.
Let’s get some data from a Wikipedia table:
library(tidyverse)
library(rvest)page <- read_html("https://en.wikipedia.org/wiki/Decolonisation_of_Africa")
independence <- page %>%
html_node(".wikitable") %>%
html_table(fill = TRUE)
independence <- independence %>%
select(-Rank) %>%
map_df(~str_remove_all(., "\\[.*\\]")) %>%
rename(country = `Country[a]`,
colonial_name = `Colonial name`,
colonial_power = `Colonial power[b]`,
independence_date = `Independence date[c]`,
first_head_of_state = `First head of state[d]`,
independence_won_through = `Independence won through`)This dataset was scraped from the following Wikipedia table. It shows when African countries gained independence from which colonial powers. In Chapter 11, I will show you how to scrape Wikipedia pages using R. For now, let’s take a look at the contents of the dataset:
independence## # A tibble: 54 x 6
## country colonial_name colonial_power independence_da… first_head_of_s…
## <chr> <chr> <chr> <chr> <chr>
## 1 Liberia Liberia United States 26 July 1847 Joseph Jenkins …
## 2 South … Cape Colony … United Kingdom 31 May 1910 Louis Botha
## 3 Egypt Sultanate of… United Kingdom 28 February 1922 Fuad I
## 4 Eritrea Italian Erit… Italy 10 February 1947 Haile Selassie
## 5 Libya British Mili… United Kingdo… 24 December 1951 Idris
## 6 Sudan Anglo-Egypti… United Kingdo… 1 January 1956 Ismail al-Azhari
## 7 Tunisia French Prote… France 20 March 1956 Muhammad VIII a…
## 8 Morocco French Prote… France Spain 2 March 19567 A… Mohammed V
## 9 Ghana Gold Coast United Kingdom 6 March 1957 Kwame Nkrumah
## 10 Guinea French West … France 2 October 1958 Ahmed Sékou Tou…
## # … with 44 more rows, and 1 more variable: independence_won_through <chr>as you can see, the date of independence is in a format that might make it difficult to answer questions such as Which African countries gained independence before 1960 ? for two reasons. First of all, the date uses the name of the month instead of the number of the month (well, this is not such a big deal, but still), and second of all the type of the independence day column is character and not “date”. So our first task is to correctly define the column as being of type date, while making sure that R understands that January is supposed to be “01”, and so on.
Using {lubridate}
There are several helpful functions included in {lubridate} to convert columns to dates. For instance
if the column you want to convert is of the form “2012-11-21”, then you would use the function ymd(),
for “year-month-day”. If, however the column is “2012-21-11”, then you would use ydm(). There’s
a few of these helper functions, and they can handle a lot of different formats for dates. In our case,
having the name of the month instead of the number might seem quite problematic, but it turns out
that this is a case that {lubridate} handles painfully:
library(lubridate)##
## Attaching package: 'lubridate'## The following object is masked from 'package:base':
##
## dateindependence <- independence %>%
mutate(independence_date = dmy(independence_date))## Warning: 5 failed to parse.Some dates failed to parse, for instance for Morocco. This is because these countries have several independence dates; this means that the string to convert looks like:
"2 March 1956
7 April 1956
10 April 1958
4 January 1969"which obviously cannot be converted by {lubridate} without further manipulation. I ignore these cases for
simplicity’s sake.
Let’s take a look at the data now:
independence## # A tibble: 54 x 6
## country colonial_name colonial_power independence_da… first_head_of_s…
## <chr> <chr> <chr> <date> <chr>
## 1 Liberia Liberia United States 1847-07-26 Joseph Jenkins …
## 2 South … Cape Colony … United Kingdom 1910-05-31 Louis Botha
## 3 Egypt Sultanate of… United Kingdom 1922-02-28 Fuad I
## 4 Eritrea Italian Erit… Italy 1947-02-10 Haile Selassie
## 5 Libya British Mili… United Kingdo… 1951-12-24 Idris
## 6 Sudan Anglo-Egypti… United Kingdo… 1956-01-01 Ismail al-Azhari
## 7 Tunisia French Prote… France 1956-03-20 Muhammad VIII a…
## 8 Morocco French Prote… France Spain NA Mohammed V
## 9 Ghana Gold Coast United Kingdom 1957-03-06 Kwame Nkrumah
## 10 Guinea French West … France 1958-10-02 Ahmed Sékou Tou…
## # … with 44 more rows, and 1 more variable: independence_won_through <chr>As you can see, we now have a date column in the right format. We can now answer questions such as
Which countries gained independence before 1960? quite easily, by using the functions year(),
month() and day(). Let’s see which countries gained independence before 1960:
independence %>%
filter(year(independence_date) <= 1960) %>%
pull(country)## [1] "Liberia" "South Africa"
## [3] "Egypt" "Eritrea"
## [5] "Libya" "Sudan"
## [7] "Tunisia" "Ghana"
## [9] "Guinea" "Cameroon"
## [11] "Togo" "Mali"
## [13] "Madagascar" "Democratic Republic of the Congo"
## [15] "Benin" "Niger"
## [17] "Burkina Faso" "Ivory Coast"
## [19] "Chad" "Central African Republic"
## [21] "Republic of the Congo" "Gabon"
## [23] "Mauritania"You guessed it, year() extracts the year of the date column and converts it as a numeric so that we can work
on it. This is the same for month() or day(). Let’s try to see if countries gained their independence on
Christmas Eve:
independence %>%
filter(month(independence_date) == 12,
day(independence_date) == 24) %>%
pull(country)## [1] "Libya"Seems like Libya was the only one! You can also operate on dates. For instance, let’s compute the difference between
two dates, using the interval() column:
independence %>%
mutate(today = lubridate::today()) %>%
mutate(independent_since = interval(independence_date, today)) %>%
select(country, independent_since)## # A tibble: 54 x 2
## country independent_since
## <chr> <S4: Interval>
## 1 Liberia 1847-07-26 UTC--2019-02-10 UTC
## 2 South Africa 1910-05-31 UTC--2019-02-10 UTC
## 3 Egypt 1922-02-28 UTC--2019-02-10 UTC
## 4 Eritrea 1947-02-10 UTC--2019-02-10 UTC
## 5 Libya 1951-12-24 UTC--2019-02-10 UTC
## 6 Sudan 1956-01-01 UTC--2019-02-10 UTC
## 7 Tunisia 1956-03-20 UTC--2019-02-10 UTC
## 8 Morocco NA--NA
## 9 Ghana 1957-03-06 UTC--2019-02-10 UTC
## 10 Guinea 1958-10-02 UTC--2019-02-10 UTC
## # … with 44 more rowsThe independent_since column now contains an interval object that we can convert to years:
independence %>%
mutate(today = lubridate::today()) %>%
mutate(independent_since = interval(independence_date, today)) %>%
select(country, independent_since) %>%
mutate(years_independent = as.numeric(independent_since, "years"))## # A tibble: 54 x 3
## country independent_since years_independent
## <chr> <S4: Interval> <dbl>
## 1 Liberia 1847-07-26 UTC--2019-02-10 UTC 172.
## 2 South Africa 1910-05-31 UTC--2019-02-10 UTC 109.
## 3 Egypt 1922-02-28 UTC--2019-02-10 UTC 97.0
## 4 Eritrea 1947-02-10 UTC--2019-02-10 UTC 72
## 5 Libya 1951-12-24 UTC--2019-02-10 UTC 67.1
## 6 Sudan 1956-01-01 UTC--2019-02-10 UTC 63.1
## 7 Tunisia 1956-03-20 UTC--2019-02-10 UTC 62.9
## 8 Morocco NA--NA NA
## 9 Ghana 1957-03-06 UTC--2019-02-10 UTC 61.9
## 10 Guinea 1958-10-02 UTC--2019-02-10 UTC 60.4
## # … with 44 more rowsWe can now see for how long the last country to gain independence has been independent. Because the data is not tidy (in some cases, an African country was colonized by two powers, see Libya), I will only focus on 4 European colonial powers: Belgium, France, Portugal and the United Kingdom:
independence %>%
filter(colonial_power %in% c("Belgium", "France", "Portugal", "United Kingdom")) %>%
mutate(today = lubridate::today()) %>%
mutate(independent_since = interval(independence_date, today)) %>%
mutate(years_independent = as.numeric(independent_since, "years")) %>%
group_by(colonial_power) %>%
summarise(last_colony_independent_for = min(years_independent, na.rm = TRUE))## # A tibble: 4 x 2
## colonial_power last_colony_independent_for
## <chr> <dbl>
## 1 Belgium 56.6
## 2 France 41.6
## 3 Portugal 43.2
## 4 United Kingdom 42.6{lubridate} contains many more functions. If you often work with dates, duration or interval data, {lubridate}
is a package that you have to master.
Hope you enjoyed! If you found this blog post useful, you might want to follow me on twitter for blog post updates and buy me an espresso or paypal.me.
