I’m very new to web scraping and R programming in general and I am trying to put together a data frame of reviews of a random company in order to practice sentiment analysis. The website I am using for this project is Trustpilot and the random company is Netflix.

While I can successfully scrape the text I want (reviewer name & review text), I am unable to add a column for the dates which is really important as well. Basically the code works without the date column in the data frame, but if it’s added then it returns the three variables but no observations.

    url <- "https://uk.trustpilot.com/review/www.netflix.com?page=1"

map_df(1:24, function(i) {

  # progress indicator

  pg <- read_html(sprintf(url, i))

 data.frame(reviewer=html_text(html_nodes(pg, ".consumer-info__details__name")), 
                review=html_text(html_nodes(pg, ".review-info__body__text")),
                  date=html_text(html_nodes(pg, "date")),

}) -> mjreviews

How can I make this code work?