Bigram

The title of a paper often contains stop words, such as articles, prepositions and pronouns, which usually convey no interesting physical meaning. They need to be filtered.

Behold the of s and the.

Let’s declutter a bit by removing the stop words.

3-gram

title_trigram <- d %>%  
  select(Title) %>%  
  unnest_tokens(trigram, Title, token="ngrams", n=3) 


title_trigram %>%  
  head(5) %>%  
  kable()

This snippet shows how to generate n-gram using tidytext and tidyverse. Yes, kable from knitr presents more readible table.

trigram_separated <- title_trigram %>%
  separate(trigram, c("word1", "word2", "word3"), sep = " ") %>%  
  filter(!word1 %in% stop_words$word,
         !word2 %in% stop_words$word,
         !word3 %in% stop_words$word
         ) 

trigram_count <- trigram_separated %>%  
  count(word1, word2, word3, sort=TRUE) 


trigram_count %>%  
  head(10) %>%  
  kable()

Let’s take a look at the top 100 3-word phrases in the titles. Find your own position.

## # A tibble: 20 x 2
##    trigram                                n
##    <chr>                              <int>
##  1 "hot\nmix\nasphalt"                  209
##  2 "warm\nmix\nasphalt"                 147
##  3 "reclaimed\nasphalt\npavement"       143
##  4 "modified\nasphalt\nbinders"          62
##  5 "crumb\nrubber\nmodified"             46
##  6 "roller\ncompacted\nconcrete"         44
##  7 "modified\nasphalt\nbinder"           41
##  8 "recycled\nasphalt\npavement"         39
##  9 "rubber\nmodified\nasphalt"           39
## 10 "mix\nasphalt\nmixtures"              38
## 11 "sbs\nmodified\nasphalt"              38
## 12 "recycled\nconcrete\naggregate"       37
## 13 "freeze\nthaw\ncycles"                34
## 14 "asphalt\nconcrete\nmixtures"         33
## 15 "asphalt\nmixture\nbased"             33
## 16 "life\ncycle\nassessment"             33
## 17 "modified\nasphalt\nmixture"          32
## 18 "polymer\nmodified\nasphalt"          32
## 19 "finite\nelement\nmethod"             30
## 20 "mechanistic\nempirical\npavement"    30

Top 20 ranked trigrams:

Trigram (“three words”) that ranked 21-50:

You may be interested in the correlation among the words. So, a word network may help. I generated this word network using trigrams occurred more than 10 times among all the papers.

Productive teams

Good job top-1 and -2 teams. Sigh, team matters. Note that Chinese researchers looks strong in these two areas, too!

Productive individuals

Area of the each rectangle is to the scale of the number of papers published. The numbers only reflect the authors’ publications on the selected key words. So, some scholars may have significantly more work.

There are some productive young scholars but more are established figures.

Performance of well-known “big figures”?

Let’s generate a graph for the famous scholars in the field. When will you be on this list? See how many papers take to be on this list. At least 10 decent journal papers per year, that takes a big team!

Tools used

What I used for data processing and visualization?