April 5, 2016

structural topic modeling

Structural topic modeling

How to use the stm package (from the stm vignette): stm workflow

The topic modeling part

Find topics in your data!

install.packages("stm", "SnowballC") # probably new
install.packages("dplyr", "tidyr") # if you don't have them already

Getting my data ready. Note that your data should be a data frame where each row has one document. You should have a column called “documents” that has all of the text. Any other variables can be added as additional columns.

df <- read.table("/Users/TARDIS/Documents/STUDIES/context_word_seg/utt_orth_phon_KEY.txt", header=1, sep="\t", stringsAsFactors=F, quote="", comment.char ="") # this gets used for word-lists contexts, it will get over-written for other contexts
library(dplyr); library(tidyr)
data <- df %>%
  select(-phon) %>%
  extract(col=utt, into=c("child", "age_weeks"), regex="^([[:alpha:]]{2})([[:digit:]]{2})")
data$temp <- gl(n=ceiling(nrow(data)/30), k=30)[1:nrow(data)]
data <- group_by(data, child, age_weeks, temp) %>%
  summarize(documents=paste(orth, collapse=" ")) %>%
  select(-temp)

Now bring it to stm for processing there.

library(stm)

## stm v1.1.3 (2016-01-14) successfully loaded. See ?stm for help.

processed <- textProcessor(data$documents, metadata = data)

## Building corpus... 
## Converting to Lower Case... 
## Removing stopwords... 
## Removing numbers... 
## Removing punctuation... 
## Stemming... 
## Creating Output...

out <- prepDocuments(processed$documents, processed$vocab, processed$meta) # removes infrequent terms depending on user-set parameter lower.thresh (the minimum number of documents a word needs to appear in in order for the word to be kept within the vocabulary)

## Removing 504 of 1222 terms (504 of 9742 tokens) due to frequency 
## Your corpus now has 460 documents, 718 terms and 9238 tokens.

Take a look at those messages. I left everything at default here, but you may or may not want to, depending on your research question.

Now let’s get some topics!! (This might take a looooong time to run.)

fit0 <- stm(out$documents, # the documents
            out$vocab, # the words
            K = 10, # 10 topics
            max.em.its = 75, # set to run for a maximum of 75 EM iterations
            data = out$meta, # all the variables (we're not actually including any predictors in this model, though)
            init.type = "Spectral")

## Beginning Initialization.
##   Calculating the gram matrix...
##   Finding anchor words...
##      ..........
##   Recovering initialization...
##      .......
## Initialization complete.
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 1 (approx. per word bound = -5.240) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 2 (approx. per word bound = -4.973, relative change = 5.097e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 3 (approx. per word bound = -4.893, relative change = 1.603e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 4 (approx. per word bound = -4.865, relative change = 5.813e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 5 (approx. per word bound = -4.851, relative change = 2.764e-03) 
## Topic 1: come, darl, yes, ssh, bath 
##  Topic 2: look, want, hey, like, can 
##  Topic 3: dear, hmm, come, want, smile 
##  Topic 4: come, now, tell, got, yes 
##  Topic 5: hello, hmm, boo, gillian, hey 
##  Topic 6: yes, come, hey, can, tell 
##  Topic 7: yes, got, tell, come, can 
##  Topic 8: hey, mummi, yes, dear, hannah 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, hey, come, got 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 6 (approx. per word bound = -4.844, relative change = 1.453e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 7 (approx. per word bound = -4.840, relative change = 8.955e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 8 (approx. per word bound = -4.837, relative change = 5.594e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 9 (approx. per word bound = -4.835, relative change = 4.705e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 10 (approx. per word bound = -4.833, relative change = 4.139e-04) 
## Topic 1: come, darl, yes, bath, ssh 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, good, want, smile 
##  Topic 4: come, now, got, alright, know 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, come, well, tell, can 
##  Topic 7: yes, got, tell, come, want 
##  Topic 8: hey, mummi, yes, smile, hannah 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, clever, hey, come 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 11 (approx. per word bound = -4.832, relative change = 3.135e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 12 (approx. per word bound = -4.830, relative change = 2.621e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 13 (approx. per word bound = -4.829, relative change = 2.242e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 14 (approx. per word bound = -4.828, relative change = 2.136e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 15 (approx. per word bound = -4.827, relative change = 2.183e-04) 
## Topic 1: darl, come, yes, bath, alright 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, good, got, smile 
##  Topic 4: come, now, alright, know, got 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, well, come, tell, can 
##  Topic 7: yes, got, tell, come, want 
##  Topic 8: hey, mummi, smile, yes, hannah 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, clever, hey, boy 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 16 (approx. per word bound = -4.826, relative change = 2.327e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 17 (approx. per word bound = -4.825, relative change = 2.282e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 18 (approx. per word bound = -4.824, relative change = 1.872e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 19 (approx. per word bound = -4.823, relative change = 1.281e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 20 (approx. per word bound = -4.823, relative change = 9.551e-05) 
## Topic 1: darl, come, yes, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, good, got, girl 
##  Topic 4: come, now, know, alright, got 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, well, come, tell, can 
##  Topic 7: yes, got, tell, come, two 
##  Topic 8: hey, mummi, smile, hannah, yes 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, clever, hey, come 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 21 (approx. per word bound = -4.823, relative change = 8.411e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 22 (approx. per word bound = -4.822, relative change = 9.121e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 23 (approx. per word bound = -4.822, relative change = 9.178e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 24 (approx. per word bound = -4.821, relative change = 8.362e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 25 (approx. per word bound = -4.821, relative change = 6.609e-05) 
## Topic 1: darl, come, yes, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, good, got, girl 
##  Topic 4: come, now, know, alright, get 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, come, well, tell, can 
##  Topic 7: yes, got, tell, two, come 
##  Topic 8: hey, mummi, smile, hannah, yes 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, clever, hey, well 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 26 (approx. per word bound = -4.821, relative change = 6.486e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 27 (approx. per word bound = -4.820, relative change = 7.197e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 28 (approx. per word bound = -4.820, relative change = 8.080e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 29 (approx. per word bound = -4.819, relative change = 8.420e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 30 (approx. per word bound = -4.819, relative change = 8.447e-05) 
## Topic 1: darl, come, yes, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, good, got, girl 
##  Topic 4: come, now, know, alright, get 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, come, tell, well, can 
##  Topic 7: yes, tell, got, two, come 
##  Topic 8: hey, smile, mummi, hannah, look 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, clever, well, hey 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 31 (approx. per word bound = -4.819, relative change = 6.516e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 32 (approx. per word bound = -4.818, relative change = 6.256e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 33 (approx. per word bound = -4.818, relative change = 7.905e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 34 (approx. per word bound = -4.818, relative change = 6.690e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 35 (approx. per word bound = -4.818, relative change = 4.168e-05) 
## Topic 1: darl, come, yes, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, got, good, girl 
##  Topic 4: come, now, know, alright, want 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, come, tell, well, can 
##  Topic 7: yes, tell, got, two, one 
##  Topic 8: hey, smile, mummi, hannah, look 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, well, clever, hey 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 36 (approx. per word bound = -4.817, relative change = 3.416e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 37 (approx. per word bound = -4.817, relative change = 3.527e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 38 (approx. per word bound = -4.817, relative change = 3.889e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 39 (approx. per word bound = -4.817, relative change = 4.008e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 40 (approx. per word bound = -4.817, relative change = 3.828e-05) 
## Topic 1: darl, yes, come, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, got, good, girl 
##  Topic 4: come, now, know, alright, want 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, tell, come, can, well 
##  Topic 7: yes, got, tell, two, one 
##  Topic 8: hey, smile, mummi, look, hannah 
##  Topic 9: mummi, yes, girl, hello, hannah 
##  Topic 10: tickl, yes, well, clever, christoph 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 41 (approx. per word bound = -4.817, relative change = 3.309e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 42 (approx. per word bound = -4.816, relative change = 3.105e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 43 (approx. per word bound = -4.816, relative change = 2.992e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 44 (approx. per word bound = -4.816, relative change = 2.875e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 45 (approx. per word bound = -4.816, relative change = 2.589e-05) 
## Topic 1: darl, yes, come, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, got, good, girl 
##  Topic 4: come, now, know, alright, want 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, tell, come, can, stori 
##  Topic 7: yes, got, tell, two, one 
##  Topic 8: hey, smile, mummi, look, hannah 
##  Topic 9: mummi, yes, girl, hello, alright 
##  Topic 10: tickl, well, yes, clever, christoph 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 46 (approx. per word bound = -4.816, relative change = 1.884e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 47 (approx. per word bound = -4.816, relative change = 1.983e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 48 (approx. per word bound = -4.816, relative change = 1.887e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 49 (approx. per word bound = -4.816, relative change = 1.402e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 50 (approx. per word bound = -4.816, relative change = 1.453e-05) 
## Topic 1: darl, yes, come, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, got, good, girl 
##  Topic 4: come, now, know, alright, want 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, tell, come, can, stori 
##  Topic 7: yes, got, tell, two, one 
##  Topic 8: hey, smile, mummi, look, hannah 
##  Topic 9: mummi, yes, girl, hello, alright 
##  Topic 10: tickl, well, yes, clever, christoph 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 51 (approx. per word bound = -4.815, relative change = 1.471e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 52 (approx. per word bound = -4.815, relative change = 1.221e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 53 (approx. per word bound = -4.815, relative change = 1.442e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 54 (approx. per word bound = -4.815, relative change = 1.921e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 55 (approx. per word bound = -4.815, relative change = 1.180e-05) 
## Topic 1: darl, yes, come, bath, clean 
##  Topic 2: look, want, can, like, nice 
##  Topic 3: dear, come, got, good, girl 
##  Topic 4: come, now, alright, know, want 
##  Topic 5: hello, hmm, boo, gillian, matter 
##  Topic 6: yes, tell, come, can, stori 
##  Topic 7: yes, got, two, tell, can 
##  Topic 8: hey, smile, mummi, look, hannah 
##  Topic 9: mummi, yes, girl, hello, alright 
##  Topic 10: tickl, well, yes, clever, christoph 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Model Converged

Note: “The default is init.type =”LDA" but in practice researchers on personal computers with vocabularies less that 10,000 can utilize the spectral initialization successfully." And spectral initialization is better, so you should do that. If you have a very large dataset, read up on how to correctly use the LDA option in the stm vignette.

Yay! Let’s look:

labelTopics(fit0)

## Topic 1 Top Words:
##       Highest Prob: darl, yes, come, bath, clean, nice, put 
##       FREX: clean, bath, wee, pet, darl, dirti, dri 
##       Lift: rid, took, vesti, armi, bobo, petal, soapi 
##       Score: bobo, juli, darl, clean, pet, bath, wee 
## Topic 2 Top Words:
##       Highest Prob: look, want, can, like, nice, littl, make 
##       FREX: lewi, make, bit, foot, rattl, play, push 
##       Lift: bite, cupboard, cupsi, lew, lewi, newspap, pour 
##       Score: juli, newspap, lewi, foot, charl, boy, joseph 
## Topic 3 Top Words:
##       Highest Prob: dear, come, got, good, girl, windi, littl 
##       FREX: dear, windi, sleepi, bless, whoop, head, downstair 
##       Lift: unhappi, milki, gracious, hurri, downstair, sock, dear 
##       Score: juli, unhappi, dear, windi, sleepi, head, fatti 
## Topic 4 Top Words:
##       Highest Prob: come, now, alright, know, want, get, got 
##       FREX: keith, shut, readi, bib, attent, bottl, first 
##       Lift: cake, dessert, dream, five, self, allow, attent 
##       Score: self, keith, crocodil, alright, attent, first, juic 
## Topic 5 Top Words:
##       Highest Prob: hello, hmm, boo, gillian, matter, want, cheeki 
##       FREX: boo, gillian, cheeki, hmm, hello, ticki, monkey 
##       Lift: pinch, poke, cheeki, gillian, boo, mayb, moan 
##       Score: cheeki, juli, hello, boo, gillian, hmm, ticki 
## Topic 6 Top Words:
##       Highest Prob: yes, tell, come, can, stori, mum, good 
##       FREX: quack, stori, thumb, tell, finger, suck, scalp 
##       Lift: bodi, cmon, concentr, pretend, scalp, silent, sober 
##       Score: concentr, juli, quack, tell, scalp, stori, clap 
## Topic 7 Top Words:
##       Highest Prob: yes, got, two, tell, can, one, bad 
##       FREX: cold, two, pretti, nose, walk, happi, bad 
##       Lift: color, nuddi, cardigan, cold, mother, punch, tale 
##       Score: cold, juli, pram, yes, two, nose, pretti 
## Topic 8 Top Words:
##       Highest Prob: hey, smile, mummi, look, hannah, hold, yes 
##       FREX: shh, hey, smile, lambchop, chou, hold, hannah 
##       Lift: contempl, fine, juli, wiggler, shh, botti, friend 
##       Score: juli, wiggler, hey, hannah, lambchop, shh, smile 
## Topic 9 Top Words:
##       Highest Prob: mummi, yes, girl, hello, alright, hannah, big 
##       FREX: girli, mummi, sweetheart, alright, girl, iron, stretch 
##       Lift: globe, pick, prove, rubber, scream, splish, wobbl 
##       Score: juli, splish, hannah, hello, mummi, girl, alright 
## Topic 10 Top Words:
##       Highest Prob: tickl, well, yes, clever, christoph, kick, boy 
##       FREX: tickl, christoph, well, bash, clever, parrot, boy 
##       Lift: ahhphroooowp, tickl, bash, plenti, kitten, music, chris 
##       Score: ahhphroooowp, juli, tickl, well, christoph, clever, boy

For more information on FREX and high probability rankings, see Roberts et al. (2013, 2015, 2014); Lucas et al. (2015). For more information on score, see the lda R package. For more information on lift, see Taddy (2013).

plot.STM(fit0, type = "labels")

plot.STM(fit0, type = "summary")

# topicCorr(fit0) # lots of output :)
round(topicCorr(fit0)$cor, 2) # just the correlations between topics

##        [,1]  [,2]  [,3]  [,4]  [,5]  [,6]  [,7]  [,8]  [,9] [,10]
##  [1,]  1.00 -0.24  0.21 -0.08 -0.21 -0.11  0.00 -0.12  0.02 -0.15
##  [2,] -0.24  1.00 -0.29 -0.03 -0.08 -0.15 -0.10 -0.10 -0.22 -0.06
##  [3,]  0.21 -0.29  1.00 -0.11  0.00 -0.08  0.08 -0.12 -0.18 -0.11
##  [4,] -0.08 -0.03 -0.11  1.00 -0.19 -0.04 -0.10 -0.15 -0.15 -0.09
##  [5,] -0.21 -0.08  0.00 -0.19  1.00 -0.18 -0.14 -0.05 -0.26 -0.05
##  [6,] -0.11 -0.15 -0.08 -0.04 -0.18  1.00 -0.12 -0.10 -0.21  0.02
##  [7,]  0.00 -0.10  0.08 -0.10 -0.14 -0.12  1.00 -0.14 -0.17 -0.05
##  [8,] -0.12 -0.10 -0.12 -0.15 -0.05 -0.10 -0.14  1.00 -0.02 -0.13
##  [9,]  0.02 -0.22 -0.18 -0.15 -0.26 -0.21 -0.17 -0.02  1.00 -0.21
## [10,] -0.15 -0.06 -0.11 -0.09 -0.05  0.02 -0.05 -0.13 -0.21  1.00

How many topics?

The right answer depends on your corpus, the size of your documents, and your research question. Even when you have a clear research question, it still may be tricky to decide how many topics will make the most sense. Especially when you’re starting out and you don’t necessarily know how the chracteristics of your corpus/documents will affect your topic models, you might want to try out several different numbers of topics. You can run several models with different numbers of topics and compare them to help you figure out what makes the most sense using the searchk() function. This code runs a model with 7 topics and another with 10.

ntopics <- searchK(out$documents, out$vocab, K = c(7, 10), data = meta)

## Beginning Initialization.
##   Calculating the gram matrix...
##   Finding anchor words...
##      .......
##   Recovering initialization...
##      .......
## Initialization complete.
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 1 (approx. per word bound = -5.320) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 2 (approx. per word bound = -5.187, relative change = 2.500e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 3 (approx. per word bound = -5.110, relative change = 1.477e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 4 (approx. per word bound = -5.068, relative change = 8.138e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 5 (approx. per word bound = -5.041, relative change = 5.343e-03) 
## Topic 1: clever, look, like, boy, want 
##  Topic 2: come, know, matter, hmm, littl 
##  Topic 3: dear, well, want, like, just 
##  Topic 4: hello, smile, boo, hmm, good 
##  Topic 5: yes, tickl, can, hey, dear 
##  Topic 6: come, dear, got, good, want 
##  Topic 7: hey, mummi, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 6 (approx. per word bound = -5.021, relative change = 4.082e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 7 (approx. per word bound = -5.005, relative change = 3.107e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 8 (approx. per word bound = -4.994, relative change = 2.305e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 9 (approx. per word bound = -4.984, relative change = 1.851e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 10 (approx. per word bound = -4.977, relative change = 1.517e-03) 
## Topic 1: look, want, like, clever, nice 
##  Topic 2: come, know, matter, bath, now 
##  Topic 3: dear, want, well, like, just 
##  Topic 4: hello, hmm, smile, boo, look 
##  Topic 5: yes, tickl, hey, can, tell 
##  Topic 6: dear, come, got, good, tell 
##  Topic 7: hey, mummi, girl, hannah, alright 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 11 (approx. per word bound = -4.971, relative change = 1.184e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 12 (approx. per word bound = -4.966, relative change = 9.019e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 13 (approx. per word bound = -4.963, relative change = 7.212e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 14 (approx. per word bound = -4.960, relative change = 6.433e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 15 (approx. per word bound = -4.957, relative change = 6.157e-04) 
## Topic 1: look, want, like, can, clever 
##  Topic 2: come, know, now, bath, matter 
##  Topic 3: want, like, get, just, put 
##  Topic 4: hello, hmm, smile, boo, look 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, good, tell 
##  Topic 7: hey, mummi, girl, hannah, alright 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 16 (approx. per word bound = -4.954, relative change = 5.870e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 17 (approx. per word bound = -4.951, relative change = 5.736e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 18 (approx. per word bound = -4.948, relative change = 5.389e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 19 (approx. per word bound = -4.946, relative change = 4.811e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 20 (approx. per word bound = -4.944, relative change = 4.405e-04) 
## Topic 1: look, well, like, can, want 
##  Topic 2: come, know, now, bath, hey 
##  Topic 3: get, want, like, put, just 
##  Topic 4: hello, hmm, smile, boo, look 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: hey, mummi, girl, hannah, yes 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 21 (approx. per word bound = -4.942, relative change = 4.003e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 22 (approx. per word bound = -4.940, relative change = 3.532e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 23 (approx. per word bound = -4.938, relative change = 3.193e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 24 (approx. per word bound = -4.937, relative change = 3.207e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 25 (approx. per word bound = -4.935, relative change = 3.742e-04) 
## Topic 1: look, well, can, like, nice 
##  Topic 2: come, know, now, hey, will 
##  Topic 3: get, want, like, put, babi 
##  Topic 4: hello, hmm, boo, smile, hey 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: hey, mummi, girl, hannah, yes 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 26 (approx. per word bound = -4.933, relative change = 3.510e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 27 (approx. per word bound = -4.932, relative change = 2.393e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 28 (approx. per word bound = -4.931, relative change = 1.867e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 29 (approx. per word bound = -4.930, relative change = 1.659e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 30 (approx. per word bound = -4.930, relative change = 1.534e-04) 
## Topic 1: well, look, can, like, nice 
##  Topic 2: come, know, now, hey, will 
##  Topic 3: get, want, like, come, put 
##  Topic 4: hello, hmm, boo, smile, hey 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: hey, mummi, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 31 (approx. per word bound = -4.929, relative change = 1.376e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 32 (approx. per word bound = -4.928, relative change = 1.238e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 33 (approx. per word bound = -4.928, relative change = 1.124e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 34 (approx. per word bound = -4.927, relative change = 1.029e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 35 (approx. per word bound = -4.927, relative change = 9.005e-05) 
## Topic 1: well, look, can, like, nice 
##  Topic 2: come, know, now, hey, will 
##  Topic 3: get, come, want, like, put 
##  Topic 4: hello, hmm, boo, smile, hey 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: hey, mummi, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 36 (approx. per word bound = -4.926, relative change = 7.793e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 37 (approx. per word bound = -4.926, relative change = 7.025e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 38 (approx. per word bound = -4.926, relative change = 6.147e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 39 (approx. per word bound = -4.925, relative change = 5.097e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 40 (approx. per word bound = -4.925, relative change = 4.265e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, now, know, hey, say 
##  Topic 3: get, come, want, like, put 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: hey, mummi, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 41 (approx. per word bound = -4.925, relative change = 4.111e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 42 (approx. per word bound = -4.925, relative change = 4.715e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 43 (approx. per word bound = -4.924, relative change = 5.951e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 44 (approx. per word bound = -4.924, relative change = 5.967e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 45 (approx. per word bound = -4.924, relative change = 5.443e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, now, hey, know, say 
##  Topic 3: come, get, want, like, got 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: hey, mummi, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 46 (approx. per word bound = -4.924, relative change = 5.656e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 47 (approx. per word bound = -4.923, relative change = 5.992e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 48 (approx. per word bound = -4.923, relative change = 6.316e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 49 (approx. per word bound = -4.923, relative change = 6.208e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 50 (approx. per word bound = -4.922, relative change = 5.230e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, want, like, got 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 51 (approx. per word bound = -4.922, relative change = 4.456e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 52 (approx. per word bound = -4.922, relative change = 3.996e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 53 (approx. per word bound = -4.922, relative change = 3.662e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 54 (approx. per word bound = -4.922, relative change = 3.427e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 55 (approx. per word bound = -4.922, relative change = 3.327e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, want, like, got 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, one, tell 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 56 (approx. per word bound = -4.921, relative change = 3.032e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 57 (approx. per word bound = -4.921, relative change = 2.993e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 58 (approx. per word bound = -4.921, relative change = 3.491e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 59 (approx. per word bound = -4.921, relative change = 3.423e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 60 (approx. per word bound = -4.921, relative change = 3.104e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, like, got, want 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, tell, one 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 61 (approx. per word bound = -4.921, relative change = 2.903e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 62 (approx. per word bound = -4.921, relative change = 2.340e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 63 (approx. per word bound = -4.920, relative change = 2.198e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 64 (approx. per word bound = -4.920, relative change = 2.342e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 65 (approx. per word bound = -4.920, relative change = 2.282e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, like, got, want 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, tell, one 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 66 (approx. per word bound = -4.920, relative change = 2.428e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 67 (approx. per word bound = -4.920, relative change = 2.718e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 68 (approx. per word bound = -4.920, relative change = 2.945e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 69 (approx. per word bound = -4.920, relative change = 2.687e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 70 (approx. per word bound = -4.920, relative change = 2.376e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, got, like, want 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, tell, one 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 71 (approx. per word bound = -4.919, relative change = 2.175e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 72 (approx. per word bound = -4.919, relative change = 2.256e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 73 (approx. per word bound = -4.919, relative change = 2.462e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 74 (approx. per word bound = -4.919, relative change = 2.732e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 75 (approx. per word bound = -4.919, relative change = 2.787e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, got, like, want 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, tell, one 
##  Topic 6: dear, come, got, tell, good 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 76 (approx. per word bound = -4.919, relative change = 2.709e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 77 (approx. per word bound = -4.919, relative change = 2.806e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 78 (approx. per word bound = -4.918, relative change = 3.203e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 79 (approx. per word bound = -4.918, relative change = 4.204e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 80 (approx. per word bound = -4.918, relative change = 5.751e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, got, like, want 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, hey, tell, one 
##  Topic 6: dear, come, got, good, tell 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 81 (approx. per word bound = -4.918, relative change = 6.615e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 82 (approx. per word bound = -4.917, relative change = 4.844e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 83 (approx. per word bound = -4.917, relative change = 2.848e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 84 (approx. per word bound = -4.917, relative change = 1.913e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 85 (approx. per word bound = -4.917, relative change = 1.439e-05) 
## Topic 1: well, can, look, like, nice 
##  Topic 2: come, hey, now, know, say 
##  Topic 3: come, get, got, like, want 
##  Topic 4: hello, hmm, boo, hey, smile 
##  Topic 5: yes, tickl, tell, hey, one 
##  Topic 6: dear, come, got, good, tell 
##  Topic 7: mummi, hey, yes, girl, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 86 (approx. per word bound = -4.917, relative change = 1.444e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 87 (approx. per word bound = -4.917, relative change = 1.642e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 88 (approx. per word bound = -4.917, relative change = 1.134e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Model Converged 
## Beginning Initialization.
##   Calculating the gram matrix...
##   Finding anchor words...
##      ..........
##   Recovering initialization...
##      .......
## Initialization complete.
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 1 (approx. per word bound = -5.279) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 2 (approx. per word bound = -5.038, relative change = 4.560e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 3 (approx. per word bound = -4.941, relative change = 1.922e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 4 (approx. per word bound = -4.904, relative change = 7.610e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 5 (approx. per word bound = -4.882, relative change = 4.401e-03) 
## Topic 1: yes, can, well, boy, clever 
##  Topic 2: yes, come, tell, got, good 
##  Topic 3: boo, say, hey, kick, want 
##  Topic 4: look, hey, see, littl, can 
##  Topic 5: hey, yes, mummi, hannah, look 
##  Topic 6: yes, tickl, come, dear, tell 
##  Topic 7: hello, hmm, hey, gillian, got 
##  Topic 8: dear, hey, alright, come, mummi 
##  Topic 9: come, mummi, big, yes, now 
##  Topic 10: mummi, yes, hannah, girl, nice 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 6 (approx. per word bound = -4.869, relative change = 2.573e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 7 (approx. per word bound = -4.861, relative change = 1.692e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 8 (approx. per word bound = -4.856, relative change = 1.166e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 9 (approx. per word bound = -4.851, relative change = 1.002e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 10 (approx. per word bound = -4.846, relative change = 8.826e-04) 
## Topic 1: yes, can, boy, clever, nice 
##  Topic 2: yes, come, tell, got, good 
##  Topic 3: boo, well, say, hey, kick 
##  Topic 4: look, see, littl, want, like 
##  Topic 5: hey, yes, mummi, hannah, hold 
##  Topic 6: yes, tickl, come, dear, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, hey, alright, come, darl 
##  Topic 9: come, big, mummi, now, smile 
##  Topic 10: mummi, yes, girl, hannah, nice 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 11 (approx. per word bound = -4.843, relative change = 7.865e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 12 (approx. per word bound = -4.839, relative change = 6.566e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 13 (approx. per word bound = -4.837, relative change = 5.541e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 14 (approx. per word bound = -4.835, relative change = 3.795e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 15 (approx. per word bound = -4.833, relative change = 3.252e-04) 
## Topic 1: can, yes, boy, clever, nice 
##  Topic 2: yes, come, tell, got, good 
##  Topic 3: well, boo, say, come, yes 
##  Topic 4: look, want, see, littl, like 
##  Topic 5: hey, yes, mummi, hannah, smile 
##  Topic 6: yes, tickl, come, dear, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, hey, alright, come, darl 
##  Topic 9: come, big, now, smile, mummi 
##  Topic 10: mummi, yes, girl, hannah, good 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 16 (approx. per word bound = -4.832, relative change = 3.171e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 17 (approx. per word bound = -4.831, relative change = 1.815e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 18 (approx. per word bound = -4.830, relative change = 9.873e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 19 (approx. per word bound = -4.830, relative change = 7.631e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 20 (approx. per word bound = -4.830, relative change = 8.756e-05) 
## Topic 1: can, yes, boy, clever, nice 
##  Topic 2: yes, come, tell, got, good 
##  Topic 3: well, boo, say, come, yes 
##  Topic 4: want, look, see, littl, nice 
##  Topic 5: hey, yes, mummi, hannah, smile 
##  Topic 6: yes, tickl, come, dear, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, alright, hey, darl 
##  Topic 9: come, big, now, smile, fat 
##  Topic 10: mummi, girl, yes, hannah, good 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 21 (approx. per word bound = -4.829, relative change = 1.390e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 22 (approx. per word bound = -4.828, relative change = 1.597e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 23 (approx. per word bound = -4.828, relative change = 1.278e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 24 (approx. per word bound = -4.827, relative change = 1.158e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 25 (approx. per word bound = -4.826, relative change = 1.131e-04) 
## Topic 1: can, yes, boy, clever, like 
##  Topic 2: yes, come, tell, good, got 
##  Topic 3: well, boo, say, come, yes 
##  Topic 4: want, look, see, littl, nice 
##  Topic 5: hey, yes, mummi, hannah, smile 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, alright, darl, ssh 
##  Topic 9: come, big, now, smile, fat 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 26 (approx. per word bound = -4.826, relative change = 8.500e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 27 (approx. per word bound = -4.826, relative change = 8.340e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 28 (approx. per word bound = -4.825, relative change = 8.284e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 29 (approx. per word bound = -4.825, relative change = 1.009e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 30 (approx. per word bound = -4.824, relative change = 1.199e-04) 
## Topic 1: can, yes, boy, clever, like 
##  Topic 2: yes, come, tell, good, got 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, alright, darl, ssh 
##  Topic 9: come, big, now, smile, get 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 31 (approx. per word bound = -4.824, relative change = 1.118e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 32 (approx. per word bound = -4.823, relative change = 1.126e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 33 (approx. per word bound = -4.823, relative change = 1.102e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 34 (approx. per word bound = -4.822, relative change = 1.108e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 35 (approx. per word bound = -4.822, relative change = 9.561e-05) 
## Topic 1: can, yes, clever, boy, like 
##  Topic 2: yes, come, tell, good, got 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, alright, got 
##  Topic 9: come, big, now, smile, get 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 36 (approx. per word bound = -4.821, relative change = 7.619e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 37 (approx. per word bound = -4.821, relative change = 8.315e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 38 (approx. per word bound = -4.820, relative change = 8.645e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 39 (approx. per word bound = -4.820, relative change = 6.973e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 40 (approx. per word bound = -4.820, relative change = 6.576e-05) 
## Topic 1: can, yes, clever, boy, like 
##  Topic 2: yes, come, tell, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, alright, got 
##  Topic 9: come, big, now, smile, get 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 41 (approx. per word bound = -4.819, relative change = 8.137e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 42 (approx. per word bound = -4.819, relative change = 9.252e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 43 (approx. per word bound = -4.818, relative change = 9.444e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 44 (approx. per word bound = -4.818, relative change = 8.206e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 45 (approx. per word bound = -4.818, relative change = 6.200e-05) 
## Topic 1: can, yes, clever, like, boy 
##  Topic 2: yes, come, tell, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, smile, mummi 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, alright, got 
##  Topic 9: come, big, now, smile, get 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 46 (approx. per word bound = -4.818, relative change = 5.607e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 47 (approx. per word bound = -4.817, relative change = 5.721e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 48 (approx. per word bound = -4.817, relative change = 5.900e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 49 (approx. per word bound = -4.817, relative change = 6.012e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 50 (approx. per word bound = -4.816, relative change = 6.268e-05) 
## Topic 1: can, yes, like, clever, tri 
##  Topic 2: yes, come, tell, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, smile, mummi 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 51 (approx. per word bound = -4.816, relative change = 5.711e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 52 (approx. per word bound = -4.816, relative change = 4.735e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 53 (approx. per word bound = -4.816, relative change = 4.113e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 54 (approx. per word bound = -4.815, relative change = 4.369e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 55 (approx. per word bound = -4.815, relative change = 5.259e-05) 
## Topic 1: can, yes, like, clever, tri 
##  Topic 2: yes, tell, come, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, smile, mummi 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 56 (approx. per word bound = -4.815, relative change = 4.744e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 57 (approx. per word bound = -4.815, relative change = 4.242e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 58 (approx. per word bound = -4.815, relative change = 5.003e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 59 (approx. per word bound = -4.814, relative change = 6.421e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 60 (approx. per word bound = -4.814, relative change = 6.686e-05) 
## Topic 1: can, yes, like, clever, tri 
##  Topic 2: yes, tell, come, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, smile, mummi 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 61 (approx. per word bound = -4.814, relative change = 6.098e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 62 (approx. per word bound = -4.813, relative change = 6.503e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 63 (approx. per word bound = -4.813, relative change = 7.060e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 64 (approx. per word bound = -4.813, relative change = 5.916e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 65 (approx. per word bound = -4.812, relative change = 4.233e-05) 
## Topic 1: can, yes, like, tri, clever 
##  Topic 2: yes, tell, come, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, smile, mummi 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 66 (approx. per word bound = -4.812, relative change = 3.007e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 67 (approx. per word bound = -4.812, relative change = 2.289e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 68 (approx. per word bound = -4.812, relative change = 2.108e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 69 (approx. per word bound = -4.812, relative change = 3.370e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 70 (approx. per word bound = -4.812, relative change = 5.149e-05) 
## Topic 1: can, yes, like, nice, tri 
##  Topic 2: yes, tell, come, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, smile, mummi 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 71 (approx. per word bound = -4.811, relative change = 4.136e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 72 (approx. per word bound = -4.811, relative change = 3.405e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 73 (approx. per word bound = -4.811, relative change = 3.108e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 74 (approx. per word bound = -4.811, relative change = 2.621e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 75 (approx. per word bound = -4.811, relative change = 3.038e-05) 
## Topic 1: can, yes, like, nice, tri 
##  Topic 2: yes, tell, come, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, mummi, smile 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 76 (approx. per word bound = -4.811, relative change = 3.308e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 77 (approx. per word bound = -4.811, relative change = 3.611e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 78 (approx. per word bound = -4.810, relative change = 3.891e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 79 (approx. per word bound = -4.810, relative change = 3.397e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 80 (approx. per word bound = -4.810, relative change = 2.829e-05) 
## Topic 1: can, yes, like, nice, tri 
##  Topic 2: yes, tell, come, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, mummi, smile 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 81 (approx. per word bound = -4.810, relative change = 3.003e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 82 (approx. per word bound = -4.810, relative change = 4.447e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 83 (approx. per word bound = -4.810, relative change = 4.149e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 84 (approx. per word bound = -4.809, relative change = 3.027e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 85 (approx. per word bound = -4.809, relative change = 2.585e-05) 
## Topic 1: can, yes, like, nice, tri 
##  Topic 2: yes, tell, come, good, mum 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, look, mummi, smile 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 86 (approx. per word bound = -4.809, relative change = 2.408e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 87 (approx. per word bound = -4.809, relative change = 2.769e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 88 (approx. per word bound = -4.809, relative change = 3.707e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 89 (approx. per word bound = -4.809, relative change = 4.295e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 90 (approx. per word bound = -4.808, relative change = 3.493e-05) 
## Topic 1: can, yes, like, nice, tri 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hannah, hello 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 91 (approx. per word bound = -4.808, relative change = 3.001e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 92 (approx. per word bound = -4.808, relative change = 2.956e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 93 (approx. per word bound = -4.808, relative change = 3.409e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 94 (approx. per word bound = -4.808, relative change = 3.833e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 95 (approx. per word bound = -4.808, relative change = 3.496e-05) 
## Topic 1: can, yes, like, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, tell, got 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 96 (approx. per word bound = -4.808, relative change = 3.052e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 97 (approx. per word bound = -4.807, relative change = 3.032e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 98 (approx. per word bound = -4.807, relative change = 3.527e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 99 (approx. per word bound = -4.807, relative change = 2.665e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 100 (approx. per word bound = -4.807, relative change = 1.909e-05) 
## Topic 1: can, like, yes, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 101 (approx. per word bound = -4.807, relative change = 1.807e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 102 (approx. per word bound = -4.807, relative change = 1.882e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 103 (approx. per word bound = -4.807, relative change = 2.111e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 104 (approx. per word bound = -4.807, relative change = 2.265e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 105 (approx. per word bound = -4.806, relative change = 2.207e-05) 
## Topic 1: can, like, yes, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, make 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 106 (approx. per word bound = -4.806, relative change = 2.141e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 107 (approx. per word bound = -4.806, relative change = 1.712e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 108 (approx. per word bound = -4.806, relative change = 1.351e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 109 (approx. per word bound = -4.806, relative change = 1.212e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 110 (approx. per word bound = -4.806, relative change = 1.050e-05) 
## Topic 1: can, like, yes, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, nice 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 111 (approx. per word bound = -4.806, relative change = 1.141e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 112 (approx. per word bound = -4.806, relative change = 1.101e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 113 (approx. per word bound = -4.806, relative change = 1.211e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 114 (approx. per word bound = -4.806, relative change = 1.544e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 115 (approx. per word bound = -4.806, relative change = 1.829e-05) 
## Topic 1: can, like, yes, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, nice 
##  Topic 5: hey, yes, mummi, look, smile 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, big, get, smile 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 116 (approx. per word bound = -4.806, relative change = 2.202e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 117 (approx. per word bound = -4.806, relative change = 2.304e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 118 (approx. per word bound = -4.805, relative change = 2.126e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 119 (approx. per word bound = -4.805, relative change = 1.923e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 120 (approx. per word bound = -4.805, relative change = 1.987e-05) 
## Topic 1: can, like, yes, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, nice 
##  Topic 5: hey, yes, mummi, smile, look 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, get, big, know 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 121 (approx. per word bound = -4.805, relative change = 3.045e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 122 (approx. per word bound = -4.805, relative change = 2.173e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 123 (approx. per word bound = -4.805, relative change = 1.557e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 124 (approx. per word bound = -4.805, relative change = 1.552e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 125 (approx. per word bound = -4.805, relative change = 1.616e-05) 
## Topic 1: can, like, yes, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, nice 
##  Topic 5: hey, yes, mummi, smile, look 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, get, big, know 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 126 (approx. per word bound = -4.805, relative change = 1.695e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 127 (approx. per word bound = -4.805, relative change = 1.937e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 128 (approx. per word bound = -4.805, relative change = 2.153e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 129 (approx. per word bound = -4.804, relative change = 1.865e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 130 (approx. per word bound = -4.804, relative change = 1.280e-05) 
## Topic 1: can, like, yes, tri, nice 
##  Topic 2: yes, tell, come, good, one 
##  Topic 3: well, boo, say, yes, come 
##  Topic 4: want, look, see, littl, nice 
##  Topic 5: hey, yes, mummi, smile, look 
##  Topic 6: yes, tickl, come, got, tell 
##  Topic 7: hello, hmm, gillian, got, matter 
##  Topic 8: dear, come, darl, got, alright 
##  Topic 9: come, now, get, big, know 
##  Topic 10: mummi, yes, girl, hello, hannah 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Completing Iteration 131 (approx. per word bound = -4.804, relative change = 1.213e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## Completed M-Step. 
## Model Converged

plot(ntopics)

The structural part

The idea is that you can use the idea of topic modeling within a larger model that includes predictors, etc. that you think may change what the topics are like or how often particular topics show up. There are two different flavors of predictors to think about: those that change how much each topic shows up (prevalence), and those that change

For example (from the stm vignette), maybe you have lots of posts from a handful of political blogs, and you have the blogs coded as either liberal or conserative, and you know the date each post was written. You might expect that some topics are more likely to be discussed more often on liberal blogs vs. conservative ones. That’s a difference in prevalence. Or you might think that some topics will be more or less likely as time goes on (e.g. maybe you expect an increase in posts about a certain candidate as he or she wins more primaries). Again, that’s prevalence.

If you put in no predictors at all, the model reduces to a fast implementation of the Correlated Topic Model (Blei and Lafferty 2007). In other words, it’s regular ol’ LDA, but the topics themselves are allowed to covary.

fit1 <- stm(out$documents, # the documents
            out$vocab, # the words
            content =~ child, 
            prevalence =~ age_weeks,
            K = 10, # 10 topics
            max.em.its = 75, # set to run for a maximum of 75 EM iterations
            data = out$meta, # all the variables (we're not actually including any predictors in this model, though)
            init.type = "Spectral")

## Warning in sparse.model.matrix(termobj, data = data): variable 'age_weeks'
## converted to a factor

## Beginning Initialization.
##   Calculating the gram matrix...
##   Finding anchor words...
##      ..........
##   Recovering initialization...
##      .......
## Initialization complete.
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 1 (approx. per word bound = -5.240) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 2 (approx. per word bound = -4.760, relative change = 9.166e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 3 (approx. per word bound = -4.664, relative change = 2.011e-02) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 4 (approx. per word bound = -4.621, relative change = 9.143e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 5 (approx. per word bound = -4.598, relative change = 5.012e-03) 
## Topic 1: clean, head, bath, bobo, fatti 
##  Topic 2: goin, shake, oop, hand, watch 
##  Topic 3: interest, smile, bet, bye, wonder 
##  Topic 4: sorri, outsid, rude, chang, home 
##  Topic 5: yesterday, boo, puppet, cheeki, hot 
##  Topic 6: els, mouth, believ, blow, bounc 
##  Topic 7: walk, cold, bad, three, said 
##  Topic 8: smile, cheek, star, light, hey 
##  Topic 9: swim, teeth, drop, rub, soft 
##  Topic 10: tickl, done, feet, tick, tri 
## Aspect 1: gorgeous, grumpi, shush, hmm, moan 
##  Aspect 2: christoph, hour, microphon, allow, suck 
##  Aspect 3: precious, hide, silent, mop, plenti 
##  Aspect 4: keith, somat, violent, pup, grandma 
##  Aspect 5: mummi, quiet, busi, prove, nosh 
##  Aspect 6: lulu, cradl, appl, mobil, pussycat 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 6 (approx. per word bound = -4.586, relative change = 2.599e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 7 (approx. per word bound = -4.579, relative change = 1.676e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 8 (approx. per word bound = -4.573, relative change = 1.288e-03) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 9 (approx. per word bound = -4.569, relative change = 8.567e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 10 (approx. per word bound = -4.566, relative change = 5.746e-04) 
## Topic 1: bath, clean, head, cold, ssh 
##  Topic 2: shake, goin, watch, oop, hand 
##  Topic 3: wonder, interest, bet, forget, hungri 
##  Topic 4: sorri, babi, mum, chang, outsid 
##  Topic 5: yesterday, cheeki, boo, hot, puppet 
##  Topic 6: blow, els, bounc, can, quack 
##  Topic 7: walk, three, said, bad, pretti 
##  Topic 8: smile, light, star, cheek, hey 
##  Topic 9: swim, rub, drop, rusk, exercis 
##  Topic 10: tickl, done, feet, tick, tri 
## Aspect 1: hmm, grumpi, gorgeous, moan, fright 
##  Aspect 2: christoph, throat, allow, microphon, suck 
##  Aspect 3: precious, plenti, silent, curl, mop 
##  Aspect 4: keith, somat, pup, violent, grandma 
##  Aspect 5: mummi, prove, quiet, busi, nosh 
##  Aspect 6: lulu, chime, pussycat, mobil, cradl 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (5 seconds). 
## Completing Iteration 11 (approx. per word bound = -4.563, relative change = 6.549e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 12 (approx. per word bound = -4.561, relative change = 4.785e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 13 (approx. per word bound = -4.559, relative change = 4.990e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 14 (approx. per word bound = -4.557, relative change = 3.450e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 15 (approx. per word bound = -4.556, relative change = 2.834e-04) 
## Topic 1: bath, ssh, clean, cold, head 
##  Topic 2: shake, goin, watch, hand, quit 
##  Topic 3: wonder, interest, bet, hungri, forget 
##  Topic 4: sorri, mum, chang, babi, outsid 
##  Topic 5: cheeki, yesterday, boo, hot, puppet 
##  Topic 6: blow, els, can, dinner, sore 
##  Topic 7: walk, said, hiccough, stand, bad 
##  Topic 8: smile, light, star, cheek, pat 
##  Topic 9: swim, rub, exercis, drop, rusk 
##  Topic 10: tickl, feet, done, tick, tri 
## Aspect 1: grumpi, pinch, gorgeous, hmm, moan 
##  Aspect 2: christoph, music, throat, allow, microphon 
##  Aspect 3: precious, sock, silent, plenti, curl 
##  Aspect 4: keith, pup, somat, violent, grandma 
##  Aspect 5: mummi, prove, contempl, busi, quiet 
##  Aspect 6: lulu, cradl, chime, mobil, pussycat 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 16 (approx. per word bound = -4.554, relative change = 4.654e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 17 (approx. per word bound = -4.554, relative change = 3.129e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 18 (approx. per word bound = -4.552, relative change = 3.226e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 19 (approx. per word bound = -4.550, relative change = 4.019e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 20 (approx. per word bound = -4.548, relative change = 5.125e-04) 
## Topic 1: bath, ssh, clean, cold, head 
##  Topic 2: shake, goin, watch, bop, quit 
##  Topic 3: wonder, hungri, bet, forget, windi 
##  Topic 4: babi, outsid, mum, sorri, chang 
##  Topic 5: cheeki, boo, yesterday, hot, puppet 
##  Topic 6: blow, els, can, bounc, dribbl 
##  Topic 7: walk, said, stand, hiccough, three 
##  Topic 8: smile, light, window, star, pat 
##  Topic 9: swim, rub, rusk, drop, exercis 
##  Topic 10: tickl, done, feet, tick, tri 
## Aspect 1: grumpi, gorgeous, hmm, pinch, moan 
##  Aspect 2: christoph, music, throat, allow, swallow 
##  Aspect 3: precious, silent, sock, gettin, plenti 
##  Aspect 4: somat, pup, violent, grandma, aint 
##  Aspect 5: mummi, prove, quiet, busi, contempl 
##  Aspect 6: cradl, chime, pussycat, mobil, appl 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 21 (approx. per word bound = -4.547, relative change = 2.434e-04) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Completing Iteration 22 (approx. per word bound = -4.547, relative change = 9.134e-05) 
## ...................................................................................................................
## Completed E-Step (0 seconds). 
## ......................................................................................................
## Completed M-Step (6 seconds). 
## Model Converged

sageLabels(fit1) # only works with content covariates in the model
plot(fit1)