The Language of Brands in Social Media

We base our approach of identifying topics of conversation on the topic modeling approach using LDA, a generative probabilistic model that allows clustering words into groups (namely, "topics") on the basis of other words to which they relate. This results in groups that tend to have similar or related meanings. More specifically, according to LDA, each document contains a mixture of a small set of topics, and each word (observed) within the document is attributable to one of the (unobserved) topics. The topics underlying these words are latent dimensions. With this model, the researcher can infer topics (words along with their probability of being in a given topic) on the basis of statistical distributions of words across documents. The words in a topic are topic definitions - namely, the model posteriors that indicate the probability of a topic given a word p(topic|word).

Beyond topic modeling, our interest is in examining how different topics correlate with different brands, and we use two approaches to construct these relationships. We determine this by assessing the average topic probabilities (averaged at the brand level). We propose various applications stemming from this first approach. The second approach to assessing brand–topic relationships involves a significance testing framework, which identifies the significance associated with positively and negatively correlated topics for each brand. We use the term "differential language analysis" to refer to this second approach. A typical application of differential language analysis is the correlation of language usage in UGC with known characteristics of the people posting, such as age, gender, or personality, with the ultimate goal of predicting these on the basis of language characteristics. For example, leveraging distinctive words, phrases, and topics, computational linguists can predict gender, age, personality, emotion, sentiment, and flirting. Monroe, Colaresi, and Quinn examine how political conflicts can be understood by examining language used by Republicans and Democrats. In the present context, we use differential language analysis to identify the topics that most distinguish brands/brand characteristics; this in turn forms the basis for identifying a set of topics that is uniquely related to each brand.

The framework used here can be described in distinct stages, namely (1) preprocessing and tokenization of the data, (2) generating and extracting topics using LDA, and (3) linking topics and brands using both average topic probabilities and significance tests using logistic regressions. We briefly outline each of these stages next, as it pertains to our empirical data analysis. Figure 1 provides a graphical summary of the key steps in this approach.

Figure 1. Overview of framework for mapping online brand image using differential language analysis of social media conversations.

Preprocessing and Tokenization

Preprocessing is a key step before topic modeling. Typically, this involves extracting words or phrases (one- to three-word sequences more likely to occur together than by chance). In our case, given that we examined Twitter data (restricted to 140 characters per message), we elected to focus primarily on 1-grams (single words) rather than phrases. We used Schwartz et al.'s social media tokenizer, which appropriately handles extra-linguistic social media content such as hashtags, emoticons, and "at-mentions" of other users.

In the second step, we calculate each brand's use of a topic, or the probability of a topic's usage at the brand level. We follow Griffiths and Steyver, who use a generative model and envision each document as produced by choosing a distribution over topics. Alternatively, they view each document as having a latent structure consisting of a set of topics and use LDA to uncover the topics within each document (Equations 5 and 7 of Griffiths and Steyvers highlight the use of LDA to uncover topics within documents).

In our setting, each message (which is an individual message pertaining to a brand) can be viewed as a mixture of topics. We derive the average topic probability for each message pertaining to a brand, and the average of topic probabilities for each message for each brand provides us the average brand–topic probability. To some extent, the conceptual framework for brand–topic relationships is also akin to author–topic models, where brands (instead of authors) are linked to various topics.

We also use a second approach to assess the brand–topic relationships by estimating the significant positive and negative topic correlates of each brand. To do this, we conduct significance testing to identify which topics significantly predict that a message pertains to a given brand through a series of logistic regressions, where the outcome of interest was whether a given message pertained to a particular brand name (vs. not), and used message-topic probability as an independent variable, as summarized next:

(1)

We use LDA outputs from MALLET as well as differential language analysis toolkit to assess differences in linguistic features across groups or factors. A key advantage of the Schwartz et al. toolkit is that it uses the Benjamini and Hochberg correction for false discovery rates by default. Given the large number of regressions needed to estimate each brand–topic relationship, this correction is a critical step in ensuring the validity of significance testing.

The data set contains 134,953 tweets about specific brands (i.e., including a brand's Twitter handle) from a larger corpus of 1.2 million tweets collected over a three-week period beginning mid-February 2015 using Twitter application programming interface. We identified messages that referenced brand names (and explicitly included brand Twitter handles). Our data set of messages referenced 193 brand names in a range of categories. These categories included airlines, banks, beer, health and beauty, beverages, clothing, coffee, computers, consumer/household goods, credit cards, online and brick-and-mortar retailers, entertainment, fast food/restaurants, food, beverage, grocery, hotels, household goods, insurance, shoes, television shows, soft drinks, sports, grocery stores, telecommunications, and travel. An initial frequency analysis of the 1.2 million tweets across brands demonstrated significant differences in the total number of messages. This situation poses a challenge for topic modeling because of the likelihood that the topic model we estimate will be dominated by a few brands with a large volume of tweets. Therefore, we used random samples of 2,000 Tweets per brand, which we then subjected to a series of preprocessing steps described subsequently.

To reduce noise in the data (to ensure that a tweet was actually about a particular brand), we restricted our focus to tweets that listed a defined Twitter handle pertaining to the brand. We also considered an alternative approach that involved searching for tweets containing the brand name itself. However, this process also yielded messages that pertained to commonly used words in the English language for certain brand names (e.g., visa, tide) that were unrelated to the brand name. Identifying the brand-related tweets would involve manually combing through vast numbers of social media messages or spending time separately developing a brand disambiguator. Instead, our approach of using the Twitter handle accurately identified brand-related tweets while minimizing the need for a human coder to comb through the social media posts to remove any unrelated posts. Thus, we believed our approach was justifiable because it is scalable to new or larger data.

Our results are based on brand name handles contained within the tweets, but the topic models excluded the actual brand names and handles themselves, to concentrate on the context in which the brand appeared and not to have the name itself skew the results (i.e., putting all brands on equal footing). After the preprocessing steps, the average sample size per brand was around 700 tweets (SD = 286). We then estimated topic models based on this overall sample (N = 134,953).

Data Processing

Constraints based on number of tweets per user

Twitter bots and trolls can skew results, and identifying which posts originated from legitimate users and which were posted by trolls or bots is difficult. One potential way to pinpoint these bots and trolls is to identify Twitter users who have a high volume of posts within a certain period. As our data set was already constrained to a three-week period, we removed all tweets beyond the tenth tweet in the data set for any given user. Although this removal may not eliminate bots entirely, it significantly minimizes the issue without completely removing legitimate users who post frequently (we compromised between Type 1 and Type 2 errors for identifying bots).

Filtering tweets with special characters or URLs and sponsored tweets

We removed 1-grams, which are characterized by special characters (e.g., %@%), or tweets that included URLs (%http% or %<URL>%). We also removed any sponsored tweets or promotional messages originating from the brand (i.e., messages that indicate that the tweets are sponsored or are ads).

Removing stop words

Stop words are commonly occurring words such as "the," "is," "at," "which," and "on". Because our concern was with content rather than style, consistent with standard approaches using LDA, we removed these from the data before extracting topics.

Removing brand names

We removed the actual brand names present in the tweets, as these brand names do not have direct relevance to our assessment of topics. To do this, we searched for all instances of the brand names in tweets identified for the brand and removed these words. The removal of brand names also has an important rationale from an empirical standpoint. Brand names appear excessively frequently in the data and do not fall within the Dirichlet distributions assumed by the LDA model. Because brand names are the criteria for tweet collection, such words are overrepresented in the data and essentially become outliers in the distribution of word counts. To demonstrate this further, we ran the model again including brand names and found that the topics observed were less meaningful. For brevity, we do not present these results, but they are available on request.

Creating a topic file

After filtering out the brand names, we focused on extracting topics from a pooled sample consisting of all brands. To choose the number of topics, we specified the number of topics that LDA can generate. Across multiple topic models based on 25, 50, 100, 200, 250, 500, 1,000, and 2,000 topics, the 100-topic model generated the ideal solution, based on perplexity, coherence scores, and a qualitative assessment of the topics generated.

Model selection using perplexity and coherence

To identify the optimal number of topics that we need to choose, we calculated these scores across a range of models estimated by specifying different numbers of topics (25, 50, 100, 200, 250, 500, 1,000, and 2,000). We calculated the perplexity score using the following formula:

$\mathrm{PP}(\mathrm{W})=\mathrm{P}\left(\mathrm{w}_1, \mathrm{w}_2, \ldots, \mathrm{w}_{\mathrm{N}}\right)^{1 / \mathrm{N}}=\mathrm{N} \sqrt{\frac{1}{\mathrm{p}\left(\mathrm{w}_1 \mathrm{w}_2 \ldots \mathrm{w}_{\mathrm{N}}\right)}}$

(2)

Alternatively, this specification can be rewritten as PP(W) = 2^−l.

To calculate this, we first estimated a topic model by specifying a number of topics (e.g., 100, 200). We then read all the messages to calculate N (total corpus size or number of messages). From this topic model estimation, we outputted the probabilities associated with each message calculated as log[p(w₁,w₂,w₃)] = log[p(w₁) × p(w₂) × p(w₃)] = log[p(w₁)] + log[p(w₂)] + log[p(w₃)], and so on, until arriving at log p(w_N), where N is the number of messages and W₁ is the first message.

In general, lower perplexity scores indicate a better model. We provide the perplexity scores across a range of topic models in Figure WA1 of the Web Appendix. As we examine the perplexity scores, across the number of topics, the 200-topic model has the lowest score of 18.0. This is identical to the 250-topic model score (18.0), with the 100-topic model also providing similar, though slightly higher, scores (18.3). Because the 100-topic model is a more parsimonious representation without a significant trade-off in terms of the perplexity calculation, we chose the 100-topic model based on this analysis.

A second metric we examined is the coherence score, or topic coherence, which is based on the semantic coherence of the words. Research has found ways to automatically measure topic coherence with humanlike accuracy using a score based on pointwise mutual information. We calculated the average coherence score of each of the topic models across various numbers of topics (see Web Appendix Figure WA2). We considered two measures of topic coherence: C_v and C_umass. Based on C_umass, the 50-topic model also had the best topic coherence, which was closest to zero (–980.6). This was followed by the 100-topic model (–996.7). Optimizing across the two metrics of perplexity and coherence required us to make some trade-offs. We deemed the 100-topic model as offering the best combination of these two metrics.

Visualizing the data

We used word clouds to visualize the topics. Clusters of words that are semantically related form topics, and each brand is associated with the topics to varying degrees. Within the word cloud, the size corresponds to the prevalence of the word within the topic (i.e., p(word|topic)).

Evaluating brand positioning using topic probabilities

A straightforward approach to assessing the relationships between topics and brands is to use the topic probabilities averaged at the brand level. We conduct a correlation analysis of the various brands using the average topic probability across the 100 topics. We specify these correlations as one approach that managers can use to identify brands that are similar or dissimilar on the basis of social media topics.

Assessing lexical differences across brands using significance testing

One of the goals of our research is to use social media language and topics to predict the presence or absence of brand names in a conversation. After uncovering topics and connecting these topics with brands, we use Schwartz et al.'s software to conduct regressions of topic use across brands, via significance testing (see Appendix A for details on this step). We then use this analysis of positive and negative topic correlates of a given brand and describe applications for brand management derived from this analysis.

Results

Given our open-vocabulary approach, we present words associated with each of the 100 topic IDs. We do not focus on labeling these topics in this article, though there are approaches available to do so using the incidence of words within a topic. Rather, our focus is on using these topics of conversation as a way to identify brand similarities and differences. However, a few general observations are worth noting regarding the types of topics described:

1. Positive or negative incidents: Some of the topics identified described a news story or feature, some of which were positive (e.g., topic 17) and others negative (e.g., topic 21).
2. Activities and lifestyles: Topics were also identified by different activities and lifestyles, including music and entertainment (topic 82) and holidays/special occasions (topic 73).

From the output of the topic models, we introduce temporal topic variability (TTV), which can help accurately "predict" brand preference shifts that occur in the future. TTV is a summary of changes in relationships between specific social media topics and a given brand, and we predict that these can affect future shifts in brand preferences. Thus, we empirically test the likelihood that shifts in topic–brand relationships will have an impact on future changes in brand preference. Note that the topic meanings are kept constant and the changes in relationship between specific topics and brands over time is what constitutes this metric. In other words, we address the following questions: if a brand such as Google observes high variability expressed in social media topics in period t (as measured by changes in topic probabilities associated with a brand from the previous period to the current period), will this result in a greater likelihood of brand preference shifts in period t + 1? Will these shifts be present even after we control for category differences?

To answer these questions, and to identify brand preference shifts, we integrated the social media data with brand outcomes data from Young & Rubicam's Brand Asset Valuator (BAV) survey for a future period (here, "future" refers to the period t + 1 vs. t). Using these brand outcomes as dependent variables, we regressed these against a current-period change in social media topic probabilities (we define the current period subsequently). We controlled for category random effects in our model.

We considered three brand outcomes: brand preferences, net promoter score, and percentage of lapsed users. Brand preferences are measured across the same set of brands in the BAV data, using a survey approach. Specifically, the BAV survey asks respondents to review a brand preference question and to select one of the four response categories regarding each brand ("the one I prefer," "one of several I’d consider," "would only consider when no alternative," or "would never consider"). We viewed those who selected the first two responses as those who prefer the brand, and the percentage of those who indicated preference formed the dependent variable. The net promoter score was based on the percentage of respondents who said that they would recommend the brand to a friend, and lapsed users were the percentage who indicated that they had lapsed in brand usage. If TTV can predict future shifts in these important brand outcomes, this would mean that our proposed approach functions as an early warning system to measure changes in a brand's positioning in the marketplace.

We gathered additional data from Twitter, using the same Twitter handles but going back to the 2011–2015 time frame. We used the same 100-topic solution previously generated and searched for topic terms that we had identified using the analysis described previously. We then examined the probabilities of topics for each brand and each year and calculated the average difference in probabilities for each year relative to the previous year (e.g., 2012 [vs. 2011], 2013 [vs. 2012], 2014 [vs. 2013], 2015 [vs. 2014]) for each of the 100 topics for each brand. We then took the sum of the absolute value of differences in topic probabilities. Our metric of brand–topic associations for brand x across topics k and time t is

$\text{TTV of brand x }=\sum_{\mathbf{k}=1}^K\left\{\sum_{\mathbf{t}}^{\mathrm{T}}\left[\operatorname{abs}\left(\text { topic } \mathbf{k} \text { prob }_{(\mathbf{t})}\right)-\left(\text { topic k pro } \mathrm{b}_{\mathrm{t-1}}\right)\right]\right\}$

(3)

Turning to the BAV data, we calculated changes in future brand preference as

$=\frac{\left[a b s\left(\text { brand preference }_{t+1}\right)-\left(\text { brand preferenc } \mathrm{e}_t\right)\right]}{2}$

(4)

Note that in this specification, TTV is regressed against a future change in brand preference (and change in net promoter scores, and change in percentage of lapsed users). We estimated the model using an instrumental variable regression using generalized method-of-moments specification (we use STATA's procedure to invoke an instrumental variable regression with generalized method-of-moments estimation and cluster-robust standard errors;). This specification included lagged change in brand preference as an endogenous regressor and used lagged changes in brand equity dimensions of BAV brand stature and BAV brand strength as instruments for change in brand preference (see Table WA2 in the Web Appendix for results). We used cluster robust standard errors to account for category effects. In this model, we found that TTV was a significant predictor of all three models (future change in brand preference: β = 45.09, p < .01; future change in net promoter score: β = 21.20, p < .01; future change in percentage of lapsed users: β = 18.84, p < .01). In an alternative specification, also estimated the model using maximum likelihood and used a fixed-effects specification to control for category effects. We found that the TTV indicator was a marginally significant predictor of future period brand preference volatility (β = 12.73, p = .06); it was not a significant predictor or future change in net promoter score (β = 5.79, n.s.) but was a significant predictor of future change in percentage of lapsed users (β = 13.54, p < .05).

Overall, the results of this analysis using TTV provides further evidence that identifying topics from social media and assessing how their associations with brands shift over time (while keeping the topic meanings the same) can help predict how a brand is likely to be perceived in the future. The results also confirm that the suggested approach to extracting brand image information from social media can be used as a predictive tool to forecast future shifts in key brand metrics such as future brand preferences and percentages of lapsed users. This type of predictive validation is a key step in establishing construct validity, in that our metric effectively correlates with other, known approaches to measurement of brand image that are costly and time-consuming. It provides insights into the nature of brand image by identifying topics of conversation that relate to a brand. In addition, the TTV concept provides a novel approach to predicting future shifts in brand image. In the next section, we outline various applications of the differential language analysis of social media topics as they relate to various brands and describe the implications for data-driven brand management decision making.