1
Twitter mood predicts the stock market.
Johan Bollen1,?,Huina Mao1,?,Xiao-Jun Zeng2.
?: authors made equal contributions.
Abstract—Behavioral economics tells us that emotions can
profoundly affect individual behavior and decision-making. Does
this also apply to societies at large, i.e. can societies experience
mood states that affect their collective decision making? By
extension is the public mood correlated or even predictive of
economic indicators? Here we investigate whether measurements
of collective mood states derived from large-scale Twitter feeds
are correlated to the value of the Dow Jones Industrial Average
(DJIA) over time. We analyze the text content of daily Twitter
feeds by two mood tracking tools, namely OpinionFinder that
measures positive vs. negative mood and Google-Profile of Mood
States (GPOMS) that measures mood in terms of 6 dimensions
(Calm, Alert, Sure, Vital, Kind, and Happy). We cross-validate
the resulting mood time series by comparing their ability to
detect the public’s response to the presidential election and
Thanksgiving day in 2008. A Granger causality analysis and
a Self-Organizing Fuzzy Neural Network are then used to
investigate the hypothesis that public mood states, as measured by
the OpinionFinder and GPOMS mood time series, are predictive
of changes in DJIA closing values. Our results indicate that the
accuracy of DJIA predictions can be significantly improved by
the inclusion of specific public mood dimensions but not others.
We find an accuracy of 87.6% in predicting the daily up and
down changes in the closing values of the DJIA and a reduction
of the Mean Average Percentage Error by more than 6%.
Index Terms—stock market prediction — twitter — mood
analysis.
I. INTRODUCTION
STOCK market prediction has attracted much attentionfrom academia as well as business. But can the stock
market really be predicted? Early research on stock market
prediction [1], [2], [3] was based on random walk theory
and the Efficient Market Hypothesis (EMH) [4]. According
to the EMH stock market prices are largely driven by new
information, i.e. news, rather than present and past prices.
Since news is unpredictable, stock market prices will follow a
random walk pattern and cannot be predicted with more than
50 percent accuracy [5].
There are two problems with EMH. First, numerous studies
show that stock market prices do not follow a random walk
and can indeed to some degree be predicted [5], [6], [7], [8]
thereby calling into question EMH’s basic assumptions. Sec-
ond, recent research suggests that news may be unpredictable
but that very early indicators can be extracted from online
social media (blogs, Twitter feeds, etc) to predict changes
in various economic and commercial indicators. This may
conceivably also be the case for the stock market. For example,
[11] shows how online chat activity predicts book sales. [12]
uses assessments of blog sentiment to predict movie sales.
[15] predict future product sales using a Probabilistic Latent
Semantic Analysis (PLSA) model to extract indicators of
sentiment from blogs. In addition, Google search queries have
been shown to provide early indicators of disease infection
rates and consumer spending [14]. [9] investigates the relations
between breaking financial news and stock price changes.
Most recently [13] provide a ground-breaking demonstration
of how public sentiment related to movies, as expressed on
Twitter, can actually predict box office receipts.
Although news most certainly influences stock market
prices, public mood states or sentiment may play an equally
important role. We know from psychological research that
emotions, in addition to information, play an significant role
in human decision-making [16], [18], [39]. Behavioral finance
has provided further proof that financial decisions are sig-
nificantly driven by emotion and mood [19]. It is therefore
reasonable to assume that the public mood and sentiment can
drive stock market values as much as news. This is supported
by recent research by [10] who extract an indicator of public
anxiety from LiveJournal posts and investigate whether its
variations can predict S&P500 values.
However, if it is our goal to study how public mood
influences the stock markets, we need reliable, scalable and
early assessments of the public mood at a time-scale and
resolution appropriate for practical stock market prediction.
Large surveys of public mood over representative samples of
the population are generally expensive and time-consuming
to conduct, cf. Gallup’s opinion polls and various consumer
and well-being indices. Some have therefore proposed indirect
assessment of public mood or sentiment from the results of
soccer games [20] and from weather conditions [21]. The
accuracy of these methods is however limited by the low
degree to which the chosen indicators are expected to be
correlated with public mood.
Over the past 5 years significant progress has been made
in sentiment tracking techniques that extract indicators of
public mood directly from social media content such as blog
content [10], [12], [15], [17] and in particular large-scale
Twitter feeds [22]. Although each so-called tweet, i.e. an
individual user post, is limited to only 140 characters, the
aggregate of millions of tweets submitted to Twitter at any
given time may provide an accurate representation of public
mood and sentiment. This has led to the development of real-
time sentiment-tracking indicators such as [17] and “Pulse of
Nation”1.
In this paper we investigate whether public sentiment, as
expressed in large-scale collections of daily Twitter posts, can
be used to predict the stock market. We use two tools to
measure variations in the public mood from tweets submitted
1http://www.ccs.neu.edu/home/amislove/twittermood/
ar
X
iv
:1
01
0.
30
03
v1
[
cs
.C
E]
1
4 O
ct
20
10
2
to the Twitter service from February 28, 2008 to December 19,
2008. The first tool, OpinionFinder, analyses the text content
of tweets submitted on a given day to provide a positive
vs. negative daily time series of public mood. The second
tool, GPOMS, similarly analyses the text content of tweets to
generate a six-dimensional daily time series of public mood
to provide a more detailed view of changes in public along
a variety of different mood dimensions. The resulting public
mood time series are correlated to the Dow Jones Industrial
Average (DJIA) to assess their ability to predict changes in
the DJIA over time. Our results indicate that the prediction
accuracy of standard stock market prediction models is signif-
icantly improved when certain mood dimensions are included,
but not others. In particular variations along the public mood
dimensions of Calm and Happiness as measured by GPOMS
seem to have a predictive effect, but not general happiness as
measured by the OpinionFinder tool.
II. RESULTS
A. Data and methods overview
We obtained a collection of public tweets that was recorded
from February 28 to December 19th, 2008 (9,853,498 tweets
posted by approximately 2.7M users). For each tweet these
records provide a tweet identifier, the date-time of the sub-
mission (GMT+0), its submission type, and the text content
of the Tweet which is by design limited to 140 characters.
After removal of stop-words and punctuation, we group all
tweets that were submitted on the same date. We only take
into account tweets that contain explicit statements of their
author’s mood states, i.e. those that match the expressions “i
feel”,“i am feeling”,“i’m feeling”,“i dont feel”, “I’m”, “Im”,
“I am”, and “makes me”. In order to avoid spam messages and
other information-oriented tweets, we also filter out tweets that
match the regular expressions ”http:” or ”www.”
As shown in Fig. 1 we then proceed in three phases. In the
first phase, we subject the collections of daily tweets to 2 mood
assessment tools: (1) OpinionFinder which measures positive
vs. negative mood from text content, and (2) GPOMS which
measures 6 different mood dimensions from text content.
This results in a total of 7 public mood time series, one
generated by OpinionFinder and six generated by GPOMS,
each representing a potentially different aspect of the public’s
mood on a given day. In addition, we extract a time series
of daily DJIA closing-values from Yahoo! Finance. In the
second phase, we investigate the hypothesis that public mood
as measured by GPOMS and OpinionFinder is predictive of
future DJIA values. We use a Granger causality analysis in
which we correlate DJIA values to GPOMs and OF values
of the past n days. In the third phase, we deploy a Self-
Organizing Fuzzy Neural Network model to test the hypothesis
that the prediction accuracy of DJIA prediction models can be
improved by including measurements of public mood. We are
not interested in proposing an optimal DJIA prediction model,
but to assess the effects of including public mood information
on the accuracy of a “baseline” prediction model.
Twitter
feed ~
(1) OpinionFinder
(2) G-POMS (6 dim.)
Mood indicators (daily)
DJIA ~
Stock market (daily)
(3) DJIA
Granger
causality
-n (lag)
F-statistic
p-value
text
analysis
normalization
SOFNN
predicted
value MAPE
Direction %
1
2
t-1
t-2
t-3
3
t=0
value
feb28
2008
apr may jun jul aug sep oct nov dec dec20
2008
(1) OF ~
GPOMS
(2) Granger Causality analysis
(3) SOFNN training test
Methodology
Data sets and timeline
Fig. 1. Diagram outlining 3 phases of methodology and corresponding data
sets: (1) creation and validation of OpinionFinder and GPOMS public mood
time series from October 2008 to December 2008 (Presidential Election and
Thanksgiving), (2) use of Granger causality analysis to determine correlation
between DJIA, OpinionFinder and GPOMS public mood from August 2008 to
December 2008, and (3) training of a Self-Organizing Fuzzy Neural Network
to predict DJIA values on the basis of various combinations of past DJIA
values and OF and GPOMS public mood data from March 2008 to December
2008.
B. Generating public mood time series: OpinionFinder and
GPOMS
OpinionFinder (OF)2 is a publicly available software pack-
age for sentiment analysis that can be applied to determine
sentence-level subjectivity [25], i.e. to identify the emo-
tional polarity (positive or negative) of sentences. It has
been successfully used to analyze the emotional content of
large collections of tweets [26] by using the OF lexicon to
determine the ratio of positive versus negative tweets on a
given day. The resulting time series were shown to correlate
with the Consumer Confidence Index from Gallup3 and the
Reuters/University of Michigan Surveys of Consumers4 over
a given period of time. We adopt OF’s subjective lexicon that
has been established upon previous work [37], [38], [24]. We
select positive and negative words that are marked as either
“weak” and “strong” from the OF sentiment lexicon resulting
in a list of 2718 positive and 4912 negative words. For each
tweet we determine whether it contains any number of negative
and positive terms from the OF lexicon. For each occurrence
we increase the score of either negative or positive tweets by
1 and calculate the ratio of positive vs. negative messages for
the tweets posted on the same day t.
Like many sentiment analysis tools OF adheres to a unidi-
mensional model of mood, making binary distinctions between
positive and negative sentiment [23]. This may however ignore
the rich, multi-dimensional structure of human mood. To
capture additional dimensions of public mood we created a
second mood analysis tools, labeled GPOMS, that can measure
human mood states in terms of 6 different mood dimensions,
namely Calm, Alert, Sure, Vital, Kind and Happy. GPOMS’
2http://www.cs.pitt.edu/mpqa/opinionfinderrelease/
3http://www.gallup.com/poll/122840/Gallup-Daily-Economic-Indexes.aspx
4http://www.sca.isr.umich.edu/
3
mood dimensions and lexicon are derived from an existing
and well-vetted psychometric instrument, namely the Profile
of Mood States (POMS-bi)[32], [33]. To make it applicable
to Twitter mood analysis we expanded the original 72 terms
of the POMS questionnaire to a lexicon of 964 associated
terms by analyzing word co-occurrences in a collection of
2.5 billion 4- and 5-grams5 computed by Google in 2006
from approximately 1 trillion word tokens observed in publicly
accessible Webpages [35], [36]. The enlarged lexicon of 964
terms thus allows GPOMS to capture a much wider variety of
naturally occurring mood terms in Tweets and map them to
their respective POMS mood dimensions. We match the terms
used in each tweet against this lexicon. Each tweet term that
matches an n-gram term is mapped back to its original POMS
terms (in accordance with its co-occurence weight) and via the
POMS scoring table to its respective POMS dimension. The
score of each POMS mood dimension is thus determined as the
weighted sum of the co-occurence weights of each tweet term
that matched the GPOMS lexicon. All data sets and methods
are available on our project web site6.
To enable the comparison of OF and GPOMS time series we
normalize them to z-scores on the basis of a local mean and
standard deviation within a sliding window of k days before
and after the particular date. For example, the z-score of time
series Xt, denoted ZXt , is defined as:
ZXt =
Xt − x¯(Xt±k)
σ(Xt±k)
(1)
where x¯(Xt±k) and σ(Dt±k) represent the mean and stan-
dard deviation of the time series within the period [t−k, t+k].
This normalization causes all time series to fluctuate around a
zero mean and be expressed on a scale of 1 standard deviation.
C. Cross-validating OF and GPOMS time series against large
socio-cultural events
We first validate the ability of OF and GPOMS to capture
various aspects of public mood. To do so we apply them
to tweets posted in a 3-month period from October 5, 2008
to December 5, 2008. This period was chosen specifically
because it includes several socio-cultural events that may have
had a unique, significant and complex effect on public mood
namely the U.S presidential election (November 4, 2008) and
Thanksgiving (November 27, 2008). The OF and GPOMS
measurements can therefore be cross-validated against the
expected emotional responses to these events. The resulting
mood time series are shown in Fig. 2 and are expressed in
z-scores as given by in Eq. 1.
Fig. 2 shows that the OF successfully identifies the public’s
emotional response to the Presidential election on November
4th and Thanksgiving on November 27th. In both cases OF
marks a significant, but short-lived uptick in positive sentiment
specific to those days.
The GPOMS results reveal a more differentiated public
mood response to the events in the three-day period surround-
ing the election day (November 4, 2008). November 3, 2008 is
5n-grams are frequently occurring sequences of terms in text of length n,
for example “we are the robots” could be a frequent 4-gram.
6http://terramood.informatics.indiana.edu/data
characterized by a significant drop in Calm indicating highly
elevated levels of public anxiety. Election Day itself is char-
acterized by a reversal of Calm scores indicating a significant
reduction in public anxiety, in conjunction with a significant
increases of Vital, Happy as well as Kind scores. The latter
indicates a public that is energized, happy and friendly on
election day. On November 5, these GPOMS dimensions con-
tinue to indicate positive mood levels, in particular high levels
of Calm, Sure, Vital and Happy. After November 5, all mood
dimensions gradually return to the baseline. The public mood
response to Thanksgiving on November 27, 2008 provides a
counterpart to the differentiated response to the Presidential
election. On Thanksgiving day we find a spike in Happy
values, indicating high levels of public happiness. However,
no other mood dimensions are elevated on November 27.
Furthermore, the spike in Happy values is limited to the one
day, i.e. we find no significant mood response the day before
or after Thanksgiving.
1.25
1.75
OpinionFinder day after
election Thanksgiving
-1
1
pre- election
anxiety
CALM
-1
1
ALERT
-1
1
election
results
SURE
1
1
pre! election
energyVITAL
-1
-1 KIND
-1
1
Thanksgiving
happiness
HAPPY
Oct 22 Oct 29 Nov 05 Nov 12 Nov 19 Nov 26
z-
sc
or
es
Fig. 2. Tracking public mood states from tweets posted between October
2008 to December 2008 shows public responses to presidential election and
thanksgiving.
A visual comparison of Fig. 2 suggests that GPOMS’
Happy dimension best approximates the mood trend provided
by OpinionFinder. To quantitatively determine the relations
between GPOMS’s mood dimensions and the OF mood trends,
we test the correlation between the trend obtained from OF
lexicon and the six dimensions of GPOMS using multiple
regression. The regression model is shown in Eq. 2.
YOF = α+
n∑
i
βiXi + �t (2)
where X1, X2, X3, X4, X5 and X6 represent the mood time
series obtained from the 6 GPOMS dimensions, respectively
4
TABLE I
MULTIPLE REGRESSION RESULTS FOR OPINIONFINDER VS. 6 GPOMS
MOOD DIMENSIONS.
Parameters Coeff. Std.Err. t p
Calm (X1) 1.731 1.348 1.284 0.20460
Alert (X2) 0.199 2.319 0.086 0.932
Sure (X3) 3.897 0.613 6.356 4.25e-08 ? ? ?
Vital (X4) 1.763 0.595 2.965 0.004??
Kind (X5) 1.687 1.377 1.226 0.226
Happy (X6) 2.770 0.578 4.790 1.30e-05 ??
Summary Residual Std.Err Adj.R2 F6,55 p
0.078 0.683 22.93 2.382e-13
(p-value < 0.001: ? ? ?, p-value < 0.05: ??, p-value < 0.1: ?)
Calm, Alert, Sure, Vital, Kind and Happy.
The multiple linear regression results are provided in
Table I (coefficient and p-values), and indicate that YOF
is significantly correlated with X3 (Sure), X4 (Vital) and
X6 (Happy), but not with X1 (Calm), X2 (Alert) and X5
(Kind). We therefore conclude that certain GPOMS mood
dimension partially overlap with the mood values provided by
OpinionFinder, but not necessarily all mood dimensions that
may be important in describing the various components of
public mood e.g. the varied mood response to the Presidential
election. The GPOMS thus provides a unique perspective on
public mood states not captured by uni-dimensional tools such
as OpinionFinder.
D. Bivariate Granger Causality Analysis of Mood vs. DJIA
prices
After establishing that our mood time series responds to
significant socio-cultural events such as the Presidential elec-
tion and Thanksgiving, we are concerned with the question
whether other variations of the public’s mood state correlate
with changes in the stock market, in particular DJIA closing
values. To answer this question, we apply the econometric
technique of Granger causality analysis to the daily time
series produced by GPOMS and OpinionFinder vs. the DJIA.
Granger causality analysis rests on the assumption that if a
variable X causes Y then changes in X will systematically
occur before changes in Y . We will thus find that the lagged
values of X will exhibit a statistically significant correlation
with Y . Correlation however does not prove causation. We
therefore use Granger causality analysis in a similar fashion
to [10]; we are not testing actual causation but whether one
time series has predictive information about the other or not7.
Our DJIA time series, denoted Dt, is defined to reflect daily
changes in stock market value, i.e. its values are the delta
between day t and day t− 1: Dt = DJIAt −DJIAt−1. To
test whether our mood time series predicts changes in stock
market values we compare the variance explained by two linear
models as shown in Eq. 3 and Eq. 4. The first model (L1)
uses only n lagged values of Dt, i.e. (Dt−1, · · · , Dt−n) for
prediction, while the second model L2 uses the n lagged values
of both Dt and the GPOMS plus the OpinionFinder mood time
series denoted Xt−1, · · · , Xt−n.
7[10] uses only one mood index, namely Anxiety, but we investigate the
relation between DJIA values and all Twitter mood dimensions measured by
GPO