FABBS Doctoral Dissertation Research Excellence Award

Tessa Charlesworth, Harvard University, Department of Psychology

Patterns of Long-Term Change in Implicit Social Cognition

Tessa Charlesworth, Harvard University, Department of Psychology


To navigate the social world, humans developed the adaptive capacity to rapidly form attitudes  and beliefs about the social groups around them. Yet to be fully adaptive attitudes and beliefs must  be capable of change; as history unfolds to reveal new social roles for each group, new  representations of groups in the media, or new legislation around group rights so too should  attitudes and beliefs update in response. Although past work has demonstrated long-term change  in explicit, self-reported cognition, evidence has so far remained elusive for durable transformations in implicit social cognition that is more automatic, less controllable and indirectly assessed. The current dissertation integrates four papers to provide the first comprehensive  investigation of long-term change in implicit social cognition. Using data from 6+ million implicit  attitude tests collected since 2007 and 65+ million words from multiple decades of archived text,  the results newly reveal that long-term change in implicit social cognition is possible, but not  inevitable. Such diverse patterns of change and stability can be used to guide new theory on why,  how, and for whom implicit cognition evolves over time, while also guiding new policies and  practices that help motivate progressive social change.

Project Summary 

Every day, we encounter people who differ from us on a network of dimensions, whether  nationality, gender, age, race, sexual orientation, or more. To navigate such complexity, we rely  on forming attitudes (i.e., good/bad evaluations) and beliefs (i.e., semantic associations with attributes such as “intelligence” or “athletic” [1]). These attitudes and beliefs help us determine  whom to approach or avoid, whom to follow or lead, even whom to learn from or teach [2]. Yet to  be truly adaptive, attitudes and beliefs must be capable of change, updating in response to new  information not only over the short-term [3], but also over long-term spans of years or decades to  keep pace with historical transformations. 

For decades, social scientists have documented evidence of long-term change in explicit, self reported cognitions: for instance, in 1958, only 4% of White Americans explicitly approved of  Black–White marriages; by 2013, 87% of White Americans approved of such unions [4].  However, change in explicit social cognition (ESC) cannot provide a complete understanding of  societal change. After all, much of what goes on in our minds is not available to conscious  introspection and explicit reports [5–7]. Additionally, even if one were able to introspect on  explicit cognitions, one may be unwilling to report true beliefs and attitudes for fear of appearing  prejudiced [8]. Change in ESC could therefore be a manifestation of increasing pressures to control  one’s bias rather than a change in the mental representation of groups. The study of change in implicit social cognition (ISC) – attitudes and beliefs that are relatively more automatic, less  controlled, and assessed indirectly – can overcome the concerns of introspective access and social  desirability, ultimately providing a complementary portrait of social change. To that end, the  current dissertation integrates four papers to outline new methods, empirical records, and  emerging theories for understanding long-term change in implicit attitudes and beliefs. 

To date, the study of change in ISC has focused on short-term malleability, revealing that ISC can  shift over a few minutes [9, 10] but will rapidly snap back to original levels after a short delay  [11]. Until the current dissertation, the question of durable, long-term change in ISC remained  elusive, in part because of the methodological demands of studying multiple attitudes continuously  over multiple years. In Chapters 1 and 2, I present a new approach of studying long-term ISC  change by applying timeseries analyses to aggregated data of 8 different attitude and belief topics (e.g., race, sexuality, age) from 6+ million respondents collected continuously through an online  website over 2007-2018. In using such massive, aggregated cross-sectional data, I model patterns  of fine-grained, non-linear change to identify not only whether, but also when, for whom, and  where change occurs. Then, to enable rigorous tests of these emerging hypotheses, Chapter 3 introduces a methodology – word embeddings applied to archived text – that can greatly expand  the record of change in social cognition across diverse communities (e.g., children vs. adults),  hundreds of attitude topics, and timespans of 200 years or more.


Chapters 1 and 2: Documenting change in ISC (2007-2018). To study long-term change in ISC,  I prepared and analyzed cross-sectional data from the Project Implicit demonstration website,  which has collected data continuously, minute-by-minute on implicit (using the Implicit  Association Test, IAT; [12]) and explicit (self-report Likerts) measures of attitudes and beliefs from volunteer respondents since the early 2000s. Data were obtained from 6+ million respondents  for one of 8 topics (attitudes towards race, skin-tone, sexuality, age, disability, and body weight,  as well as stereotypes associating men-science/women-arts and men-work/women-home). Chapter  1 documents overall patterns of change at the societal-level for all topics (six attitudes measured  2007-2016 and two stereotypes measured 2007-2018) [13, 14]. Chapter 2 digs deeper into the  pervasiveness of such change by testing demographic differences across respondent age, political  ideology, race, gender, education, religion, and sexuality [15]. 

To address unique features of Project Implicit data, I used Autoregressive Integrated Moving  Average (ARIMA) time series models [16]. ARIMA models have several advantages over  previous methods (e.g., linear regressions; [17, 18]). Specifically, ARIMA models account for  autocorrelations (dependencies), capture non-linear trends and seasonal variation (systematic rises  or falls in some months), and can offer forecasts that can be used to test model validity and  persistence of long-term trends [19]. ARIMA models were fit to the aggregated monthly timeseries  of all implicit attitude and belief topics. Importantly, analyses also controlled for sample change  by weighting for similar demographic composition over time; additional robustness checks (e.g.,  sub-setting the data in various ways) provided further confidence in all inferences. 

Chapter 3: Expanding methods for studying ISC change. Chapter 3 [20] set out to demonstrate  how novel methods from Natural Language Processing (NLP), specifically word embeddings  applied to archived texts, could be used in the social sciences. Word embeddings are numeric vector representations of word meaning in which words close in meaning (e.g., men-science) have  vector representations close together, while words far in meaning (e.g., women-science) have  vectors far apart. By comparing the relative closeness of word vectors (e.g., men-science vs.  women-science) we get an index of the strength of biases reflected in text [21]. This word  embedding approach – called the Word Embeddings Association Test (WEAT) – is analogous to  the aforementioned indirect measures of human implicit cognition (the IAT) and can reveal how  biases are present in diverse societal language. 

I used word embeddings to quantify the pervasiveness of multiple gender biases (science/arts,  work/home, math/reading, and bad/good, as well as associations with over 600 traits and 300  occupations) throughout texts that varied in time (1900s versus 2000s), format (books, TV/movies,  conversation transcripts), and audience (children versus adults). Meta-analyses were then used to  identify the overall and relative strength of gender biases in these diverse texts. Most relevant to  the study of long-term change, meta-regression comparisons focused on whether gender biases  were stronger in texts from historical time periods (e.g., classic books) or contemporary language  (e.g., contemporary TV shows). 


Chapter 1: Long-term change in ISC is possible. Across more than a decade of data, results  showed that long-term, durable change in implicit social cognition is indeed possible: implicit race,  skin-tone, and sexuality attitudes, as well as implicit stereotypes associating men-science/women arts and men-work/women-home, all showed meaningful decreases in bias over time. The fastest  change occurred in implicit sexuality attitudes, which dropped by 33% between 2007-2016, and  were even forecast to touch neutrality (no bias) in as little as 9 years from 2016, if past trends persisted. Relatively slower change was seen for race, skin-tone, and gender stereotypes, which  dropped by 13-17%, but even this change reinforced the possibility for transformation in automatic  judgments of social groups. 

However, long-term change in ISC was not inevitable: implicit age and disability attitudes have  been stable over more than a decade, shifting by less than 5% and not predicted to touch neutrality  even within the next 150 years. Most concerningly, implicit body weight attitudes increased in  bias over time by as much as 40%. The diversity of patterns – with some attitudes changing towards  neutrality, others remaining stable, and yet others increasing in bias – shows that implicit attitude  change reflect which group topics have been prioritized by society (e.g., race, sexuality) and  received numerous efforts for intervention and change, versus which topics have been largely  neglected (e.g., age, disability). Notably, this sensitivity and variability in ISC contrasts with  explicit attitudes and beliefs, which all consistently dropped towards neutrality from 2007-2018  perhaps because of general increasing norms against expressing explicit prejudices. 

Chapter 2: Long-term change in ISC is widespread across people. Comparisons across 32  demographic differences showed that patterns of change and stability in ISC were impressively  widespread across most people. Regardless of respondent gender, education, or religion, attitudes  that were changing (race, sexuality) were changing for everyone, while attitudes that were stable  (age) were stable for everyone. Two exceptions nevertheless emerged to this general pattern of  demographic consistency: on both implicit race and sexuality attitudes younger respondents  dropped faster towards neutrality than older respondents, and liberal respondents dropped faster  than conservative respondents. As discussed below, such results help point to the most likely  sources of widespread, macro-level transformation in implicit attitudes and beliefs. 

Chapter 3: New methods demonstrate pervasive biases in text. In line with the truly widespread, societal nature of implicit cognition, the new methods in Chapter 3 (using word embeddings  applied to 65+ million words) showed that gender biases were broadly pervasive throughout  society and diverse texts. Texts from both children and adults, and from both more historical and  contemporary time periods, revealed the presence of well-studied gender stereotypes (e.g., men science/women-arts) at similar magnitudes. Nevertheless, results from a broader sample space of  stereotypes (e.g., associations to 300 occupation titles) showed that text-based biases of  occupations (e.g., woman-nurse, man-carpenter) have shifted over time, with more recent corpora  showing weaker biases than historical corpora. Such findings reinforce that change indirectly measured, societal biases is possible for some topics, but not inevitable for all. 


Through three chapters, integrating four papers [13–15, 20], I present novel methods, empirical  records, and initial theoretical hypotheses that help form the foundation for understanding long term, societal change in implicit social cognition (ISC). The results show, for the first time, that  long-term ISC change is indeed possible; far from being the rigid “cognitive monster”, as  originally speculated [22], ISC can undergo meaningful transformations, dropping in bias by as  much as 33% over just a decade. Equally crucial, however, is the lack of change for some  attitudes: implicit attitudes about age and disability, for example, were forecast to take over 150  years to touch neutrality. Thus, the rate and direction of change in ISC appears sensitive to the topic at hand. Such diversity in patterns forms the basis for new hypotheses on the factors that  will predict where ISC change occurs, including factors such as the degree to which the attitude  or stereotype topic is societally prioritized by the public, and the overall magnitude of the  attitude to begin with. 

Importantly, any explanation for long-term implicit attitude and stereotype change must also  contend with the remarkable consistency of change across people and places. Although there are some differences in the rate of change (e.g., younger respondents move faster than older  respondents), the fact that implicit attitudes are becoming more neutral in every U.S. state, across  both men and women, less-educated and more-educated, religious and non-religious, even liberal  and conservative respondents, newly shows that ISC change is a collective phenomenon. The  results thus point to the primary role of macro-level variables that affect all people in a society  (e.g., federal legislation on same-gender marriage, widespread exposure to Black Lives Matter  protests) above and beyond meso-level demographic variables that would motivate change or  resistance in just some groups. 

Emerging hypotheses of ISC change require a dataset to test generalizability across many more  attitude topics and much longer timespans that encompass the range of hypothesized events. Yet,  so far, the study of attitude and stereotype change has remained limited to whatever data  happened to be collected by social scientists in laboratories and surveys. Chapter 3 employed  new word embeddings methods to overcome the constraints of data from the here and now. The  research showed that such methods are indeed valid and can quantify the pervasiveness of known  gender biases. Now, we can use such methods applied to text spanning hundreds of groups, and  across timespans encompassing significant events such as the Civil Rights movement, elections,  or wars, ultimately yielding new and rigorous tests of our theories of social change. 

To conclude, this dissertation provides the first comprehensive portrait of long-term change in  implicit attitudes and beliefs, revealing nuances of which representations change, at what rate,  and for whom. In the process, the results have revealed insights into how our automatic,  indirectly-assessed cognitions about social groups are deeply interwoven with social structures:  societal implicit attitudes are indeed capable of both adaptively responding to and shaping our  broader environment. 

Impact Statement  

We stand in a time of significant upheaval and change in our society: in the midst of a global pandemic, a rise of right-wing populism, and a worldwide call for racial equity, there is renewed attention to the question of social progress. At its heart, my dissertation addresses this fundamental concern of progression in how humans think and feel about other social groups by  compiling the first empirical records to show that change is possible even in our more implicit  attitudes and stereotypes. Such a record provides the groundwork both for theory development and for guiding applied practice and policies that will maintain or initiate movement towards  more equal attitudes and beliefs of all social groups.