I situate my research at the intersection of emerging media, media effects, and computational social science. Currently, I am pursuing two major lines of research. First, today’s media landscape is characterized by an increasing prevalence of visual media. My research is among the pioneering attempts to apply computer vision methods, such as facial recognition and computational aesthetics, to investigate the production, diffusion, and effects of digital visual media. My second line of scholarship focuses on science communication. I am particularly interested in how emerging AI technologies such as facial recognition and automation are connected to debates about inequality and social justice and how these factors affect public opinions.
Automated image analysis has received increasing attention in social scientific research, yet existing scholarship has mostly covered the application of supervised learning to classify images into predefined categories. This study focuses on the task of unsupervised image clustering, which aims to automatically discover categories from unlabelled image data. We first review the steps to perform image clustering and then focus on one key challenge in this task—finding intermediate representations of images. We present several methods of extracting intermediate image representations, including the bag-of-visual-words model, self-supervised learning, and transfer learning (in particular, feature extraction with pretrained models). We compare these methods using various visual datasets, including images related to protests in China from Weibo, images about climate change on Instagram, and profile images of the Russian Internet Research Agency on Twitter. In addition, we propose a systematic way to interpret and validate clustering solutions. Results show that transfer learning significantly outperforms the other methods. The dataset used in the pretrained model critically determines what categories the algorithms can discover.
Athec: A Python Library for Computational Aesthetic Analysis of Visual Media in Social Science Research [Forthcoming in Computational Communication Research] [Preprint]
Visual aesthetics are related to a broad range of communication and psychological outcomes, yet the tools of computational aesthetic analysis are not widely available in the community of social science scholars. This article addresses this gap and provides a tutorial for social scientists to measure a broad range of hand-crafted aesthetic attributes of visual media, such as colorfulness and visual complexity. It introduces Athec, a Python library developed for computational aesthetic analysis in social science research, which can be readily applied by future researchers. In addition, a case study applies Athec to compare the visual aesthetics of Instagram posts from the two candidates in the 2016 US presidential election, Hillary Clinton and Donald Trump, showing how amateurishness and authenticity are reflected in politicians’ visual messages. With tools of computational aesthetic analysis, communication researchers can better understand the antecedents and outcomes of visual aesthetics beyond the content of visual media.
While previous research has revealed an ideological divide in Americans’ perceptions of COVID-19, specific ideological components can additionally explain public reactions to the pandemic. With two surveys–one sample of crowdsourced workers (N = 482) and a nationally representative sample of American adults (N = 7,449)–this research investigates how multiple ideological facets simultaneously predict individuals’ reactions to COVID-19. Results demonstrate that social dominance orientation and libertarianism are two important ideological sources that predict more dismissal of COVID-19 and less support for government measures. Right-wing authoritarianism was negatively correlated with COVID-19 concern and support for government actions, but suppression effects could exist. The effects of ideological variables were largely consistent when trust in science was considered. This study highlights the role of specific ideological components in contributing to the political divide regarding attitudes toward the COVID-19 pandemic beyond the liberal–conservative identification.
Anatomy of Audience Duplication Networks: How Individual Characteristics Differentially Contribute to Fragmentation in News Consumption and Trust [Published in New Media & Society]
While partisan selective exposure could drive audience fragmentation, other individual factors might also differentiate news diets. This study applies a method that disentangles the differential contributions of the individual characteristics of audience duplication networks. By analyzing a nationally representative survey about US adults’ media use in 2019 (N = 12,043), we demonstrate that news fragmentation is driven by a myriad of individual factors, such as gender, race, and religiosity. Partisanship is still an important driver. We also distinguish between media exposure and media trust, showing that many cross-cutting ties in co-exposure networks disappear when media trust is considered. We conclude that audience fragmentation research should extend beyond ideological selectivity and additionally investigate how and why other individual-level preferences differentially contribute to fragmentation both in news exposure and news trust.
What Makes Politicians’ Instagram Posts Popular? Analyzing Social Media Strategies of Candidates and Office Holders with Computer Vision [Published in The International Journal of Press/Politics] [PDF][SAGE]
Previous research on the success of politicians’ messages on social media has so far focused on a limited number of platforms, especially Facebook and Twitter, and predominately studied the effects of textual content. This research reported here applies computer vision analysis to a total of 59,020 image posts published by 172 Instagram accounts of U.S. politicians, both candidates and office holders, and examines how visual attributes influence audience engagement such as likes and comments. In particular, this study introduces an unsupervised approach that combines transfer learning and clustering techniques to discover hidden categories from large-scale visual data. The results reveal that different self-personalization strategies in visual media, for example, images featuring politicians in private, nonpolitical settings, showing faces, and displaying emotions, generally increase audience engagement. Yet, a significant portion of politician’s Instagram posts still fell into the traditional, “politics-as-usual” type of political communication, showing professional settings and activities. The analysis explains how self-personalization is embodied in specific visual portrayals and how different self-presentation strategies affect audience engagement on a popular but less studied social media platform.
The Ideological Divide in Public Perceptions of Self-driving Cars [Published in Public Understanding of Science] [Preprint][SAGE]
Applications in artificial intelligence such as self-driving cars may profoundly transform our society, yet emerging technologies are frequently faced with suspicion or even hostility. Meanwhile, public opinions about scientific issues are increasingly polarized along the ideological line. By analyzing a nationally representative panel in the United States, we reveal an emerging ideological divide in public reactions to self-driving cars. Compared with liberals and Democrats, conservatives and Republicans express more concern about autonomous vehicles and more support for restrictively regulating autonomous vehicles. This ideological gap is largely driven by social conservatism. Moreover, both familiarity with driverless vehicles and scientific literacy reduce respondents’ concerns over driverless vehicles and support for regulation policies. Still, the effects of familiarity and scientific literacy are weaker among social conservatives, indicating that people may assimilate new information in a biased manner that promotes their worldviews.
Same Candidates, Different Faces: Uncovering Media Bias in Visual Portrayals of Presidential Candidates with Computer Vision [Published in the Journal of Communication] [PDF] [Oxford]
How do today’s partisan media outlets produce ideological bias in their visual coverage of political candidates? Applying computer vision techniques, this study examined 13,026 images from 15 news websites about the two candidates in the 2016 U.S. presidential election. The analysis unveils a set of visual attributes (e.g., facial expressions, face size, skin condition) that were adopted by media outlets of varying ideologies to differentially portray these two candidates. In addition, this study recruited 596 crowdsourced workers to rate a subset of 1,200 images and demonstrated that some visual features also effectively shape viewers’ perceptions of media slant and impressions of the candidates. For example, Clinton was portrayed with more expressions of happiness, which rendered her as more favorable, whereas Trump was associated with more expressions of anger, which made him look less positive but more dominant. These differences in facial expressions varied in line with media outlets’ political leanings.
Feast for the Eyes: Effects of Food Perceptions and Computer Vision Features on Food Photo Popularity [Published in the International Journal of Communication] [PDF]
The widely circulated food photos online have become an important part of our visual culture. Combining human ratings of food characteristics and computational analysis of visual aesthetics, we examined what contributed to the aesthetic appeal of a diversity of food photographs (N = 300) and likes and comments they received in an artificial newsfeed from participants (N = 399). The results revealed that people tended to like and share images containing tasty foods. Both healthy and unhealthy foods were able to gain likes. Aesthetic appeal and specific visual features, such as the use of arousing colors and different components of visual complexity, also influenced the popularity of food images. This work demonstrates the potential of applying computer vision methods in visual analysis, offers insights into image virality, and provides practical guidelines for communicating healthy eating.
Time Travel with One Click: Effects of Digital Filters on Perceptions of Photographs [Published in Proceedings of the ACM CHI Conference on Human Factors in Computing Systems (ranked 1st in human-computer interaction)] [PDF] [Slides] [ACM]
Today’s digital photographs are being heavily “filtered.” By simple clicks on mobile apps like Hipstamatic and Instagram, users can easily apply digital filters to their pictures to create effects such as faux-vintage and light leaks. To understand the potential impacts of photo filters, we conducted an online experiment and investigated how the use of the black-and-white and film-style photo filters changed viewers’ perceptions and descriptions of photographs. We found that photo filters substantially increased viewers’ perceived temporal distances to photographs. Participants also tended to describe analogue-style photos more interpretively and tentatively than unfiltered ones, indicating an increase in construal levels. We suggest that the widely used photo filter is not just a tool to change aesthetics; it also adds a layer of history, meaning, and defamiliarization to photographs, allowing users to construct a mental distance in images that deviates from everyday experiences. We offer insights into the psychology of visual styles and implications for designing filter apps and photo-sharing platforms.
The Influence of Weight-of-evidence Strategies on Audience Perceptions of (Un)certainty When Media Cover Contested Science [Published in Public Understanding of Science] [PDF]
Controversy in science news accounts attracts audiences and draws attention to important science issues. But sometimes covering multiple sides of a science issue does the audience a disservice. Counterbalancing a truth claim backed by strong scientific support with a poorly backed argument can unnecessarily heighten audience perceptions of uncertainty. At the same time, journalistic norms often constrain reporters to “get both sides of the story” even when there is little debate in the scientific community about which truth claim is most valid. In this study, we look at whether highlighting the way in which experts are arrayed across truth claims—a strategy we label “weight-of-evidence reporting”—can attenuate heightened perceptions of uncertainty that can result from coverage of conflicting claims. The results of our study suggest weight-of-evidence strategies can indeed play a role in reducing some of the uncertainty audiences may perceive when encountering lop-sided truth claims.