I situate my research at the intersection of emerging media, media effects, and computational social science. In particular, my scholarship addresses several important transitions in today’s digital media environment. First, today’s media landscape is characterized by an increasing prevalence of visual media. My research is among the pioneering attempts in the field of communication to apply cutting-edge computer vision methods, such as facial recognition and computational aesthetics, to investigate the production, diffusion, and effects of digital visual media.
In addition, in today’s high-choice media environment, the success of a media message critically depends on its likelihood of inviting audience attention in the first place and subsequent sharing. Through large-scale web scraping or collaboration with industrial companies, my second line of research uses “digital footprints” left by digital media users to identify factors that affect information propagation.
Applying computer vision techniques such as facial recognition and emotional analysis, this study examines 13,026 images from 15 news websites about the two candidates (Clinton and Trump) in the 2016 U.S. presidential election. By comparing visual coverage of these two candidates in media outlets across the liberal–conservative spectrum, this study revealed the specific visual representations adopted by partisan media to reflect their ideological positions. In addition, this study also recruited crowdsourced workers to investigate which visual features could indeed influence audience perceptions of candidates. This research contributes to our field by simultaneously advancing theories in visual bias as well as methods of analyzing visual data on a large scale.
Feast for the Eyes: Effects of Food Perceptions and Computer Vision Features on Food Photo Popularity [Published in the International Journal of Communication] [PDF]
The widely circulated food photos online have become an important part of our visual culture. This study combined human ratings of food characteristics and computational analysis of visual aesthetics to investigate factors that contributed to the aesthetic appeal of a diversity of food photographs and likes and comments they received in a newsfeed from participants. This work demonstrates the potential of applying computer vision methods in visual analysis, offers insights into image virality, and provides practical guidelines for communicating healthy eating.
Time Travel with One Click: Effects of Digital Filters on Perceptions of Photographs [Published in Proceedings of the ACM CHI Conference on Human Factors in Computing Systems (ranked 1st in human-computer interaction)] [PDF] [Slides] [ACM]
By simple clicks on mobile apps like Hipstamatic and Instagram, users can easily apply digital filters to their pictures to create effects such as faux-vintage and light leaks. To study the potential impacts of photo filters, this study conducted an online experiment and investigated how the use of the black-and-white and film-style photo filters changed viewers’ perceptions and descriptions of photographs. This study offers insights into the psychology of visual styles and implications for designing filter apps and photo-sharing platforms.
The Influence of Weight-of-evidence Strategies on Audience Perceptions of (Un)certainty When Media Cover Contested Science [Published in Public Understanding of Science] [PDF]
Controversy in science news accounts attracts audiences and draws attention to important science issues. But sometimes covering multiple sides of a science issue does the audience a disservice. Counterbalancing a truth claim backed by strong scientific support with a poorly backed argument can unnecessarily heighten audience perceptions of uncertainty. At the same time, journalistic norms often constrain reporters to “get both sides of the story” even when there is little debate in the scientific community about which truth claim is most valid. This experimental study looked at whether highlighting the way in which experts were arrayed across truth claims—a strategy labelled as “weight-of-evidence reporting”—could attenuate heightened perceptions of uncertainty that resulted from coverage of conflicting claims.