User Engagement With Online News: Conceptualizing Interactivity and Exploring The Relationship Between Online News Videos and User Comments [PDF]

Citation preview

545073 research-article2014



NMS0010.1177/1461444814545073New Media & SocietyKsiazek et al.



Article



User engagement with online news: Conceptualizing interactivity and exploring the relationship between online news videos and user comments



new media & society 2016, Vol. 18(3) 502­–520 © The Author(s) 2014 Reprints and permissions: sagepub.co.uk/journalsPermissions.nav DOI: 10.1177/1461444814545073 nms.sagepub.com



Thomas B Ksiazek Villanova University, USA



Limor Peer



Yale University, USA



Kevin Lessard



Villanova University, USA



Abstract With the emergence and rapid acceptance of online news come new and varied opportunities for user engagement with content, along with alternative metrics for capturing those behaviors. This study focuses on interactive engagement with online news videos. We propose a theoretical framework for conceptualizing user engagement on a continuum from exposure to interactivity. Furthermore, we make a distinction between user–content (e.g. commenting) and user–user (e.g. replying to another user’s comment) modes of interaction. We then explore publicly available measures of these concepts and test a series of hypotheses to explore commenting and conversational behaviors in response to YouTube news videos. We conclude by discussing the theoretical and practical implications for advancing our understanding of user engagement with online news.



Corresponding author: Thomas B Ksiazek, Department of Communication, Villanova University, Garey Hall B2a, 800 Lancaster Avenue, Villanova, PA 18940, USA. Email: [email protected]



Ksiazek et al.



503



Keywords Digital journalism, engagement, interactivity, news videos, online news, user comments, YouTube



The news media landscape has undergone drastic changes over the past decade, as digital platforms are creating unprecedented competition for all legacy, or traditional, media (i.e. newspapers and magazines, as well as broadcast TV and radio news). All of this is part of a broader trend toward convergence in media industries, defined by features such as cross-platform availability of content, less organizational control over production and distribution, greater user empowerment, lower barriers to entry, and increased participation and interactivity among consumers and producers of content. In terms of sustainability and audience building, some (Hayes, 2008; Webster, 2011) describe this industry transition as shifting from a “push” to “pull” model of mediated communication, especially for video content. The suggestion is that producers in the past relied on a linear model and would simply push content through a limited number of distribution channels with the expectation that users would pay attention. Now, producers operate in a highly competitive and nonlinear environment, characterized by features such as time-shifting technologies and “on demand” access that enable greater user control over when, where, and how they access media. In this context, news organizations must do more to pull users in, offering something not only more useful, informative, compelling, and gratifying than their competition, but also more engaging. Interactivity is one of the unique defining features of engagement with online news (Deuze, 2003). MacGregor (2007) argues, “To engage is a new value, but not really a news value. This focus on the quality of interaction heralds change” (p. 293). As such, indicators of interactive engagement are complementing more traditional measures of exposure (circulation, ratings, etc.) in understanding the behavior of news audiences. The notion of engaging users by encouraging them to interact with both content and other users has had substantial traction in marketing and advertising circles for some time, not least because these activities can now be captured and examined via social media tools (Napoli, 2011; Peterson and Carrabis, 2008). With the challenges facing many news organizations, the ability of journalists to offer engaging content to attract, or “pull,” users would seem to be more important than ever. This study aims to better understand indicators of engagement with online news, specifically the commenting and conversational practices of users in response to online news videos. In what follows, we clarify the concept of engagement and propose a framework for conceptualizing interactivity and its centrality to the understanding of engagement with online news. Next, we explore various indicators of user behavior to illustrate the usefulness of interactivity as an organizing principle. This involves disentangling popularity and exposure metrics from measures of interactivity, each of which constitutes a differing level of engagement. We draw on publicly available online behavioral data to explore the relationships among these metrics, and we integrate these data with a content analysis of news videos to analyze the relationships between content



504



new media & society 18(3)



features and interactivity. We contrast commenting practices between “hard” and “soft” news, “objective” and “biased” news, and professional and citizen journalism. The central goal of this study is to further our understanding of the diverse ways that users experience and interact with the news.



User engagement and interactivity Engagement is a broad phenomenon that describes all sorts of user attention and involvement with media (see Napoli, 2011: 97–98). At its most basic level, it begins with exposure, but most perceive engagement as constituted of both psychological and behavioral experiences (Brodie et al., 2011; Calder et al., 2009; Malthouse and Calder, 2011; Napoli, 2011; Oh et al., 2010), and it can be a property of the users, the medium, or both (O’Brien and Toms, 2008). A common thread in the research on engagement is the involvement, real or perceived, of the user in either producing, consuming, or disseminating information (Deuze, 2003; Hargittai and Walejko, 2008; Jenkins, 2006; Napoli, 2011; Oh et al., 2010). This involvement is often expressed as interactivity, which is understood as a fundamental component of the broader phenomenon of engagement. Research on interactivity often highlights the multi-directional flow of information. The interactive nature of online media enables the audience to not only receive information but also disseminate and remake it (Jenkins, 2006; Thurman, 2008). Consistent with this line of thinking, Schultz (1999) defines interactivity as a “chain of interrelated messages” in a two-way or reactive system of communication that allows for input from individuals on all sides of the message. Recent research has applied a uses and gratifications perspective to understand the drivers of interactivity. Ruggiero (2000) notes the potential for interactive media to fulfill certain psychological needs. Subsequent empirical research finds three primary motivations for interactivity: information seeking, socialization or social interaction, and entertainment. This work further distinguishes motives for user–content and user–user interactivity (discussed below), where the former is driven by the need for information and the latter is motivated by the need for social interaction (Chung and Yoo, 2008; Ko et al., 2005; Yoo, 2011). Additionally, a need for entertainment appears to drive both forms of interactivity (Chung and Yoo, 2008; Yoo, 2011). This theme is also central to the work of Clay Shirky (2010), who argues that the Internet enables new forms of collaboration as it affords people an easy entrée into cultural production, as opposed to mere consumption. Regarding online journalism specifically, Schmitz Weiss and De Macedo Higgins Joyce (2009) suggest the Internet offers unique opportunities for interactivity because it “allows for a closer relationship with the audience; a shortened social space” between media producer and consumer (p. 593). Related, immediacy is a primary characteristic of interactivity in online news media, one that allows users to publicly register their response to a news article or video (Lange, 2007). Commenting on news stories is a way users can engage with content that they did not have available to them until recently. We argue that these comments, as well as replies from other users to already posted comments, can be seen as representative of high-level interactive engagement between the user and content and between the users themselves, respectively.



Ksiazek et al.



505



We distinguish interactivity with online news videos in two broad ways, building on theoretical work with interactivity in a variety of online contexts (Chung, 2008; Chung and Yoo, 2008; Kiousis, 2002; McMillan, 2002, 2005; Massey and Levy, 1999; Nagar, 2011; Schultz, 2000; Stromer-Galley, 2000; Yoo, 2011). User–content interactivity involves a user interacting with content and producers, such as posting an initial comment to a video thread. This represents a basic form of feedback for the content creator. Alternatively, user–user interactivity consists of interactions among two or more users, such as a user replying to another comment already posted on the video thread by a different user. This back and forth among viewers of the news video is representative of a dialogue, or conversation, between the commenters. Both forms of interactivity signal a highly engaged user, where the individual is not just exposed to content, but demonstrates evidence of cognitive processing. The practice of commenting highlights an active user that is challenging, supporting, or at the very least reflecting on the news. While it is one thing to simply read or watch a news story, making the decision to publicly contribute your reaction or opinion in response to the story indicates an individual that is more invested, aware, and attentive—in other words, more engaged—with the content. To sum up, we view the ability to act, interact, and co-create online, as a key characteristic of online media, which distinguishes it from other media platforms. We theorize a continuum of engagement, from exposure to interactivity, where more (quantity) and better (quality) ways to interact with content and with other users indicate deeper engagement. We now turn to a discussion of how these concepts are measured.



Measuring media use: From exposure to engagement Despite the growing body of work on engagement and interactivity with the news, the vast majority of research on news consumption has traditionally relied on rather simplistic measures of raw exposure, with user behavior captured by gross metrics like ratings, circulation, and, more recently, page views and the like. Extending this line of inquiry, we argue that interactivity is distinct from popularity (i.e. exposure), though it is related. We agree with Beckett (2010) that engagement is a composite of various metrics of exposure and interactive behaviors, but make the case for thinking about engagement as a continuum that ranges from less (popularity or exposure) to more (interactivity) user engagement. Research on audience behavior has long been interested in examining general exposure to media, both from a practical standpoint for its producers and also for the theoretical implications it brings forth. The delivery of news online contributes to a more refined measurement of usage for two reasons: (a) it allows measurement at the level of an individual story rather than the entire news product (e.g. newspaper, television news broadcast), and (b) it provides more precise ways to measure whether a specific piece of content was consumed. These new measures of usage, which aim to capture popularity in some form, typically consist of raw measures of exposure (e.g. page views, traffic, hits, etc.). For instance, Chatzopoulou et al. (2010) treat the number of views of a YouTube video as a basic index of popularity. While these measures of exposure are certainly important, they are limited in what they tell us beyond whether or not a user encountered the content. Without some measure of engagement, we cannot assess how users respond to what is in front of them. In the



506



new media & society 18(3)



past, researchers relied on surveys, self-reports, or letters to the editor, but the Internet has dramatically broadened the range of behaviors vis-a-vis content: emailing, sharing, “liking,” recommending, social bookmarking, and so on. Indeed, we are increasingly seeing the integration of traditional exposure metrics and new indicators of engagement to provide a more complete picture of user behavior (Napoli, 2011). We distinguish here between “viewing” and other common popularity indicators, such as “rating,” “ranking,” “favoriting,” and “liking.” These behaviors suggest a different level of user experience, one that involves some action on the part of the user (Deuze, 2003). While viewing is a classic measure of exposure, the other metrics essentially operationalize the concept of recommendation. Moreover, while all forms of online use require some activity, we argue that viewing requires minimal interactivity and does not necessarily signal engagement. Instead, exposure is but one aspect of engagement. Likewise, recommendation metrics capture very basic user activity (i.e. clicking), but it is questionable whether they represent deeper engagement. Interactive features available online, such as direct feedback, comment sections, and interactions with other users, allow users to take a much more active (and interactive) role in the user experience, and thus they clearly suggest deeper engagement with the content. Like page views, these features are publicly available and can be tracked, making it possible to more accurately measure these types of social behaviors. Recent scholarly work is beginning to take advantage of publicly available user comments and highlights a growing interest in this phenomenon. Nagar (2011) analyzed user comments on mainstream news sites as an expression of public opinion and forum for the surveillance and exchange of ideas. Larsson (2011) studied commenting as one of several interactive features available on Swedish newspaper websites, finding the option varies across types of users in terms of appreciation and use. For instance, “Prosumers” were active commenters, while “Lurkers” were more likely to read others’ comments without partaking. Hujanen and Pietikäinen (2004) studied commenting as a new mode of participation, although they found that Finnish youth rarely engaged in this practice when visiting news websites. This work exemplifies a new trajectory in research on news consumption that focuses on new ways of understanding user behavior. As digital platforms afford us greater ability to track behavior in more subtle ways, we can disentangle popularity and interactivity, examine the relationship between them, and better understand these alternative measures of engagement. We proceed to analyze several different indicators of popularity (from viewing to recommendation metrics), along with the two modes of interactivity, user–content and user–user. The analysis will shed light on the relationships among interactivity and popularity metrics, as well as the role that certain content features may play in helping to explain these measures of user behavior. In the next section, we outline three sets of hypotheses. The first deals with the relationship between popularity and interactivity, while the last two make inferences about the role of content in explaining interactivity.



Research hypotheses The first goal of our study is to examine the two types of interactive engagement (user– content and user–user) in relation to broad patterns of popularity. As both types are increasingly integrated in the study of user behavior, we propose two hypotheses to



Ksiazek et al.



507



better understand the relationship between popularity and interactivity. The predicted relationships are based on a common notion that more popular content is also more engaging. Siersdorfer et al. (2010) focused on the comment sections of YouTube videos, but with an emphasis on popularity (i.e. “likes”) of comments, ultimately finding that positive comments are associated with greater popularity. Studying the behavioral characteristics of users on the social networking site Digg, Jamali and Rangwala (2009) also link interactivity with popularity ratings; they found that the time of the comment (i.e. time passed since original post) as well as the actual word length of a post or comment was related to high levels of popularity. Lee et al. (2010) proposed a model to predict the popularity of online content derived from the lifetime of threads as well as the number of comments on such threads, where longer lifetime/more comments leads to greater popularity. Taken together, these studies offer empirical justification consistent with the theoretical link between popularity and interactive engagement. Therefore, we propose the following hypotheses: H1a: There is a positive relationship between popularity and comments (user–content interaction). H1b: There is a positive relationship between popularity and conversation (user–user interaction). A secondary goal of this study is to explore the relationship between interactivity and a variety of content characteristics. In the following sections, we review two primary distinctions made in the research on journalism content, hard/soft news and objective/ biased news, and propose additional hypotheses to explore the relationship between interactive engagement and these content features.



News: hard versus soft In recent studies of news content, the provision and selection of hard versus soft news is an area that is garnering a great deal of research attention (Baum, 2002, 2003, 2004; Boczkowski, 2009; Boczkowski and Peer, 2011; Gans, 1980; Hamilton, 2004; Patterson, 2003; Prior, 2003; Schaudt and Carpenter, 2009; Tewksbury, 2003; Zaller, 2003). For instance, Boczkowski and Peer (2011) found a “choice gap,” where journalists provide more hard news than users seem to desire. Despite this gap, other research suggests that news organizations are still trending toward offering more soft news, overall. Hamilton (2004) argues that profit motivations lead to more soft news across all types of news organizations, where the quest for marginal viewers that prefer soft news provides an economic explanation for the rise in soft news over recent years. As we witness the “softening of news” (Boczkowski and Peer, 2011), many wonder about the social and political implications of less public affairs content. For instance, will this lead to a less politically involved citizenry? What about the impact on informed voters? How does this contribute to or detract from the health of a deliberative democracy? To date, most research on hard/soft news has explored content (subject and format), production, antecedents, and outcomes. Comparing the frequency of comments and conversation posted to hard and soft news videos would offer an additional contribution to



508



new media & society 18(3)



this body of research, specifically as it relates to the generative deliberation prospects of each content type. Based on past evidence of a preference for soft news among audience members, and assuming a positive relationship between popularity and interactivity, we should see a positive relationship between more popular content (i.e. soft news) and both types of interactive engagement. We propose the following hypotheses: H2a: Soft news videos will exhibit more comments (user–content interaction) than hard news videos. H2b: Soft news videos will exhibit more conversation (user–user interaction) than hard news videos.



Views: Bias versus objectivity In addition to exploring hard and soft news content, we also compare interactivity across several indicators of bias and objectivity in news videos. The “objectivity norm” has been a feature of modern journalism since the late 19th and early 20th century (Schudson, 2001). Recently, however, many scholars wonder whether market forces and economic imperatives resulting from an environment of increased competition, niche content, and declining profits will lead to a shift away from objective news. A recent study of online news videos showed that biased content was more popular than objective content (Peer and Ksiazek, 2011). Similar to the previous hypotheses, this suggests that biased news videos, because they are more popular, should also exhibit a positive relationship with interactive engagement. Thus, we offer the following hypotheses: H3a: Videos that break from the objectivity norm will exhibit more comments (user– content interaction). H3b: Videos that break from the objectivity norm will exhibit more conversation (user–user interaction).



Method In order to test these hypotheses, we collected a sample of YouTube news videos over a 3-month period. The resulting dataset includes measures of popularity and commenting behaviors, as well as hand-coded content characteristics for a subsample of the videos. The following sections describe the sampling and data collection, details of the specific measures used in this study, and the analytical procedures used to test the hypotheses.



Sampling and data collection The dynamic nature of the Web makes it extremely difficult to select a purely random sample of news videos. To cope with this, we first chose a single site to study, as even a sample of sites would not be random. YouTube seemed a natural choice, as it has long offered various measures of user behavior (Burgess and Green, 2009; Cheng et al., 2008; Jarboe, 2011). It is the most popular online video property in the United States, with over 136 million unique viewers—a figure that is about three times that of the next closest competitor—and



Ksiazek et al.



509



about 16.5 billion total streams during May 2012 (NielsenWire, 2012, 22 June). It is also an aggregator of video content, thus offering a site that includes videos from a variety of news outlets throughout the world, both professional and amateur (Pew Research Center, 2012). Next, we sampled a composite week, a form of cluster sampling that is common in content analysis of television programs. We set our parameters for March–June 2008 and selected a random Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, and Saturday during that time frame. We crawled the pages of the top 100 videos in YouTube’s “News” category, ranked by number of views, for each of the 7 days. These videos include both repurposed news clips (e.g. a 60-Minutes segment uploaded to YouTube) along with videos made exclusively for the web, which are often produced by amateur or citizen journalists. We collected publicly available popularity and interactivity metrics for each video. This provided our initial dataset of 700 news videos to test H1a and H1b. We also collected a smaller subsample of videos that consisted of the top 40 news videos for 5 days in the March–June 2008 time frame.1 These 191 videos were handcoded for a variety of content and production characteristics, and crawled for popularity and interactivity metrics.2 These data were used to test H2a, H2b, H3a, and H3b.



Measures In order to test the first two hypotheses about the relationships between popularity and interactivity, we used five variables measured for all 700 videos. To measure the popularity of each video, we used three indicators that are publicly available on YouTube: Number of Views is a measure of raw exposure to a video (i.e. the number of times a video has been viewed), Number of Favorited captures the number of times a video is marked as a favorite, and Number of Ratings indicates the number of times a video received a rating. To gauge the level of interactivity for each video, we used two variables. Number of Comments represents the total number of comments posted directly to a video (i.e. by entering text in the “Respond to this video” text box) and indicates the degree of user–content interactivity for a given video. To measure user–user interactivity, we captured the number of replies within the set of comments for each video. A reply to an existing comment is registered when a user chooses to click the “Reply” button associated with a particular comment rather than post a comment as described above. Instead of using the absolute number of replies, we created a Ratio of Replies variable, which represents the proportion of replies to total comments for a given video, and thus offers a more conservative measure than the raw number of replies.3 Table 1 provides descriptive statistics for all five variables. While the values presented in the table are real values, all variables were log transformed to address skewed distributions before conducting the analyses. To test the remaining hypotheses, we used the same interactive engagement metrics above, first comparing them across hard and soft news videos, then between biased and objective videos. Four journalism graduate students coded the content of the subsample of videos. The students received extensive training and were closely supervised. They followed a strict protocol and achieved an overall percent agreement score of 82% and a Kn score of .72 (see Peer and Ksiazek, 2011).4 Videos were classified as hard or soft news by recoding an existing “content topic” variable in our dataset. Hard news consisted of the following topics: politics, US elections,



510



new media & society 18(3)



Table 1.  Descriptive statistics. Min   Number of Comments (User–content interactivity) Ratio of Replies (User–user interactivity) Number of Views Number of Favorited Number of Ratings     Number of Comments (User–content interactivity) Ratio of Replies (User–user interactivity)



.00 .00 201.00 .00 .00



.00 .00



Max



Mean



SD



Popularity ←→ Comments N = 700 3375.00 78.013 213.61 .84



.19



.18



772,637.00 12,083.47 37,832.76 2209.00 31.32 142.70 3035.00 72.44 174.94 Content ←→ Comments N = 191 3375.00 145.8115 337.17332 .64



.1879



.15327



SD: standard deviation. Note: fairness, sourcing, and agenda are nominal variables and thus are not included.



economics and business, foreign affairs, War on Terror, crime, and natural disasters. Soft news included the following topics: sports, arts, entertainment, culture, science, technology, health, medicine, fitness, weather, sex, animals, and weird/humorous. Videos were characterized as biased or objective based on three hand-coded variables: Fairness, Sourcing, and Agenda. Fairness indicates whether the video presents more than one side of the relevant issue, with multiple sides indicating objectivity. Sourcing asks whether the video uses an outside/secondary source for information, balance, or an alternative point of view, where the use of sources is an indicator of objectivity. Agenda asks coders to determine whether the video seems overly opinionated or seems to be promoting an agenda, where the lack of an agenda characterizes more objective videos.5



Analysis The unit of analysis in our study is the video, such that each popularity and interactivity metric as well as all content variables are measured for each individual video. To test H1a and H1b, we analyzed the correlations among the popularity and interactivity measures. To test the other hypotheses, we used t-tests to compare the mean scores for both interactivity variables across different types of content. For H2a/H2b, we compared the mean number of comments and ratio of replies between hard and soft news videos. For H3a/H3b, we did the same between objective and biased videos, using fairness, sourcing, and agenda as our indicators of the relative objectivity of a video.



Results The first two hypotheses predict a positive relationship between popularity and commenting (H1a), as well as conversation (H1b). Hypothesis 1a is supported (see Table 2).



511



Ksiazek et al.



Table 2.  Pearson correlation matrix for popularity, user–content interactivity, and user–user interactivity variables (n = 700 videos).



Number of Comments Ratio of Replies Number of Views Number of Favorited Number of Ratings



Number of Comments



Ratio of Replies



–



−.500* –



Number of Number of Number of Views Favorited Ratings .470* −.181* –



.511* −.215* .522* –



.594* −.207* .527* .663* –



*p