Coding Procedures, Texts Analyzed, and Reliability

This webpage reports the results of studies of 2008 campaign messages. That research has not been published. However, we also provide results from our past research, which has been published, on earlier presidential campaign message. What follows is a description of our method, content analysis. This is followed by a table which lists the message forms we’ve studied, the campaigns included, the number of candidates, the number of individual messages analyzed, and the reliability of our data. Finally, we provide a list of references to the places where this past research has been published.

Procedures for Analyzing Campaign Message Content

Three steps were used in the content analysis of these campaign messages. First, the candidates’ statements in a message were unitized into themes, the unit of analysis. Themes are referred to throughout this text as “utterances,” “comments,” “statements,” “remarks” or “claims.” Berelson (1952) defined a theme as “an assertion about a subject” (p. 18). Similarly, Holsti (1969) stated that a theme is “a single assertion about some subject” (p. 116). In this study a theme is basically a argument or a claim about the candidates. Because talk is elliptical, themes can be as short as a phrase or as long as several sentences.

The second step in the procedure codes (classifies, content analyzes) each theme’s function, using these rules:

Acclaims portray the candidate in a favorable light.
Attacks portray the opposing candidate in an unfavorable light.
Defenses respond to attacks, attempting to repair the candidate’s reputation.

Finally, coders classified the topic of each theme:

Policy statements concern governmental action (past, current, or future) and problems amenable to governmental action.
Character statements address characteristics, traits, abilities, or attributes of the candidates.

The texts were content analyzed by two coders. Training began with consideration of previous research employing the functional approach. Definitions and examples of each category from political campaign texts were supplied to coders. A set of coding rules explained how to apply these categories to texts. Coders practiced coding related texts. Intercoder reliability was calculated on a subset of 10% of the texts. You can see information about the sample of presidential messages and reliability here.

Primary and general TV spot data from Benoit (1999) and Benoit, McHale, Hansen, Pier, and McGuire (2003).

Primary debate data from Benoit, Pier, Brazeal, McHale, Klyukovksi, and Airne (2002) and Benoit, McHale, Hansen, Pier, and McGuire (2003).

Primary and general brochure data from Benoit & Stein (2003).

Acceptance address data from Benoit, Wells, Pier, and Blaney (1999) and Benoit, McHale, Hansen, Pier, and McGuire (2003).

General debate data from Benoit and Brazeal (2002); Benoit and Harthcock (2000b); Benoit and Wells (1996), Benoit, McHale, Hansen, Pier, and McGuire (2003); and Wells (1999).

Radio spot data from Blaney (2000) and Benoit, McHale, Hansen, Pier, and McGuire (2003).

Analyzing News Coverage

We followed the procedures developed for our study of New York Times coverage of campaigns (Benoit, Stein, & Hansen, 2005).  We categorized each theme into one of six basic topics: policy and character (from Functional Theory) as well as horse race, voters’ reactions, scandal, and election information. We further analyzed each theme of horse race coverage into one of eight sub-categories: strategy, campaign events, polls, predictions, endorsements, vote choice, fund raising, and spending. Themes that were reports of candidate’s defenses were coded as such; themes that were evaluative (rather than descriptive) were coded as positive (acclaims) or negative (attacks). Finally, statements in a news story were counted as unattributed (essentially from the reporter), from the candidate, from a supporter, or from another source.

References:

Benoit, W. L. (1999). Seeing spots: A functional analysis of presidential television advertisements from 1952-1996. New York: Praeger.

Benoit, W. L., Blaney, J. R., & Pier, P. M. (1998). Campaign ‘96: A functional analysis of acclaiming, attacking, and defending. New York: Praeger.

Benoit, W. L., Blaney, J. R., & Pier, P. M. (2000). Acclaiming, attacking, and defending: A functional analysis of nominating convention keynote speeches, 1960-1996. Political Communication, 17, 61-84.

Benoit, W. L., & Brazeal, L. M. (2002). A functional analysis of the 1988 Bush-Dukakis presidential debates. Argumentation and Advocacy, 38, 219-233.

Benoit, W. L., & Harthcock, A. (1999). Functions of the Great Debates: Acclaims, attacks, and defense in the 1960 presidential debates. Communication Monographs, 66, 341-357.

Benoit, W. L., McHale, J. P., Hansen, G. J., Pier, P. M., & McGuire, J. (2003). Campaign 2000: A functional analysis of the presidential campaign at the dawn of the new millennium. Lanhan, MD: Rowman & Littlefield.

Benoit, W. L., Pier, P. M., Brazeal, L., McHale, J. P., Klyukovski, A., & Airne, D. (2002). The primary decision: A functional analysis of debates in presidential primaries. Westport, CT: Praeger.

Benoit, W. L., Stein, K. A., & Hansen, G. J. (2005). New York Times coverage of presidential campaigns, 1952-2000. Journalism & Mass Communication Quarterly, 82, 356-376.

Benoit, W. L., & Wells, W. T. (1996). Candidates in conflict: Persuasive attack and defense in the 1992 presidential debates. Tuscaloosa: University of Alabama Press.

Benoit, W. L., Wells, W. T., Pier, P. M., & Blaney, J. R. (1999). Acclaiming, attacking, and defending in nomination convention acceptance addresses, 1960-1996. Quarterly Journal of Speech, 85, 247-267.

Berelson, B. (1952). Content analysis for the social sciences and humanities. Reading, MA: Addison-Wesley.

Blaney, J. R. (2001). Radio and the functional theory of political communication. Paper presented at the Central States Communication Association, Cincinnati, OH.

Holsti, O. (1952). Content analysis in communication research. New York: Free Press.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrica, 33, 159-174.

O’Keefe, D. J. (1977). Two concepts of argument. Journal of the American Forensic Association, 13, 121-128.