The goal of establishing computational rhetorics as a field of research and practice is to use computer algorithms to find rhetorical moves in discourse. While any interpretive method is heuristic in nature, relying on the meaning-making orientations of the analyst, my hope is that computers can do much of this global, exegetical work themselves, leaving humans to do more fine-grained readings. In pursuit of this project, I have created an app called the Hedge-O-Matic.
The Hedge-O-Matic uses Naive Bayes machine-learning algorithms to classify tokenized sentences as either a hedge or a non-hedge with confidence ratings. In this blog post, I test the Hedge-O-Matic on the recently released Wells Report, which details the findings of the National Football League’s investigation into the workplace harassment of Miami Dolphins tackle, Jonathan Martin by guard Ritchie Incognito and other members of the Miami Dolphins organization.
What do I mean by hedge?
I define a hedge as a rhetorical move intended to make a statement less definitive in order to qualify claims or mitigate controversy. For this post, I further divide hedging into two moods: interpersonal hedges and propositional hedges. An interpersonal hedge would involve those expressions that dampen the certainty of an assertion through a qualification. For example, “I don’t like the color scheme of this site, but that’s just my personal opinion.” Such a criticism could be hurtful to this blogger, so the potential critic characterizes the opinion as something more idiosyncratic than unilateral. The motivation here is to ward off bad feelings.
A propositional hedge seeks to qualify an argument made about reality, in which the evidence for the claim is less than certain. For example, a scientist studying the effect of vitamin C on the common cold might say, “In 75% of trials, the introduction of vitamin C showed a decrease in the contraction of the common cold, but more studies need to be conducted.”
At this time, the Hedge-O-Matic is primarily trained on propositional hedges that would normally be found in formal or academic writing as opposed to interpersonal communications.
How the Hedge-O-Matic Works
The Hedge-O-Matic is a typical Bayesean classifier that tags discourse according to what it determines is the most likely category based on a given set of features. The following outlines the method:
- Each sentence of the dataset is manually coded as either a hedge or non-hedge
- These sentences are normalized so that function words such as prepositions, articles, and punctuation are removed
- Key features are extracted from the hedge and non-hedge sets and trains the Naive Bayes algorithm
- The unseen or “test” data undergoes the same normalization and feature set extraction as the training set
- The features of the test data are compared to the training data and scored.
- If the features of the test sentence are predominantly shared by those of the existing hedge training set, the test sentence is classified as a hedge. If the predominant features are shared by the non-hedge training set, then it is classified as a non-hedge.
Currently, the training set for the Hedge-O-Matic is comprised of 150+ academic, science articles. This choice of genre has a significant influence on the results of the Hedge-O-Matic. Because of the conventions used in academic writing, the Hedge-O-Matic works best when testing writing of similar genres and styles. It works less well with interpersonal dialogues on an online discussion board or with novels or poetry.
When tested with unseen text from academic, science journals, the Hedge-O-Matic has an accuracy range of 76-82%.
The Wells Report
The Wells Report is a 144 page document commissioned by the National Football League in its efforts to investigate the workplace harassment of Miami Dolphins tackle, Jonathan Martin. I chose the Wells report because of its availability and because I presumed that the legalistic genre of the document would contain a high incidence of hedging moves as investigators and witnesses measured their claims.
I inputed the full text of the Wells report minus the table of contents and the index. Thus, headers and sub-headers are present in the data. You can download the full output of the Hedge-O-Matic here as a csv file. In the remainder of this post, I will highlight particular aspects of the output and suggest future directions of this computational rhetorics project
After processing, the Hedge-O-Matic found that the Wells report consists of 1568 sentences, 52% of which are non-hedges and 48% are hedges. This suggests that the Wells Report is a particular hedgey document, but I would caution people here to not overdetermine these findings (and, yes, I am hedging here). As I have mentioned, the Hedge-O-Matic is trained on academic, science articles, not legal documents, so many of the features that inform the training set might not apply to the Wells Report. In addition, at this stage of development, the Hedge-O-Matic only assigns designations of “hedge” and “non_hedge” to sentences. This means that more neutral sentences that the Hedge-O-Matic is 50% sure of will take one of these designations.
That said, the Hedge-O-Matic is particularly confident about certain kinds of statements found in the Wells Report. for example:
|13||Incognito contends that the conduct about which Martin complains was part of locker room banter meant in good fun and that Martin was a willing and active participant in verbal sparring, never letting on that he was hurt by it.||hedge||0.812468602|
|15||According to our consulting expert, a psychologist who focuses on matters of workplace conduct, such a reaction is consistent with the behavior of a victim of abusive treatment.||hedge||0.806094195|
|28||That the same taunts might have bounced off a different person is beside the point.||hedge||0.766130995|
We see in sentence 13 an instance of claim being reported. The word “contends” signals an argument that has yet to be proven, and, is, at least in my coding bias, a hedge. Moreover, in sentence 15, the phrase, “such a reaction is consistent” issues a comparison that calls for readers to judge the probability that reaction is that of abused victim because of similarity. This contrasts with statements such as, “This behavior is a result of abuse.”
Sentence 28 is more complicated. The Hedge-O-Matic is not so confidence that it is a hedge, and with good reason. In my reading, it is not a hedge. The authors of the Wells Report are clear in their judgements. What is leading the Hedge-O-Matic astray is the use of the modal “might,” which often is used in hedging maneuvers.
As for non-hedges:
|37||After learning of the existence of a voicemail message that Incognito had left for Martin months earlier, in which Incognito used a racist slur and made vulgar taunts, the Dolphins suspended Incognito.||non_hedge||0.612489107|
|42||We received full cooperation from the NFL, the National Football League Players Association (or “NFLPA,” the union that represents NFL players) and the Miami Dolphins.||non_hedge||0.579707853|
|43||We interviewed every then-current Dolphins player and coach as well as the team’s management, including the owner and Chairman, Stephen Ross, and former General Manager, Jeff Ireland—more than 100 interviews in In addition to Mr. Wells, the Paul, Weiss team included the following attorneys: Brad S. Karp, Bruce Birenboim, David W. Brown, Maria H. Keane, Kira A. Davis, Moira E. Weinberg, Marques S. Tracy, 6 total.||non_hedge||0.991719362|
Not surprisingly, the non-hedgey sentences here encompass descriptions of actions taken or previously recorded. Sentence 37 is a matter of fact established by the Dolphin’s actions. Sentences 42 and 43 are clear descriptions of actions conducted by the Wells Report investigators. Interesting, but perhaps not-surprising, level of cooperation obtained by the Wells report from the 3 organizations with the most stake and power in the proceedings–the NFL, the NFL Player’s Union, and the Miami Dolphins–is described in non-hedgey declarative language.
What else might we claim about how actors are described in this report?
My research into this document is only preliminary and is meant to be suggestive of what computational rhetorics can accomplish. However, I would like to draw people’s attention to the latter parts of the document that account for the actions of Dolphins Head Coach, Joe Philbin.
|1480||After learning about the April 6 voicemail, Coach Philbin and the Executive Vice President of Football Administration had a meeting with Incognito, during which Incognito admitted he had left the message in question and that he had made other vulgar or racially charged statements to Martin.||non_hedge||0.965031663|
|1481||Reportedly, Incognito also said that profane language was common in the locker room and that admonishments from 136 Coach Philbin “would not change the way we’re going to speak to one another.” By this time, Coach Philbin also had learned about the fine book and Martin’s payment of a hefty “fine” for having skipped the Las Vegas trip.||hedge||0.615453719||1482||Coach Philbin told us that he was incensed and that he told Incognito that such conduct was “absolutely ridiculous” and “not The Dolphins then issued another press release, announcing that they had received notification from Martin’s representatives “about allegations of player misconduct” and had asked the league to conduct a review.||non_hedge||0.923698271|
|1483||At approximately 11:30 pm on the evening of November 3, Philbin informed Incognito of his indefinite suspension for “conduct detrimental to the team.” The Dolphins then issued their third press statement of the day to announce Incognito’s suspension and to affirm that the team “believe[d] in maintaining a culture of respect for one another.” On Monday, November 4, ESPN published a transcript of Incognito’s April 6 voicemail message.||non_hedge||0.882715736|
|1484||The same day, in a press conference, Coach Philbin took responsibility for the atmosphere in the Dolphins workplace.||non_hedge||0.969616452|
|1486||He stated that he took allegations of harassment seriously and would not tolerate “any type of conduct and behavior that detracts from” players’ ability Initially, Dolphins players did not comment publicly about Martin’s departure from the team, evidently because they had been instructed not to discuss the situation with the press.||non_hedge||0.669376853|
Without making generalization about the power structure of the NFL and the Miami Dolphins or the agenda of the Wells Report investigators, let me say that it is intriguing to me how the words of certain institutional actors in this scandal are framed by the Wells Report, and that those experiencing the most precarity in their employment situations tend to be ascribed more hedgey behavior. The distinction is clear when comparing sentence 1,481 with the remaining sentences in that section. Incognito’s actions in this bit of narrative are reputed. Compare that with the actions of Philbin and the Dolphin organization. Philbin “took responsibility.” The Dolphins “issued” their press release. Philbin “told” the investigators.
To be clear, there are numerous instances in the report in which Jonathan Martin and Ritchie Incognito’s communications are framed in non-hedgey ways. And a more thorough analysis of the findings is merited as well as a reconfiguration of the Hedge-O-Matic’s sentence boundary detectors. Those who examined the csv output will note that many cells are filled with multiple sentences. This is a result of 2 things: inconsistent punctuation in the source text and periods within quotations marks not breaking upon tokenization.