Volume 67, Number 3, August 2020 l Technical Communication 5

Applied Research

Practitioner’s Takeaway:

• Technical Communication (TC) statistically corresponded with content related to knowledge and information management and design. Overall, both topics appeared in the sample less frequently than expected. Other research topics; including editing, usability, and design; were also underrepresented throughout the sample, problematizing what

research content practitioners have available to them.

• Compared to the other four journals, TC published the least amount of content directed toward academics. Instead, the journal’s content focused on writers, managers, and designers.

• TC was one of three journals to statistically correspond with content written by multiple authors.

Content and Authorship Patterns in Technical Communication Journals (1996–2017): A Quantitative Content Analysis By Ryan K. Boettger and Erin Friess

Purpose: The maturity of technical communication merits a comprehensive, longitudinal analysis of the content published in its leading journals and the scholars who produce this research. Although reflexive research is common in the sciences and social sciences, few studies have analyzed the body of research in technical communication. Clarity on content and authorship patterns can help position the field for future relevance and sustainability. Method: We conducted a quantitative content analysis on 672 articles published in five leading technical communication journals from 1996–2017. Articles were coded on nine content variables related to primary topic, primary audience, and authorship. We subsequently conducted a correspondence analysis on the variables to identify how specific content areas associated with the journals. Results: Content and authorship patterns were near identical to the patterns found in the field 30 years prior. The journals published content primarily focused on rhetoric, genre, pedagogy, and diversity. In contrast, field-defining topics—usability/ UX, comprehension, design, and editing and style—appeared in the sample less than expected. A majority of research was single-authored and written by female first authors; further, a majority of the first authors had academic affiliations in the United States. Conclusion: Scholars must consider if these content and authorship patterns are the products of deliberate choices and, if so, if this is the field’s inevitable trajectory for the next 30 years. We argue that certain topics are being overproduced while other topics that established the field are being underproduced and, in some cases, being assumed by other disciplines. Keywords: content analysis, correspondence analysis, research, technical communication, technical writing

ABSTRACT

6 Technical Communication l Volume 67, Number 3, August 2020

CONTENT AND AUTHORSHIP PATTERNS

Applied Research

BACKGROUND

State-of-the-discipline studies are common to many fields, including public administration (Lynn & Wildavsky, 1990), public policy (Bunea & Baumgartner, 2014), political science (Kacmar & Baron, 1999), group communication (Frey, 1994), and digital studies (Kirschenbaum & Werner, 2014). These periodic assessments identify the values, boundaries, and research priorities of a particular field over a designated timeframe.

Technical communication merits the same analysis as these other academic disciplines, particularly when it has been suggested that the field lacks a cohesive identity. Rude (2009) noted several reasons for the field’s unformed disciplinary identity, including the placement of our programs (often in traditional English literature-based departments); how we distinguish our questions and methods from other, more established disciplines; and our relative newness as a legitimate academic discipline with its own interconnecting theories and practices (p. 177). In fact, technical communication has been described as a “young” discipline for at least the last 30 years (Blakeslee, 2009; Carver, 1998; Garrison, 2014; Haselkorn, 1997; Hayhoe, 2006; Wahlstron, 1988). This youth has enabled scholars to freeform their definition of technical communication, the content areas that merit investigation, and the methods used to expand its body of knowledge. Rude (2009) extended this observation, pointing out that scholars are often redefining, reenvisioning, or rethinking the field. The consistent use of the prefix re “implies an established identity that should now be modified, but it also reflects a failure to pin down the characteristics” (p. 188). As a result, technical communication is recognized for its diversity, but this diversity has proven “very difficult to define or to circumscribe” (Rainey, 1999, p. 524).

St. Amant and Melonçon (2016) recently argued that technical communication’s inability to define itself has hampered its legitimacy. They described an incommensurability problem in which “nothing seems shared or common” and one that “undermines . . . our power to act, engage, and develop as a field” (pp. 3–4). They suggested that technical communication was “doom[ed] . . . to fail unless we can change the field’s perspective of what we consider common ground” (p. 4). Rude (2009) raised similar concerns

years earlier, motivating her development of four areas of related research questions that could better define technical communication as well as distinguish its scholarship from other disciplines. These four areas included disciplinarity, pedagogy, practice, and social change (p. 176).

Disciplinarity—or, how shall we know ourselves?— is perhaps most relevant to this study, and the research focused on disciplinarity can take many forms. Most of the related technical communication scholarship focuses on research methods (e.g., Boettger & Lam, 2013; Lam & Boettger, 2017; Melonçon & St.Amant, 2018). Rude (2009) acknowledged the value of methods but cautioned that we borrow them from so many other disciplines that their study alone does not always reveal what is unique and disguisable to technical communication. Our study pivots from these methods-driven studies, but we use the results to complement our findings. Instead, we report both the content (or topic) and authorship patterns within technical communication journals over a 22-year period. Rude (2009) too wrote that the study of topics alone could offer little significance without the presence specific research questions; however, we argue that these analyses contribute a more holistic understanding of where technical communication has been, where the field is now, and where the field could go. Further, we believe the scholars who are studying these topics reflect how the field has developed and perhaps how it might need to be redefined.

To respond to these areas, we conducted a quantitative content analysis on a random sample of 672 articles published in the five leading technical communication journals from 1996–2017. This approach and the resulting data is a step toward determining what the field has published in recent decades. This current research aims to assess and contextualize the disciplinarity of the field (and, to a lesser extent, its pedagogy, practice, and social change) through an analysis of both the content of the research and the characteristics of the authors to better frame the field’s “visibility, identity, status, and sustainability” (Rude, 2009, p. 207). To explore these issues, we designed the study to identify the primary content areas, authorship characteristics, and collaboration patterns within these journals over the last 22 years.

Volume 67, Number 3, August 2020 l Technical Communication 7

Ryan K. Boettger and Erin Friess

Applied Research

LITERATURE REVIEW

State-of-the-discipline studies are a critical research component, as they afford the opportunity to assess the health of a field and to identify patterns for comparative assessments. Further, state-of-the-discipline studies that focus on content can identify what is of apparent value to the field. With relatively few publication outlets focused exclusively on technical communication, the content of the research journals is an argument as to what is of value to the field. At present, our five leading journals typically publish 4–6 pieces of scholarship over the course in each of their four issues every year. This only allows 80–100 opportunities to address the content demands and alignment issues that are vital to the future of technical communication.

Additionally, state-of-the-discipline studies that investigate authorship characteristics such as gender and professional affiliation can ascertain the degree to which publications align with a field’s claims of diversity (Eigenberg & Whalley, 2015; Fox et al., 2016; Gomes et al., 2016; Raptis, 1992; Siddiqui, 1997). Understanding a field’s collaborative patterns can frame arguments for the acceptance of collaborative work to promotion and tenure boards who, in some disciplines, have favored sole-authored work over collaborative pursuits (Abbasi et al., 2012; Ezema & Asogwa, 2014; Katz & Martin, 1997; Perianes-Rodríguez et al., 2010). However, in technical communication, little research has analyzed the content areas (or the topics) and authorship characteristics of the field’s research.

Technical communication scholars have only recently begun to reflect on its existing body of research, in part because the field’s age did not provide a sufficient amount of longitudinal data. Current studies typically focus on research methods (Boettger & Lam, 2013; Brammer & Galloway, 2007; Melonçon & St.Amant, 2018). A recent study reported that 37% of articles published in the five leading technical communication journals over a five-year period were empirical (Melonçon & St.Amant, 2018). This study built from an early definition of empirical research, which describes or measures an observable phenomenon in a systematic way (MacNealy, 1999). The coding for this current article also applied this definition. Further, almost 60% of this empirical research was published in either Transactions on Professional Communication or Technical Communication. These results are potentially

relevant to the present study as both journals are affiliated with professional organizations and associated with content that addresses practitioner audiences (Smith, 2000a, 2000b). Similarly, specific content areas are associated with specific research approaches. Scholarship on collaboration and usability/UX were typically empirical, whereas scholarship on rhetoric, pedagogy, and genre were typically non-empirical (Lam & Boettger, 2017).

The content-related studies (studies that assess what areas or topics technical communication covers) produced in technical communication can typically be organized into two categories. The first encompasses a collection of self-reflective studies conducted as integrated literature reviews or anecdotal assessments (Brammer & Galloway, 2007; Fine, 1996; Forman, 1998; Malone, 2007; Rogers, 1995). Although these studies have offered focused examinations into specific content areas and phenomena, they have often done so without citation analyses, scientometrics, or other rigorous and replicable means for assessment. The second category focuses on technical communication doctoral research. Two studies have examined the types of doctoral research produced from 1965–1990 and 1989–1998, respectively, and found emphasis on pedagogical, rhetorical, and compositional areas (Rainey, 1999; Rainey & Kelly, 1992). Additionally, Cook et al. (2003) conducted a survey in which recent technical communication doctoral graduates self- reported the topics of their dissertation research; they found that rhetoric, culture, and pedagogy were among the most reported content areas. The results from these latter studies inform our own analysis as these doctoral students were likely new tenure-track researchers during our 22-year time period.

The field’s most longitudinal examination of journal content remains the citation analyses by Smith (2000a, 2000b). Smith conducted a citation analysis on 10 years’ worth of technical communication publications (including the same five journals analyzed in the present study) and found that the content areas were “broadly identified as professional issues (defining technical communication, pedagogy, and research methods), rhetoric and the rhetorics of communities, document design and technology issues, and workplace communication” (p. 427). Smith’s analysis also noted content differences among the five journals. Technical Communication Quarterly and Journal of Business and

8 Technical Communication l Volume 67, Number 3, August 2020

Applied Research

CONTENT AND AUTHORSHIP PATTERNS

Technical Communication were identified as the leading publications for authors with academic affiliations as well as the forums for the field’s more theoretical discussions. As noted earlier, TPC and TC both associated with scholarship from the point of view of the practitioner. TPC also associated with research focused on communication with subject-matter experts and TC associated with research on design. Additional rigorous (and contemporary) research on technical communication content areas are needed to enable a broader understanding of what, exactly, technical communication currently is to its scholars.

State-of-the-discipline studies have also investigated the authorship characteristics (e.g., Gomes et al., 2016; Raptis, 1992; Siddiqui, 1997) and collaborative patterns (e.g., Abbasi et al., 2012; Katz & Martin, 1997; Perianes- Rodríguez et al., 2010). These results summarize the professional and personal characteristics of a field’s scholars as well as provide insight into the value of collaborative research and patterns, such as the frequency that advisees publish with their dissertation advisors.

Only a few technical communication studies have addressed authorship characteristics. The earlier cited survey of dissertation authors also examined the diversity of the authors and found that more women than men completed technical communication dissertations, and 93% of these authors self-identified their ethnicity as “White” (Cook et al., 2003). Of the 18 schools represented by the respondents, all were based in the US and all but two were large public institutions (i.e., more than 20,000 students). In a study of technical communication research journals (the same five journals reviewed in this present study), Smith’s (2000a, 2000b) longitudinal citation analyses found that approximately one third of the data was produced by more than one author and that more scholarship was produced by males than females (Smith, 2000a, 2000b). Authorship statistics of TPC over a 25-year period found that about a third of the articles were written by two or more authors with collaborations trending upward longitudinally in their sample (Brammer & Galloway, 2007). A subsequent analysis of four technical communication journals over a five-year period found that approximately two thirds of the articles were produced by more than one author (Lam, 2014). As a contrast, authorship patterns in JBTC indicated that almost 80% of their publications were single-authored and 62% of the lead

authors were females (Burnett, 2003). These authorship patterns suggest variation among the five leading journals as well as trends that have developed over the last several decades. Additional rigorous research is needed to enable a broader understanding of who authors technical communication research and how those authorship characteristics align with technical communication’s “growth area” of diversity (Johnson et al., 2018, p. xix).

Therefore, we continued to explore these issues through the following research questions:

RQ1. What are the primary content areas covered in technical communication journals, and who are the primary audiences that benefit most from this content?

RQ2. What are the authorship characteristics of these journal article writers?

RQ3. What are the collaboration patterns among authors? What patterns prevail in certain journals and on particular topics?

METHODS

Our primary method was content analysis. We define content analysis as “a research technique for making replicable and valid inferences from texts (and other meaningful matter) in the contexts of their use” (Krippendorff, 2012, p. 18). Content analysis has been modified for qualitative inquiry; however, our application is quantitative and meaning was identified through valid measurement rules and relational inferences via statistical methods (Boettger & Palmer, 2010; Neuendorf, 2016). The general framework for quantitative content analysis includes identifying the sample, developing a coding scheme, norming raters, and analyzing data.

The timeframe for this analysis began with content published in 1996, which is roughly when Smith (2000ab) concluded the timeframe for her bibliometric studies. We concluded the timeframe in 2017, which, at the time of coding, provided the latest complete volume of each journal. We analyzed content from five journals: Journal of Business and Technical Communication (JBTC), Journal of Technical Writing

Volume 67, Number 3, August 2020 l Technical Communication 9

Ryan K. Boettger and Erin Friess

Applied Research

and Communication (JTWC), Technical Communication (TC), Technical Communication Quarterly (TCQ), and IEEE Transactions on Professional Communication (TPC). We selected these journals for analysis because they were all published for the entirety of the designated time period, were included in Smith’s previous studies, and have been identified as the leading forums for technical communication scholarship (Boettger & Lam, 2013; Carliner et al., 2011; Lowry et al., 2007; Smith, 2000a, 2000b). The field has expanded its number of journals, and technical communication scholars published in other forums but focusing on the five leading journals provides the parameters necessary for longitudinal study.

Our sample included 672 articles published in five leading technical communication journals from 1996– 2017. We began with 2,148 articles, or every peer- reviewed article published during the 22-year period. Each article was numbered in a MS-Excel spreadsheet, and we used the random number formula to identify the sample for analysis. The random selection of the sample retained the representative number of articles published by each journal. As an example, TC published 20.4% of the articles in the population

(n = 439), and the journal represented 19% (n = 127) of the present study’s sample. The remaining sample included 112 articles from JBTC, 133 from JTWC, 137 from TCQ, and 163 from TPC. We manually coded 31.3% of the corpus, which is slightly above the ideal sample size for yielding a 95% confidence level with a 3.5% margin of error.

We manually coded the sample on nine content variables: journal, year, primary topic, primary audience, authorship, gender, affiliation type, geographic affiliation, and world region. Variables were selected based on their presence in previous studies in technical communication and related fields (Boettger et al., 2015; Boettger & Friess, 2016; Boettger et al., 2014; Boettger & Lam, 2013; Brammer & Galloway, 2007; Carliner et al., 2011; Lowry et al., 2007; Juzwik et al., 2006; St. Clair Martin et al., 2012; Tansey et al., 2012). Table I includes a description of each variable and its levels.

Operationalization best practices related to survey and experimental research also apply to measurement in content analysis. This includes the development of mutually exclusive coding categories where each recording unit fits into only one category on a given score dimension (Neuendorf, 2016). This practice

Table I. Variable and variable levels considered in the present study

Variable Description

Journal Recorded the forum of the article as JBTC, JTWC, TC, TCQ, or TPC.

Year Recorded the year the article was published (e.g., 1996–2017).

Primary Topic Classified the primary topic of each article as assessment, collaboration, communication strategies, comprehension, design, diversity, editing and style, genre, professionalization, knowledge and information management, pedagogy, research design, rhetoric, technology, or usability and user experience.

Primary Audience

Classified the primary audience who would most benefit from reading each article as academic, business owner, consultant, editor, general, manager, student, visual communicator, other, senior writer/content strategist, or writer/content developer.

Authorship Classified the authorship of each article as single-, co-, or multi-authored.

Gender Classified the first author as either female or male based on the pronouns used in the author’s biography.

Affiliation Type Classified the affiliation of the first author as academic or industry/government.

Geographic Affiliation

Classified the geographic affiliation of the first author as national or international.

World Region Classified the world region of the first author as Africa/Middle East, Asia, Australia, Central and South America, Europe, or North America.

10 Technical Communication l Volume 67, Number 3, August 2020

Applied Research

CONTENT AND AUTHORSHIP PATTERNS

enables different coders to arrive at the same results and provides a common instrument to facilitate data comparison across multiple studies. In fact, the codes developed by the researchers for this and earlier studies have also been applied by other researchers (e.g., Hannah & Lam, 2016).

Our codebook was finalized after 12 drafts and norming sessions with a separate sample and among three researchers. Previous research describes the development of these codebooks, particularly how we developed and refined the mutually exclusive codes for primary topic and primary audience codes (Boettger et al., 2015; Boettger & Friess, 2016; Boettger et al., 2014). For example, identifying mutually exclusive categories for primary topic proved challenging. We initially coded a small sample using four different schemas before arriving at the final approach. When our first codebook was developed, the classification scheme for the STC Body of Knowledge (and later applied in Carliner et al., 2011) was still evolving and contained several coding options. The large number of possible codes proved challenging to sort into mutually exclusive categories, norm across multiple raters, and analyze for noticeable patterns. We encountered similar issues with the coding scheme used by the eServer Technical Communication Library (tc.eserver.org). In addition, we considered the keywords that prospective authors choose to classify their submissions to journals using the ScholarOne Manuscript system (e.g., audience analysis, linguistic research, listening persuasion/proposals). We found these keywords proved helpful to each journal in identifying appropriate manuscript reviewers but more difficult to apply usefully and consistently to all the major journals in technical communication. In the end, our aim was to create a codebook that addressed the diversity of scholarship in the field across multiple publication venues.

We acknowledge that a piece of technical communication scholarship does not always neatly fit into a single category, but the abstraction and isolation of variables is a vital step to any scientific method. No one study can address every nuance of a phenomenon; however, our results include a consistent application of codes that were developed with attention to validity and reliability. Therefore, these codes can be applied to other data samples for comparison, contributing to the growth rather than the stagnation of a particular research conversation.

For this study, we collapsed the earlier developed codes of gender and intercultural communication codes into a more encompassing diversity code. This update was in response to the focus on diversity and inclusion in recent technical communication scholarship. Twenty percent of the current sample was re-coded for inter-rater reliability. Agreement between the study’s authors was 84.6% (using Krippendorff’s alpha coefficient) and within the recommended range (Watt & van den Burg, 1995).

Data were analyzed with descriptive statistics, contingency table analyses, and correspondence analyses. Contingency table analyses correlate multivariate frequency distributions, allowing researchers to statistically compare distributions of non-numerical data. For this study, we ran a binomial, a type of contingency table analysis that tests the statistical significance of deviations from theoretically expected distributions in two categories. We also ran the chi-square test to test two-way table associations.

Correspondence analysis (or CA) is a geometric technique used to analyze multi-way tables containing some measure of correspondence between the rows and columns (Greenacre, 2007). The most useful component of CA is its ability to visually organize the data into central and peripheral instances. CA is not an inferential measure and does not determine statistical significance. Statistical output provides a chi-square value that reflects the overall interaction between the rows and columns, but the researchers must consult other statistical output to properly interpret the results. Throughout this paper, we only report CAs that had a significant chi-square value of ? 0.05, and, like previous researchers, we reviewed other output to determine between-variable relationships (e.g., Boettger & Friess, 2016; Boettger & Lam, 2013; Friess, 2018; Lam & Boettger, 2017).

RESULTS

The results are organized around the three research questions.

RQ1: Content and Audience What are the primary content areas covered in technical communication journals, and who are the primary audiences that benefit most from this content?

https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftc.eserver.org%2F&data=02%7C01%7CRyan.Boettger%40unt.edu%7C937fa9d849b04d6a87f208d6a19bdca3%7C70de199207c6480fa318a1afcba03983%7C0%7C0%7C636874088136990296&sdata=hjP%2BMXvC9OBPvYrRsJJ6DcistXRLh1A7DRfPCUBJyaw%3D&reserved=0
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftc.eserver.org%2F&data=02%7C01%7CRyan.Boettger%40unt.edu%7C937fa9d849b04d6a87f208d6a19bdca3%7C70de199207c6480fa318a1afcba03983%7C0%7C0%7C636874088136990296&sdata=hjP%2BMXvC9OBPvYrRsJJ6DcistXRLh1A7DRfPCUBJyaw%3D&reserved=0
Volume 67, Number 3, August 2020 l Technical Communication 11

Ryan K. Boettger and Erin Friess

Applied Research

Primary topic Overall, the journals published content primarily focused on rhetoric, pedagogy, and genre (see Table II). A contingency table analysis determined how evenly distributed the primary topics were across the journals. Our null hypothesis assumed that if all topics were evenly distributed, 44.8 articles on each topic would have appeared within the 22-year period. This number was derived by dividing the sample size by the number of primary topics (i.e., 672/15). As hypothesized, not every content area was equally represented in the journals, and it is this result that focuses much of our analysis. The far-right columns of Table II list the observed frequencies of the topics and the related p-values.

Articles on rhetoric, pedagogy, genre, and diversity appeared in the journals at a higher than expected frequency. In other words, these areas appeared significantly more often than 44.8 times in the sample. These topics comprised 49.4% of the overall sample and were dispersed in all five journals. Articles

on usability/UX, comprehension, knowledge and information management, research design, design, and editing and style appeared in the journals less frequently than expected. In other words, these areas appeared significantly less often than 44.8 times in the sample. The remaining five topics were not significantly distributed and, therefore, appeared within the journals as frequently as expected.

Primary topic and journal A correspondence analysis (CA) identified a significant relationship between primary topic and journal (?2 = 171.005 p < 0.00). Seven associations were identified from the statistical output. The strongest correspondence was between TCQ and rhetoric (see Figure I). TCQ published 42.4% of the rhetoric articles in our sample (see Table I). Next, TPC corresponded with collaboration and communication strategies. The journal published 52.8% and 40%, respectively, of the articles on both topics. JTWC corresponded with genre and pedagogy. Table II. Frequencies and contingency table analysis results of primary topic and journal Primary Topic Journal Frequency P binomial JBTC JTWC TC TCQ TPC Rhetoric 26 15 9 39 3 92 0.00* Pedagogy 14 19 7 … Read more Applied Sciences Architecture and Design Biology Business & Finance Chemistry Computer Science Geography Geology Education Engineering English Environmental science Spanish Government History Human Resource Management Information Systems Law Literature Mathematics Nursing Physics Political Science Psychology Reading Science Social Science Home Homework Answers Blog Archive Tags Reviews Contact google+twitterfacebook Copyright © 2021 HomeworkMarket.com