Figure 1. Publications by Year

Taxonomy Construction

A taxonomy is an analysis tool that facilitates identification, naming, and classification of objects. For this study, three taxonomies are constructed. Each taxonomy has two main categories. Each main category has subcategories. The taxonomies are constructed using Microsoft Excel97® spreadsheet software, with the main categories placed on the X- and Y-axes. The subcategories are placed in columns along the X-axis and rows along the Y-axis, as shown in Table 1.

The intersection of each column and row is a cell in the taxonomy. Each cell is assigned a distinct number. Documents are evaluated and assigned to cells according to their best fit in the X and Y categories and subcategories. Cell numbers are then used to analyze the document set. The categories and subcategories for each taxonomy are discussed in detail in the following sections.

The eighty-six documents in the retrieval set were examined individually as they were retrieved and entered into the database, then analyzed as a body three separate times, once for each taxonomy. Each document is categorized by assigning the number from the most representative cell in each taxonomy. The cell numbers were entered into corresponding fields created in the Microsoft Access97® document database. The database fields were imported into Microsoft Excel97® for statistical analysis.

The Wulf Taxonomy and Analysis

Wulf’s (1989) White Paper identifies three discipline classes that needed to contribute to research about the collaboratory:

• Computer Science (CS)
• Computer/Communication Engineering (CCE)
• Social, Behavioral, Economics (SBE)

and three areas that needed research focus:

• Instrumentation
• Colleagues
• Data

A 3x3 taxonomy based on Wulf's disciplines and focus was drafted, and an analysis of the document retrieval set undertaken. For multiple-author publications, the discipline of the lead author was used. The discipline of the lead author was determined by information provided with the article, usually an affiliation in the byline, author biographical note, or email address. In the event the primary affiliation of the author was not determinable, the article was placed in the most logical category based on the journal of publication or topic of research.

It became quickly apparent it would be impossible to maintain precise distinction between the first two of Wulf’s discipline categories, so the Computer Science and Computer/Communications Engineering categories were collapsed into a single category, CS/CCE. Two additional categories emerged during document analysis and coding, and were created. The first new category, Library Information Science (LIS) was added because of researcher preference and curiosity. The second new category, Other, was added because many of the disciplines did not fit the Wulf categories neatly. Library Information Science includes authors whose primary affiliation is either a library or information agency, library school, or an information school or department. Disciplines assigned to the Other category includes authors from government, journalism, medicine, education, chemistry, botany, physics, biology, and chemistry, among others. Abbreviations for the revised discipline categories are provided in Table 2. The eighty-six documents were categorized into the reconstructed 4x3 Wulf taxonomy without problem, and a second coding was performed to assure consistency.

Figure 2. Percentage of Publications by Discipline

Figure 3. Publications by Discipline and Focus

The "Other" category of disciplines produced 30%, or twenty-six of the eighty-six documents, including eighteen Instrumentation documents, six Colleague documents, and two Data documents. The number and percentage of Other disciplines confirms Wulf's assumption that the collaboratory would be an inherently interdisciplinary environment constructed by multiple disciplines. Because the Other category was added during data analysis and contains a significant number of articles, a second taxonomy based on Haddow's (1997) types of publications is constructed.

The Haddow Taxonomy and Analysis

Many of the documents in the eighty-six document retrieval set were not research articles in the traditional sense. Haddow's (1997) review of the nature of journals of librarianship offers a second taxonomy into which the collaboratory articles are placed for a more exact contextual analysis.

Haddow relied on Price, and Windsor and Windsor's definition of 'scholarly' literature as hinging on the number of citations an article includes, with those publications having citations classified as scholarly, and those without citations not scholarly. She recalls Beal's description of library publications as either "glad tidings, testimony, or research," type articles, and Saracevic and Perk's further classification of "news-type articles."

For this analysis, several of Haddow's (1997) classifications were combined. Articles concerned with visions, speculations, and success stories, and which contained no or minimal citations, are placed in the "Glad Tidings and Testimony" category. Articles with substantial citations, published in scholarly journals, that develop theory or hypothesis or provide detailed analysis, development, or implementation details are placed in the "Research" category. Articles concerning announcements of funding, partnership formations, commercial implementations or software or hardware rollouts and the like, are placed in the "News-type" category. Table 4 illustrates the placement of the articles in the retrieval set within the article type taxonomy.

Figure 4. Article Type

 

 

Figure 5. Article Type and Year

Two Glad Tidings articles and the collaboratory's first Research article were published in 1989, the year Wulf's White Paper was presented at the Rockefeller Workshop. There were no publications in 1990. The first official collaboratory-specific federal research funding was announced in 1990, and a surge of News-Type articles followed. News-Type publications dominate the collaboratory literature for the next four years, until 1995, when, for the first and only time in the ten year history of the collaboratory, more Research articles were published than any other type.

News-Type publications began four consecutive years of increased frequency in 1994, reaching a high of thirteen News-Type articles in 1998, a year that also saw the most number of Glad Tidings and Testimony articles. Quantitatively speaking, the seven research articles published in 1995 mark the research high-point of the decade. The year 1995 also marks the beginning of an overall upswing in the total number of publications, with a peak 20 articles published in 1998, the last year included in this study.

Figure 5 draws attention to three two-year "waves" in publications: 1988-1990, 1990-1992, 1992-1994. These waves roughly correlate with announcements of funding for collaboratory research. These early indicators of trends, waves, and possible correlation in the types of articles indicates a potential research agenda interested in analyzing media effects, or publication affects in relation to federal funding, or by other aspects of mass media, mass communication, or political science theories.

Wulf noted in his White Paper that the research leading to the collaboratory is uncoordinated. So is the federal government's funding documents related to collaboratory research. Full fiscal documents could not be retrieved during the five-month data gathering period of this study because, like the research documents, they are buried here and there in the volumes of data, and use an uncoordinated vocabulary with undeveloped access portals. For example, of the forty-one NSF projects funded in the 1998 Knowledge and Distributed Intelligence Initiative (KDI), a $350 million, seven-year research program intended to specifically advance research about the collaboratory, only one funded proposal uses the word "collaboratory" in its title. A fulltext fielded search of the abstracts of the NSF funded research database for keyword "collaboratory" produces seventeen funded projects, dating from 1990, including the one described above.

(See an html-ized version of the NSF Comma-separated-value (CSV) concantenated dataset of 1999 KDI awards at www.intertwining.org/collaboratory/1998_Awards.htm.
The search page for NSF-funded research projects is at http://www.nsf.gov/verity/srchawdf.htm.
The KDI webpage is at http://www.ehr.nsf.gov/kdi/ )

Descriptive Statistical Analysis

Figure 6 shows a moderately strong positive linear trend (R² = 0.7533) in the number of collaboratory publications over time. Granted, the dataset is too small for rigorous inference. Nevertheless, it is clear that the concept of the collaboratory, as reflected by a decade of representative literature, has sustained positive interest.

Figure 6. Trends Analysis, Publications over Time

Many of the fifty News-Type publications identified by the Articles Type taxonomy were written by or about individual collaboratory participants who are actively engaged in scholarly or scientific research, and most are clearly attempts to foster popular understanding and garner popular support for the notion of the collaboratory. Simultaneous with the rise in News-Type publications in the last half of the decade, there was a rise in Glad Tidings and Testimony publications but a decrease in the number of Research articles. An average of one Glad Tidings and Testimony article was published each year. Many of these Glad Tidings and Testimony articles were also written by or about a core group of researchers or projects, indicating a concerted public relations effort. Future analysis might include publication production by author or project, and citation and co-citation analysis to reveal the "personalities" of the collaboratory.

Discipline x Focus x Article Type Co-analysis

This section analyzes the twenty-two research-type articles identified by the Haddow Taxonomy by focus and discipline of the Wulf Taxonomy. Figure 7 shows the subset of research-type articles by research focus. Fifteen of the twenty-two research-type articles focus on Implementation, six focus on Colleagues, and one is about Data.

 Figure 7. Focus of Research Articles

Figure 8 shows the twenty-two research-type articles by discipline of the primary author. Seven of the articles, or 32%, originate from Computer Science/Computer Communication Engineering (CS/CCE). Nine of the research-type articles, the largest portion, are published by the Social, Behavioral, and Economics disciplines (SBE). Library Information Science (LIS) produced three of the twenty-two research-type articles, and Other disciplines produced three research-type articles. It is likely that at least some of the articles were co-authored by scholars from different disciplines.

Figure 8. Discipline of Research Articles

Co-author citation and co-citation analysis would add another dimension to understanding the dataset as it reflects collaboratory research between the disciplines, but, that, too, is outside the focus of this Study. Figure 9, Discipline of Research Articles, also shows that half of the eighteen articles produced by the Social, Behavioral, and Economics disciplines (revealed in Figure 3) are research-type articles and represent 41% of the total research-type publications.

Table 5 provides a numbers and percentages overview of total publications, research-type publications, and research publications as percent of total publications, by discipline. SBE produced the highest percentage (41%) of the total research publications in the collaboratory literature, but produced relatively few of the total overall publications, while CS/CCE produced the highest number (34) of overall publications but a significantly lower percentage (21%) of them are research.

Table 5. Frequencies and Percentages of Articles by Discipline

Returning again to Figure 3, Computer Science/Computer Communications Engineering disciplines produced 40% of the total documents, thirty of which focus on Implementation, and two each on Colleagues and Data. Coanalysis with the Article Type shows that seven, or 21% of CS/CCE's articles are Research, and that they represent 32% of the total Research articles and 8% of the total collaboratory literature.

The Discipline x Focus Taxonomy revealed that "Other" disciplines produced twenty-six of the eighty-six documents in the collaboratory literature, and that eighteen of those articles are about Implementation, six are about Colleagues, and two are about Data. Coanalysis using the Article Type Taxonomy shows that three, or 12% of the "Other" disciplines' publications are Research publications, and that they represent 14% of the Research articles. Other disciplines contribute 3% to the total Research literature of the collaboratory.

The Discipline x Focus Taxonomy reveals that Library Information Science produced eight of the sixty-eight documents in the collaboratory literature, five of those eight publications are about Implementation, two are about Colleagues, and one is about Data. Coanalysis with the Article Type Taxonomy reveals three of LIS's eight articles, or 38%, are research-type articles and that LIS Research contributes 14% to the total research literature, and 3% to the total collaboratory literature.

This wearisome deconstruction of the collaboratory Literature exceeds Miller's (1956) "Magical Number Seven, Plus or Minus Two" limit of human information processing capacity, and forbids elegant graphing, so an alternative reconstruction is undertaken.

Figure 9 compares percent of disciplinary contributions to total publications (n=86) and percent of disciplinary contribution to research-type publications (n=22). LIS and SBE each produce proportionally higher percentages of research-type articles in their total publications than do the CS/CCE and Other disciplines.

Figure 9. Percent of Total and Research Publications by Discipline

When LIS and SBE are collapsed into the larger category of "Social Sciences," and CS/CCE and Other are collapsed into the larger category of "Hard Sciences" as in Figure 10, an inverse relation between percent of total and percent of research-type publications is more dramatically revealed. The social sciences produced more research articles as a percentage of their total publications than did the hard sciences. However, if the percentages are converted to numbers, as in Figure 11, the data show that the hard sciences produced more overall publications than did the social science, but the social sciences produced a relatively equal number of articles that are research.

(Although the "Other" Category contains several contributions from discipliens perhaps better classified as "Social Science" than "Hard Science," such as journalism and economics, the contributions of these disciplines was minimal….only one article each. Thus, further reclassification is not deemed necessary.)

Figure 10. Percentage of Total and Research-Type Articles, by Discipline

Figure 11. Number of Total and Research-Type Articles, by Discipline

Conclusion

Wulf (1988) stressed the importance of disciplinarily-balanced research and relative equality of contribution to research aimed at developing the collaboratory, and assumed that research leading to the collaboratory would reflect this balance and equality of contribution. This chapter constructed taxonomies from Wulf's White Paper (1988) and Haddow's (1997) document types to analyze the collaboratory literature for disciplinary contribution, research focus, and article types, and to test those assumptions.