"New Criteria for New Media"
Version 2.2, January 2007
Published in Leonardo (Cambridge) 42, no. 1 (spring 2009). Free download.
Authors: Joline Blais, Jon Ippolito, and Owen Smith in collaboration with Steve Evans and Nate Stormer.
ABSTRACT: An argument for redefining promotion and tenure criteria for faculty in new media departments of today's universities. This version also includes an appendix with sample criteria.
Recognition and achievement in the field of new media must be measured by standards as high as but different from those in established artistic or scientific disciplines. As the reports from the American Council of Learned Societies[1], the Modern Language Association[2], and the University of Maine[3] recommend, promotion and tenure guidelines must be revised to encourage the creative and innovative use of technology if universities are to remain competitive in the 21st century.
The following points summarize some of the key areas in which new media research departs from traditional academic scholarship, with the aim of providing a rationale for specific criteria for promotion and tenure detailed elsewhere.
The differences between traditional and new media excellence lie in both form and content. The hard-copy format of traditional review documentation, such as photocopies or slides, is insufficient for evaluating new media work; screenshots do little justice to electronic projects based on innovative interactive or participatory design. As the MLA puts it, "evaluative bodies should review faculty members' work in the medium in which it was produced. For example, Web-based projects should be viewed online, not in printed form."[4]
Further complicating the evaluation of new media achievements is the fact that they are often interdisciplinary, as reflected by the current University of Maine New Media faculty, whose backgrounds range from engineering to computer science to fine art to photojournalism to literature. For example, while art professors typically divide clearly into critical (Art History) and creative (Studio Art) faculties, new media's brief history often requires its practitioners to develop a critical context for their own creative work. This is why the majority of pre-eminent new media critics are also artists.[5] It is also why new media research spans numerous genres, from critical essays to political activism to community-building to software design.
These differences may require evaluators of new media artist-researchers to look beyond the usual standards applicable in other disciplines. As noted by a 2003 National Academies report:
Ironically, the National Academies study found that the highest benchmark for success in traditional academic departments, publication in peer-reviewed journals, is less relevant to success in new media--and empirically less an accurate measure of stature in the field--than more supple or timely forms of intellectual exposition:
Leonardo magazine (MIT Press) is currently the only print magazine universally recognized as a peer-reviewed journal about new media. There are currently a handful of networked peer-reviewed journals devoted to new media, such as Leonardo's Electronic Almanac (Cambridge), Fibreculture (Sydney), and First Monday (Chicago). Yet the field's most prominent print publishers and research archivists[8] have acknowledged a 15-25 year lag and limited exposure that makes print publications far less relevant for new media research. Although promising new paradigms for distributed publication are on the horizon, at the time of writing these systems are only in the planning stage.[9] Finally, as the MLA warns, participation in electronic scholarship should not place extra demands on a researcher[10]; an accomplishment in new media research should substitute for a print article or monograph, not merely supplement them.
Given the accessibility and timeliness required for new media research, the following measures of recognition should be prioritized in the evaluation of new media research candidates:
1. Invited / edited publications
Invitations to publish in edited electronic journals or printed magazines and books should be recognized as the kind of peer influence that in other fields would be signaled by acceptance in peer-reviewed journals.
2. Live conferences
The 2003 National Academies study concludes that conferences on new media, both face-to-face and virtual, offer a more useful and in some cases more prestigious venue for exposition than academic journals:
New forms of conference archiving--such as archived Webcasts--add value and exposure to the research presented at conferences.
3. Citations
Citations are a valuable and versatile measure of peer influence because they may come from or point to a variety of genres, from Web sites to databases to books in print. Examples include citations in:
a. Electronic archives and recognition networks, such as the publicly accessible databases maintained by the Daniel Langlois Foundation (Montreal), the V2 organization (Rotterdam), the Database of Virtual Art (Berlin), and the Media Art Net database (Karlsruhe).
b. Books, printed journals, and newspapers. These are easier to find now, thanks to Google Scholar, Google Print, and Amazon's "look inside the book" feature.
c. Syllabi and other pedagogical contexts. Google searches on .edu domains and citations of the author's work in syllabi from outside universities can measure the academic currency of an individual researcher or her ideas. In the sciences, readings or projects cited on a syllabus are likely to be popular textbooks, but in an emerging field like new media, such recognition is a more valid marker of relevance.
4. Download / visitor counts
Downloads and other traffic-related statistics represent a measure of influence that has gained importance in the online community recently. As a 2005 open access study[13] concludes:
5. Impact in online discussions
Email discussion lists are the proving grounds of new media discourse. They vary greatly in tone and substance, but even the least moderated of such lists can subject their authors to rigorous--and at times withering--scrutiny.[14] Measures such as the number of list subscribers, geographic scope, the presence or absence of moderation, and the number of replies triggered by a given contribution can give a sense of the importance of each discussion list.[15]
6. Impact in the real world
While magazine columns and newspaper editorials may have little standing in traditional academic subjects, one of the strengths of new media are their relevance to a daily life that is increasingly inflected by the relentless proliferation of technologies. Even counting Google search returns on the author's name or statistically improbable phrases can be a measure of real-world impact[16]. By privileging new media research with direct effect on local or global communities, the university can remain relevant in an age where much research takes place outside the ivory tower.
8. Net-native recognition metrics
Peer-evaluated online communities may invent their own measures of member evaluation, in which case they may be relevant to a researcher who participates in those communities. Examples of such self-policing communities include Slashdot, The Pool, Open Theory, and the Distributed Learning Project. The MLA pins the responsibility for learning these new metrics on reviewers rather than the reviewed.[17] Given the mutability of such metrics, however, promotion and tenure candidates may be called upon to explain and give context to these metrics for their reviewers. Again, efforts to educate a scholar's colleagues about new media should be considered part of that scholar's research, not supplemental to it.
9. Reference letters
Letters of recommendation from outside referees are an important compensation for the irrelevance of traditional recognition venues. Nevertheless, it is insufficient merely to solicit such letters from professors tenured in new media at other universities, since so few exist. More valuable is to use the measures outlined in this document to identify pre-eminent figures in new media, or to require new media promotion and tenure candidates to identify such figures and supply evidence that they qualify according to the criteria above.
[1] The ACLS recommends "policies for tenure and promotion that recognize and reward digital scholarship and scholarly communication; recognition should be given not only to scholarship that uses the humanities and social science cyberinfrastructure but also to scholarship that contributes to its design, construction, and growth....We might expect younger colleagues to use new technologies with greater fluency and ease, but with tenure at stake, they will also be more risk-averse....Senior scholars now have both the opportunity and the responsibility to take certain risks, first among which is to condone risk taking in their junior colleagues and their graduate students, making sure that such endeavors are appropriately rewarded." "Our Cultural Commonwealth," report by the ACLS Commission on Cyberinfrastructure for the Humanities and Social Sciences, 29 July 2006, http://www.acls.org/cyberinfrastructure/cyber.htm, accessed January 2, 2007.
[2] "Departments and institutions should recognize the legitimacy of scholarship produced in new media, whether by individuals or in collaboration, and create procedures for evaluating these forms of scholarship." December 2006 report of the MLA Task Force on Evaluating Scholarship for Tenure and Promotion, http://www.mla.org/tenure_promotion, accessed January 2, 2007.
[3] "The Commission encourages each department on campus, as well as the University as a whole, to examine promotion and tenure criteria to recognize and reward innovative uses of technology in teaching, research and service....the University needs to consider the criteria and standards used in the promotion and tenure process. The Commission encourages each department and the University as a whole to consider whether faculty efforts in this area are recognized, valued, and/or encouraged." November 2003 report of the University of Maine Commission on Information Technologies, accessed at http://www.umaine.edu/documents/CIT.pdf on May 2, 2004.
[4] MLA Committee on Information Technology. "Guidelines for Evaluating Work with Digital Media in the Modern Languages." 20 May 2000. ADE Bulletin 132 (2002): 94”"95. 82, mirrored at http://www.mla.org/guidelines_evaluation_digital, accessed 2 January, 2007.
[5] A brief sampling of new media theorist-practitioners includes Simon Biggs (Cambridge University), Matthew Fuller (Piet Zwart Institute, Rotterdam), Mary Flanagan (Hunter), Alexander Galloway (NYU), Kenneth Goldberg (Berkeley), Eduardo Kac (Art Institute of Chicago), Natalie Jeremijenko (UCSD), Raphael Lozano-Hemmer (Karlstad University, Sweden), Lev Manovich (UCSD), Randall Packer (American University), Richard Rinehart (Berkeley), and Jeffrey Shaw (ZKM).
[6] National Research Council, Beyond Productivity: Information Technology, Innovation, and Creativity (Washington, DC: The National Academies Press, 2003), pp. 8-9.
[7] National Research Council, op. cit., p. 188.
[8] These estimates are from MIT's Roger Malina (Director of Leonardo magazine) and the Daniel Langlois Foundation's Alain Depocas (Director of the Centre for Documentation + Research), and are mirrored at http://cordova.asap.um.maine.edu/wiki/index.php/Standards_of_Recognition.
[9] The Interarchive project is a possible model for distributed publication; see http://newmedia.umaine.edu/interarchive.
[10] "Change in favor of a more capacious conception of scholarship, which we strongly endorse, should not mean ever-wider demands on faculty members, most especially those coming up for tenure and promotion." MLA Task Force on Evaluating Scholarship for Tenure and Promotion, op. cit., p. 21.
[11] National Research Council, op. cit., pp. 8-9.
[12] National Research Council, op. cit., p. 188.
[13] Tim Brody and Stevan Harnad, "Earlier Web Usage Statistics as Predictors of Later Citation Impact", http://eprints.ecs.soton.ac.uk/10647/, accessed 5 March 2005.
[14] This recent [http://www.nettime.org/Lists-Archives/nettime-l-0504/msg00051.html] rejoinder by Morlock Elloi on the <nettime> list exemplifies the expectations of such online forums:
> If you have any past publications that might help me understand your point of view, I would gladly read them.
While I understand that in paidspeakerworld the weight of the argument is computed as (volume of publications) x (number of speeches), on nettime and elsewhere closer to reality arguments stand for themselves.
[15] Electronic and email texts also have a currency acknowledged by leading institutions in the field. As of December 21, 2005, one of the premiere bibliographic indices in new media, the Langlois Foundation's CR+D database, included the following indexation for "Jon Ippolito":
* Author of 10 documents
* Subject of 48 documents
* Participant to 21 events
* Organizer of 2 events
Of the 10 documents by the author indexed, 1 is from an email list and 2 are parts of Web sites. In the case of artist and critic Alexander Galloway, the relevance of his online texts is even more striking: although by 2005 he was the author of several journal articles and an important book from MIT Press, the two documents that represented his writing in the CR+D database were both from email lists.
[16] A statistically significant number of Google returns, eg > 30, may be a necessary but insufficient condition for confirming global impact.
[17] "In evaluating scholarship for tenure and promotion, committees and administrators must take responsibility for becoming fully aware both of the mechanisms of oversight and assessment that already govern the production of a great deal of digital scholarship and of the well-established role of new media in humanities research. It is of course convenient when electronic scholarly editing and writing are clearly analogous to their print counterparts. But when new media make new forms of scholarship possible, those forms can be assessed with the same rigor used to judge scholarly quality in print media. We must have the flexibility to ensure that as new sources and instruments for knowing develop, the meaning of scholarship can expand and remain relevant to our changing times." MLA Task Force on Evaluating Scholarship for Tenure and Promotion, op. cit., p. 46.
New Media Department, University of Maine
Promotion and Tenure Guidelines
Addendum: Criteria by Category
Version 3.0
ABSTRACT: This appendix provides sample guidelines for evaluating faculty in new media, in this case drawn from the University of Maine's New Media Department guidelines. The outline follows the official University of Maine template for promotion and tenure activity reports, citing examples of the kinds of new media accomplishments that qualify for each category. Because of the rapid pace of innovation in electronic formats, this list must remain partial, since it is impossible to predict what new recognition mechanisms may be relevant a few years from now.
New media pedagogy must be light on its feet to stay relevant. Below are some instructional activities that serve as important supplements to regular courses on the new media curriculum.
B. Other Teaching Activities:
1. Independent Study, directed research, etc. (list by course number)
Because new media's tools and topics proliferate too quickly to be captured by any one curriculum, faculty are encouraged to teach independent studies when students want to explore research areas not on a current syllabus.
In addition, New Media student and faculty projects often reach beyond the walls of the classroom into the real world. The new media program recognizes the value of directed research in which faculty involve students in outside collaborations for artistic or commercial purposes, as well as faculty members who facilitate students exposure to or participation in national and international exhibitions, conferences, and other venues.
C. Curriculum and Course Development:
1. Curriculum
During its building years, the new media program expects its faculty to contribute more to curriculum development than expected in other departments. This work may take the form of course proposals, curriculum proposals, or curriculum subcommittee membership.
2. Courses
Given the quick pace of new media evolution, the program recognizes exceptional value in developing courses that explore new pedagogies or emerging technologies.
It is understood that New Media faculty may spend a significant portion of their research or course preparation time learning an emerging technology, such as a new programming language, with the understanding that such knowledge may lay the groundwork for future research or new courses. This groundwork is not "brushing up on skills," but experimenting with promising yet unproven systems, codes, or devices.
Good collaborators are critical to thriving research ecosystems. Candidates are encouraged to list any collaborative roles they have played in publications and other activities, such as conceptual architect, approach designer, release engineer, or matchmaker (eg, introducing two other researchers whose collaboration results in a publication). Each new media department may choose to weight these various roles according to its own priorities.
A. Publications
1. Books/Monographs:
Networked or rich-media publications such as extended blogs, DVDs, or CD-ROMS should be included if they constitute a sustained investigation of a particular topic.
2. Refereed Journal Articles:
In a new media context, a "closed peer-review" article includes invited contributions to edited print journals and networked journals. The format of these contributions may go beyond the form of a written essay to include podcasts, videoblogs, and other forms of archival media.
An "open peer-review" article includes contributions to self-policing publication networks, where the quality or relevance of contributions are subject to community debate and evaluation.
3. Chapters of Books/Monographs (please indicate if invited or juried):
Essays or chapters in edited volumes are more important in new media than the sciences, for these edited volumes establish standards for discourse in emergent subdisciplines of new media.
This category should also include invited contributions to edited, single-issue networked publications.
4. Edited Volumes:
This category includes coordinating or managing a multi-user discussion list, whether accessible via email or Web.
This category also includes the conception, design, engineering, and/or editing of organized media collections, including film festivals , networked databases , and publications.
5. Technical Reports/Book Reviews:
This category includes networked reports and reviews.
6. Other Publications (e.g. editorials, working papers, etc.):
This category includes essays published to email lists, including all contributions to discussions sparked by the publication of that essay.
B. Creative Activities, Exhibitions, and Performance Related Activities (please indicate whether regional, international, national, solo, group, invited or juried):
1. Exhibitions:
This category includes networked exhibitions hosted by brick-and-mortar institutions or independent organizations , and can include online exhibitions as well as physical installations.
a Participating
b. Curated
2. Performance Related Activities:
This category includes political design, social software, and interactive performance.
3. Creative Writing and Poetry:
This category includes literature in all its forms, both analogue and digital, in print or online.
C. Professional Presentations and Posters (please indicate if regional, national, or international):
1. Conferences and Discussions organized
Researchers in new media at this point in its development are actively filling in gaps in the awareness of new media's own history, a critical vocabulary, and other intellectual frameworks already in place in other fields. The new media program recognizes the value that organizing private and public events have for the field as a whole and, when local, for our students.
2. Presentations
As studies of new media have argued, presenting research at prestigious conferences can be more important than publishing it.
While there is no substitute for in-person gatherings, teleconferences are gradually becoming an important venue for conference presentations, though they vary in degree of formality and organization.
A. Service to University
1. Department:
As a fledgling program with a high student-to-teacher ratio, the new media program requires an unusual amount of innovation and labor from its faculty, which should be taken into consideration when evaluating faculty contributions to other areas.
3. University:
Because new media promise to change the methods of many academic disciplines, faculty are encouraged to lend their voice to interdisciplinary committees and work with other departments to envision and develop programs that integrate new media into their own practices.
B. Service to the Public (e.g. Service on state commissions, public schools, civic groups, consulting, media interviews, public presentations):
New media can be especially effective in transforming local cultures as well as global ones. Faculty research in this area can be distinguished from traditional academic "service" by its innovative, activist, or performative character.
A. Press
Given the limitations of publishing new media research in academic journals, recognition from the press in the form of articles or interviews about a researcher's work can be a valuable indicator of influence.
1. Print and broadcast press
This category includes outside sources such as general-interest newspapers, radio or TV spots, and specialized journals or magazines.
2. Electronic press
This category includes articles in online journals as well as blogs.
B. Citations
Only general citations go here; citations to document the relevance and achievement of specific projects should accompany the entries on that research above.
1. Print citations
Although they are not as timely as electronic citations, citations in books on new media can suggest a measure of a researcher's influence and relevance to the field.
2. Electronic citations
One measure of influence in academia can be suggested by citations in other university syllabi. (See the breakdown in PART 3.)