RIP: Show What You Tell

Rich Internet Publications:
"Show What You Tell"

Leen Breure
Utrecht University, Netherlands
Department of Information and Computing Sciences
leen@cs.uu.nl

Hans Voorbij
Utrecht University, Netherlands
Department of Information and Computing Sciences
hansv@cs.uu.nl

Maarten Hoogerwerf
Royal Netherlands Academy of Arts and Sciences (KNAW)
Data Archiving and Networked Services (DANS)
maarten.hoogerwerf@dans.knaw.nl

Abstract

The journal article is still the basis of scholarly communication. This genre, however, largely adheres to the rules of the printed publication and does not meet the requirements of this age of digital Web publishing. Today we do not need to restrict ourselves any longer to communicating the results of the research process only. We can also allow readers to inspect the underlying data online, to publish their own comments and, using a variety of multimedia content, to be witness to intermediary stages of the scientific discovery process. This development has stimulated the transformation of the conventional article: when published in a digital format, it is more and more enhanced with data sets, photos, videos, interactive maps and animations; these enhancements affect its structure and layout. A variety of new publication formats is appearing, some of which can be no longer adequately described as simply "enhanced" publications. They are rather to be conceived as a new genre, for which we propose the term Rich Internet Publication (RIP), analogue to the well-known concept of Rich Internet Application. Both share features of information integration, visualization and exploration (i.e. non-linear reading), typical for hypermedia products. This paper relates apparently quite different variants of scholarly publishing by highlighting common elements of data access and delineates the gradation of publication enhancement.

RIPs do not constitute a sharply delimited category, but are part of a broad spectrum, which starts with regular enhanced publications closely resembling their printed counterparts, and ends with high-quality multimedia presentations having more in common with Web applications than with the conventional journal article. We distinguish two subcategories: RIP type I is primarily based on a linear text, but fully integrated with multimedia content and tools to access and analyze data, while RIP type II is more image-driven, has a user interface with more graphic elements and encourages explorative, non-linear reading.

The production of enhanced publications and RIPs is not yet a straightforward process. It requires extra effort from the author, which is currently insufficiently rewarded. This may change when funding agencies get more interested in research products that go beyond the level of textual publications. Dedicated tools for construction of RIPs are equally important, which requires consensus on architecture and infrastructure. Development of these tools could fit in with the recently started research line of adding semantic metadata to object-based enhanced publications. Moreover, the creation of a RIP will rely on the author's basic competencies of e-scholarship. When authors start creating RIPs on a larger scale, the process of exchanging and preserving them has to be supported. Usually, a RIP is not a single static file, which can be downloaded and attached to an email, but a set of related components. Preserving the content's integrity will be a major concern.  

1. Introduction

The established system of scholarly communication is based on the journal article. As a genre, it still closely adheres to the rules of the printed publication, in spite of the fact that most journals have already migrated to some digital format with PDF as common denominator. The impact of information technology, however, is rapidly changing the ways in which the scientific process is recorded. Today, research is best served by a digital communication system that captures "the full scholarly record" (Van de Sompel et al. 2004). This should include not only text with figures, tables and references to literature, but also datasets, simulations, software, and dynamic knowledge representations. The current state of information technology allows the recording of the complete scientific discovery process, as Jane Hunter has made clear in her seminal work on Scientific Publication Packages. It implies not only data capturing and storage, but also an adequate semantic layer together with search and retrieval facilities (Hunter 2006).  

This much richer way of scholarly communication requires a dedicated technological infrastructure and a new mental attitude that still have to be established, but one may already observe an increasing tendency to blur the boundary between publication and data. The BioLit project, for example, aims to connect articles in the Public Library of Science (PLoS) with the Protein Data Bank (PDB). PLoS consists of a large body of text covering biology, medicine, computational biology, genetics, pathology and a variety of other fields, while the PDB contains all publicly accessible three-dimensional structures of biological macromolecules. Tools implemented in PLoS enable the user to explore the wealth of information through the database, rather than through the article (Fink and Bourne 2007).

Elsevier took a similar step. Beginning with the first issue of Cell in January 2010, Cell Press launched a new format for the online presentation of all research articles. The various publication sections are accessible by tabs (see figure 1). A single unified hierarchical structure incorporates the core text and supplemental material, so that readers can drill down through the layers on basis of their current task, their level of expertise and interest. The new format (introduced through a prototype already the year before) has been received with mixed feelings for quite different reasons. It is appreciated because of the visual format that is more geared to reading online (People talking 2010). Criticism concerns the "interface trickery", which some people perceive as a divide between the publisher's more richly marked-up version and the one held by the author in word processing format (Sefton 2009). The user interface requires special technology and is not generic enough, for example, it relies heavily on JavaScript, it lacks a general hierarchical structure, data are not presented in RDF format, and the table of contents implemented as a set of tabs works only with a limited number of headings (Carvill 2009).    

The Article of the Future

Figure 1: The new Cell user interface. Outstanding features are the explicit layering and internal integration of the information displayed. In addition to the textual summary a graphical abstract is included, together with a video on which the authors introduce their findings. The tabs on top correspond with the regular division of a scientific article, while sections like 'Results' and 'Data' provide ample room for charts. Figures in the 'Data' section are linked to the body of text so that each chart can be viewed in sync with its own context. The 'Comments' tab opens a blog-like web page for reactions (see: http://www.cell.com/abstract/S0092-8674%2809%2901439-1).

The same innovative spirit pervades other Elsevier journals, like Historical Geography, Applied Geography, and Computers, Environment and Urban Systems, which recently have published some multimedia articles. Many of the online journals have got a tab 'Figures/Tables' next to 'Article', which contains images and, sometimes, a video (see also Elsevier's Geography, Planning and Development portal).

A comparable concept of information delivery has been adopted for medical decision-support data, as in Best Practice, published by the BMJ Group. A tab-based user interface gives access to step-by-step structured information around the patient consultation, covering diagnosis, prognosis, treatment and prevention. A dense network of hyperlinks combines research evidence, guidelines and expert opinion into a comprehensive manual, which can be extended with local guidelines. 

Archaeology is an other field where a large research community shares lots of data, for which purpose online databases are used. Although electronic publishing has not yet become widely accepted, this domain has developed its own dedicated solutions (Xia 2006). The web-based system Open Context integrates textual documents with images of excavation material. It is a free, open access channel for electronic publication, created as a means for scholars and students to easily find and reuse content created by others. Starting from a Google Maps interface with a timeline, one can select a project. This comes with a detailed description and related inventory of finds, each extensively documented through a set of photos. A bit more complicated is Online Cultural Heritage Research Environment (OCHRE), developed at the University of Chicago, and targeted at a varied group of historical disciplines, including both archaeologists and philologists. It requires a Java client application, which communicates with a backend database, being a central, XML-based repository, in which data from multiple sources have been integrated following the data warehouse concept (see figure 2).

 OCHRE

Figure 2: Aramaic tablets in the Persepolis Fortification Archive, published through OCHRE (see: http://ochre.lib.uchicago.edu/PFA_Online/).

Other humanities disciplines are moving into the same direction, however, in general with more reluctance (Boonstra, Breure and Doorn 2004). Recently, the Center for Studies in Higher Education (CSHE) in Berkeley published a report on e-scholarship, which includes, amongst others, a case study on the field of history. Although the book continues to be the mainstay of historical publication (Harley 2010), data sharing and data presentation are also here the germs of a new approach. Adding the underlying dataset to a publication greatly enhances the possibilities of evaluating the author's conclusions and starting new research. The image of the past becomes more complete by taking different perspectives: textual sources are only one category of historical evidence. A new model of historical scholarship has been proposed, in which textual and visual modes of presentation are merged, like in the Virtual Jamestown web site. A combination of drawings, sketches, artifacts, maps, and first-hand accounts re-creates life in this Indian settlement (Shifflett 2007).

This kind of visualization goes beyond mere illustration and emphasizes the use of multimedia tools and techniques as the primary means of analysis and presentation (Barish and Daley 2009, Borgman 2009). The idea has been most vigorously formulated by a group of scholars at the University of California, Los Angeles, (UCLA) in the Digital Humanities Manifesto (Presner and Johanson 2009 and Presner 2010). This statement contains the following paragraph, which drastically reformulates the traditional relationship between text and visuals:

"Digital Humanities is not a unified field but an array of convergent practices that explore a universe in which: a) print is no longer the exclusive or the normative medium in which knowledge is produced and/or disseminated; instead, print finds itself absorbed into new, multimedia configurations; and b) digital tools, techniques, and media have altered the production and dissemination of knowledge in the arts, human and social sciences." (version 2, paragraph 9).

Visualization is equally important in the life sciences. In biology, computer-based visualization is widely used to help understand and communicate data, to generate ideas and to gain insight into biological processes (O’Donoghue 2010). Ideally, it allows seamless moving over a broad range of scales, from tissue, cellular to molecular level. Visualization may help scholars not to get overwhelmed by the enormous mass of data. Biological knowledge is currently distributed across multiple heterogeneous databases, which have different focuses and different ways of information organization. Many visualization tools are Web-based and allow the construction of integrated web applications for data mining and browsing (Tao 2004).

To summarize: peer-reviewed journal articles are still the gold standard for communicating the results of scientific research and experimentation, but in this age of digital Web publishing neither authors nor their readers still find them entirely sufficient (Smith 2006). We see the old genre in transition, destabilized by the potentials of the Web and burdened with new features like data access, interactivity and visualization, without having yet grown into a new genre that fully acknowledges the added functionality. The supplementary material published is not intended as a simple illustration to enliven the scholarly discourse. It should allow the reader to watch, to witness, to experience and to analyze the subject in question. Nowadays, we want to take a closer look at the scientific discovery process. Major obstacles are the deluge of data, which greatly surpass the capacity of print journals, and the lack of integrated access, i.e. the connection between online databases and repositories on the one side, and publications on the other side.

In this article, we want to address the question what this new genre would look like, provided that we may speak of a new genre at all. Treloar raised the question whether hypermedia scholarly journals are a new genre or just an old genre in a new medium, or, in other words, whether the transition is an evolution or a revolution. He concludes that we see a gradual evolution in the functions of the scholarly journal and a faster evolution in its form, but not a revolution (Treloar 2000). We do not conceive of the "new genre" as a single monolithic model, but rather as a loose set of functionalities co-occurring and clustering in many variants and many different implementations. From the perspective of information technology, these functionalities may be expressed as design patterns, i.e. as reusable solutions to commonly occurring presentation problems; on the implementation level, they will correspond to reusable components. The appearances of the new genre cover a gliding scale, with slightly enhanced traditional publications at the low end, and more sophisticated interactive, multimedia publications at the high-end, as will be discussed in the next section. 

2. From Enhanced Publications to Rich Internet Publications

Most novel forms of digital scientific publication can be grouped under the concept of enhanced publication. The term refers to the old genre, which has been enhanced with additional material. The enhanced publication may be an article in a journal, a thesis, a report, a memorandum or a chapter in a book that involves scientific or scholarly research and contains an interpretation or analysis of primary data or something derived from it. The supplementary material will consist of research data, models, algorithms, illustrative images, metadata sets or post-publication data such as comments or rankings. The option of changing post-publication data allows an enhanced publication to develop over the course of time (DRIVER). The system's architecture is supposed to have an object-based structure with explicit links between the objects. An object can be an article or part thereof, a data set, an image, a movie, a comment, a module or a link to information in a database (Woutersen-Windhouwer and Brandsma 2009, Godtsenhoven et al. 2009).

So, among the benefits of enhanced publications are the possibility to incorporate data that would otherwise not be added to a publication (because of room and money constraints) and the sharing of research data within a scientific community. However, there are drawbacks as well. Creating and maintaining online databases and the electronic publishing itself are expensive. It takes additional time of authors (without a proportional rewarding), whereas publishers have to deal with the costs of curation and storage, particularly where unique, non-experimental data are concerned as in archaeology (Adema 2009). Other hurdles have loomed up in their wake, like findability, encoding of semantics, durability, copyright handling and data structure (e.g. Van de Sompel et al. 2004; Adema 2009).

Up to now much attention has been given to storage, findability and retrieval of enhanced publications, while the presentational aspects have remained relatively underexposed. As a logical implication of the trend to more multimedia-based scholarly communication one may also enlarge the concept of enhanced publication itself. We conceive it as a continuum, with (1) at a low-end text-based articles with illustrations, aggregated data display and optional links to data packages (section 2.1), (2) somewhere in the middle publications with a more advanced user interface like that of Cell (section 2.2), and (3) at the high end presentations with high-quality multimedia and a rich interactivity. Formats closer to the high end will be more integrated and contain more multimedia and interactivity (section 2.3). Currently, the semantic layer is most clearly present in the demos of enhanced publications at the low-end, but we assume the practice of adding metadata for identification and retrieval will further extend along the spectrum in the coming years.

2.1. The Low-end

As stated above, the first category of enhanced publications mostly consists of text plus supplementary material, rather loosely connected in the user interface. A good example is the Journal of Archaeology in the Low Countries (JALC). As the name suggests, JALC covers the region of The Netherlands, Belgium and adjacent areas. It is published primarily as an e-journal (by Amsterdam University Press) with the option of printing on demand. As part of a trial project some journal contributions have been worked up into enhanced publications. The enhancements consist mostly of larger pop-up versions of images and tables, and a small number of interactives like a dynamic presentation of burial, in which different layers and objects can be made visible or hidden at choice. Interactivity is also used in tables by applying filters on underlying data sets (see figure 3).

JALC

Figure 3: Fragment from L. Troubleyn et al. (2009) "Consumption patterns and living conditions inside Het Steen, the late medieval prison of Malines (Mechelen, Belgium)". Journal of Archaeology in the Low Countries, No. 1-2, http://dpc.uba.uva.nl/jalc/01/nr02/a02.
On clicking on the link 'Dynamic content' the interactive table is displayed in a new tab or window: the user can select one or more categories of animal remains to be included in the view.

An other proof of concept is made as part of the European DRIVER-project (Digital Repository Infrastructure Vision for European Research). This so-called DRIVER II-demonstrator offers a comprehensive view on the different component objects in the enhanced publication: the text is available as PDF and, dependent on the subject, data sets can be displayed in an aggregated format or queried through a user-friendly database interface (see figure 4). DRIVER has the objective to establish a cohesive, pan-European infrastructure of digital repositories, offering sophisticated functionality services to both researchers and the general public. The information to be provided includes all three categories: research data, extra materials and post-publication data (like commentaries and ranking). Currently, publishers offer only one or two of these categories; they publish extra materials (such as movies) separately from the publication, often without a link between these related objects, and rarely support the option to add post-publication data (Enhanced Publications 2009, Godtsenhoven et al. 2009). This should change with enhanced publications, components of which are provided with metadata and interlinked with each other.

DRIVER

Figure 4: Example of an enhanced publication about journalists in the Netherlands, made as part of the DRIVER II demonstrator by the Data Archiving and Networked Services (DANS) in the Netherlands (http://driver2.dans.knaw.nl/demonstrator/html/index.html). This enhanced publication consists of the data from the survey and the book. The book itself is available as a complete file and a collection of separate chapters. SPSS data are accessible through 'Resources'.

The DRIVER II demonstrator takes advantage of the Object Reuse and Exchange standard by the Open Archives Initiative (OAI-ORE). OAI-ORE enables the definition and identification of aggregations, also called compound digital objects, which may combine distributed resources with multiple media types including text, images, data, and video. The Australian Aus-e-Lit project has worked into this direction with LORE (Literature Object Re-use and Exchange), a Firefox extension which allows users to create OAI-ORE compliant compound objects, and publish, edit, search and retrieve them (see figure 5). A user may gather web resources, organize and tag them, describe links between them and make notes about them via the graphical editor. He can also discuss topics and resources by replying to annotations created by others.

 LORE

Figure 5: Example of compound object creation in LORE (see: http://www.itee.uq.edu.au/~eresearch/projects/aus-e-lit/).

2.2. The Spectrum's Middle

We reserve the middle part of the spectrum for publications with a higher degree of integration. The main text is carefully geared to interactives, like slideshows, image galleries, maps, and query interfaces, creating a synergetic functionality, which comes close to that of a real Web application. Integration concerns quite different aspects, such as internal linking between information objects and the functional combination of user interface components on the presentation level. At the low-end such elements are usually spread over different windows, each operating in isolation; for example, the display of images and text may be not as nicely synchronized as in the Cell application, where charts can be browsed and viewed in context (see caption figure 1). Text and maps can be synchronized as well. Most Geographic Information Systems will display text when the user clicks on a point on the map. In an enhanced publication the opposite will be more natural: an interactive map displays information as response to clicks in the text (see figure 6).

 Geo-narrative

Figure 6: Demonstrator of a geo-narrative from the Cultural Landscape Explorer project (Information and Computing Sciences, Utrecht University). Clicking on a link in the text (in this case 'Utrecht') highlights this location on the map (see: http://www.cs.uu.nl/research/projects/i-cult/CLE/3Geonar/Geonar.htm).

The always-limited screen space will be the major obstacle and will often enforce a simple set of controls next to the publication's main text, with a link to a more detailed version. Although not a typical enhanced publication the Perseus Digital Library provides an example of this pattern: studying classical works, the user can load the vocabulary tool, which appears in summary format next to the text displayed, while a more advanced version is displayed in a separate window (see figure 7).

 Perseus

Figure 7: A page from the Perseus Digital Library, displaying the beginning of Caesar's De Bello Gallico. The vocabulary tool (on the right) currently shows the top 50% of words sorted by weighted frequency. A more detailed display with advanced options is accessed through the link 'Study all vocabulary in this passage'. Note also the graphic text locator on top, a visual component which we may also expect to find in more advanced enhanced publications.

Both examples demonstrate an other aspect, which is the logical correlate of integration. Online reading and exploration of content presuppose some kind of ordering, mostly a hierarchy of levels. The top level is the main text of the publication, which has many entry points to lower levels, containing more detailed textual information or data. As a consequence, the linear structure of the publication, still dominant at the low-end of the scale, is up to a certain degree abandoned in this type. Reading in this way implies exploring information resources, which may happen in an uncontrolled order. Authors cannot be sure any longer, that all the information they supplied, is actually taken in by the readers in time. This situation is comparable to games and virtual environments, where designers have to control the player's behavior through perceptual opportunities, which attract and guide the user to essential spots, reward him and elicit particular behaviors (Fencott 2002; Schell 2005; see also section 3.2).

We mentioned already Cell, which is a good example of a journal in the middle of the spectrum. Among many regular online publications Oxford University Press produces a few e-journals that fall into this category too, like the Oxford Islamic Studies Online and the Oxford African American Studies Center (see figure 8). Each article is linked in detail to various repositories and comes with extensive search facilities and timelines. For example, in the Oxford Islamic Studies Online one has a choice between a Main Search, a Biography Search, an Image Search, a Primary Source Search, a Bibliography Search, a Qur'an Search and a Concordance Search. After highlighting a word in the text a user may perform an expanded search through the repository, which produces a list of related articles. The timelines too - each covering a certain aspect - have events hyperlinked to relevant articles.

African American Studies Center

Figure 8: Snapshot of the Oxford African American Studies Center: clicking on an event on the timeline brings up a related article.

2.3. The High-end

At the high-end of the spectrum we find interactive multimedia applications with visual narratives and a non-linear structure making exploration more attractive than the primarily text-based variants of the previous category, which largely rely on linear reading. These interactive multimedia applications are not appropriate for regular publishing of research data because of the considerable investment of time and money required for authoring and production. They are created as experiments in digital scholarship or they are typically used as showcases for larger projects targeted at a broad audience beyond the group of domain experts. Fine examples are available in the portfolio of web studios as Terra Incognita and Second Story. They are clearly modeled after Rich Internet Applications (RIAs) and it may be a matter of discussion whether these products are to be labeled as 'publications' or 'applications'.

As Mullet (2003) has pointed out, a Rich Internet Application creates a seamless user experience through immediate responses and smooth transitions between different states. It keeps focus, presenting a complete story about a single subject without distracting information, and it is aware of user requirements by appropriate filtering. This requires some heuristics for organizing the content presentation. This is also made clear by Terra Incognita and Second Story in their reflections on the experiential rich media applications they created (Marable 2000, 2004; Johnson 2003, 2005). Reflecting on his experience in producing cultural heritage applications for the Web Marable (2004) has formulated an Integrated On-line Exhibit Model with three tiers:

  1. Experience, a narrative-driven layer with emotionally engaging and entertaining elements.
  2. Exhibit, the majority of interpretive and educational content in a mixture of linear and non-linear modes. Here, visitors can explore topics and themes.
  3. Research (or archive) comprising the exhibition sources, like historical documents, works of art, etc.

There are two additional key elements in the model: multiple entry points, which give different audiences access to the layer of content that best meets their needs, and connecting story lines encouraging visitors to follow a theme or topic between the layers (see figure 9). The layered structure shows some similarity with the more text based counterparts in the middle of our scale (section 2.2).

 Genographic

Figure 9: The Genographic Project, web application by Terra Incognita. In this project for National Geographic, visitors can explore the history book hidden within their genetic code. Visitors can even submit samples of their own DNA and explore the routes of their own ancestors, right on the web site (see: http://www.terraincognita.com/production.php?topic=science_nature&title=the_genographic_project).

Marable's model comes close to the structure of Visualizing Cultures, a MIT project started in 2002 (see figure 10). This Web platform (with content currently mainly about Japan) enables scholars to examine large bodies of previously inaccessible images and to compose texts with high-quality multimedia. The site contains a number of "units", each generally comprised of four sections:

  1. An essay, written by a scholar and driven by the visuals themselves. Images are isolated and juxtaposed to highlight diverse perspectives and points and are supposed to be read by the user, who is stimulated to ask questions about the content displayed.
  2. Visual narratives, which are dominated by images. They have the look and feel of old magic lantern slides: long strips with pictures, which can be moved from left to right; here is more to look at than to read.
  3. Access to an image database, mostly in the form of an image gallery.
  4. Video and animation, short clips that include author commentaries, interviews, tours, animation, and archival source footage.

Visualizing Cultures

Figure 10: Homepage of Visualizing Cultures. To the right an icon view of the units in the project (see text above). See: http://ocw.mit.edu/ans7870/21f/21f.027/home/index.html

Our own work in this category includes a RIP about late-medieval commemoration of the dead in the convent Mariënpoel, near the Dutch town of Leiden. It was created as part of the Medieval Memoria Online (MeMO) project. This project aims to facilitate research of the medieval culture of remembrance (memoria). MeMO will make available to researchers descriptions of text sources and of objects that are usually studied by art historians in particular (memorial works of art and tomb monuments).The application centers around a painting showing four generations of the family by whom the convent was founded. This painting hung in the convent's church, as a daily reminder of praying for the souls of the deceased. The RIP uses the painting as a stepping-stone to the presentation of the complex intertwining of the religious and political life in the late Middle Ages (see figure 11).

The presentation is inspired by the style of Rich Internet Applications as made by Terra Incognita and Second Story. It has four levels. The user enters the application through a graphic layer on top, which is to be explored and consists of reproductions of paintings, maps and original historical documents. This activity takes the user to the second level, which is largely text-based, with illustrations and interactives. On the third level the reader can zoom into images, e.g. photos of charters or other textual sources. Relevant passages are highlighted and provided with a transcription and translation. The lowest level comprises research notes with detailed comments, like extensive footnotes in printed publications. This way of publishing was chosen, not only to excite a broader interest for the subject itself, but also to demonstrate how conclusions were achieved and how art objects may be interpreted in connection with other, textual sources, thus showing the importance of crossing the borders of academic disciplines such as history, art history and literary history.

Mariënpoel

Figure 11: A Rich Internet Publication about religious and political aspects of late medieval commemoration of the dead: the case of the Van Zwieten family and the convent Mariënpoel, near Leiden, Netherlands (Utrecht University; see: http://www.cs.uu.nl/research/projects/i-cult/CLE/6Memoria/).

2.4. The Concept of Rich Internet Publications

The examples discussed above show, that the more one approaches the high end of the spectrum, the more the old genre seems to be worn out (see figure 12). The basic body of text is first enriched with visuals (images, video, animation), content is hyperlinked into a network, which is easily traversed into all directions by advanced search facilities, thus affecting the typical linear structure of printed publications. Finally, the text itself may be superseded by a multimedia layer to provoke a more direct experience than verbal descriptions can do. This kind of e-scholarly communication urges an author "to show what he tells": literally, through visuals, and figuratively, by access to underlying research data.

We think that enhanced publications with this set of features deserve a more appropriate name, which expresses their novel nature. As an analogon to Rich Internet Applications (RIA) we propose the term 'Rich Internet Publications' (RIP) for the middle and the high end of the spectrum. We do not believe in a strict classification and rigid criteria: the more features are present, the more the predicate RIP is justified. The short anthology of enhanced publications reveals at least two important variants:

  • Type I (or shortly RIP-I) is primarily text-based; Cell and the Oxford Islamic Studies Online are good examples. The text is not highly adapted to online reading; it could also appear in a print journal in almost the same format. The difference is in the rich enhancements.
  • Type II (or RIP-II) is more image-based, like the Web applications by Terra Incognita, Second Story and Visualizing Cultures. Here images and graphics determine the overall presentation, which, of course, does not exclude large text parts also occurring in RIP-I.

Note, that this classification only concerns the presentation of information, and does not imply, that instances at the high-end have now already the same semantics as the enhanced publication demonstrators at the low-end. For compatibility with enhanced publications, one can describe the different components of RIP-I or RIP-II instances using, for example, OAI-ORE.  

The spectrum of RIP

Figure 12: The spectrum of enhanced publications. Key words are meant indicative only. This classification is made from the perspective of information presentation. Currently, Rich Internet Publications may lack the detailed semantic layer of enhanced publications as with DRIVER II and LORE. 

3. The production of Rich Internet Publications

3.1. The Social Context

Classifying enhanced publications is a rather academic exercise. What really matters is how we can create them on a large scale and in an affordable and durable way. The reception of Elsevier's Article of the Future shows, that not everybody wants to delegate this work to a publisher. However, moving up to the high-end of the spectrum is far from easy for authors. In addition to the semantic infrastructure envisaged by projects like DRIVER, a set of tools is required to ease the job; but above all, an administrative system is needed that remunerates the extra efforts. As long as the traditional peer reviewed text is the only thing that really does matter when it comes to scientific rankings, one may expect that many authors will consider multimedia enhancements at best an embellishment that takes too much time to create. This may change, when Current Research Information Systems (CRIS) become more widespread (even pan-European, see euroCRIS), are linked to institutional repositories (Godtsenhoven et al. 2009), and get a more refined granularity acknowledging objects below the level of the final publication.

For some years our own research has been focused on affordable ways of production, of both type I and type II RIPs positioned in the field of cultural heritage and cultural landscapes (Breure et al. 2008a, 2008b, de Boer et al. 2009). Although a dedicated tool set is still lacking, technology, in our experience, has not turned out to be the major obstacle. Visual writing is a much bigger problem for academics who have been trained all their life in producing texts. Even if a research procedure can be demonstrated through a series of images, authors may initially fall back to the deeply rooted tendency of describing objects or events, rather than showing what they want to tell by adding text to visuals. Writing for new media has to be learned, just as we all have mastered the composition of traditional scholarly publications. Providing RIP models in combination with some training in e-scholarship (Barish and Daley 2009) and coaching in projects will help, provided authors will clearly understand their gains. We have achieved good results by working in very small teams, consisting of a scholar and one or two highly qualified developers, who understood the content matter very well and, at the same time, did all design and implementation work. This agile style of development kept communication lines extremely short, and helped authors to understand the technology used and to improve their competencies in this respect. In exchange for their extra efforts they acquired an interesting "showcase" of their research project, which turned out to be a sufficient stimulus for their active participation.

3.2. Writing for RIP

Visual writing means making images stand for words, as Friedmann stated (Friedmann 2006). He describes a strategy for interactive media, which could be roughly followed for RIPs too (in particular for the development of type II). Preferably, such guidelines would be merged with model-driven approaches for Rich Internet Applications (Preciado et al. 2007). Because this requires an extensive treatment of its own, we shall leave it aside and focus on the production of type I RIPs.

For RIP-I it is most convenient to start from a conventional paper and move the content from textual to interactive-visual mode. Some theoretical and methodological guidance may be found in the social semiotic framework of Bezemer and Kress (2008), which we shall apply here in a rather eclectic way. Social semiotics investigates human signifying practices in specific social and cultural circumstances, and tries to explain meaning making as a social practice. Bezemer and Kress phrased their theory in the context of secondary school teaching. Transduction is the term they use for the process of moving content across modes, in contrast with transformation being a change within the same mode, like rearranging fragments within a text. Image, writing, layout, speech are all examples of modes, each having its own resources (e.g. photos, words and phrases, fonts). As the authors point out, changing media involves recontextualization: in our case, a RIP will probably have also an audience slightly different from the readers of the printed version, and will certainly be used in a different way thanks to its new capabilities. This implies several actions:

  1. Selection: We have to map the topics to a new structure, which entails selecting text together with visual resources and (aggregated) data for different levels of detail. To accommodate online reading we will need at least a summary and a full text level, but a visual outline may be included as well. Deeper in the application the user may have access to the full data set. Such a hierarchy is a recurrent pattern in the examples discussed above (Cell, Terra Incognita, Visualizing Cultures). A flexible outline tool is of great help for text selection. It must provide a preview of the outline to be read on screen and mark up the selected text in some way or another. Not everything may be relevant in the new context, the requirements of the audience may be different and the resources will not be the same. So, one must ask questions like: What is the goal of the RIP in comparison with the print version? How is content most aptly represented and what visual resources are available for the RIP? How can readers be guided through the branching content? Some elementary form of storyboarding (for example in PowerPoint) is needed to set down the results of these selections.
  2. Arrangement: The new juxtaposition of text, images, animations, and tools may change the meaning and function of all components involved. It will create new perceptual opportunities (Fencott 2002), which influence the readers' behavior. The reading path has to be reconsidered: is it still linear, or is the user expected to alternate reading with free exploration of the hyperlinked content? Can you be sure about what the user has read on a certain point? Patterns of arrangement could have implicit connotations. For example, Kress and Van Leeuwen suggested that in our western society the left is the region of the ‘given’ and the right the region of the ‘new’; and that the top is the region of the ‘ideal’, whereas the bottom depicts the ‘real’ (Kress and Van Leeuwen 2006).
     
  3. Foregrounding: In the textual mode we have to rely on rhetoric constructions and layout principles to put an idea in the foreground. Visual resources allow new, powerful emphasis of the content, which may or may not be in line with the textual discourse from which we started. They should not distract the reader's attention, as often happens when visual elements are added for the sake of attractiveness. Highlighting topics by visualization may have the drawback that more information is required than available, a problem we came across frequently. For example, one can easily describe medieval wetlands in which a peat dome was situated, but a sound visualization of its shape could be quite difficult because no reliable traces are left. However, an illustrator must cope with this lack of information. Bezemer and Kress called this phenomenon epistemological commitment. Every mode imposes a different set of such commitments, which should be considered as part of the design process.
     
  4. Social repositioning: Bezemer and Kress observed, that social relations between teacher and students were reconstructed as a consequence of transduction and recontextualization; a new pedagogy emerged. Comparing RIP with regular articles we may expect that the role of the author will change. In the textual mode the author can only act authoritatively, referring to evidence through data tables, charts and bibliographic notes. In the interactive-visual mode his role is extended to that of a guide, who helps the reader to observe the discovery process and re-analyze the results. The reader becomes a user and, because of the explorative facilities, the selector of its own learning material, a role reserved to the author alone in the textual mode.

3.3. Implementing a RIP

This is not the place to delve into programming languages and to discuss the pros and cons of RIA platforms like Flash, Silverlight, OpenLaszlo, JavaFX and HTML5. However, the practical realization of the RIP concept depends highly on easy to use software tools, which could be used by an e-scholar with basic knowledge of information technology. The latter would comprise, for example, skills to create and edit images, understanding XML and experience with XML editors. Although there are some authoring packages (like Pachyderm for cultural heritage) that come close, the required software is generally not yet available as a ready-made toolbox.

Although extra effort is unavoidable, it seems crucial to keep the authoring process simple. One may think of something like the IKEA-concept: practicality, good value, simplicity, and easy assembling with a minimum set of tools. Let us imagine, that an author having an article in mind (or having already mostly completed) goes to the "store", a Web application, which will ask questions and let him make choices. At the end, the program will have classified the publication in making. It advises a style and format and provides a list of downloadable components, all XML-based, which can be customized by the author by means of a simple XML editor. The author will enter the main text according to an XML schema, preferably through a user-friendly interface hiding technical details, and specify the images that are to appear in a gallery or slideshow. Display preferences and parameters can be supplied in XML format as well. Visual material will need some editing, which requires some basic knowledge of image processing. Finally, all components can be uploaded and will function together as a RIP-II.

We may be sure that practice will be more complicated, in particular when a high-quality RIP is required, but it would be a challenge to keep it as straightforward as this. Like IKEA, we have to give our time to design interchangeable parts with uniform interfaces, which are pre-assembled as far as possible. Today, the tool kit would ideally be limited to an XML editor and a run-of-the-mill image-processing package, or, more user-friendly, a graphical user interface may be provided for building these mashups, as with Yahoo! Pipes. In the future both, XML creation and basic image processing, should be supported by wizards and WYSIWYG tools.

Currently we are working on a prototype based on this vision, which will be part of the front-end of Alfalab, a collaborative framework project of the Royal Netherlands Academy of Arts and Sciences (KNAW). This explores the chances for virtual research collaboration in the Humanities on basis of a supporting digital infrastructure. Alfalab comprises a virtual laboratory with online tools and access to a dataset registry (Van Zundert 2009). A RIP-I is being developed now as a gentle introduction to the geographic tools of the lab. A printed publication based on research with geo-data and geospatial tools is made online readable and enhanced with access to tools and data samples. The RIP will function as a kind of tutorial, on the one hand demonstrating what can be achieved with these techniques, and on the other hand allowing users to experiment themselves with the lab resources. The emphasis is on an integrated environment, in which a user can easily switch between taking in information and learning by doing. At the same time, this is a first experiment in building a RIP in the modular way described. 

3.4 Durability

The lifecycle of a traditional publication does not end after it has been created and read: it is also supposed to be cited and it needs to be available for verification. The same requirements apply to RIPs. Citing a RIP will not be too difficult, provided that a title, author details and a persistent identifier have been added. However, permanent availability may be a problem. Publishers, institutional repositories and national libraries already have a hard job ensuring that traditional publications are permanently available and readable. Guaranteeing access to the underlying research data is an even harder task for the various data archives due to the quantity and the heterogeneous form of the data (Doorenbosch and Sierman, 2010). Preserving RIPs will become more difficult when all components are highly integrated. In that case one may either archive a complete RIP as a single package or delegate preservation of the individual components to the appropriate archives. The first option will be a complex task for one archive, given the heterogeneity of the components. The second option will be a organizational challenge, given that all components need to be reintegrated upon retrieval.

4. Conclusion

In this short overview of enhanced publications we have proposed to identify some at first sight quite different species living on the Web as members of the same genus, for which we introduced the name 'Rich Internet Publication' (RIP), analogue to the more well known concept of Rich Internet Applications. A RIP can be regarded as a "super-enhanced" publication, in which the typical structure of the printed article is more or less abandoned. In type I RIPs the linear text structure is still dominant, while type II comes closer to a Web application.

We believe that a sharp definition with precise criteria is difficult to construct and not productive in practice. Key features of Rich Internet Publications are integration, visualization and exploration. The message conveyed can be studied on different levels of detail, varying from a summary, to full text and research data, which are fully connected by hyperlink paths. The main text is integrated with images, interactive info graphics, access to facilities for searching information, inside the publication itself and outside in related publications. Findability requires the further implementation of a semantic layer as with the OAI-ORE demonstrators. Integration comprises also the synergetic juxtaposition and interaction of text and other components within the same screen space. In the more graphic variants, text may become subordinated to graphics and images: visuals function no longer as mere illustrations, but offer a view on the process of discovery. In such a case, text is in service of the visuals by explaining them and providing instructions for exploration.

The realization of enhanced publications and of RIPs in particular will depend on what kind of institutional requirements are to be met by research projects, whether the monitoring will be restricted to publications or extended to objects on deeper level as well. Rewarding the extra effort is a critical success factor. Equally important is a dedicated tool set and information architecture. Investigation on easy and affordable productions of RIPs should be connected with the research line on the semantics of enhanced publications. For both purposes, findability and publication, a mash-up structure of reusable information objects seems to be most productive. Just like with complex database applications, a RIP-II will mostly require the help of technical experts. However, an e-scholar could be empowered to assemble a RIP-I with relatively simple tools. The design of these tools requires more joint research and agreements on guidelines and standards. Finally, it is important to consider how long-term availability of the RIPs can be facilitated.

5. References

Publications

Websites