The Use of Visual Artifacts in the User-Informed Development of an Educational Digital Library Collection: Khoo: JoDI

The Use of Visual Artifacts in the User-Informed Development of an Educational Digital Library Collection

Michael Khoo
Department of Communication, UCB 270,
University of Colorado at Boulder, Boulder, CO 80309, USA
Email: michael.khoo@ucsu.Colorado.edu

Abstract

Digital libraries are complex sociotechnical artifacts. As such they will be understood and treated in different ways by the different groups that interact with them. The different understandings of these groups will be rooted in the differing tacit, underlying 'technological frames' that they will have of digital libraries as technologies. In cases where developers and users are both involved in the development of digital library collections, and where the frames of developers and users differ significantly, this can result in difficulties in the collection development process. It is important, therefore, to acknowledge that such differences can exist between developers and users, and to find ways to identify, describe, and mediate them. The paper describes the case of the Digital Water Education Library (DWEL). DWEL was an example of community-led collections development, in which users - in this case educators - were involved in the design and development of its collection. An ethnographic and communications-based analysis of DWEL's organizational communication revealed the existence of different technological frames among the developers and the users of DWEL, differences which impeded the progress of the project. These differences were exacerbated by the project's distributed organizational structure and reliance on network communication technologies for the bulk of its organizational communication. The paper describes how these differences were mediated, in part through the sharing of 'boundary objects' - graphic representations of the project's structures and processes - among the developers and users, and how these representations subsequently informed the development of an online tool that represented some of the developers' knowledge to the users.

Keywords: anthropology, boundary objects, case study, digital libraries, ethnography, organizational communication

1 Introduction: technological frames theory

Digital libraries are complex sociotechnical artifacts. Design, implementation and use involve a range of actors and technologies interacting in a variety of contexts. Studies of sociotechnical artifacts have demonstrated many times that the development of these artifacts can be influenced in unpredictable ways by a range of social and technological contingencies, such as the different understandings of the artifact - the 'technological frames' (Orlikowski and Gash 1994) - that actors bring to their interactions with each other and with the technology. Significant differences in actors' frames can result in misunderstandings, poor communication between project members, and ineffective decision-making related to technology design and implementation.

Technological frames theory studies the shared frames of reference underlying individual and collective perceptions of technology. There are several versions of technological frames theory (Bijker 1995, Orlikowski and Gash 1994). Here, I will focus on the work of Orlikowski and Gash who derive their theory partly from Goffman's theory of frames (Goffman 1974) and Giddens' theory of structuration (Giddens 1984). Orlikowski and Gash argue that:

an understanding of people's interpretations of technology is critical to understanding their interaction with it. To interact with technology, people have to make sense of it; and in this sense-making process, they develop particular assumptions, expectations, and knowledge of the technology, which then serve to shape subsequent actions toward it. While these interpretations become taken-for-granted and are rarely brought to the surface and reflected on, they nevertheless remain significant in influencing how actors in organizations think about and act toward technology. (p. 175)

These sense-making interpretations constitute 'technological frames of reference', according to Orlikowski and Gash "built-up repertoire[s] of tacit knowledge that [are] used to impose structure upon, and impart meaning to, otherwise ambiguous social and situational information to facilitate understanding". Technological frames are "flexible in structure and content, having variable dimensions that shift in salience and content by context and over time. They are structured more as webs of meanings than as linear, ordered graphs" (p. 176). They are "shared by members of a group having a particular interaction with some technology" (p. 203), suggesting that different technological frames can be associated with particular stakeholder groups, communities of practice, and so on.

Technological frames can be either 'congruent' or 'incongruent', depending on the nature and extent of differences among frames. Congruent frames are not necessarily identical, but they are related in ways that imply similar expectations of a technology. Incongruent frames, on the other hand, Orlikowski and Gash continue, imply "important differences in expectations, assumptions, or knowledge about some key aspects of technology ... We expect that where incongruent technological frames exist, organizations are likely to experience difficulties and conflicts around developing ... and using technologies" (p. 180).

Khoo 2001 suggested the existence of incommensurate technological frames, in which the concepts of one frame are not understandable in terms of the concepts of the other frame (Kuhn 1970). This situation has consequences for organizational communication. Orlikowski and Gash's model allows for communication across incongruent frames, and indeed recommends communication for sorting out the differences among organizational groups holding incongruent frames. In the case of incommensurate frames, which can be constituted differently at an ontological level, there will be a greater conceptual distance to go between incommensurate frames than between incongruent frames. In terms of organizational communication it can pay to examine situations where incommensurate technological frames may be present.

Technological frames theory can provide a powerful framework for analyzing interactions between developers and users in digital libraries, who can embrace different technological frames. Users are often interested in how technologies can support their existing 'traditional' practices, while developers often focus on the 'transcendent' possibilities of a technology (Ehn 1988). Khoo 2001 discussed how those with less digital library experience (such as users) can see digital libraries in terms of traditional libraries that are being computerized, while those with more digital library experience (such as developers) will see digital libraries as technologies with possibilities not necessarily related to bricks-and-mortar libraries. Both will bring their respective frames to their organizational interactions, and for successful interactions to occur, these differences will have to be mediated.

Maintaining good communication between holders of different frames is particularly important for digital libraries such as DWEL that involve users in their collection development. Close attention has to be paid to the design of the infrastructure that will support this communication. This paper describes how the communication of the DWEL project was analysed from a technological frames point of view, and the interventions that followed from this analysis. These interventions included the generation of a series of visual artifacts and graphic representations that focused on surfacing the developers' understandings of digital libraries as technologies, and making these available to the users who were developing and building the DWEL collection in the form of boundary objects (Star and Griesemer 1989).

2 Digital libraries

Digital libraries are "Collections of digital resources selected according to certain criteria, organized in a certain logical fashion, and made accessible for retrieval over distributed computer networks" (Kochtanek et al. 2001).

The 'digital' part of a digital library may very generally be understood as any collection of digital resources that are made available for a wider audience of users, usually over the Internet. Online access can be provided by existing 'bricks and mortar' libraries (such as that of a university), and also by libraries that exist solely on the Internet. Not everyone may have access to all of these resources; while many digital libraries are open for all to use, others may require payment of a membership fee to access their resources. Other digital services may also be added to a digital library. These may include the provision of support documentation (downloadable 'How to use the library' brochures, and so on), an e-mail or 'live chat' reference desk, descriptions of special collections, etc.

The 'library' part of a digital library means that digital libraries should provide their users with a range of library-like functions. The scope of the collection, for instance, i.e. the range of material held by the library, should be apparent to the user as soon as they enter the library, whether it is a general scope, or a specialized scope. The collection itself should consist of resources and documents that have been coherently indexed along a number of dimensions using a 'controlled vocabulary' of agreed upon indexing terms, so as to permit searching and browsing of the collection. Finally, once found, the relevant resources should be deliverable, either by viewing on a computer screen, or by downloading to a hard drive.

Significant research into digital libraries in both computer and social sciences dates back to the 1990s, following the development of the Internet, HTML and Web browsers (Arms 2000). More recently, the US government has funded a variety of digital library research initiatives, with the National Science Foundation, through its Digital Libraries Initiative, placing emphasis on developing collections by and for specialized communities of scholars and educational practitioners.

3 Digital Water Education Library

The Digital Water Education Library (DWEL) was funded by the NSF as a two year research project, from 2002-2003. The aim of the project was to generate an Internet-accessible indexed collection of digital educational resources relevant to teaching about the subject of water, including texts, images, films, animations, sound files, and other forms of digital media; as well as to develop the supporting documents such as scope statements that would help to inform the development of the project. On completion DWEL was to be accessioned into the Digital Library for Earth System Education (DLESE), forming a component of the NSF's vision for the next generation of educational infrastructure in the USA, part of a national network of digital libraries that will provide universal access to a wide range of quality educational resources.

The DWEL project members were expected to come up with more than just the resources that would make up the collection; they would also be responsible for generating the institutional guidelines - the scope documents, the review criteria, the work practices and organizational structure - that would support these volunteer cataloguers in their collecting and cataloguing activities. When it was time for the DWEL collection to be accessioned into DLESE, it would be accompanied by these latter documents, which would act as guarantors for the overall quality of the collection as a whole. This way, DLESE would not have to review each of the 500 resources that DWEL had collected, but would instead review the documents that outlined the criteria through which each resource had to be filtered before it was allowed to become part of the DWEL collection.

To generate the collection, DWEL recruited approximately two dozen school teachers and informal educators (the numbers changed as people left or joined the project). The project used educators rather than professional cataloguers to develop its collection because it was thought that the people best suited to identifying good quality Web-based educational resources are representatives of the end-users of those resources. Cost was also a consideration: professional cataloguing is expensive work, and while the DWEL participants received a small stipend for their work, this only went a little way to compensating them for the actual time they put into the project.

3.1 Organisational structure

The DWEL project had a hierarchical structure with clearly outlined paths of communication and authority between the developers and the other project members (Figure 1). At the top of the hierarchy were the project developers, the Principal Investigators (PIs), who were advised by an Advisory Board, and who managed four 'working groups', each consisting of a team leader and between three and five members. The working groups carried out the bulk of the cataloguing, with the working group leaders serving as organizational liaisons between their group members and the PIs. The PIs and the working group leaders and members all had access to a project WebCT site (section 3.1.1) that would support their project communication.

Figure 1

Figure 1. DWEL internal organisation chart

To ensure the collection would contain resources appropriate to a range of student ages, there were four working groups that each focused on collecting resources suitable for their age range:

  • 'K-4,' or kindergarten and grades 2, 3, and 4

  • '5-8,' or middle school, or grades 5, 6, 7 and 8

  • '9-12' (high school, or grades 9, 10, 11 and 12

  • informal education

These educators lived across the USA, and while they were flown into Boulder for workshops on two occasions (at the beginning and half-way through the project), most of their work was carried out online. The tasks of collection building and development in DWEL were thus distributed in both time and space, with the DWEL project members engaged in asynchronous distributed interaction. To engage in the selection and cataloguing tasks just described, they used a series of proprietary and custom electronic tools.

3.1.1 WebCT

A discussion tool was based on, WebCT, a proprietary software package that provided a range of interaction spaces accessible through a Web browser. Using a user name and a password to log on at a special WebCT Web site, project members could take part in threaded discussions on topics related to the project, post relevant documents and view the documents that others have uploaded to the server, and take part in chat sessions. They could also message each other, although the messages stay within the WebCT environment and could not be broadcast to outside accounts. The project PIs hoped that WebCT would support group discussions regarding the development of overall criteria that could be used to judge the appropriateness of educational Web sites for the DWEL collection, and also of the development of DWEL's collection scope document.

3.1.2 DLESE cataloging system

A cataloguing tool, the DLESE Cataloging System (DCS), was available through a browser window and allowed members of the DWEL project to enter the URL of the resource they were cataloguing, along with a detailed description of that resource. These descriptions were built up by selecting terms from 'controlled vocabularies', i.e. by selecting one or more pre-defined descriptors from either a menu or a list of buttons, and also by providing text descriptions of the resource entered into boxes.

Both of these tools, the DLESE cataloguing tool and WebCT, were complex and were introduced to the working groups in a series of introductory workshops organized in Boulder at the start of the project in January 2002.

3.1.3 Project workflow

Using these tools, the working groups searched the Web for sites, or parts of sites, with the following characteristics:

  • themed around water

  • exceptional educational content

  • technologically functional

  • passed certain basic preliminary filters (e.g. ease of use, lack of political agenda, etc.)

Assuming that the Web site had not been previously catalogued by someone else, which could be confirmed by searching for the URL of the resource in the existing catalogue, the user then:

  • accessed the online cataloguing tool

  • described the Web site by entering free-text data, or selections from controlled a range of controlled vocabularies, in a number of fields.

In the case of the DWEL project, the first few records submitted were assessed by cataloguers working at DLESE for accuracy, and were sometimes sent back for review and reworking.

4 Case study

This section analyses the DWEL project from the theoretical perspective of technological frames theory. Note that while the account is presented in approximate chronological order, and appears to give the impression that the analysis proceeded in terms of problem identification, problem resolution, further problem identification, further problem resolution, etc., in practice the analyses and interventions did not proceed in an such obviously 'logical fashion': there were false starts, wrong interpretations, detours, and overlapping periods of both analysis and intervention.

As has been described, the DWEL 'community' consists of approximately 40 individuals:

  • About six project 'developers' and 'managers', all of whom have had some digital library experience, some of it extensive.

  • Approximately 24 'users' and cataloguers, predominantly primary and secondary school teachers, who share an interest in water education, but who generally have no prior digital library experience, and who were divided into four working groups.

  • Another ten or so support staff and advisors whom the other project members could contact and draw upon for advice

Each of the working groups was responsible for selecting, reviewing and cataloguing appropriate online educational resources for their age range, as well as for designing a series of evaluation and review criteria for assessing and accessioning resources into the collection.

The DWEL project commenced officially in January 2002 with an initial four-day series of meetings and workshops held in Boulder in January 2002, where all project members met each other and were trained in the software they would be using over the coming two years. Although financially burdensome, with approximately 30 people flown in from around the USA and accommodated for up to four days in Boulder, the workshops were deemed vital for the success of the project, a chance for all to meet their fellow project members, especially the members of the specific working groups that they would be a part of. After the meetings, the working group members returned home, and began to build the DWEL collection.

Despite an initial surge of interest in using WebCT, traffic on the site began to exhibit a downward trend that became more marked during the second and third months of the project. By the middle of March, working group use of WebCT had more or less stopped, as had the contributions to the cataloguing tool. Although the first few months had been set aside as a time for working group members to become acquainted with each other and their tasks via WebCT, it appeared that these conversations were not taking place.

4.1 Early analyses

By April, the PIs had initiated a series of telephone conference calls among themselves in which they began to explore potential hypotheses for the lack of activity in the project. Their early analyses generated several possible explanations:

  • Lack of time: it was hypothesized that project members, as busy educators, did not have enough time to devote to their cataloguing tasks.

  • Lack of communication: it was hypothesized that project members were not talking together frequently enough in WebCT about the project. Such conversations were necessary, it was thought, for each of the project members to develop the deep understanding of collection development that would then help them in their cataloguing activities.

  • Technological complexity: it was thought that the use of multiple tools and multiple browser windows to view an item, look at the discussions concerning it in WebCT, and catalogue the item, was a technological barrier to participation.

  • Task complexity: on a number of occasions, the working group members stated that they found their task and the instructions they were given by the PIs to be confusing.

These early analyses were reinforced by the results of a questionnaire sent out to all project members after the first four months, which suggested that some of the main stumbling blocks to increased involvement in the project were:

  • a lack of time to complete the assigned tasks

  • the confusing nature of the instructions given by the PIs

  • the complex and overwhelming nature of the task itself.

4.2 Early interventions

As the initial realization that something was not going right in the project, the PIs embarked on a series of interventions designed to stimulate WebCT discussion and cataloguing activity.

4.2.1 Communication interventions

The PIs supplemented the discussions in WebCT with several other forms of communication:

  • Group e-mailings were initiated that broadcast messages to all project members. As the addresses of all members were included in the 'To:' field, every member thus was able to communicate with every other member through use of the 'Reply to all' e-mail function.

  • Regular telephone conferences were established both between the PIs and DLESE staff, and between the PIs and the working group leaders. The former often discussed and analysed problems in the project, while the latter allowed the PIs to brief the group leaders on the instructions that they would then give to their respective working group members in WebCT.

4.2.2 Workflow interventions

Early intervention focused on breaking down the large overall task of collection development and building it into smaller, more easily understandable sub-components. Particular attention was paid to the breaking down of the concept of 'water' into a series of interlinked concepts based on various geoscientific 'spheres', i.e. the atmosphere, exosphere, cryosphere, geosphere, biosphere, and hydrosphere. These six separate spheres were represented in terms of a 'concept map' (Figure 2) that showed how they related to each other and also to the overall aims of the DWEL project.

Figure 2

Figure 2. DWEL concept map

After the PIs had distributed this concept map to the project members, they also translated it into a linear 'timetable' that outlined the focus of the cataloguing activity for the remainder of the project (Figure 3). Thus, for instance, the project members would spend the period from June to July working on collecting and cataloguing resources related to the subject of 'groundwater'.

Figure 3

Figure 3. DWEL 'Campaigns' structure

Both the DWEL concept map and the campaign map represented useful ways to visualize the overall scope of the project in terms of both subject domain and timeline, in terms of smaller, coherently themed and linked sub-components.

The early interventions by the PIs did produce an increase in WebCT discussions and cataloguing activity, but this increase was not permanent; subsequently a further and more detailed ethnographic analysis was carried out from the point of view of technological frames theory.

4.3 Technological frames analysis

Ongoing anthropological and communications-based analyses of the project's interactions by the author suggested a further explanation for the lack of activity: that the PIs and the working group leaders and members involved in the project had different tacit understandings - 'technological frames' - of what a digital library was. This analysis suggested there was a conceptual gap between the developers of the DWEL project and the working groups, regarding the technological structure, scope and purpose of the DWEL project. In particular, it was proposed that these groups were embracing what Ehn (1988) has called 'transcendent' and 'traditional' definitions, and that the presence of these differing definitions was impeding the progress of the project.

Empirical confirmation for this conceptual gap was sought in the analysis of various forms of DWEL discourse, including:

  • A text of the DWEL project proposal to NSDL.

  • Transcriptions of video recordings of various sessions of the DWEL meetings held in January 2002, including the DWEL Advisory Board meeting, and workshop discussions between the PIs and the working groups.

  • Discussion threads in WebCT

These data were analyzed using a latent semantic analysis technique, 'Centering Resonance Analysis' (CRA) (Corman et al. 2002). CRA assumes that 'communicators speak or write coherently by creating utterances that deploy a stream of words comprising centers (more specifically, noun phrases) in a strategic way, creating a semantic structure of words'. CRA parses texts into noun phrases, and then calculates an index of significance for each noun based on the frequency of its occurrence in relation to all other nouns in the text. CRA results can be output in several ways, including spreadsheets of indices of significance, and graphs of networks based on these indices. In the analysis of the DWEL project's organizational communication, attention was focused on the ranking of significant nouns as they appeared in the spreadsheets, summarized in the tables below.

The CRA output suggested there were distinct differences in the discourse of the DWEL project PIs and others involved in the project. In particular, the PIs used the word 'resource' extensively, whereas other members of the project, such as the Advisory Board members, and the working groups, did not. In the DWEL project proposal drawn up by the PIs, for instance, 'resource' was the most significant noun (Table 1).

Table 1: CRA analysis of the DWEL project proposal

resource
collection
DWEL
water
student
science
research
project
digital
teacher
DLESE
material


In the day-long Advisory Board meeting, attended by both the PIs and a range of educational experts who had agreed to act as advisors to the project, 'resource' was used frequently by all three of the PIs, but was almost absent from the discourse of the advisors ('AB') (Table 2).

Table 2: CRA analysis of the Advisory Board meeting

PI 'A'
thing
stuff
good
resource
people
look
DLESE
work
catalogue
group
way
level

PI 'B'
thing
resource
people
work
group
right
different
water
kind
issue
sort
process

PI 'C'
resource
sort
thing
people
process
collection
DLESE
water
site
metadata
way
point

AB 'A'
web
site
people
good
subject
activity
specific
year
minor
developer
cast
point

AB 'B'
lesson
teacher
child
thing
development
test
literature
grade
student
fiction
go
background

AB 'C'
work
group
right
powerful
use
question
tool
teacher
place
background
X
research

AB 'D'
kind
thing
science
work
use
example
project
judge
national
year
student
point

AB 'E'
thing
stuff
science
work
workflow
way
process
school
level
just
train
kind


The same was true in the case of interaction between the PIs and the working group members (WGs) in the DWEL workshops; again, 'resource' featured prominently in the discourse of the PIs, but not in the discourse of the working groups (Table 3).

Table 3: CRA analysis of the DWEL workshops

PIs
thing
resource
great
site
time
student
year
information
look
group
go
idea

WGs
site
thing
child
teacher
good
group
standard
data
different
student
level
point


Finally, in the WebCT discussions of the working groups in the months after the workshops, 'resource' did begin to feature in working groups' discourse, but not as highly as in the PIs' discourse (Table 4).

Table 4: CRA analysis of the WebCT

site
water
DWEL
teacher
work
science
project
group
resource
education
DLESE
material


The CRA results suggested there were distinct differences between the PIs and the working groups regarding the way they thought about DWEL as a technology. In particular, the PIs seemed to be using 'resource' in a wide-ranging, abstract and polysemic way, to refer to a range of actions, practices, and concepts related to the creation of databases or repositories of metadata records. The working groups, on the other hand, were hardly using 'resource' at all. It was therefore concluded that the PIs were talking about DWEL in terms of a technological frame that the working groups were either not using, and/or did not understand. While the working groups were (as was hoped) recognizing exemplary examples of digital classroom resources when they came across them on the Web, at the same time, lacking the understanding embodied in the PIs' technological frame and use of 'resource', they had difficulty in understanding the processes whereby a list of such resources would then be turned into a structured, coherent, and catalogued library collection of use to others.

If this was the case, it would be of benefit to the DWEL project if the working groups could be helped to understand the PIs' technological frame, and what the latter meant by 'resource'. What did this frame consist of? Reviewing examples of the PIs' discourse, it became apparent that it embraced a wide and complex range of inter-related actions that, taken in toto, constituted a tacit understanding of how a resource moved from being a digital artifact available on the Web to being an item in the DWEL collection described with metadata and that conformed to the DWEL collection scope. The complexity of this technological frame can be seen in the following examples of the PIs' discourse. The opening paragraph of the original DWEL project proposal drafted by the PIs states, for instance:

Imagine an interdisciplinary educational resource that allows you to rapidly discover the instructional materials you need connects you to real-time or archived Earth databases; delivers resources in a format that can readily be used in your classroom; and includes the training you need to use these resources. Towards this end we propose to develop a collection of digital materials and resources built around the theme of "Water in the Earth System" (WES).

'Resource' is mentioned here four times, each time in multiple and varying contexts. It is first mentioned in the sense of a digital library collection, as a search engine, and as a database interface; it is then referred to as something delivered by the digital library just referred to, as well as something to be used in the classroom; before in the last sentence it is referred to as a component of a collection of digital materials.

In another example the proposal states:

WES staff will work closely with the DLESE Program Center (DPC) to catalog K-12 water resources with basic descriptive metadata (i.e. title, description, grade level, resource type, etc.) so that they can easily be incorporated into the DLESE collection.

and

One goal of the DWEL research is to understand community participation processes with respect to collection, review, cataloging, and adaptation of resources for inclusion in the DWEL collection.

Here, a 'resource' appears as something to be catalogued, described, incorporated into the DLESE collection; and also something to be collected, reviewed, catalogued, adapted and included in the DWEL collection.

This polysemy of 'resource' arises in part from the wordsmithing involved in crafting as much information as possible into the limits of a project proposal. It also occurred, however, in spoken discourse in a more diffuse manner, although in many ways this diffuseness only underscores the complex ways in which the PIs seemed to be comfortable using 'resource' among themselves. The following statements were all made by one or other of the PIs at various points in the Advisory Board meeting:

PI: The DWEL collection is one product outcome, and that means some number of resources organized amongst some thematic criteria, and where the metadata is adequately generated to a very high standard, but we also want to contribute to the DLESE broad.

Here, a resource is something that is: part of a collection; organized according to thematic criteria; described by metadata; contributed to the DLESE collection.

PI: I think also that what we're seeing is that, uhm, resource creators can actually use a lot of help in understanding copyright too ... if you think about a digital library model, the assumption is you have all these nifty components, and someone else can uh use these components to create a new resource and then share it back. Well they can't actually share back the resource unless those components were released under an IP policy that allowed the creation of derivative works and none of the standard copyrights allow that so the whole digital library thing, the whole food chain, falls apart right there because the resource creators are using the wrong licensing models.

According to this PI, a resource is something that is: created by someone; used; re-used to make new resources; shared; and licensed. Even more examples of the dimensions of a resource are provided in the following extracts:

PI: All of these resources are designed differently, with heterogeneous format or structure, and so every time a user encounters one, it's a whole new learning curve to try to comprehend that particular resource, and the metadata can play a very useful role in scaffolding that comprehension process.

PI: What we are creating is not the resource but the metadata that describes the resource so this stuff, this is the value added, that we have gone through it, not only have we reviewed the resource, but that we've enhanced it with this additional information.

PI: Maybe we should devote our efforts to examine these resources more thoroughly. And the next phase is that we go through a review process, where they as part of the review process pull out technical requirements, copyright issues, and that should be put into the metadata right there, you don't want to have to engage with the resource at that level.

These extracts (and there were many more) included references to a whole range of practices associated with 'resource', ranging from the original creation and development of a resource, through its identification as a suitable candidate for DWEL, its review, its description with metadata, its accessioning to a collection, and its use and reuse. Taken together these practices constitute, I suggest, a significant part of the technological frame of the PIs, a frame that referred to the technological possibilities of digital libraries, and a frame that also, in its complexity, was obscure to the working groups, who already had their own technological frame, one associated with the traditional uses of library materials in educational settings.

4.4 Intervention: representing the collection

The technological frames analysis of the first few months of the DWEL project suggested the problems that the arose in the project could be explained in part by assuming the PIs had failed to explain their technological frame of digital libraries to the working groups (who already had their own technological frame). As a technological frame, the PIs' awareness of the processes of digital library collection development had assumed a tacit, ontological and 'given' status for them; and while the PIs had assumed that their frame was also 'obvious' to the educators, it was not.

It was decided therefore that further work had to be done within the project in order to surface the knowledge of the PIs with regard to the process of collection development, and the practices connected with 'resource', and to make this knowledge available to the educators in useful ways. Ethnographic studies of engineers such as Bucciarelli (1994) and Henderson (1999) have noted the importance that shared visual representations can play in the design process; and Star and Griesemer (1989) have described in detail the useful role that visual artifacts can play in mediating between the various members of heterogeneous and interdisciplinary scientific communities. Several significant redesigns of DWEL's project infrastructure were thus undertaken over the next months, aimed at surfacing the technological frames and tacit assumptions of the PIs in ways that would be accessible to the educators.

Significant work was put into representing the scope of the DWEL collection in terms of a number of themed sub-topic concept maps. These maps would help guide the working groups through many of the decisions that they would have to make, when searching for and then reviewing resources suitable for inclusion in the DWEL collection. Gradually, each sub-topic was represented by a concept map that contained all the subjects thought to be of relevance to that subtopic; a total of nine of sub-topic concept maps were thus generated, a full set of which is provided below. Note that the subject matter of most of these concept maps was derived in turn from the overall concept map of the DWEL project described above (see Figures 2 and 3).

Each of the concept maps was distributed to the whole of the project, and could be referred to by the working groups when they were collecting resources to satisfy a particular campaign theme. As graphic representations of clusters of sub-topics and sub-sub-topics, they provided an easy way visually to assess whether a particular educational resource satisfied the scope of a particular sub-topic of the DWEL collection. As can be seen from Figure 4, the form of these maps varied, but in general they all represented attempts to parse a particular subject domain related to water into finer and finer categories, until the granularity of the bottom-level categories was focused enough to inform the collection and review of educational resources. For instance, in the 'atmosphere' concept map (top row, left), 'atmosphere' is parsed into 'human activities,' 'hazards,' 'causes of precipitation,' 'physics,' 'chemistry,' 'water cycle,' and 'forms of water'; and each of these sub-topics is then parsed again, with for instance 'causes of precipitation' being parsed into 'coalescence,' 'supercooling,' and 'cloud seeding.'

Atmosphere Cryosphere Groundwater

Left to right: 'Atmosphere,' 'Cryosphere,' and 'Groundwater' concept maps


Life Oceans Surface Water

Left to right: 'Life,' 'Oceans,' and 'Surface Water' concept maps


Water Policy Water Quality Water Use

Left to right: 'Water Policy,' 'Water Quality,' and 'Water Use' concept maps

Figure 4. Water sub-topic concept maps (full size images are supplemented)

4.5 Intervention: representing the workflow

The DWEL workflow was also decomposed into individual components, identifying the steps project members had to go through, both individually and as members of a team, to generate one catalogue record. These steps were formalized into the series of diagrams reproduced in Figure 5.

High-level representation of workflow

(i) High-level representation of workflow


step 1 step 2A step 2B step 3A step 3B

(ii)-(vi) Pre-cataloguing workflow steps (steps 1, 2A, 2B, 3A, and 3B in diagrams)


step 4A step 4B

(vii)-(viii) Cataloguing workflow steps (steps 4A, and 4B in diagrams)


step 4C

(ix) Post-cataloguing workflow step (step 4C in diagram)

Figure 5. Representations of DWEL workflow (full size images are supplemented)

Two things are notable about this series of diagrams. First, there is a large number of often complex steps involved in the DWEL's workflow processes, from identifying initial candidates for the accession to the collection, to obtaining consensus among working group members as to the best of these candidates, to accomplishing the actual cataloguing. Second, the technological support originally offered by the DWEL project to its members in carrying out their tasks was limited to the DLESE Cataloguing System, that is supporting to the steps 4A and 4B outlined in Figure 5 (vii) and (viii) (third row). From a technological frames perspective, however, steps 1 to 3B (second row) can also be seen as essential for the development of the DWEL collection and as such also part of the technological frame of the PIs. While these steps may have been reasonably obvious to the PIs, their form and function seemed opaque to most of the working group members.

Steps 1 to 3B were thus necessary for the successful generation of DWEL catalogue records, but in the original configuration of the project there was no technological support for the working group members to carry out these steps. WebCT had been provided, but this was as a general communication tool, for the working group members to have conversations as to how these steps might be carried out. On its own, WebCT did not function as a tool to guide the working groups through these earlier workflow steps.

4.6 Intervention: the DWEL Work Hub

It was therefore decided to provide technological support for DWEL project members to achieve steps 1-3B of the workflow process. At this point, a team of Web developers based at Colorado State University used the workflow documents illustrated in Figure 5 to inform the design of a suite of online tools known as the DWEL Work Hub (Figure 6).

Figure 6

Figure 6. DWEL Work Hub (full size image is supplemented)

The DWEL Work Hub consisted of a series of linked, structured Web pages that guided the DWEL working group members through all the steps outlined in the workflow diagrams. It thus provided technological support for a number of the workflow steps that preceded the use of the actual cataloguing tool. Note that in the screenshot in Figure 6, use of the 'Cataloging Tool' - that is, the DCS - occurs only at Stage V (5) of the process, after there has already been considerable activity among working group members related to the identification, allocation among working groups, and evaluation of possible candidates for cataloguing.

The Work Hub also incorporated the concept maps that the PIs had been producing, that outlined the various sub-topics with regard to 'water' that the DWEL collection was supposed to cover. The Work Hub used these as conceptual frameworks that could help guide the working group members in their collecting tasks (see section 4.4). The Work Hub can therefore be seen as embodying, in the form of an artifact, the technological frame knowledge of the PIs regarding the steps involved in the generation of both single catalogue records and also coherent collections of such records, making this knowledge available to the working group members in the form of a structured artifact.

The implementation of the DWEL Work Hub was supported both by documentation available on the Work Hub site, and also by a series of training workshops held in Colorado in January 2003, in which the working group members received detailed instructions in how to use the new tool. Subsequent use of the tool by the working groups correlated with a significant stabilization in the rate of cataloguing activity across the project. Whereas before cataloguing activity was characterized by short bursts of productivity stimulated by the introduction of new work regimes and processes by the project PIs, bursts of activity that were always followed by longer periods of inactivity, in the six months after the DWEL Work Hub was introduced cataloguing saw a slow but steady improvement in activity, as working group members adopted to the work schedule of sub-topic campaigns drawn up during the first months of the project (see Figure 3).

One explanation for this stablization of DWEL project productivity lies in the ways in which the Work Hub systematically guided the working group members through a workflow process that, at the beginning of the DWEL project, was only implicitly present in the project's organizational structure. While this workflow process was apparent to the PIs, this was in the form of a technological frame, a frame that needed to be reflexively identified by the PIs and then represented in graphic and artifact form for it to become apparent to the working group members.

5 Discussion and conclusion

Work is often organized in tacit ways, and distributed teams have to make extra efforts to make their work processes explicit to all concerned. This is especially true if people are embracing different technological frames; unless these frames are surfaced in some way, there will be a danger that people will feel as if they are operating in a conceptual vacuum, and thus do nothing.

The analysis of distributed work described in this paper has been informed by Orlikowsi and Gash's (1994) theory of 'technological frames': "particular assumptions, expectations, and knowledge of the technology, which then serve to shape subsequent actions toward it. While these interpretations become taken-for-granted and are rarely brought to the surface and reflected on, they nevertheless remain significant in influencing how actors in organizations think about and act toward technology." The research has described the existence of various forms of technological frames regarding digital libraries amongst DWEL's PIs and working groups, namely a more technologically oriented view of digital libraries, and a more utilitarian view often oriented towards classroom practice, respectively. In the case of the DWEL project, the paper has described how the project PIs and developers did not necessarily recognize they were embracing different technological frames, but that nevertheless the project was experiencing a number of organizational difficulties that could be explained by the presence of differing technological frames.

It was not until after considerable qualitative research had been carried out into the DWEL project's communication and practices that these differences were recognised. This prompted the DWEL project PIs to make their own understandings of the project specifically and reflexively concrete and thus available to the working group members, first in the form of various graphical representations of the project's structure and processes, and then in the form of the DWEL Work Hub. These concrete representations can be considered as 'boundary objects' that act as translation points between the different developer and user technological frames within DWEL.

As digital libraries grow more complex, organizational communication will form an increasingly important component of digital library design, especially where this involves developer-user conversations, for instance in the pursuit of collection design and development. The paer has introduced one analytical framework for informing the design of digital library project organizational communication - technological frames theory - and illustrated its application with an example from the DWEL project, where interventions based on a technological frames analysis generated changes that produced significant improvements in the project's workflow and productivity.

Acknowledgements

I am grateful to the members of the DWEL project for allowing me to observe and record their meetings and communication. This research was funded under NSF grant #DUE 0121724.

References

Arms, W.

(2000) Digital Libraries (MIT Press: Cambridge, MA)

Bijker, W. (1995) Of Bicycles, Bakelites, and Bulbs: Towards a Theory of Sociotechnical Change (MIT Press: Cambridge, MA)

Bucciarelli, L. (1994) Designing Engineers (MIT Press: Cambridge, MA)

Corman, S., Kuhn, T., McPhee, R. and Dooley, K. (2002) "Studying complex discursive systems: Centering resonance analysis of communication". Human Communication Research, 28(2), 157-206

Ehn, P. (1988) Work-Oriented Design of Computer Artifacts (Arbetslivscentrum: Stokholm)

Giddens, A. (1984) The Constitution of Society (University of California Press: Berkeley)

Goffman, E. (1974) Frame Analysis (New Harper and Row: New York)

Henderson, K. (1999) On Line and on Paper: Visual Representations, Visual Culture, and Computer Graphics in Design Engineering (MIT Press: Cambridge, MA)

Khoo, M. (2001) "Community design of DLESE's collections review policy: A technological frames analysis". Proceedings of ACM/IEEE Joint Conference on Digital Libraries 2001 (New York: ACM Press), pp. 157-164

Kuhn, T. (1970) The Structure of Scientific Revolutions (University of Chicago Press: Chicago)

Kochtanek, T., Hein, K. and Kassim, A. (2001) "A Digital library resource Web site: Project DL". Online Information Review, 25(1), 29-40

Orlikowski, W. and Gash, D. (1994) "Technological frames: Making sense of IT in organizations". ACM Transactions on Information Systems, 12(2), 174-207

Star, S. and Griesemer, J. (1989) "Institutional ecology, 'translations' and boundary objects: Amateurs and professionals in Berkeley's Museum of Vertebrate Zoology, 1907-39". Social Studies of Science, 19, 387-420