Lost in social space: Information retrieval issues in Web 1.5

Jon Dron, University of Brighton, UK/Athabasca University, Canada

Terry Anderson, Athabasca University, Canada

Abstract

This paper is concerned with the application of Web 2.0 technologies within a conventional institutional learning setting. After considering the affordances of Web 2.0 technologies vs Web 1.0 technologies and a framework for viewing social software in terms of groups, networks and collectives, we describe an instance of trying to use Elgg, a rich social application, to support a distance-taught course within a conventional face-to-face university. A number of issues are identified, some of which are related to Elgg’s interface but the biggest of which relate to the tensions between top-down and bottom-up control and the shifting contexts of personal, group, network and collective modes of engagement. These problems suggest that, in their current form, social technologies pose intractable difficulties in information organisation and retrieval when used for formal learning. We propose a range of solutions that make use of the wisdom of the crowd combined with human intervention. This paper addresses and extends themes explored in SIRTEL 07.

Introduction

Information retrieval has one or more dimensions that may include a technical component (e.g. search algorithms), an organisational component (how information is structured) and/or a psychological component (how information is presented so that humans can find what they are looking for easily). In this paper we explore the latter two dimensions as they apply in an e-learning environment.

One of the typical roles of a teacher is to draw the learner’s attention to relevant documents, acting in many ways like a librarian. However, a notable benefit of e-learning, especially in a web environment, is that it becomes trivially easy to provide more resources from which to learn than a single learner could sensibly manage. In such circumstances (e.g. in resource-based learning), the learner is required to choose between competing options and must attempt to find the best path. In such an event, the problem becomes one of information retrieval – selecting relevant resources while ignoring those that are less so. This is not just about the traditional information retrieval task of finding relevant documents but is also about finding useful people, tracking relevant dialogue and seeking useful paths through a range of resources.

A significant benefit of e-learning is that it can enable greater autonomy in learners ( (Moore, 1993) (Garrison & Anderson, 2003) (J. Dron, 2006) . To some extent this is because they may typically choose the time and place that they learn but, equally, it can make it easier for learners to choose the paths that they follow and the people that they engage with on those paths. For example, the sensitive provision of hypertext links, especially when appropriately annotated, can enable learners to select paths that suit their needs, rather than simply following a linear sequence.

Adaptive systems can make that selection easier (Brusilovsky, 2001). The study of adaptive systems has a long history in technology aided instruction (see for example (Winn, 2003) ) and has received increasing interest as the need for adapting the multitude of learning resources available on the web for instant and relevant individual use in lifelong learning (Hummel et al., 2007) (Koper, 2005) . A particularly significant sub-class of such systems employ social adaptation. For example, tag clouds can provide advice on what the crowd finds interesting, collaborative filters can help to select resources and people of relevance, and reputation systems can indicate who is reliable or an expert in the area. Even simple rating and comment systems can help learners to choose more appropriate or effective paths.

In addition to assisting in the retrieval of information, social systems can play a significant role in its creation and organisation. For example, through technologies such wikis, collaborative tagging, resource sharing and, to an extent, blogging, learners can play a significant role in the generation of a knowledge structure. Moreover, every interaction with the system can play a role in affecting a system’s structure. Mayfield (2006) has organised these kinds of interaction into a loose twelve-aspect hierarchy of participation that starts with reading (picked up through referral logs and so on) and ends in refactoring, collaborating, moderating and leading. To a large extent it is this that distinguishes what is vulgarly known as Web 2.0 (the participative web) from Web 1.0.

Web 1.0

In a traditional Web 1.0 learning environment such as that provided by learning management systems such as WebCT and Blackboard, structure is largely provided by a combination of the efforts of system designers/programmers, the administrators of the system and teachers/instructional designers. Contributions from students tend to be limited to small group contexts, discussion forums and, occasionally, tightly constrained activities defined by the course designer. The participative feedback loop is controlled and filtered primarily by the teacher, occasionally assisted by logs generated by the system.

Pedagogically, there is virtually nothing new in what such platforms offer when compared with hand-built systems developed in the 1990s. Arguably their greatest selling-point when compared with the environments of yesteryear is their ease of use, which they achieve primarily through reducing the choices available to authors when compared with those available in hand-crafted environments. This in turn means that a significant part of the decision-making process including learning designs and activities is delegated to the creators of the learning environment. As structure influences behaviour (Senge, 1993), sometimes significantly, this tends to a relatively limited subset of pedagogies imagined as use-cases by the software designers (Siemens, 2004) . The tools are usually fairly flexible if sufficient effort and skill is employed and, consequently, many other approaches are possible, but most people have a tendency to take the default paths. For example, Dron (2006) observes that well over 99% of several thousand Blackboard-based courses use the default course entry point (announcements), even though this is not mandated and is easy to change.

Web 1.0 teaching and learning tools assume that education can take place behind the closed doors or walled garden of a virtual campus or classroom. Security and control issues dominate thinking and in typical ‘horseless carriage” style, courses are frequently created that mimic the resources and practice of the classroom of the past. Although there is evidence that this model ‘works’ at least to the extend that learning outcomes are equivalent or better than to those attained in classroom contexts (Bernard et al., 2004; Olson & Wisher, 2002; Sitzmann, Kraiger, & Stewart, 2006), there has been little change in pedagogy nor has there been a major increase in learning productivity or accomplishment (Zemsky & Massey, 2005)  

Web 1.0 online education may be seen as, largely, an extension of the on-campus classroom. The (usually) teacher-driven creation and control of the course is supported by a host of relatively familiar (to both teachers and students) classroom tools, now made available online. The tracking tools of the LMS allow teachers to see who has participated much like they have scanned faces in the campus classroom. Teacher-talk predominates, though the asynchronous nature of the discourse has been shown to reduce percentage of teacher to student contribution (Ahern, Peck, & Laycock, 1992). Student contribution during the course is accessible only to those enrolled in the course and only during the time that the course is in progress.

As we increasingly discover the limitations and constraints of the first generation educational platforms of the 1990s, social technologies and other Web 2.0 features such as mashups and looser aggregations of tools that are collectively part of a Personal Learning Environment (PLEs) have at last reached the mainstream.

Web 2.0

The general trend towards universal authorship and a richer, social, more interactive experience of the Web has been characterised by O’ Reilly as Web 2.0 (O'Reilly, 2005) . While the authors (like many others) dislike the term, not least because there is no great qualitative difference between many technologies of yesteryear and those that are seen as typical of Web 2.0, it is valid as a means of identifying a distinctive cultural shift in emphasis from total control of website authorship by the technorati to a gentle relinquishing of control to the masses – the user-generated Web. This shift has only recently begun to diffuse into the world of higher education, where rigid institutional structures and protective traditions combine with occasionally justified fear and reticence on the part of universities and colleges has meant that power relationships have been maintained through the use of very structured learning management systems and their ilk. They are representatives of what Darby (2003) refers to as first generation systems, automating existing modes of teaching and, in the process, ossifying such modes.

We are still in early days of construction, exploration and testing of Web 2.0 applications in formal education. Much of the literature consists of reports of interventions in both classroom and distance education contexts of applications such as blogs, Wikis, folksonomie tagging, profile and social systems (see for example (Franklin & van Harmelen, 2007). Early studies have looked at gender differences in blogging behaviour (Pedersen & Macafee, 2007), the reduction in feelings of isolation and alienation by distance learners (Dickey, 2004), comparisons of disclosure activities (Govani & Pashley, 2005) and development of social capital among Facebook users in educational contexts (Ellison, Steinfield, & Lampe, 2007). We are also beginning to see the development of theoretical models for introducing, evaluating and using web 2.0 tools – notably blogs and the need to consider factors such as perceptions of, and need for, an audience; perceptions of, and need for, community; the utility of, and need for comments; presentational style of the blog content; overarching factors related to the technological context; and the pedagogical context of the course” (Kerawalla, Minocha, Kirkup, & Conole, in press). Despite a growing literature base, it is apparent that we are in exporation mode and in such contexts the use of case studies, with rich contextual detail is a most appropriate research strategy yin (Yin, 2003)

The most salient features of Web 2.0 for educational application are the qualitative increase in openness and the facilities to give more learner control. Web 2.0 tools often (but far from exclusively) assume that learning takes place as much outside as inside the virtual classroom and thus is open to the contribution, input and influence of learners, of external communities of practice, of graduates and even the contribution of students from past courses.  The teacher’s role in such circumstances changes from supplier and filter of content, to one of orchestrator, mentor, adviser and assessor – an informed co-traveller more than a guide.

Web 1.5

New technologies in education rarely completely change custom and practice. Rather they are most often incorporated into practice and in the process can be described as disruptive (Christensen, 1997). These effects either result in the innovation being abandoned or its gradual incorporation into the mix of practices that mark the diverse forms of formal education. We and others have written of some of the difficulties and challenges of this transition, in particular the clash of cultures and structures between the top-down, institutional model of teaching and learning and the bottom-up, learner-led models of Web 2.0  (Dron, 2007; Dron & Anderson, 2007). In this paper we will be observing an instance of this clash of cultures and technologies through a theoretical lens that takes a multi-dimensional view of the Many, drawing into tight focus the tensions between the teacher-dominated world of Web 1.0 and the learner-dominated world of Web 2.0.  In particular, it is observed that these tensions lead, almost inevitably, to being lost in social space: of encountering difficulties in finding people and resources within the social environment. We will be offering some solutions, some technological, some methodological, and we shall be identifying some hopeful paths as well as warning of the dangers of following some current paths.

Implementing Alternatives: Elgg at the University of Brighton

Largely in response to the limitations and constraints of traditional learning management systems, in particular the strongly teacher-oriented model that they embody, an instance of Elgg, an open source  social networking system, was installed at the University of Brighton. The system has been rolled out at an institutional level  with over 30,000 users in total, of whom around a half have actively contributed, if only by logging in and setting up a basic profile. A far smaller percentage (less than 5%) are regular, active users but this has been sufficient to drive a thriving and active community.

Bottom-up information structure

Built around people and communities as opposed to content, Elgg offers facilities for community creation, network discovery, blogging, podcasting, file sharing, link sharing and many other social facilities.

In contrast to the content and structure-orientation of institutional learning management systems, Elgg enables connections between people, dialogues, informal group formation, collaborative content creation, story telling and other human-centred, community-related activities. The structures that do develop are largely bottom up. They are represented by features such as tag clouds, lists ordered by date, popularity or relevance, networks formed by individual decisions to join communities and clusters built from connections. However, it does not have to be that way and the system has the flexibility to support islands of organised collaboration, such as traditional courses, simply through the creation of a Community , a technical nomenclature that Elgg uses to describe a shared social space. Note that, for the rest of this paper, we will be capitalising this specific use of the word ‘community’ when talking about Elgg Communities, to distinguish it from the more generic use of the word. In many senses, Elgg Communities act like individuals within the system, each having its own profile, blog, file store, media space, bookmarks and other optional plugin components such as wikis. Membership of a Community may be more or less tightly controlled by its owner, from click-to-join through to fully moderated or limited to a pre-specified set of users.

Although certainly not the only Web 2.0 social networking system, Elgg is arguably the most popular environment developed with e-learning and e-education applications in mind. There will be other platforms that emerge for e-learning applications, but we think that many of issues dealt with in this case study will be confronted by educators at all levels and using many different social platforms

Groups, Networks and Collectives

Elgg supports forms of the Many which, in traditional institutional learning, are not formalised or recognised. We have identified three distinct kinds of the Many that are enabled by social software such as Elgg: the Group, the Network and the Collective (Dron & Anderson, 2007). In addition to these aggregations, such software supports personal spaces. The potential relationships between them are shown in Figure 1.

groups, networks and collectives

Figure 1 - the relationship between groups, networks and collectives

Groups

Groups are the traditional organizational context of educational communities or classes, typically having a well-defined membership, a clear focus, a hierarchical composition and a distinct border between activities of the group and those of the larger community. This kind of grouping is well supported in Elgg, which enables the creation (by anyone, not just by teachers) of separate and potentially closed group spaces in the form of Communities.


Networks

The ability to link others as friends or to join more or less formal communities on an ad hoc basis creates a network of relationships between people and the groups of which they are a part, going well beyond the formal groupings formerly found in online formal educational communities. Being part of a network is often a weaker, less readily classifiable relationship than membership of a group. Networks can evolve, sometimes quite surprisingly. We may follow links to friends who have friends who have friends with interests that spark new ideas and connections. We may drift in and out of communities, sometimes actively engaging, often peripherally participating, frequently lurking. Discovery of networks can be facilitated through software that identifies similarities, typically through similar tagging behaviour and subject matter. This leads us to our final aggregation: the collective.

Collectives

Even more tenuous is our connection with collectives. Collectives are characterised by software-mediated aggregation: they are not about connections, but instead are formed by grouping people and their largely independent activities into sets. A typical product and instance of a collective is the tag cloud. Use of tags by more than one (often unconnected) person leads to tag clouds showing text with different emphases depending on popularity of the tag. We thus see a snapshot of an aspect of the collective mind. Following tags in a larger typeface leads us to individually created resources and may lead us into networks and groups. Elgg offers rich support for tags and fairly flexible ways to use and show tag clouds. Other collective instantiations include collaborative filters, search engines employing variations on the PageRank algorithm and group rating systems. In each case, software combines the implicit or explicit behaviours and opinions of the many to offer an emergent structure which is not determined by a single designer and which cannot easily be predicted in advance.

Using Elgg for institutional learning: A case study

The following sections contain a case study in which social software was used to support learning in a credited online course. Yin (2003) argues that case study is the most effective form of investigation when used to answer research questions focussed on ‘how’ or ‘why’ rather than “how much?’ and when the context of investigation is contemporary or rapidly changing. In addition case studies have also proven especially relevant and valued in contexts in which results are important and familiar to practicing professionals. Effective use of social software in formal education contexts matches all of these criteria, indicating a fit between research methodology and the problem addressed in this paper.

The purpose of this study was to implement and explore the implications of attempting to explicitly employ group, network and collective strategies in a concrete course setting. We were primarily attempting to discover the strengths and weaknesses of these aggregations within a formal context, but the results led to some surprising revelations regarding ways and means of presenting and discovering people and information within spaces that mix these aggregations.

About the course

An online final-year optional undergraduate course on Developments in Learning Technologies (CI326) has been running at the University of Brighton for three years. Its cohorts are primarily drawn from students in the areas of computing and digital media. Conceived from the start as a social, learner-led course, it has employed a pedagogy very firmly based on communication, reflection and active engagement. Its syllabus is research-oriented, and is loose and flexible, driven almost as much in pacing and content by the students as by the tutor. Students are encouraged to research widely, both through literature and practice, and contribute their findings to the group. From the start, the primary means of communication has been through blogs and wikis, with some limited use of conventional discussion forums, online chat and webinars. Little static content is provided apart from some problem-based exercises, questions and help with the learning process and assessment.

In its first year, the course was run primarily through an extended Blackboard system, with add-in Building Blocks providing Blog and Wiki functionality. The system was entirely closed, available only to members of the course. As such, despite use of (loosely) Web 2.0 technologies, the dynamics and style of the course were firmly rooted in traditional CMC approaches to online learning and only supportive of the group mode of the Many.

In its second year, much of the communication was shifted outside of the Blackboard environment. An Elgg-based Community was created in which the main tasks were posted for each week and much of the dialogue (through blog comments) occurred. Students maintained their own blogs, tagging entries relevant to the course with its course code (CI326). The rich trust model provided by Elgg enabled them to limit access to the group if they wished, or to make it available to logged on users or to be entirely public. This iteration began to straddle the borders of groups, networks and collectives, but the tight structure provided through the Blackboard system imposed a pattern of behaviour that was still fundamentally controlled by the tutor. For example, Blackboard held all details and mechanics of assessment, was used to make announcements (including, fairly unsuccessfully, via SMS), hosted threaded discussions and contained all course documentation. Weaknesses in the wikis available as plugins for Elgg also meant that Blackboard continued to be the location for the course wiki. Though integrated through single sign-on and the use of system variables to pass identities from Blackboard to Elgg, it was an uncomfortable hybrid, with various bits of functionality (most notably discussions but also some content presentation) duplicated between the systems. It was also hard to maintain, necessitating the duplication of changes in one to the other, and not always easy for students to navigate, search and retrieve and explore due in part to different navigational metaphors and processes between the two.

As a result, and to explore the potential of social network systems to support formal learning, virtually all of the content and process in the most recent iteration were migrated to Elgg. Blackboard became a simple departure point, with no significant contribution to the structure of the course and little use of its technologies apart from a single real-time session employing its virtual classroom. The course author (first author of this paper) paid some attention to introducing a little used top-down order into Elgg by adding some structure to the Community site, including the provision of a menu of fixed tags (blog categories), a Community page containing media feeds submitted by the tutor and the learners, a listing of recent postings, information about the course, a course tag cloud and some fixed links to further pages (all stored elsewhere on the Elgg site) such as schedules, assessment and other course documents (figure 2). However, it is important to note that these interventions only affected the course Community itself, which is a small and densely connected part of the broader university network of users and Communities supported by Elgg.

The course blog was used to provide weekly tasks, revealed a week at a time (although an outline was also provided so there were no large surprises). This emergent design served two purposes:

  1. On the one hand, it offered a responsive mode of teaching, adaptable to the changing context of the larger environment. For example, during the course the judgement in favour of Blackboard’s patent claim against Desire2Learn came to the fore, provoking significant discussion at a timely point in the course.
  2. It enabled a responsive adjustment to learner needs. For instance, when the students were overwhelmed by other workloads, it allowed the tutor to relax the pressure by providing a catch-up week and, when learners expressed an interest in using more synchronous technologies, it made it easy to cater for those needs by scheduling in more synchronous sessions.

By and large, the tutor’s weekly postings followed a consistent pattern, with a specific activity and a (usually related) theory of the week to read about. Examples include setting up a Moodle site, discussing the strengths and weaknesses of reusable learning objects, engaging in a realtime webconference, using a multi-way video conferencing tool, discussing the Blackboard patent claim, attending a web presentation on educational social software and contributing to a wiki of learning technologies (hosted through an Elgg plugin). It was suggested that students should blog about these tasks and readings but it was not a requirement, and any posts that might be linked in some way to the course were acceptable, such as reflections on other classes, newpaper articles, TV snippets and so on.

Collaborative bookmarking was enabled for the Community and seeded with some annotated URLs. The students were mildly encouraged to add others, though this facility was only used by one student.

Students were expected to write something related to the course in their personal blogs at least once per week. It was felt that personal blogs were more appropriate than the course blog for these reflections, partly because of the convenience of having each student’s work in one place, but more importantly to give students a strong sense of ownership of their work by making it a part of their own personal space. These blogs were explicitly unassessed, but it was made clear that marks would be lost if they fell behind. This was intended to ensure engagement without overly constraining the process through assessment. The intention was that students would not be so afraid to make mistakes and would not feel too constrained in what and how they wrote, yet they would still feel obligation to participate actively in the course  Students were shown how to make course-related blog posts visible only to others on the course if they so wished. This enabled them to keep posts for this course separate from any others that they might choose to make and allowed them to experience the benefits of the more tightly knit, trustworthy group without network intrusion.

The main assessed task allowed students to choose a topic that interested them and either to write a conventional research essay, or to produce an e-learning artefact with a shorter essay explaining its theoretical underpinnings.

Seventeen students enrolled on the course, with a diverse range of backgrounds and motivations. Some were attracted more to the subject matter, while others were attracted more to the self-regulated way of learning. All were in their final semester at the university and were taking traditional face-to-face courses as well as being engaged in personal projects. Two made only a couple of postings near the start then silently dropped out, leaving 15 actively engaged throughout.

In keeping with traditional group-oriented approaches, the tutor played a moderately active role through dialogue within the main Community, email responses, real-time sessions and comments on students’ personal blogs.

Figure 2 shows the entry page for the CI326 course Community, indicating some of the complexity of the different modes of the many that govern its dynamics.

CI326 community entry page

Figure 2: the entry point to the CI326 community, showing overlapping spaces

 

Positive outcomes

Note: comments in italics in this and the next section are the students’ own evaluations gleaned from the formal evaluations and from focussed chat sessions conducted online. The original spellings and punctuation have been preserved. Note that ‘module’ is the term used at the University of Brighton to refer to what is known more commonly worldwide as a ‘course’.

By most conventional criteria, the course was a success. As in previous iterations, the students greatly enjoyed the loosely heutagogic approach (Hase & Kenyon, 2000) that gives control over not only the time and place of learning, but also a significant amount of the content, components of the assessment and, within bounds, the degree to which they engage socially. This was found to be highly motivating:

“I really enjoyed the amount of freedom within the module, the openness to researching your own ideas etc really added to this module”

Of course, it should be noted that students may have selected this course because of the freedom it offered and that the two who dropped out may have been positively demotivated by this form. Unfortunately it was not possible to contact them to discover the reasons they stopped contributing.

The document-centric mode of discussion encouraged by blogs meant that those who wished to take a more social approach were able to engage in discussion, while those who did not could take a more publication-oriented approach. As has been observed in other computer-mediated-communication oriented courses (Anderson, 2006), around 20% of the students were very active in discussion, while the rest tailed off rapidly with those making zero-to-two voluntary contributions making up around 50% of the cohort. However, because it was required of them, all of these posted regularly to their blogs and engaged in specified discussion activities.

The quality of submitted work was typically high, reflecting the motivation that comes from having significant control over its form and content as well as the fact that this was an optional course that had been explicitly selected from a range of options. The average grade for those who finished, ignoring one case of plagiarism, was B+, where C+ would be the norm within the university. There was great diversity in subject matter and style to the assessed work. Students took good advantage of the freedom offered to explore areas that were of personal interest.

Although they found the workload high and had difficulties fitting the self-directed working patterns required for online learning with those of conventional lecture/tutorial based courses, by far the majority of students found it highly rewarding:

“Although the workload was a little too much at times, I thoroughly enjoyed this module.”

They greatly valued the input of the tutor and, especially, others on the course and beyond, as well as being able to (especially) look at and (to a lesser extent) to discuss the work of others:

“I did read other peoples blogs on the activities as well to help me understand it better from others who have just learnt the concept as well”

“its encouraging also when others comment on your blog, i had a few coimments”

“Overall I have enjoyed this module and have learnt a lot, from both Jon and from some of the other students.”

“Some times it felt a bit lonely working on your on, and reading a lot without others.  I found it encouraging when others commented on my post as it made me feel that someone was interested in my findings.”

The value of the broader university network, while not a dominant feature, is clear: in this recent iteration, around 25% of the comments added to student blogs were by people who had an interest in learning technologies, not because they were students of the course. Although membership of the Community was moderated by the course author, in practice anyone from within the university was granted access if they requested it and most students added at least some posts that were accessible to the broader network. The leader of the university’s Learning Technologies Group, whose interests in e-learning and social software fitted well with the course, played a particularly active role, but three others from outside the community added comments to student blogs. No one from outside the course contributed to the course blog itself, which was clearly perceived as a closed group.

Revealing and hiding posts from the group and the broader network

Because many (though far from all) of the postings had access limited to the course group by their authors, there were relatively few opportunities for more engagement from the broader university network. The spread of permissions was quite varied. While some decided on the level of access control they would use and stuck to it (whether available to the university or just the community) others varied access throughout the module. Overall, 90 posts in personal blogs were restricted to the course community, while 34 were available to the whole university. Only one made a posting completely public which, interestingly, was his assignment, something most students kept private or limited to the community. Seven students restricted all their posts to the group, one made all of them available to the entire network of logged in users and the rest used a mix of restricted and unrestricted posts. There were a number of other posts that should have appeared in personal blogs but that found their way into the main course community blog, which all inherited the default permission of being available only to the community.

Most students went beyond the confines of the course to explore and, occasionally, contribute to the wider network. It is not possible to give an accurate account of how much they would have done this anyway and how much it was encouraged by their engagement with the course, but it is probably safe to assume that most spent more time using the system than would otherwise have been the case.  Few posted non course-related articles to their blogs, but most made use of the network in some way. However, their lack of active posting to the larger network suggests that they were largely treating the CI326 community as a closed group.

Interestingly, even the two most shy and retiring of students in the context of the social activities of the course went to some effort to build a connected network of their own friends by adding others from outside the course as ‘friends’. 

Negative outcomes: lost in social space

Students were nearly unanimous in disliking the Elgg interface, some vehemently so, mostly finding it confusing and disorientating. A few who had explored further appreciated its functionality, but even they were critical of the system’s design:

” i hate it”

“dont like it”

“no organisation”

“initially i always got lost in the navigation”

“actually i did get lost  quite a bit”

“i should have explored it...but it was a little frustrating to find stuff”

“i didnt know that it did all those things”

“its not very intuitive”

“I think the hardest part of the module was the Community of Brighton. I found myself getting a bit lost and the navigation is not the best. I found out that it provides RSS feeds and Podcasts, I think would have been interesting and helpful to have had a look at these.”

Part of the reason for student dissatisfaction is that Elgg’s basic interaction design is problematic in a number of key areas. A lot of work has been conducted by the team at the University of Brighton to make improvements and a range of templates have been made available to make it more usable and attractive, but there are still flaws at the architectural level. Users frequently complain about difficulties finding options, which are sometimes hidden in unexpected places. This can have a heavy influence on behaviour. For example, to change the appearance of a Community site involves visiting one’s own preferences page rather than the Community’s preferences, which in turn means that the vast majority of Community sites tend to look the same, even when other templates are offered, because it is just too hard to work out how to change them. If Elgg were the only social networking system in the world such issues might be overlooked, but well over half of the students were active users of social networking sites such as Facebook, MySpace and Bebo, with which Elgg did not compare favourably. Several mentioned that the interface seemed clunky and primitive by comparison.

Groups and networks

There is a more fundamental problem, however, that relates back to our taxonomy of Groups, Networks and Collectives. Just as poor interaction design can lead to being lost in hyperspace, following social networks can lead to being lost in social space. Although there were some issues with general interaction design and difficulties with exploring the functionality, students particularly singled out the lack of structure as the main flaw, making it hard for them to know where to go and what to do next. To some extent this is because it is different: the course runs alongside others that use Blackboard, which through design and policy provided a very consistent interface to all other courses that the students have been taking. However, the deeper issue relates to the shifting contexts between personal, course, university and public spheres. Intentionally, Elgg’s interface does not differentiate clearly between these contexts. While the course-related aspects were fairly structured, once students moved to their individual spaces or to other Communities within the Brighton Elgg space they were set adrift. Different communities and individuals use different templates offering the same functionality via different layouts. Some emphasise the personal space, others focus on the community space. Moving between such spaces is disorientating. Without clear borders between the personal, group, network and collective space, the benefits of introducing more top-down design into the one community space were greatly diminished.  To some extent this may have been caused by the author’s attempts to introduce a different form of organisation into this course, making it harder for students to know what to expect in other areas that did not follow the same pattern. However, even in Elgg’s unmodified form, different site templates used by different individuals and communities inevitably lead to some confusion, with the same functionality and terms appearing in different positions, groupings and formats in different spaces. If the spaces were completely distinct (as they are in, say, Blackboard’s course views) this may have caused little difficulty but, in Elgg, the spaces blend and blur into each other. For example, there are (nearly) always links to one’s own space and those of one’s communities wherever one may be in the system. Perhaps worse, the spaces may be almost the same among different individuals and communities, making the shift in context barely perceptible.

An apparently minor design feature of Elgg is that, like many social networking systems, connections are made to individuals by linking to them as ‘friends’. While this is a coarse division at the best of times, in a formal academic setting it is positively odd. One effect of this within the CI326 course has been that students posting completed work that they wished to keep private, viewable only by the tutor, had to create a special access group containing only the tutor. The only way to do this within Elgg is to add the tutor as a friend. This shoe-horning of a network feature into a teacher-dominated group role highlights the problem of drift across the different modes of the many. It is not surprising that students find it confusing when these different forms of engagement are so jarringly juxtaposed together.

A similarly awkward juxtaposition affects the search facility of Elgg, but in this case the effects are more than psychologically jarring. The search facility of Elgg is strongly oriented to finding people and Communities rather than specific content. Thus, if one selects a tag, the main results shown by the search displays all the users and Communities that have used that tag, right across the system. This is a collective feature of the system, but one that does not differentiate clearly between the entire collective (recalling that this involves some 15,000 or so users) and the current context, be it in a personal or community space. For more commonly used tags such as e-learning or education , this renders the search function almost useless. At the time of writing, selecting the term e-learning , for example, leads to a page of search results that takes around an hour to load over a fast connection and, when printed, occupies over 100 A4 pages, including users and communities using the tag as well as many thousands of individual posts and files that are so-tagged.

 As an aside, a similar problem afflicts many large tag-using social sites: the results shown in the tag clouds are often remarkably stable and similar across different sites as the most popular tend towards a consistent subset (Zeldman, 2005). This in turn magnifies problems of ambiguity, metonymy, synonymy, homonymy and so on as the tags are spread thinly across multiple contexts. There is typically insufficient sensitivity to the context of use and little benefit is gained from the long tail of seldom-used tags, apart from in explicit searches and filters.

To add to these difficulties, at least some of the structure in Elgg is determined by the students themselves, mainly through tagging behaviour, posting of blog entries and development of the wiki. Unlike traditional LMS-based courses, the structure is fluid and changing. Despite the efforts of the course author to impose some top-down structure through a modified template, the chaotic jumble of the public, semi-public, community and personal space is generally a click away. For instance, although it was possible to create a tag cloud that was specific to tags used by those in the course Community, following those links would lead to results from the entire university Community. Without significant extra programming, group and network space were impossible to separate. This is a far from trivial problem when dealing with an educational community of many thousands, as observed previously when discussing the problems with searching.

Elgg automatically provides a page with links to the personal spaces of all members of the Community. As a matter of policy, with a view to linking the networked and group spaces, the course author had allowed several people from outside the course to be a part of the community, and those from the previous year’s iteration were still shown as members (though, sadly, none contributed to this year’s iteration), making it hard for students (not meeting face to face) to identify those who were active classmates. This in turn made it harder to find each other’s blogs. As a result, the network space intruded into the group space to the extent that it severely undermined the ability of the group to function. Elgg’s security model makes it impossible to provide private blog entries as an RSS feed – only public entries are shown and, as most students were not keen to share private musings outside the community, the only effective way to provide links to student postings would have been through a static, custom-built web page aggregating  contributions of course group members. In retrospect, this would have been a worthwhile investment of time, though it would still have only worked at a coarse, per-user level rather than (more usefully) displaying recent posts.

The decision to use the Community blog as the primary means of communicating tasks and news meant that sequence turned out to be an intractable issue: while the date-order provided by blog entries is a useful means of maintaining a real-time narrative of the process of the course as well as clarifying tasks for the current week, it provides limited support for those wishing to take a different route through the course, and is disrupted easily by students adding their own posts to the blog. For the course author, it has been quite difficult to apply any top-down design to the way that links and resources are presented through the blog. Indeed, the positive benefits of structure being created emergently by the group actually militate against top-down structure. Some techniques have nonetheless proved helpful in attempting to achieve a modicum of structure in the blog, including:

Elgg includes a subsystem known as the Presentation Tool, a mechanism primarily intended for the presentation of eportfolios that also lends itself to structuring information more statically and formally. Unfortunately, for reasons known only to the Elgg developers, at the time that the course ran this was not an option for Communities, only for individuals, so it was of no use in this context.

Finally, like much social software, Elgg is in a state of perpetual beta. This means that any control that may be exercised in one version may be lost in the next and lessons learnt may not transfer to the next iteration. The release of version Elgg version 1.0 later in 2008 may increase stability and usability of the platform.

Discussion

We have presented an overview of some of the problems when attempting to use a social application in a formal institutional environment. Some of the issues that have arisen are simply a result of using software that is still going through a rapid beta cycle and still lacks critical education affordances. Some could be solved with a small amount of programming or a more rigorous use of template modifications. However, it seems that some are more fundamentally rooted in the clash between a conventional group-based approach to teaching and a desire to open the system up to the broader network, as well as to leverage collective elements.

In both networks and collectives, the clusters and structures that form from the bottom up play a distinct role in affecting and, often, guiding our behaviour. Thus the Many becomes an active and distinct agent in the system, an entity with a role to play that is in many ways similar to that of learners and teachers. This in turn offers a potential route out of the constraints of the managed environments of yesteryear, enabling both learners and teachers (rather than system designers) to play more active roles in determining the structure of the environment, but equally allowing them to be guided by this emergent structure if they prefer to play a less autonomous role. In many ways, if it works well, we may find social software offering the best of both worlds, delivering both a human-centred community and a guiding structure for those not able or willing to engage in the construction of that community.

The big weakness with this argument is that the structure that develops is, as we have seen, neither necessarily nor even likely to be of pedagogical value. To some extent this can be compensated for by appropriate design. The bottom-up emergent nature of collectives and groups makes them complex adaptive systems, not unlike natural ecosystems or complex human systems such as cities. In nature, environments play a large role in determining the ecosystems that develop there. The kinds of organism that develop in the artic would seldom survive in the tropics, mountains tend to produce organisms like mountain goats and oceans tend to produce organisms like fish. While there is a vast variety of forms that can survive in a particular environment, there is an infinitely larger variety that cannot. Just as different natural environments can act as constraints on the kinds of organisms that may evolve there, we need to build learning landscapes that make educationally valuable outcomes more likely. The first author has developed a set of principles for social software development and deployment that increase the likelihood of such structures being of educational value  (Dron, 2007) Some of these are implemented well in Elgg some are implemented only poorly or not at all.

Worse still, the broader institutional landscape is not conducive to the informal and emergent nature of social software systems. It is not just a problem of shifts between different modes of the Many. Networks and collectives fit uncomfortably within an institutional ethos. Within a formal institutional course-based environment there are many constraints, including the need for assessment, the need for formal learning objectives, competition from other more content-oriented and structurally rigid courses, time restrictions (including length/period of a course and timetabling), restriction on access due to costs and prerequisites, habits of both teachers and learners, power relationships (notably that teachers tend to be assessors as well as conditioning and expectations of teacher/student relationships) and much more. The crowd does not respect institutional divisions and boundaries and yet those boundaries can have a large effect on behaviour. If, for instance, we create Communities on Elgg to support course groups, the social and organisational boundaries will parcellate those communities as surely as oceans will separate islands from each other, which goes some way towards explaining the difficulties students encountered when shifting between modes of the Many. Most of these islands will be too small to develop thriving ecosystems by themselves, especially as they often prescriptively shaped by the presence of teachers within them. In such uses, we have so far seen that the ecosystems that develop are more like gardens than jungles, tended and cultivated by well-meaning teachers. There is a constant jostling for control between the crowd and the teacher, a blend of the top-down and bottom-up that can be successful, but really offers little that is different from a well-managed discussion forum in a traditional online learning environment. The true potential benefits of the network and the collective, particularly with regard to their role as determinants of structure are not exploited. To make matters worse, most social software provides little support for the gardener. Elgg is resolutely emergent and community- and/or user-centric in its design. It offers little out of the box to assist the gardener in building a structure. While it is in principle possible to radically adapt templates and stylesheets to different needs, the skills required to do so are beyond the abilities of all but the technorati and, even then, the gardens are just a step away from the jungle that threatens to encroach at every turn. Just as the earlier generations of learning management systems constrain by taking away choice, so does the new generation as represented by Elgg.  For example, control of the form of a Community area is an all-or-nothing affair. The creator of the Community can set up some structure through the use of widgets and, where provided, alternative styles, but this role cannot be distributed to others in the Community. Items created by the Community can be presented as a discussion or as a blog, but there is no scope for hybrids. Just like the older LMS, some things are easier to do and some things are harder. Blogs are always there and always easy to update. Uploading files is easy, but making use of them between components and widgets is harder.

Perhaps most significantly of all, if the teacher is not in control then, along with the developers, the crowd dictates the form and structure of the system. If the developers have not included facilities and forms that are likely to lead to educationally useful experiences, the system will fail both as a top-down and a bottom-up design. Elgg is a good system which offers many facilities to enable learning communities to come together and structure their own environment, but that structure is loose, currently and perhaps inherently hard to navigate and not always trustworthy.

In education, groups typically inhabit controlled, safe spaces, like classrooms and equivalent virtual spaces. Tools that support networks run the risk of invading those spaces. To some extent they may be seen as the virtual equivalent of open-plan classrooms, but without the safeguards and environmental and systematic controls that keep them controllable.

Possible solutions

Separate spaces

One simple way to deal with most of the problems we have observed would be to segment the different kinds of environment, as was attempted in the previous iteration of the course that maintained structure through Blackboard and that allowed controlled excursions into the social space within that rigid framework. Alternatively, the two could be kept entirely separate, with a traditional Web 1.0 structure and virtual field trips into the social space.

The problem with these methods of parcellation is that the engagement barely rises above the level of Web 1.0. A course like CI326 works fine with such an approach, enabling much learner control and good group communication, but the potential benefits of the network and the collective are limited and constrained. Most notably, it offers little opportunity for the course to adapt and change as the community changes, apart from through direct intercession of the tutor.

A hybrid solution might be to dedicate an Elgg server (virtual or otherwise) to the course, with its own distinct and exclusive users, offering a safe and unrestrictive haven from which virtual excursions and planned incursions could be managed. While this would overcome the problems of shifting modes of the many, it would primarily be a group-oriented solution, albeit one that enables that group to act as a small collective with some linking in of the network from the external world. As a consequence, it would not enable the development of distinctive pedagogies that take advantage of the Network and the Collective. For example, the elgg environment could be opened for both reading and response to professional groups, alumni, external experts and collective resources could be developed and shared that lasted far beyond the timeframe of a course section

Better preparation

It is likely that CI326 students would have used the network and collective more effectively if greater training were given in both the practical use of the technologies and the pedagogy that was intended. Plans for the next iteration of the course include the provision of a set of readings and exercises that will ensure learners are more au fait with the capabilities of the technology and the expectations of engagement with the wider community and the process itself will make it more explicit that the broader network will play a larger role. However, there are dangers that too much teacher control of the process might negate some of the benefits. If students engage in the broader network simply because it is a course requirement then there is a risk of an uncomfortable mismatch between the emergent control of the network & collective and the structuring control of the teacher. It is likely that succeeding generations of students will themselves have greater expertise and experience with social software systems and can be expected to play more active roles on network and collective construction and maintenance,

Technological changes

Technological fixes to some of Elgg’s weaknesses will also be of benefit. It should be noted that Elgg is in a state of flux as it approaches version 1.0 and the comments here relate to a vignette in time, discussing problems that may have already been solved by the time of  publication. However, the issues raised touch upon some universal concerns and so are worthy of discussion.

Minimally, better support for aggregating user profiles and blogs would make it far simpler to maintain the walled garden for the group space while keeping a clear view of the network and collective jungle just outside it. Another valuable addition would be the ability to create sub-communities, each with their own sets of rights and access controls, inheritable from those above. This would make the move between different modes of the many less jarring, especially were it to be combined with clearer recognition of user context, particularly in terms of searching and tagging. For example, tags might be applied and used at a course and/or program level, rather than across the entire greater community. Another way that this could be achieved would be through taggable tags: for example, tags that were sub-tags of, say, CI326 would be segregated from others in the broader community. An ideal solution would probably combine sub-tags and sub-communities.

The Elgg software is due to move to version 1.0 over the summer of 2008, and early indications are that many of the basic usability issues will be addressed and that it will adopt an interface more akin to that of popular social networking sites. A particularly encouraging part of the roadmap is the increased componentisation of the software, with core functionality available in distinct modules, along with richer support for open interoperability standards and support for more variegated relationship than ‘friend’. This should enable richer mashups of both Elgg technologies and those inside and outside the university’s network. If it were possible for the teacher to select appropriate components for a given Community, including those from within the walled garden of the LMS as well as those within the system and in the public sphere, then it would be significantly easier to both improve usability and to enable safe, clearly signposted access to the rest of the network.

To maintain a component-based site, a richer set of authoring tools than those currently supplied with Elgg will be required. Several of the structural elements added to the CI326 course required direct manipulation of HTML in templates and the CSS of the course style sheets, while the tag cloud involved some PHP programming. Minimally, the Presentation tool should be made available to a Community. A simple, preferably visual, interface that enabled not only the assembly of different environments for different Communities, but also the inclusion of content and users from other sites would be ideal. To achieve this, far greater use needs to be made of standards and technologies such as OpenSocial, microformats, JSON, OPML, APML and the raft of standards brought together at dataportability.org.

Conclusion

In the past, design patterns and cultural expectations led to a teacher-centric instructivist model of learning that was antagonistic to many more learner-centric forms of education. New generations of so called Web 2.0 technologies lead to a learner-centric, constructivist model that is unsuitable for many more teacher-centric forms. Neither is universally applicable.  Sometimes, for some subjects and in some contexts and for some learners, an instructivist approach is exactly what is needed. Given this dichotomy, we have presented results here that suggest a need for clear separation of these two modes, but not as a simple bifurcation. Rather, we need to mash up the bottom-up and top-down in ways that emphasise their differences but that do not exclude participation in either. What we are seeking, on average, is neither Web 1.0 nor Web 2.0, but something in between – Web 1.5.

We abhor the structural hierarchies that pervade the first (and current) generations of learning management systems – they are probably better labelled as ‘teaching management systems.’ However, the dangers of using general purpose social tools and environments are equally great. What we need is something in between, not a new computer program but a new way of constructing environments out of other tools. We call this Web 1.5, co-created by both learners and teachers.

In keeping with the mashed up nature of this ever more connected world, little of what we suggest is new. As dataportability.org’s slogan rightly proclaims, “Invent Nothing!” What is of increasing importance is not the introduction of novel technologies and teaching patterns, but in discovering different ways of assembling what we already have in ways that match the needs of teachers and learners, giving them (rather than technology designers and maintainers) the control they need over their learning environments.

References

Ahern, T., Peck, K., & Laycock, M. (1992). The effects of teacher discourse in computer-mediated discussion. Journal of Educational Computing Research, 8 (3), 291-309.

Anderson, T. (2006). Social Software Applications in Formal Online Education. Paper presented at the The 6th IEEE International Conference on Advanced Learning Technologies, Kerkrade, Netherlands.

Bernard, R. M., Abrami, P., Lou, Y., Borokhovski, E., Wade, A., Wozney, L. W. P. A., et al. (2004). How Does Distance Education Compare to Classroom Instruction? A Meta-Analysis of the Empirical Literature. Review of Educational Research, 74 , 379.

Brusilovsky, P. (2001). Adaptive Hypermedia. User Modeling and User Adapted Interaction, 11(Ten Year Anniversary Issue), 87-110.

Christensen, C. (1997). The innovator's dilemma - When new technologies cause great firms to fail . Cambridge: Harvard University Press.

Dickey, M. (2004). The Impact of Web-Logs (Blogs) on Student Perceptions of Isolation and Alienation in a Web-Based Distance-Learning Environment. Open Learning, 19(3), 279-291.

Darby, J. (2003). UK eUniversities Worldwide: Who we are and what we want from standards. Retrieved 14/12/2003, 2003, from http://www.imsglobal.org/otf/IMS-Darby.pdf

Dron, J. (2007). Control and Constraint in E-Learning: Choosing When to Choose . Hershey, PA: Information Science Pub.

Dron, J. (2007). Designing the Undesignable: Social Software and Control. Educational Technology and Society, 10 (3), 60-71.

Dron, J., & Anderson, T. (2007). Collectives, Networks and Groups in Social Software for E-Learning. Paper presented at the Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education Quebec.

Dron, J., & Anderson, T. (2007). Collectives, Networks and Groups in Social Software for E-Learning. Paper presented at the E-Learn 2007, Quebec City, Canada.

Dron, J. (2006). Any color you like, as long as it’s Blackboard®. Paper presented at the E-Learn 2006, Hawaii.

Dron, J. (2006). Social software and the emergence of control Paper presented at the ICALT 2006, Kerkrade, Netherlands.

Ellison, N., Steinfield, C., & Lampe, C. (2007). The Benefits of Facebook "Friends:" Social Capital and College Studentsí Use of Online Social Network Sites. Journal of Computer-Mediated Communication, 12(4), 1143-1168.

Franklin, T., & van Harmelen, M. (2007). Web 2.0 for Content for Learning and Teaching in Higher Education. London: JISC.

Garrison, D. R., & Anderson, T. (2003). E-Learning in the 21st century. London: Routledge.

Govani, T., & Pashley, H. (2005). Student Awareness of the Privacy Implications while Using Facebook.Unpublished manuscript.

Hase, S., & Kenyon, C. (2000). From Andragogy to Heutagogy [Electronic Version]. ultiBASE . Retrieved 7th July 2008, from http://ultibase.rmit.edu.au/Articles/dec00/ hase2.htm

Hummel, H. G. K., Van den Berg, E. J., Berlanga, A. J., Drachsler, H., Janssen, J., Nadolski, R. J., et al. (2007). Combining social- and information-based approaches for personalised recommendation on sequencing learning activities. International Journal of Learning Technology, 3(2), 152-168.

Kerawalla, L., Minocha, S., Kirkup, G., & Conole, G. (in press). An empirically grounded framework to guide blogging in higher education. Journal of Computer Assisted Learning.

Koper, R. (2005). Designing learning networks for lifelong learners. In R. Koper & K. Tatterssall (Eds.), Learning Design (pp. 239-252). Berlin: Springer.

Mayfield, R. (2006). Power Law of Participation [Electronic Version]. Retrieved 7th July 2008, from http://ross.typepad.com/blog/2006/04/power_law_of_pa.html

Moore, M. G. (1993). Theory of transactional distance. In D. Keegan (Ed.), Theoretical Principles of Distance Education. New York: Routledge.

O'Reilly, T. (2005, 30/09/2005). What Is Web 2.0: Design Patterns and Business Models for the Next Generation of Software.   Retrieved 30/11/2006, 2006, from http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html

Olson, T., & Wisher, R. (2002). The Effectiveness of Web-Based Instruction: An Initial Inquiry. International Review of Research on Open and Distance Learning .

Pedersen, S., & Macafee, C. (2007). Gender differences in British blogging. Journal of Computer-Mediated Communication, 12(4).

Senge, P. M. (1993). The Fifth Discipline- the art and practice of the learning organisation. Chatham: Century Business.

Siemens, G. (2004). Learning Management Systems: The wrong place to start learning Retrieved 29th March 2008, from http://www.elearnspace.org/Articles/lms.htm

Sitzmann, R., Kraiger, K., Stewart, D., & R., W. (2006). THE COMPARATIVE EFFECTIVENESS OF WEB-BASED AND CLASSROOM INSTRUCTION: A META-ANALYSIS. Personnel Psychology, 59 (3), 623-647.

Winn, W. (2003). Learning in Artificial Environments: Embodiment, Embeddedness and Dynamic Adaptation Technology, Instruction, Cognition and Learning, 1, 87-114.

Yin, R. (2003). Case study research: Design and methods. Thousand Oaks Ca.: Sage.

Zeldman, L. J. (2005). Remove Forebrain and Serve: Tag Clouds II [Electronic Version]. Zeldman.com . Retrieved 7th July 2008, from http://www.zeldman.com/daily/0505a.shtml

Zemsky, R., & Massey, W. F. (2005). Stalled: E-learning as thwarted innovation. In A. Carr-Chellman (Ed.), Global perspectives on e-learning: Rhetoric and reality (pp. 241-255). Thousand Oaks, CA Sage Publications, Inc.