E-education: Landoni and Diaz: JoDI

E-education: Design and Evaluation for Teaching and Learning

Monica Landoni and Paloma Diaz*
University of Strathclyde, Glasgow, UK
Email: monica@dis.strath.ac.uk
*Universidad Carlos III de Madrid, Spain
Email: pdp@inf.uc3m.es

Recent technological developments have provided a powerful stimulus for the production of a range of electronic materials for education. A number of products and prototypes to assist teaching and learning have been produced and educational materials have been extensively published electronically, but it is still unclear to what extent all of this is of use to students and lecturers/tutors when it comes to real teaching and learning. Looking at the example of electronic books indicates not only the main reasons why electronic materials have not completely replaced the physical counterpart, but more importantly suggests how to improve the quality of the materials and tools currently available.

Development of e-books has been led primarily by technology instead of by users' requirements, and the gap between functionality and usability is sufficiently wide to justify the lack of success of the first generation of e-books. In addition, the label "electronic book" has been applied to products that share little with their paper counterparts, stretching the metaphor to the point of generating unnecessary confusion for consumers. These considerations can be easily generalised to other types of electronic learning objects where there is still lot of confusion about what electronic learning objects are, standards are yet to be found and adopted successfully, and usability issues have been almost completely neglected. For this reason we hope that the papers presented in this special issue of Journal of Digital Information will be of great interest and utility to people in education involved in the design, production, delivery and use of electronic materials and tools.

This issue evolved from a workshop hosted by the 6th European Conference on Research and Advanced Technology for Digital Libraries (ECDL), held in Rome in September 2002. One of the objectives of the ECDL conferences is to raise awareness of the various European initiatives on-going under the wide label of digital libraries research. The workshop was meant to attract the attention of a mixed and still undefined polyhedric community of designers, publishers and users of electronic material for education, where often researchers play more than one role (classically by being authors, designers, and then delivering the materials).

The workshop presented research papers focussed specifically on e-learning objects, their design, production and use. More importantly, the workshop was effective in generating discussion among participants. There was a sense of relief when the authors realised they were not isolated. They had been working as part of research groups where the e-learning component was not central but more a side-effect of dealing with techniques taken from other related and wider research areas such as information retrieval, hypertext and hypermedia design or human-computer interactions. Education was seen as just another possible application area for techniques already widely experimented with elsewhere. The workshop was intended to reinforce the idea that design and usability of e-learning objects deserve more attention, and that targetting content at audiences is crucial to ensure the success of these objects.

So while the authors and organisers of the workshop recognise the importance of the contribution of related research areas, we advocate the need to make e-education a research area per se with its own profile and status. Theories and models, standards and formats, as well as evaluation frameworks and methods have been studied and applied, and even if some have been borrowed from other disciplines they have now assumed an independent personality. To share the relief felt by the workshop participants with other researchers in the field who may still feel isolated, in this issue we present peer reviewed and revised versions of selected papers from the workshop, to reinforce our belief that this research area deserves more credit and attention.

Design and usability of e-learning objects are the main focus of the papers in this issue. It is crucial to identify needs and requirements of the target community so that the design can fulfil their needs and expectations. The acquisition of a well-defined user profile is an essential component of the design process for the successful development of e-books, for example. It has become clear from a number of studies (e.g. Wilson et al. 2002, Landoni et al. 2000) that students, even more than generic readers, appreciate the advantages of electronic materials in term of portability and overall ubiquity, but they also value legibility, presentation and good design. There is also a clear demand for extra functionalities such as smart searches and dynamic indexing. Quality and the ability to provide extra facilities not available with paper textbooks are crucial for the future of electronic publications if they are to compete, as is the need for guidelines to help designers of electronic learning materials produce more effective communication tools. For this reason evaluation is the other essential step in the production of good electronic learning materials for education. Criteria and measures for evaluation need to be introduced together with suitable methodologies to study how these materials perform an educational role. Evaluation has to focus on usability issues, like those analysed in any interactive system (including consistency, predictability, self-evidence, and so on), as well as on educational aspects related to the quality of electronic learning materials as instructional resources, including criteria such as richness, completeness, motivation, and so on (Aedo and Díaz 2001).

Three papers in this issue emphasise design and the need for appropriate functionalities to be provided to make electronic material useful for readers. Caracciolo addresses the problem of providing "smart" search facilities to fulfil readers' needs. The paper discusses the possibility of enriching the electronic book model by using ontologies in order to provide the reader with better search facilities. Ohene-Djan and Fernandes provide an insight into the functionalities that e-books should inherit from paper counterparts in order to appeal to readers. They present a model dealing with personalisation and adaptation mechanisms in a formal way. Design and guidelines for the production of learning objects are the topics of the paper by Polsani, where e-books are considered as one type of learning object, a most relevant topic in the current research in learning technologies. The author provides much-needed definitions, methodologies and guidelines for the production of such objects for which several standards have been deployed.

Discussion shifts to the use of electronic material in education and, consequentially, on usability issues and evaluation techniques in the three remaining papers. Wilson discusses some results from a study of the impact of electronic materials on teaching habits in higher education. The paper describes reactions from educators in different disciplines to the introduction of electronic materials in their everyday teaching experience. While there is still an optimistic view that electronic materials can make a difference, complaints about a lack of content and technical limitations are common in a number of disciplines.

Naber and Kohle discuss problems lecturers face when producing good electronic materials for teaching in higher education. These problems can be overcome with proper training, support and adequate authoring tools. Different types of authoring tools for the production of e-books for use in in distance education are briefly presented and evaluated by Shiratuddin et al. The authors describe a small experiment designed to evaluate the usability of various software packages and the electronic materials produced with the software.

With this selection of papers we believe the issue has achieved its main goal: to provide readers with some insights into the state-of-the-art of research in usability in e-education, showing how independent this area has become. There are still a number of open challenges, such as the holy grail of setting recognised and workable standards, and the definition of criteria for measuring how successful e-learning objects are in achieving their objectives. This is what makes research in this area such a challenge.


Aedo, I. and Díaz, P. (2001) "Evaluation criteria for hypermedia educational systems". In Computers and Education: Towards an Interconnected Society, edited by M. Ortega and J. Bravo (Kluwer Academic), pp. 45-60

Landoni, M., Crestani, F. and Melucci, M. (2000) "The Visual Book and the Hyper-TextBook: Two Electronic Books One Lesson?" RIAO 2000 (Recherche d'Informations Assistee par Ordinateur - "Computer-Assisted Information Retrieval") Conference Proceedings, Paris, April, pp. 247-265

Wilson, R., Landoni, M. and Gibb, F. (2002) "A user-centred approach to ebook design". The Electronic Library, 20 (4) 2002