INTERFACES FOR SUPPORTING OVER-THE-SHOULDER LEARNING

Michael B. Twidale
Graduate School of Library and Information Science
University of Illinois at Urbana-Champaign
501 East Daniel Street
Champaign IL 61820 USA
twidale@uiuc.edu
www.uiuc.edu/~twidale

The paper examines the concept of over-the-shoulder learning: the informal collaborative learning of a computer application that occurs in brief episodes as part of regular work. Certain design implications are explored, including the representation of the process of system use, and the consequences of regarding interfaces as media for human-human interaction.

The rapid growth of networked services, along with continually falling costs of hardware, create new opportunities to use computer applications for work and leisure. Many potential users are not computer experts, and all of us encounter new applications and upgraded systems with new features and modified interfaces. Advanced applications developed for domain experts share with e-commerce web pages intended for all shoppers the challenge of how to provide sophisticated functionalities so that users can both easily learn how to use the system, and subsequently learn to use it more effectively. If it is too difficult to learn, users may just not bother. If it is too difficult to improve, users may continue to use it inefficiently, destroying the anticipated productivity gains of the advanced features. Ease of learnability is at the core of much HCI research, and the problems with the learnability of existing systems is at the core of many missed productivity gains (Landauer 1995).

There is a substantial literature, particularly in Computer Supported Cooperative Work (CSCW) research of systems deployments that failed because of an inadequate understanding of what people actually do in their work, and the consequent development of systems that solved the wrong problems (Grudin 1989). The designers had an erroneous set of assumptions about how the work was done. This paper contends that implicit in much interface design is the assumption that people will learn about the system either on their own or in formal training sessions. But what if that assumption is not always true?

Outside of schools and universities, how do people learn how to use a piece of software? It can be very different from the way that we organize learning for students, or training in businesses. Where learning is part of working it is often brief, incremental and informal. For many people, a key form of learning is not by going on a training course, or by reading the manual or the online help, or even by fiddling with the program on their own. Instead they lean over the shoulder of a work colleague who is using the system and ask for help. Or they may ask a colleague to come over, and who leans over their shoulder and helps them out. It seems reasonable to call this phenomenon ‘Over-The-Shoulder Learning’ (OTSL). Clearly not all such interactions are successful. The colleague may not be able to help, or may be too helpful and do it all so that the ‘learner’ knows little more at the end of the interaction. It is important to study these interactions in order to understand the criteria for a more successful learning episodes, and the implications for designing interfaces that, even if they cannot actively support this kind of learning, at least do not frustrate it.

One of the findings from a study of help giving in a library was the substantial amount of learning that occurs by watching what other people do (Twidale et al. 1997). This is both by watching from a distance as strangers ‘operate’ a library, and by sitting or standing next to a colleague and informally learning how the library’s online public access catalogue or a bibliographic database works. Students appeared to far prefer this mode of learning to attending ‘introduction to the library’ talks or using the online help or paper guides. More formal help giving occurred during interactions with library staff at the Help Desk, but even there the constraints of a queue of people meant that the focus was on brief episodes rather than anything resembling a traditional lesson or systems demo. In both cases it was noticeable that an application that had been designed to be used by people on their own (the library catalogue) was actually being used collaboratively. Indeed the phenomenon was even more marked at the Help Desk where the librarian often twisted the monitor round so that what appears on the screen could be used as a resource to aid the conversation in getting from what the user vaguely wanted, to what was available in the library, and how to find it. This is not a unique occurrence. Stanford University libraries use dual back-to-back screens so that both the librarian and the patron can see the results of searches, even though they are facing each other. This supports more productive discussion about the activity.

A subsequent study (Twidale 1999) examined learning in a different setting; a small office with five staff secretarial/administrative working in cubicles. These workers also had a strong preference to learn from their colleagues rather than to figure it out on their own, or to ask a much more expert systems administrator based in another office who knew far more about computers, but less about their work tasks than their colleagues.

Bannon (1986) was the first to raise the issue of supporting informal user help in interface design. Informal communication is an important part of effective work activity (Whittaker et al. 1994). Various studies of computer programmers show how they informally learn from their peers (Berlin & Jeffries 1992). Weinberg (1971) describes how vending machines in a common room became the focus of social chat. This was perceived as time-wasting, but their removal led to a decline in productivity, because interspersed within the social chat was significant problem-solving. Other studies show that the learning does not have to solely be between computing experts (Lee 1986).

Although OTSL merits more detailed study, we already have enough information to begin an examination of the implications it might have for designing more usable systems. It leads to the following broad questions:

• Do we need to bother?
• Is it important enough to merit complicating an already confusing interface design process?
• Is a well-designed interface intended to support solitary learning sufficient to support OTSL?
• Or does effective OTSL need quite different interfaces and supporting functionalities?

The library use study revealed the vital importance of context in explanations. The more you know about what a user knows, what they want to do and what they have tried to do so far, the more effective and efficient the help-giving interaction will be. It was found that librarians had difficulties in establishing prior context (Crabtree et al. 1997). A user may try some searches and fail, before going to the help desk, but find it very difficult to explain what they have actually tried because:

• They were focussed on the goals of the task, not what they were doing to achieve the goals.

• They found the system confusing, so it was difficult to remember all the details.
• They lacked the abstractions necessary for an overview of their actions, impairing recall.
• They may have made simple mistakes like typing errors, but the mind’s tendency to auto-correct means that they did not notice them.

The office study further emphasized the importance of context. Much of this was established for free – a colleague is more likely to understand the underlying work needs that precipitated the request and so offer more appropriate technical advice. However even in this ideal case problems arose because of lack of context; the learner still has to explain what she has tried so far. A learner can also have difficulties remembering all the help being provided. One user resorted to laboriously taking detailed written notes, greatly slowing down the interaction. Another complication was that learning episodes were often interrupted by important work events. All these issues need to be taken into account when designing systems to support OTSL. Also we must remember that OTSL is in competition with the ‘real work’ of the participants. So designs to support OTSL should aim to increase the amount of learning and reduce the average length of an interaction.

Ariadne addressed one aspect of the context problem; providing a mechanism for recording a user’s prior activity. The aim was to create a visualization of the usage history that the help-giver could use in order to get a rapid overview of what the user had tried, and so derive some clues about the user’s relative sophistication with that application, and with computers in general. If the help-giver wished to examine certain aspects of the usage in detail, that would be an option, but the main representation was to be sufficiently abstract that a reasonably long session could be skimmed efficiently. In the case of Ariadne this was done using a playing card metaphor, with each card laid out left to right and containing a thumbnail of a screenshot of the interface at a certain point in time. Thus reading the thumbnails left to right gives an impression of the history of activity. The vertical positioning of each card was used to give an impression of the ‘depth’ of the searching activity – whether choosing ‘top level’ options, composing and manipulating a search, or examining results. More details including screenshots are available at: www.uiuc.edu/~twidale.

Such an approach has precedents elsewhere. Unix has a history command. It appears to have been intended for solitary use, but it can be used to support OTSL. The help-giver (if at the desk of the help receiver) can use history to find out what the user has done prior to asking for help. However in order to determine the consequences of those commands, it is necessary to mentally model the results of applying them in sequence, including the consequences of those that are in some way erroneous. The fact that history can be used to give a quick overview of state is due in part to this abstraction. Only the inputs of a command are recorded, not the results. Furthermore, Unix’s command line interface means that those inputs themselves are terse. In effect, history provides a text-based representation of the process of using a text-based command line interface. It uses a two-dimensional text display (a listing of the commands) to represent the history of a one-dimensional textual interface.

In Ariadne, we explored representations of the process of using bibliographic databases accessed via telnet. The interface to be supported was text-based but represented in two dimensions on a screen. Unlike history, we recorded and represented not only the input (a menu selection, a search query etc.; all textual and rather terse) but also the output (the resulting telnet screen, which might be a sub-menu, a list of query results, or a detailed single result). Thumbnails of familiar textual displays are particularly easy to recognize; the shapes of certain textual menus become familiar even though the actual characters are represented by perhaps a single pixel. The representation uses a two dimensional graphical display to represent the history of a two-dimensional textual interface.

Looking to the future, the challenge is to consider how to represent the history of a two-dimensional graphical interface. It is tempting to simply add in yet another dimension, say time. One could just record the entire interaction and then play it back like a movie. It is good at supporting detail but poor at supporting abstraction. The helper still has to look at all the screens in turn, albeit at a faster rate than when the user did the activity. There is no easy way to rapidly skim to get the gist of the interaction, and then choose which items to focus on in more depth. That is because there is no structuring of the data, enabling the skimming of just that structuring information, as one can skim a paper by reading only the section headings. In the case of 2D text displays, the screen often changes dramatically as the user moves through a menu hierarchy. The very power of modern graphical user interfaces means that important things may be done to the system state merely by clicking on a very small icon in an application, with very little if any overall change to the screen as a whole. Thus simply recording and thumbnailing the window is insufficient.

As noted for the case of Unix history, if the help occurs at the help-seeker’s machine, the command can provide some useful context. If the help seeker has had to go to the help-giver’s desk, this option is not normally available. Thus in addition to there being a history and for it to be easy to skim, it should also be portable. In Ariadne we experimented with emailing of context and storage on a central server. Neither is ideal. Fortunately current work on portable work contexts (intended to allow a user to work on any accessible machine as if it were her own) also offers useful potential functionality for OTSL.

Ariadne was in effect an interface to an interface. It represented not a visualization of the result of doing an information search, but a visualization of the process of doing one. The process visualization was intended to help someone to use the actual interface. It acts like a set of training wheels; a temporary attachment to support the learning process. There is a danger of infinite regress – if the interface to the interface isn’t easy to use, then maybe we also need an interface to the interface to the interface… Fortunately there are some simplifying aspects of the problem. It does not matter if the learner cannot understand the process representation, provided that it is of use to the help-giver. If the learner can understand even part of the representation (say enough to comprehend its use by the help-giver to explain actions, but not enough to operate it unaided), then it can also serve as a focus for conversation. That is, the process representation can be an asymmetric interface – used to different extents by the two users.

As well as showing what happened before the help-giving episode, a process representation can also be used to record what happens during it. This can help the problem of learners needing to take laborious notes. Additional possibilities include adding annotations either to the process representation or to the application itself to help the learner remember the new skill just acquired. Keeping a record of the entire OTSL episode is useful for the learner, but it can be tedious to search through it for a forgotten step. In the physical world, we have various ways of indexing into complex information resources. We attach bookmarks, labels and sticky notes to books, but also to items of technology such as VCRs. In using an interface, it may be useful for a learner to be able to attach a note equivalent to a menu or button, reminding her of something that the help giver said. The design constraint here is the very limited space available for such an attachment.

CSCW research looks at how to design systems to support collaborative work. In the case of OTSL, every system is potentially a CSCW application (but only in short bursts). This creates a new set of questions for CSCW researchers. Conventionally CSCW examines the development of systems to collaboration over hours or even weeks. How might CSCW applications support very brief learning interactions, including those of purportedly single-user applications? A complex collaborative interface may be worth the bother of learning if it will be used for a long time. But if its only purpose is to help you learn how to use another system, and you hope that you will only be using it for a few minutes, the amount of time you are prepared to devote to learning to use it diminishes rapidly. Thus designs that prefer speed of learning over sophisticated functionality are more likely to be useful and hence more likely to be used.

Normally in HCI we consider a human interacting with a computer. In CSCW research, there is additional focus on human-human interaction that just happens to be mediated via a computer. In OTSL, there is a conversation between two human beings about how to use a computer. Conventionally, the application in question does little to help that conversation, other than to sit there and be pointed at. One possible improvement already explored in depth is process representation. There are others. People often use a whiteboard to support discussions with colleagues. The schematics drawn on the whiteboard can make a complex argument easier to follow than just the exchange of words. Would a similar approach help OTSL? Whiteboards have the advantage of complete flexibility: the participants can write whatever they choose. A computerized whiteboard allows much faster creation, movement and rearrangement of standard schematic components, if we can create suitable ones. One approach to complement history representations would be plan and goal representations of potential future use: what we want to do and ways could try doing it. This could support complex strategic explanations which involve choosing between alternatives.

In all cases the interfaces provide an additional subtle feature. Experts in a topic have a specialist vocabulary. Thus in a very few words they can refer to very complex action sequences. Novices lack such a vocabulary. The discussion must proceed at the micro level of laborious individual steps. (An extreme case is the total computer novice for whom both the concept and the phrase "double-click" is unknown and must be explained as a set of sub-components). An interface provides an opportunity for abstraction without the overhead of previously learning the vocabulary. Instead of using the technical term for an action, the learner or expert can point to its representation, and similarly for longer action sequences, provided there is a process representation to point at. These representations also simplify the learning of the vocabulary.

There will be times when colleagues are not available and the traditional HCI approach of design for solitary understanding applies. However, this does break down, and users may need access to remote help. This can be in the form of software telephone helplines, or emailed help, or live chat via IRC. The same issues of OTSL design reapply in this context, only further complicated by the difficulties of remote collaboration, caused by the more limited bandwidth for human interaction. E-commerce presents an interesting case of the problem. Unlike complex applications, e-commerce usually aims to be accessible to a very wide audience. But how do you learn how to buy, and so to feel comfortable with the buying process? Once you know, all the design effort can go into optimizing the efficiency of the interaction. But for a nervous customer, the buying process may appear fraught with danger. It would be very interesting to find out the proportion of new purchasers who first watched a friend buy something or invited a friend over to help them in their first purchase, effectively re-introducing OTSL. Compare the problem with the case in real life (RL-commerce). In going into a new kind of store with an unfamiliar purchasing protocol, the shopper is not usually handed an instruction manual. Rather they can watch what other shoppers, totally unknown to them, are doing and how they buy things. Indeed there are stories of the first supermarkets hiring actors to push shopping carts around the store precisely to illustrate (and hence teach) this new mode of RL-commerce. Contrast this with current e-commerce, where a purchaser can feel very isolated and vulnerable, and we can begin to see how e-stores might employ OTSL techniques to help less technophilic potential purchasers.

If the findings from the two preliminary studies are replicated in different work contexts, it poses a new set of issues for HCI research. Most of the effort in HCI research and development focuses on ways to help a user understand a system on their own. For conventional interfaces, designers ask themselves questions such as: "How can we change this interface so that it is easier for the user to work out how to do what she wants?" In design with a sensitivity to OTSL there is an additional question: "How can we change this interface so that it is easier for the user to ask and receive help on how do what she wants from someone else?" It remains to be seen the degree to which these two kinds of questions yield different kinds of solutions, and how we might make trade-off decisions between them. If a significant fraction of learning is with the help of others, maybe our research and development efforts are somewhat misdirected, and resources should be diverted to design for collaborative learnability.

HCI research is concerned with two key interlinked features of computer systems: learnability and use. This paper contends that one way that people learn how to use a computer system, both initially, and subsequently to incrementally improve their skills, has been neglected. Over-the-shoulder learning carries intriguing design implications for interface developers. These include a consideration of how to design support for collaborative learnability into an interface, in addition to our existing concern with supporting individual learnability. It also raises the issue of creating explicit OTSL interfaces to support the learning of another interface, a kind of attachable interface training wheels.

Bannon, L.J. (1986). Helping users help each other. In D. A. Norman and S. W. Draper (eds). User centered system design: new perspectives on human-computer interaction. Hillsdale, NJ; Lawrence Erlbaum Associates: 399-410.

Berlin, L.M. and Jeffries, R. (1992). Consultants and Apprentices: Observations about Learning and Collaborative Problem Solving. Proceedings of ACM CSCW'92 Conference on Computer-Supported Cooperative Work, 130-137.

Crabtree, A., Twidale, M.B. O'Brien, J. & Nichols, D.M. (1997). Talking in the Library: Implications for the Design of Digital Libraries. Proceedings, ACM Digital Libraries '97, (Eds.) Allen, R.B. & Rasmussen, E. 221-228.

Grudin, J. (1989). Why groupware applications fail: problems in design and evaluation. Office Technology and People,. 4(3) 245-264.

Landauer, T. K. (1995). The trouble with computers: usefulness, usability, and productivity. MIT Press.

Lee, D. (1986). Usage pattern and sources of assistance for personal computer users. MIS Quarterly, 10(4) 313-325.

Twidale, M.B., Nichols, D.M. & Paice, C.D. (1997). Browsing is a Collaborative Process. Information Processing & Management, 33(6), 761-83.

Twidale, M.B. & Nichols, D.M. (1998). Designing Interfaces to Support Collaboration in Information Retrieval. Interacting with Computers, 10(2), 177-93.

Twidale, M.B. (1999). Over the shoulder learning: supporting brief informal learning embedded in the work context. Technical Report UIUCLIS--1999/2+CSCW http://www.lis.uiuc.edu/~twidale/pubs/otsl1.html

Weinberg, G. M. (1971). The psychology of computer programming. New York, Van Nostrand Reinhold.

Whittaker, S., Frolich, D., and Daly-Jones, O. (1994). Informal workplace communication: what is it like and how might we support it? Proceedings, CHI '94, Boston, MA, ACM Press, 131-137.