Ready-to-hand

Source orientation and persuasion in multi-device and multi-context interactions

At the Social Media Workshop, Katarina Segerståhl presented her on-going work on what she has termed extended information services or distributed user experiences — human-computer interactions that span multiple and heterogeneous devices (Segerståhl & Oinas-Kukkonen 2007). As a central example, she studies a persuasive technology service for planning, logging, reviewing, and motivating exercise: these parts of the experience are distributed across the user’s PC, mobile phone, and heart rate monitor.

In one interesting observation, Segerståhl notes that the specific user interfaces on one device can be helpful mental images even when a different device is in use: participants reported picturing their workout plan as it appeared on their laptop and using it to guide their actions during their workout, during which the obvious, physically present interface with the service was the heart rate monitor, not the earlier planning visualization. Her second focus is how to make these user experiences coherent, with clear practical applications in usability and user experience design (e.g., how can designers make the interfaces both appropriately consistent and differentiated?).

In this post, I want to connect this very interesting and relevant work with some other research at the historical and theoretical center of persuasive technology: source orientation in human-computer interaction. First, I’ll relate source orientation to the history and intellectual context of persuasive technology. Then I’ll consider how multi-device and multi-context interactions complicate source orientation.

Source orientation, social responses, and persuasive technology

As an incoming Ph.D. student at Stanford University, B.J. Fogg already had the goal of improving generalizable knowledge about how interactive technologies can change attitudes and behaviors by design. His previous graduate studies in rhetoric and literary criticism had given him understanding of one family of academic approaches to persuasion. And in running a newspaper and consulting on many document design (Schriver 1997) projects, the challenges and opportunities of designing for persuasion were to him clearly both practical and intellectually exciting.

The ongoing research of Profs. Clifford Nass and Byron Reeves attracted Fogg to Stanford to investigate just this. Nass and Reeves were studying people’s mindless social responses to information and communication technologies. Cliff Nass’s research program — called Computers as (or are) Social Actors (CASA) — was obviously relevant: if people treat computers socially, this “opens the door for computers to apply [...] social influence” to change attitudes and behaviors (Fogg 2002, p. 90). While clearly working within this program, Fogg focused on showing behavioral evidence of these responses (e.g., Fogg & Nass 1997): both because of the reliability of these measures and the standing of behavior change as a goal of practitioners.

Source orientation is central to the CASA research program — and the larger program Nass shared with Reeves. Underlying people’s mindless social responses to communication technologies is the fact that they often orient towards a proximal source rather than a distal one — even when under reflective consideration this does not make sense: people treat the box in front of them (a computer) as the source of information, rather than a (spatially and temporally) distant programmer or content creator. That is, their source orientation may not match the most relevant common cause of the the information. This means that features of the proximal source unduly influence e.g. the credibility of information presented or the effectiveness of attempts at behavior change.

For example, people will reciprocate with a particular computer if it is helpful, but not the same model running the same program right next to it (Fogg & Nass 1997, Moon 2000). Rather than orienting to the more distal program (or programmer), they orient to the box.¹

Multiple devices, Internet services, and unstable context

These source orientation effects have been repeatedly demonstrated by controlled laboratory experiments (for reviews, see Nass & Moon 2000, Sundar & Nass 2000), but this research has largely focused on interactions that do not involve multiple devices, Internet services, or use in changing contexts. How is source orientation different in human-computer interactions that have these features?

This question is of increasing practical importance because these interactions now make up a large part of our interactions with computers. If we want to describe, predict, and design for how people use computers everyday — checking their Facebook feed on their laptop and mobile phone, installing Google Desktop Search and dialing into Google 411, or taking photos with their Nokia phone and uploading them to Nokia’s Ovi Share — then we should test, extend, and/or modify our understanding of source orientation. So this topic matters for major corporations and their closely guarded brands.

So why should we expect that multiple devices, Internet services, and changing contexts of use will matter so much for source orientation? After having explained the theory and evidence above, this may already be somewhat clear, so I offer some suggestive questions.

1. If much of the experience (e.g. brand, visual style, on-screen agent) is consistent across these changes, how much will the effects of characteristics of the proximal source — the devices and contexts — be reduced?
2. What happens when the proximal device could be mindfully treated as a source (e.g., it makes its own contribution to the interaction), but so does a distance source (e.g., a server)? This could be especially interesting with different branding combination between the two (e.g., the device and service are both from Apple, or the device is from HTC and service is from Google).
3. What if the visual style or manifestation of the distal source varies substantially with the device used, perhaps taking on a style consistent with the device? This can already happen with SMS-based services, mobile Java applications, and voice agents that help you access distant media and services.

References

1. This actually is subject to a good deal of cross-cultural variation. Similar experiments with Japanese — rather than American — participants show reciprocity to groups of computers, rather than just individuals (Katagiri et al.) [↩]

2 comments August 29th, 2008 Dean Eckles

Producing, consuming, annotating (Social Mobile Media Workshop, Stanford University)

Today I’m attending the Social Mobile Media Workshop at Stanford University. It’s organized by researchers from Stanford’s HStar, Tampere University of Technology, and the Naval Postgraduate School. What follows is some still jagged thoughts that were prompted by the presentation this morning, rather than a straightforward account of the presentations.¹

A big theme of the workshop this morning has been transitions among production and consumption — and the critical role of annotations and context-awareness in enabling many of the user experiences discussed. In many ways, this workshop took me back to thinking about mobile media sharing, which was at the center of a good deal of my previous work. At Yahoo! Research Berkeley we were informed by Marc Davis’s vision of enabling “the billions of daily media consumers to become daily media producers.” With ZoneTag we used context-awareness, sociality, and simplicity to influence people to create, annotate, and share photos from their mobile phones (Ahern et al. 2006, 2007).

Enabling and encouraging these behaviors (for all media types) remains a major goal for designers of participatory media; and this was explicit at several points throughout the workshop (e.g., in Teppo Raisanen’s broad presentation on persuasive technology). This morning there was discussion about the technical requirements for consuming, capturing, and sending media. Cases that traditionally seem to strictly structure and separate production and consumption may be (1) in need of revision and increased flexibility or (2) actually already involve production and consumption together through existing tools. Media production to be part of a two-way communication, it must be consumed, whether by peers or the traditional producers.

As an example of the first case, Sarah Lewis (Stanford) highlighted the importance of making distance learning experiences reciprocal, rather than enforcing an asymmetry in what media types can be shared by different participants. In a past distance learning situation focused on the African ecosystem, it was frustrating that video was only shared from the participants at Stanford to participants at African colleges — leaving the latter to respond only via text. A prototype system, Mobltz, she and her colleagues have built is designed to change this, supporting the creation of channels of media from multiple people (which also reminded me of Kyte.tv).

As an example of the second case, Timo Koskinenen (Nokia) presented a trial of mobile media capture tools for professional journalists. In this case the work flow of what is, in the end, a media production practice, involves also consumption in the form of review of one’s own materials and other journalists, as they edit, consider what new media to capture.

Throughout the sessions themselves and conversations with participants during breaks and lunch, having good annotations continued to come up as a requirement for many of the services discussed. While I think our ZoneTag work (and the free suggested tags Web service API it provides) made a good contribution in this area, as has a wide array of other work (e.g., von Ahn & Dabbish 2004, licensed in Google Image Labeler), there is still a lot of progress to make, especially in bringing this work to market and making it something that further services can build on.

References

1. Blogging something at this level of roughness is still new for me… [↩]

2 comments August 1st, 2008 Dean Eckles

Update your Facebook status: social comparison and the availability heuristic

Over at Captology Notebook, the blog of the Stanford Persuasive Technology Lab, Enrique Allen considers features of Facebook that influence users to update their status. Among other things, he highlights how Facebook lowers barriers to updating by giving users a clear sense of something they can right (”What are you doing right now?”).

I’d like to add another part of the interface for consideration: the box in the left box of the home page that shows your current status update with the most recent updates of your friends.

This visual association of my status and the most recent status updates of my friends seems to do at least a couple things:

Influencing the frequency of updates. In this example, my status was updated a few days ago. On the other hand, the status updates from my friends were each updated under an hour ago. This juxtaposes my stale status with the fresh updates of my peers. This can prompt comparison between their frequency of updates and mine, encouraging me to update.

The choice of the most recent updates by my Facebook friends amplifies this effect. Through automatic application of the availability heuristic, this can make me overestimate how recently my friends have updated their status (and thus the frequency of status updates). For example, the Facebook friend who updated their status three minutes ago might have not updated to three weeks prior. Or many of my Facebook friends may not frequently update their status messages, but I only see (and thus have most available to mind) the most recent. This is social influence through enabling and encouraging biased social comparison with — in a sense — an imagined group of peers modeled on those with the most recent performances of the target behavior (i.e., updating status).

Influencing the content of updates. In his original post, Enrique mentions how Facebook ensures that users have the ability to update their status by giving them a question that they can answer. Similarly, this box also gives users examples from their peers to draw on.

Of course, this can all run up against trouble. If I have few Facebook friends, none of them update their status much, or those who do update their status are not well liked by me, this comparison may fail to achieve increased updates.

Consider this interface in comparison to one that either

• showed recent status updates by your closest Facebook friends, or
• showed recent status updates and the associated average period for updates of your Facebook friends that most frequently update their status.

[Update: While the screenshot above is from the "new version" of Facebook, since I captured it they have apparently removed other people's updates from this box on the home page, as Sasha pointed out in the comments. I'm not sure why they would do this, but here are couple ideas:

• make lower items in this sidebar (right column) more visable on the home page -- including the ad there
• emphasize the filter buttons at the top of the news feed (left column) as the means to seeing status updates.

Given the analysis in the original post, we can consider whether this change is worth it: does this decrease status updates? I wonder if Facebook did a A-B test of this: my money would be on this significantly reducing status updates from the home page, especially from users with friends who do update their status.]

6 comments July 29th, 2008 Dean Eckles

Reprioritizing human intelligence tasks for low latency and high throughput on Mechanical Turk

Amazon Mechanical Turk is a platform and market for human intelligence tasks (HITs) that are submitted by requesters and completed by workers (or “turkers”).  Each HIT is associated with a payment, often a few cents. This post covers some basics of Mechanical Turk and shows its lack of designed-in support for dynamic reprioritization is problematic for some uses. I also mention some other factors that influence latency and throughput.

With mTurk one can create a HIT that asks someone to rate some search results for a query, evaluate the credibility of a Wikipedia article, draw a sheep facing left, enter names for a provided color, annotate a photo of a person with pose information, or create a storyboard illustrating a new product idea. So Mechanical Turk can be used in many ways for basic research, building a training set for machine learning, or actually enabling a (perhaps prototype) service in use through a kind of Wizard-of-Oz approach. Additionally, I’ve used mTurk to code images captured by participants in a lab experiment (more on this in another post or article).

When creating HITs, a requester can specify a QuestionForm (QF) (e.g., via command line tools or an SDK) that is then presented to the worker by Amazon. This can include images, free text answers, multiple choice, etc. Additionally one can embed Flash or Java objects in it. But the easiest way of creating HITs is to use a QF and not create a Java or Flash application of one’s own. This is especially true for HITs that are handled well by the basic question form. The other option is to create an ExternalQuestion (EQ), which is hosted on one’s own server and is displayed in an iFrame. This provides greater freedom but requires additional development and it is you that host the page (though you can do so through Amazon’s S3). QF HITs (without embeds) also offer a familiar interface to workers (though it is possible to create a more efficient, custom interface by e.g. making all the targets larger). So when possible, it is often preferable to use a QF rather than an EQ.

For some of the uses of mTurk for powering a service, it can be important to minimize latency for specific HITs¹, including prioritizing particular new HITs over previously created HITs. For example, after some HIT has not been completed for a specific period after creation, it may still be important to complete it, but when it is completed may become less important. This can happen easily if there the value of a HIT being completed has a sharp drop off after some time.

This should be done while maintaining high throughput; that is, you don’t want to reduce the rate at which your HITs are completed. When there are more HITs of the same type, workers can check a box to immediately start the next HIT of the same type when they submit the current one (see screenshot). Workers will often complete many HITs of the same type in a row. So throughput can drop substantially if any workers run out of HITs of the same type at any point: they may switch to another HIT type, or if they do your HITs once more appear there will be a delay. As we’ll see, these two requirements don’t seem to be well met by the platform — or at least certain uses of it.

Mechanical Turk does not provide a mechanism for prioritizing HITs of the same type, so without deleting all but particular high-priority HITs of that type, there is not a way to ensure that some particular HIT gets done before the rest. And deleting the other HITs would hurt throughput and increase latency for any new high-priority HITs added int he near future (since workers won’t simply start these once they finish their previous HITs).

EQs allow one to avoid this problem. Unlike with QF HITs (without Flash and Java embeds), one does not have to specify the full content of the HIT in advance. When a worker accepts an EQ HIT, you can dynamically serve up the HIT you want to depending on changing priorities. But this means that you can’t take advantage of e.g. the simplicity of creating and managing data from QF HITs. So though there are ways of coping, adding dynamic reprioritization to Mechanical Turk would be a boon for time-sensitive uses.

There are, of course, other factors that influence latency and throughput on mTurk when (EQ) HITs are reprioritized. Here are a few:

• HIT and sub-tasks duration. How long does it take for workers to complete a HIT, which may be composed of multiple sub-tasks? A worker cannot be assigned a new HIT until they complete (or reject) the previous one. This can be somewhat avoided by creating longer HITs that are subdivided into dynamically selected sub-tasks. This can be done with an EQ HIT or an embedded Flash or Java application in a QF HIT. But the sub-task duration is always a limiting factor, unless one is willing to force abortion of the current sub-task, replacing it will still in progress (with an EQ, Flash, or Java).
• Available workers. How many workers are logged into mTurk and completing task? How many are currently switching HIT types? This can vary with the time of day.
• Appeal of your HITs. How much do workers like your HITs — are they fun? How much do you pay for how much you ask? How many of their completed assignments do you approve?
• Reliability. How accurate or precise must your results be? How many workers do you need to complete a HIT before you have reliable results? Do other workers need to complete meta-HITs before the data can be used?

1. I use the term HIT somewhat loosely in this article. There are at least three uses that each differ in their identity conditions. (1) There are HITs considered as human intelligence tasks, and thus divided as we divide tasks; this means that a HIT in another sense can be composed of multiple HITs in this sense (tasks or sub-tasks). (2) There are HITs in Amazon’s technical sense of the term: a HIT is something that has the same HIT ID and therefore has the same specification. In QF HITs without embeds, this means all instances (assignments) of a HIT are the same in content; but in EQ HIT this is not necessarily true, since the content can be determined when assigned. (3) Finally, there is what Amazon calls assignments, specific instances of a HITs that are only completed once. [↩]

2 comments July 24th, 2008 Dean Eckles

Naming this blog: Heidegger, Husserl, folk psychology, and HCI

The name of this blog, Ready-to-hand, is a translation of Heidegger’s term zuhanden, though interpreting Heidegger’s philosophy is not specifically a major interest of mine nor a focus here. Much has been made of the significance of phenomenology, most often Heidegger, for human-computer interaction (HCI) and interaction design (e.g., Winograd & Flores 1985, Dourish 2001). And I am generally pretty sympathetic to phenomenology as one inspiration for HCI research. I want to just note a bit about the term zuhanden and my choice of it in a larger context — of phenomenology, HCI, and a current research interest of mine: cues for assuming the intentional stance toward systems (more on this below).

The Lifeworld and ready-to-hand
Heidegger was a student of Edmund Husserl, and Heidegger’s Being and Time was to be dedicated to Husserl.¹ There is really no question of the huge influence of Husserl on Heidegger.

My major introduction to both Husserl and Heidegger was from Prof. Dagfinn Føllesdal. Føllesdal (1979) details the relationship between their philosophies. He argues for the value of seeing much of Heidegger’s philosophy “as a translation of Husserl’s”:

The key to this puzzle, and also, I think, the key to understanding what goes on in Heidegger’s philosophy, is that Heidegger’s philosophy is basically isomorphic to that of Husserl. Where Husserl speaks of the ego, Heidegger speaks of Dasein, where Husserl speaks of the noema, Heidegger speaks of the structure of Dasein’s Being-in-the-world and so on. Husserl also observed this. Several places in his copy of Being and Time Husserl wrote in the margin that Heidegger was just translating Husserl’s phenomenology into another terminology. Thus, for example, on page 13 Husserl wrote: “Heidegger transposes or transforms the constitutive phenomenological clarification of all realms of entities and universals, the total region World into the anthropological. The problematic is translation, to the ego corresponds Dasein etc. Thereby everything becomes deep-soundingly unclear, and philosophically it loses its value.” Similarly, on page 62, Husserl remarks: “What is said here is my own theory, but without a deeper justification.” (p. 369)

Heidegger and his terms have certainly been more popular and in wider use since then.

Føllesdal also highlights where the two philosophers diverge.² In particular, Heidegger gives a central role to the role of the body and action in constituting the world. While in his publications Husserl stuck to a focus on how perception constitutes the Lifeworld, Heidegger uses many examples from action.³ Our action in the world, including our skillfulness in action constitutes those objects we interact with for us.

Heidegger contrasts two modes of being (in addition to our own mode — being-in-the-world): present-at-hand and ready-to-hand (or alternatively, the occurant and the available (Dreyfus 1990)). The former is the mode of being consideration of an object as a physical thing present to us — or occurant, and Heidegger argues it constitutes the narrow focus of previous philosophical explorations of being. The latter is the stuff of every skilled action — available for action: the object becomes equipment, which can often be transparent in action, such that it becomes an extension of our body.

J.J. Gibson expresses this view in his proposal of an ecological psychology (in which perception and action are closely linked):

When in use, a tool is a sort of extension of the hand, almost an attachment to it or a part of the user’s own body, and thus is no longer a part of the environment of the user. [...] This capacity to attach something to the body suggests that the boundary between the animal and the environment is not fixed at the surface of the skin but can shift. More generally it suggests that the absolute duality of “objective” and “subjective” is false. When we consider the affordances of things, we escape this philosophical dichotomy. (1979, p. 41)

While there may be troubles ahead for this view, I think the passage captures well something we all can understand: when we use scissors, we feel the paper cutting; and when a blind person uses a cane to feel in front of them, they can directly perceive the layout of the surface in front of them.

Transparency, abstraction, opacity, intentionality
Research and design in HCI has sought at times to achieve this transparency, sometimes by drawing on our rich knowledge of and skill with the ordinary physical and social world. Metaphor in HCI (e.g., the desktop metaphor) can be seen as one widespread attempt at this (cf. Blackwell 2006). This kind of transparency does not throw abstraction out of the picture. Rather the two go hand-in-hand: the specific physical properties of the present-at-hand are abstracted away, with quickly perceived affordances for action in their place.

But other kinds of abstraction are in play in HCI as well. Interactive technologies can function as social actors and agents– with particular cues eliciting social responses that are normally applied to other people (Nass and Moon 2000, Fogg 2002). One kind of social response, not yet as widely considered in the HCI literature, is assuming the intentional stance — explanation in terms of beliefs, desires, hopes, fears, etc. — towards the system. This is a powerful, flexible, and easy predictive and explanatory strategy often also called folk psychology (Dennett 1987), which may be a tacit theory or a means of simulating other minds. We can explain other people based on what they believe and desire.

But we can also do the same for other things. To use one of Dennett’s classic examples, we can do the same for a thermostat: why did it turn the heat on? It wanted to keep the house at some level of warmth, it believed that it was becoming colder than desired, and it believed that it could make it warmer by turning on the heat. While in the case of the thermostat, this strategy doesn’t hide much complexity (we could explain it with other strategies without much trouble), it can be hugely useful when the system in question is complex or otherwise opaque to other kinds of description (e.g., it is a black box).

We might think then that perceived complexity and opacity should both be cues for adopting the intentional stance. But if the previous research on social responses to computers (not to mention the broader literature on heuristics and mindlessness) has taught us anything, it is that made objects such as computers can evoke unexpected responses through other simplier cues. Some big remaining questions that I hope to take up in future posts and research:

• What are these cues, both features of the system and situational factors?
• How can designers influence people to interpret and explain systems using folk psychology?
• What are the advantages and disadvantages of evoking the intentional stance in users?
• How should we measure the use of the intentional stance?
• How is assuming the intentional stance towards a thing different (or the same) as it having being-in-the-world as its mode of being?

1. But Husserl was Jewish, and Heidegger was himself a member of the Nazi party, so this did not happen in the first printing. [↩]
2. Dreyfus (1990) is an alternative view that takes the divergence as quite radical; he sees Føllesdal as hugely underestimating the originality of Heidegger’s thought. Instead Dreyfus characterizes Husserl as formulating so clearly the Cartesian worldview that Heidegger recognized its failings and was thus able to radically and successfully critique it. [↩]
3. It is worth noting that Husserl actually wrote about this as well, but in manuscripts, which Heidegger read years before writing Being and Time. [↩]

2 comments July 23rd, 2008 Dean Eckles

Expert users: agreement in focus from two threads of human-computer interaction research

Much of current human-computer interaction (HCI) research focuses on novice users in “walk-up and use” scenarios. I can think of three major causes for this:

1. A general shift from examining non-discretionary use to discretionary use
2. How much easier it is to find (and not train) study participants unfamiliar with a system than experts (especially with a system that is only a prototype)
3. The push from practitioners in the direction, especially with the advent of the Web, where new users just show up at your site, often deep-linked

This focus sometimes comes in for criticism, especially when #2 is taken as a main cause of the choice.

On the other hand, some research threads in HCI continue to focus on expert use. As I’ve been reading a lot of research on both human performance modeling and situated & embodied approaches to HCI, it has been interesting to note that both instead have (comparatively) a much bigger focus on the performance and experience of expert and skilled use.

Grudin’s “Three Faces of Human-Computer Interaction” does a good job of explaining the human performance modeling (HPM) side of this. HPM owes a lot to human factors historically, and while The Psychology of Human-Computer Interaction successfully brought engineering-oriented cognitive psychology to the field, it was human factors, said Stuart Card, “that we were trying to improve” (Grudin 2005, p. 7). And the focus of human factors, which arose from maximizing productivity in industrial settings like factories, has been non-discretionary use. Fundamentally, it is hard for HPM to exist without a focus on expert use because many of the differences — and thus research contributions through new interaction techniques — can only be identified and are only important for use by experts or at least trained users. Grudin notes:

A leading modeler discouraged publication of a 1984 study of a repetitive task that showed people preferred a pleasant but slower interaction technique—a result significant for discretionary use, but not for modeling aimed at maximizing performance.

Situated action and embodied interaction approaches to HCI, which Harrison, Tatar, and Senger (2007) have called the “third paradigm of HCI”, are a bit different story. While HPM research, like a good amount in traditional cognitive science generally, contributes to science and design by assimilating people to information processors with actuators, situated and embodied interaction research borrows a fundamental concern of ethnomethodology, focusing on how people actively make behaviors intelligible by assimilating them to social and rational action.

There are at least three ways this motivates the study of skilled and expert users:

1. Along with this research topic comes a methodological concern for studying behavior in context with the people who really do it. For example, to study publishing systems and technology, the existing practices of people working in such a setting of interest are of critical importance.
2. These approaches emphasize the skills we all have and the value of drawing on them for design. For example, Dourish (2001) emphasizes the skills with which we all navigate the physical and social world as a resource for design. This is not unrelated to the first way.
3. These approaches, like and through their relationships to the participatory design movement, have a political, social, and ethical interest in empowering those who will be impacted by technology, especially when otherwise its design — and the decision to adopt it — would be out of their control. Non-discretionary use in institutions is the paradigm prompting situation for this.

I don’t have a broad conclusion to make. Rather, I just find it of note and interesting that these two very different threads in HCI research stand out from much other work as similar in this regard. Some of my current research is connecting these two threads, so expect more on their relationship.

References
Dourish, P. (2001). Where the Action Is: The Foundations of Embodied Interaction. MIT Press.
Grudin, J. (2005). Three Faces of Human-Computer Interaction. IEEE Ann. Hist. Comput. 27, 4 (Oct. 2005), 46-62.
Harrison, S., Tatar, D., and Senger, P. (2007). The Three Paradigms of HCI. Extended Abstracts CHI 2007.

Add comment May 27th, 2008 Dean Eckles

Using a Wizard of Oz technique in mobile service design: probing with realistic motivations

As I’ve blogged before, I spoke at the Texting 4 Health conference on the topic of research methods for mobile messaging. One method I covered was an interesting use of Wizard of Oz techniques for designing mobile services. I’ve since started getting some of this material in writing for the Texting 4 Health book. Here is a taste of that material, minus the health-specific focus and examples.
——-
Just like the famous Wizard of Oz, one can simulate something impressive with a just a humble person behind the curtain — and use this simulation to inform design decisions. When using a Wizard of Oz technique to study a prototype, a human “wizard” carries out functions that, in a deployed application or service, would be handled by a computer. This can allow evaluating a design without fully building what can be expensive back-end parts of the system (Kelley 1984). The technique is often used in recognition-based interfaces, but it also has traditional applications to identifying usability problems and carrying out experiments in which the interaction is systematically manipulated.

Wizard of Oz techniques are well suited to prototyping mobile services, especially those using mobile messaging (SMS, MMS, voice messaging). When participants send a request, a wizard reads or listens to it and chooses the appropriate response, or just creates it on-the-fly. Since all user actions in mobile messaging are discrete messages and (depending on the application) the user can often tolerate a short delay, a few part-time wizards, such as you and a colleague, can manage a short field trial. As you’ll see, this can be used for purposes beyond many traditional uses of a Wizard of Oz.

Probing photo consumption needs with realistic motivations
One use for this technique in designing a mobile messaging service is a bit like a diary study. In designing an online and mobile photography service, we wanted to better understand what photos people wanted to view and what prompted these desires.¹ Instead of just making diary entries, participants actually made voice requests to the system for photos – and received a mobile message with photos fitting the request in return. We didn’t need to first build a robust system that could do this; a few of us served as wizards, listening to the request, doing a couple manual searches, and choosing which photos to return on demand. Though this can be done with a normal voice call, we used a mobile client application that also recorded contextual information not available via a normal voice call (e.g. location), so that participants could make context-aware requests as they saw fit (e.g. “I want too see photos of this park”)

In this case, we didn’t plan to (specifically) create a voice-based photo search system; instead, like a diary study, this technique served as a probe to understand what we should build. As a probe it provided realistic motivations for submitting requests, as the request would actually be fulfilled. This design research, in additional to other interviews and a usability study, informed our creation of Zurfer, a mobile application that supports exploring and conversing around personalized, location-aware channels of photos.
It is great if the Wizard of Oz prototype is quite similar to what you later build, but it can yield valuable insights even if not. Sometimes it is precisely these insights that can lead you to substantially change your design.

This study design can apply in designing many mobile services. As in our photos study, participants can be interviewed about the trigger for the requests (why did they want that media or information) and how satisfied they were with the (human-created) responses.²

References
Kelley, J.F. (1984). An iterative design methodology for user-friendly natural language office information applications. In ACM Trans. Inf. Syst., vol. 2, pp. 26-41.

1. This study was designed and executed at Yahoo! Research Berkeley by Shane Ahern, Nathan Good, Simon King, Mor Naaman, Rahul Nair, and myself. [↩]
2. Participants were informed that their requests would be seen by our research staff. Anonymization and strict limits of who the wizards are is necessary to protect participants’ privacy. Even if participants are not informed that a wizard is creating the responses until they are debriefed after the experiment, participants can nonetheless be notified that their responses are being reviewed by the research team. [↩]

1 comment May 5th, 2008 Dean Eckles

Riskful decisions and riskful thinking: Donald Davidson and Cliff Nass

Two personal-professional narratives that I’ve been somewhat familiar with for a while have recently highlighted for me the significance of riskful decisions and thinking in academia. I think the stories are interesting on their own, but they also emphasize some questions and concerns for the functioning of scholarly inquiry.

The first is about the American philosopher Donald Davidson, whose work has long been of great interest to me (and was the topic of my undergraduate Honors thesis). The second is about Cliff Nass (Clifford Nass), Professor of Communication at Stanford, an advisor and collaborator. The major published source I draw on for each of these narratives is an interview: for Davidson’s story, it is an interview by Ernest Lepore (2004), a critic and expositor of Davidson’s philosophy; for Cliff Nass, it is an interview by Tamara Adlin (2007). After sharing these stories, I’ll discuss some similarities and briefly discuss risk-taking in decisions and thinking.

Donald Davidson is considered one of the most important and influential philosophers of the past 60 years, and he is my personal favorite. Davidson is often described as a highly systematic philosopher — uncharacteristically so for 20th century philosophy, in that his contributions to several areas of philosophy (philosophy of language, mind, and action, semantics, and epistemology) are deeply connected in their method and the proposed theories. He is the paradigmatic programmatic philosopher of the 20th century.

Despite this, Davidson’s philosophical program did not emerge until relatively late in his career. The same is true of his publications in general. Only after accepting a tenure track position at Stanford in 1951 (which was then still up-and-coming, though quickly, in philosophy) did he begin to publish (nothing was even in the “pipeline” previous to this). This began under the wing of the younger Patrick Suppes, with whom Davidson co-authored a book (1957) on decision theory. His first philosophical article appears in 1963 (which he authored alone only through an unexpected death). As Davidson puts it in an interview with Ernest Lepore, “I was very inhibited so far as publication was concerned” and was worried “that the minute I actually published something, everyone was going to jump on me” (Davidson 2004).

Then Davidson published “Actions, Reasons and Causes” (1963), twelve years after joining the Stanford faculty. It argues against the late-Wittgensteinian dogma that reasons are not also causes. It is only with this paper that there was a publication by Davidson that drew significant attention from the community (beginning with a presentation of the paper at a meeting of the American Philosophical Association). This paper has been hugely influential and alone identified Davidson as an important thinker in the field, though he was surprised the reception was not as overwhelming as he had thought: “I didn’t realize that if you publish, as far as I can tell, no one was going to pay any attention.” Many responses, both positive and critical, did eventually come, and Davidson went on to publish many highly influential papers, reaching the height of his immense scholarly influence in the 1970s and 1980s.

Clifford Nass is widely known researcher in the psychology of human-computer interaction (HCI). With Byron Reeves, he wrote The Media Equation (1996), which presents research carried out at Stanford University on how people respond in mediated interactions (e.g. with computers and televisions) by overextending social rules normally applied to other people. This hints at the (here simplified) straight, bold line of Nass’s research program: take a finding from social psychology, replace the second human with a computer, see if you get the same results. This exact strategy has been modified and expanded from, but the general consistency of Nass’s program over many years is striking for HCI: unlike in psychology, for example, in HCI there are many investigators seeking low-hanging fruit and quickly moving on to new projects.

Nass likes to refer to his “accidental PhD”, as he hadn’t intended to get a PhD in sociology. After working for a year at Intel, he was planning to matriculate in a electrical engineering PhD program, but an unexpected death postponed that. “[J]ust to bide my time and to have some flexibility, I ended up doing a sociology degree,” says Nass. He did his dissertation on the role of pre-processing jobs in labor, taking an approach that was radical in its elimination of a role for people and that connected with contemporary research by social science outsiders doing “sociocybernetics”. With such a dissertation topic (and the dissertation itself unfinished), finding a job did not seem easy at the outset: “It’s a nutty topic. I was going to be in trouble getting jobs. I had published stuff and was doing work and all that, but my dissertation was so weird” (Adlin 2007).

There was, however, a bit of luck, well taken advantage of by Nass: the Stanford Communication Department was under construction and looking to hire some folks doing weird work. So when Nass interviewed, impressing both them and the Sociology Department, he got the job, despite knowing nothing about Communication as a discipline and having been to no conferences in the field. After beginning at Stanford, Nass was seeking a research program, as clearly there was something wrong, at least when it came to getting it accepted for academic publication, with his previous work: “I was having a terrible time getting my work accepted. In fact, to this day I’ve still never published anything off my dissertation, 20-odd years later. Because again, no field could figure out who owned the material. I got reviews like, ‘This work is offensive.’”

But Nass couldn’t settle on any normal research program. He wanted to examine how people might treat computers socially. Getting funding for this work wouldn’t have been easy, but he got a grant that the grant administrator described as the 1 of 35 given that they chose to give to the “weirdest project that was proposed”. It wasn’t all easy from there, of course. For example, it took some time to design and carry out successful experiments in this program — and even longer to get the results published. But this risk-taking in distributing this grant helped enable the work to continue.
Cliff Nass is very clear about the role riskful decisions, in admissions, hiring, and funding, played in his success:

I was very lucky. I fear that those times are gone. I really do fear to a tremendous degree that the risk-taking these people were willing to do for me, to give me an opportunity, are gone. I try to remember that. [...]

I benefited from the willingness of people to say, “We’re just going to roll the dice here.”

Of course, it isn’t just Cliff who got lucky; in a sense we all did. His work has been an important influence in HCI and has contributed to our stores of both generalizable knowledge and new lenses for approaching how we get on in the world.

What does it mean for academic research, and science generally, if this choice and ability to take these risks evaporates? There is incredible competition for academic positions now, more so in some fields than others. And the best tool in getting a job is a whole list of publications accepted in important, mainstream journals in the field. There is a lot written about the competition for academic jobs and criteria for wading through applicants to sometimes a safe option. There are case studies of families of disciplines; for example, a study of the biosciences argues that market forces are failing to create sufficient job prospects for young investigators (Freeman et al. 2001).

I won’t review them all here. Instead I suggest an article for general readers from The New York Times about state and regional colleges’ use of non-tenure track positions, which has an impact of the institutions’ bottom line and flexibility (Finder 2007). This is part of a wider trend in how tenure is used that also impacts the academic freedom and resources that scholars have to pursue new research (Richardson 1999).

Enabling riskful thinking
Hans Ulrich Gumbrecht argues that “riskful thinking” is central to the value of the humanities and arts in academia. He defines riskful thinking as investigation that can’t be expected to produce results interpretable as easy answers, but that instead is likely to produce or highlight complex and confusing phenomena and problems. But I think that this is more broadly true. Riskful thinking is critical to interdisciplinary and pre-paradigmatic sciences, or disciplines long doing normal science but in need of a shake-up. These are situations where compelling phenomena can become paradigmatic cases for study and powerful vocabularies can allow formulating new problems and theories.

What threatens riskful thinking, and how can we enable it? What is so great about riskful thinking anyway, and what makes some riskful thinking so successful, while much of it is likely to fail? At Nokia Research Center in Palo Alto, our lab head John Shen champions the importance of risk taking in industry research, but also argues that risk-taking is often misunderstood and that it is only some kinds of risk-taking that are most important to cultivate in industry research.
—
Finally, a list of Davidson-Nass similarities, just for fun:

• Both were hired to tenure track positions at Stanford, where they first did and published highly influential work
• Both are easily and widely seen as highly programmatic, having defined a clear research program challenging to currently popular approaches and beliefs in their fields
• Both had great difficulty finding early, publishable success with their research programs, even after ceasing their early work (Davidson: Plato, empirical decision theory; Nass: information processing models of the labor force)
• Both had other draws and distractions (Davidson: business school, teaching plane identification in WWII; Nass: being a professional magician, working at Intel)
• Both produced dissertations viewed by others in the discipline as odd (Davison: Quine “was a little mystified by my writing on this. He never talked to me about it.”; Nass: “my PhD thesis was so bizarre”)

References
Adlin, T. (2007). An interview with Cliff Nass. UX Pioneers. http://www.adlininc.com/uxpioneers/home_popular_row_2/interview_cliff_nass
Davidson, D. (1963). Actions, Reasons, and Causes. Journal of Philosophy, 60(23), 685-700.
Davidson, D., & Suppes, P. (1957). Decision Making: An Experimental Approach. Stanford University Press.
Finder, A. (2007, November 20). Decline of the Tenure Track Raises Concerns. The New York Times.
Freeman, R., Weinstein, E., Marincola, E., Rosenbaum, J., & Solomon, F. (2001). Careers: Competition and Careers in Biosciences. Science, 294(5550), 2293-2294.
Lepore, E. (2004). Interview with Donald Davidson. In Problems of Rationality, Oxford University Press, 2004, pp. 231-266.
Nass, C., Steuer, J., & Tauber, E. R. (1994). Computers are social actors. In Proc. of CHI 1994. ACM Press.
Reeves, B., & Nass, C. (1996). The media equation: how people treat computers, television, and new media like real people and places. Cambridge University Press.
Richardson, J. T. (1999). Tenure in the New Millenium. National Forum, 79(1), 19-23.
Sanford, J. (2000, November 17). ‘Elementary pleasures’ and ‘riskful thinking’ matter to Gumbrecht. Stanford Report.

Add comment April 29th, 2008 Dean Eckles

Laptop + shopping cart = mobile

This guy can roll up with laptop and webcam to record robots (photo CC from violetblue):

But in Silicon Valley, combining laptops and shopping carts is just a way to get chores done. When at Whole Foods in Los Altos, I saw a man pushing a shopping cart with a laptop in the part where you can sit your toddler. I suppose he was reading a recipe or something. (I, and I’m sure other Valley folks, do that on a phone.)

A bit odd, but then again, I used to be (I’ve fallen off a bit) judicious about capturing the contents of my shopping cart with ZoneTag.

Add comment March 7th, 2008 Dean Eckles

Texting 4 Health conference in review

As I blogged already, I attended and spoke at the first Texting 4 Health conference at Stanford University last week. You can see my presentation slides at SlideShare here, and the program, with links to the slides for most speakers is here.

The conference was very interesting, and there was quite the mix of participants — both speakers and others. There were medical school faculty, business people, people from NGOs and foundations, technologists, representatives of government agencies and centers, futurists, and social scientists. Everyone had something to learn — I know I did. This also made it somewhat difficult as a speaker because it is hard to know how best to reach, inform, and hold the interest of such a diverse audience: what is common ground with some is entirely new territory with others.

I think my favorite session was “Changing Health Behavior via SMS”. The methods used by the panelists to evaluate their interventions were both interesting to reflect on and good tools for persuading me of the importance and effectiveness of their work. One of my reflections was about what factors to vary in doing experiments on health interventions: there is (reasonable) focus on having a no-SMS control condition, and there are very few studies with manipulations of dimensions more fine-grained. Of course, the field is young and I understand how important true controls are in medical domains, but I think that real progress in understanding mobile messaging and designing effective interventions will require looking at more subtle and theoretically valuable manipulations.

You can see other posts about the conference here and here. And the conference Web site is also starting a blog to watch in the future.

1 comment March 7th, 2008 Dean Eckles