Paper Reading #32: Taking Advice from Intelligent Systems, The Double Edged Sword of Explanations

Kate Ehrlich, Susanna Kirk, John Patterson, Jamie Rasmussen, Steven Ross, Daniel Gruen represent IBM Research in this paper.

This paper was presented at IUI 2011.

Summary


Hypothesis
The hypothesis of this research paper was that when advice from an intelligent system is available to a user, the user tends to make different decisions.

Methods
In order to verify their hypothesis, the researchers set out to create an intelligent system that could classify various types of cybersecurity incidents. This system was known as NIMBLE (Network Intrusion Management Benefitting from Learned Expertise). Based on previous information, the NIMBLE system would attempt to classify new cybersecurity events. NIMBLE was also able to offer explanations as to why it chose the even that it did.

To complete the study, researchers found participants who were highly trained in the field of cybersecurity. All of these participants had a minimum of three years in the cybersecurity and network management field.

The researchers had these professionals complete 24 timed trials. The participants had 2 minutes to guess what type of event occurred and categorize it. NIMBLE was available to assist the participants.

Results
Researchers found that in the cases where NIMBLE suggested an answer, if the correct answer was available, correct response accuracy high. It was even higher when NIMBLE offered a justification to it's suggestion.

However, the interesting part of this study was when there were no correct choices available listed by NIMBLE. If NIMBLE offerred a justification (even though it was incorrect), researchers went with NIMBLE's suggestion most of the time.

Discussion
The results of this study is very interesting. It shows, in my opinion, that if an intelligent system presents information like it knows what it's talking about, we humans will follow what it says.
This can be very dangerous. For example, if we have an intelligent system diagnose humans, much of the time, the intelligent system will be correct. But this study shows we still need doctors to think critically. If a machine pretends to know what it's talking about, when it is incorrect, the doctor still needs to be able to form his own opinion about this.

Paper Reading #31: Identifying Emotional States using Keystroke Dynamics

Clayton Epp, Michael Lippold, and Regan L. Mandryk all represent the Department of Computer Science of the University of Saskatchewan.

This paper was presented at CHI 2011.

Summary


Hypothesis
The researcher's hypothesis in this paper was that they could identify a user's emotional state by simply using keystroke dynamics

Methods
To test their hypothesis, researchers recruited participants for the study and had them install a piece of software that would run in the background. This piece of software would measure key duration and key latency throughout the user's day-to-day activities on their PC.

The software would prompt the user through the day and ask the user how they were feeling at a certain time by asking them various questions in an emotional questionnaire. The user was then asked to type a randomly select portion from Alice in Wonderland.

The results of this entire study were put together and analyzed in an attempt to discern the user's mood through their typing habits.

Results
Researchers selected multiple features of typing sets to focus on when predicting mood. This includes keystroke duration and keystroke delay for various different keys in each key event chain.

They evaluated different models based on the user emotional state and the features of the key-entry during that area of time.

They discovered that when using these models, they were able to classify many of the emotions that were selected about 80% (and over) of the time.

Discussion
This is definitely an interesting article that could allow for several different interesting applications. One for example: when the system finds out that you're stressed or focused based on your key entry then it won't display any system notifications or it'll complete other actions to help the user stay focused.

An example of a fun application might be where a user is asked to type a given paragraph and based on how they type the paragraph, music that matches the user's mood will be played.

Paper Reading #30: Life "modes" in social media

Fatih Kursat Ozenc represented the School of Design from Carnegie Mellon University.
Shelly D. Farnham represented Yahoo!.

This paper was presented at CHI 2011.

Summary


Hypothesis
The researcher's hypothesis was that people organize their life into different groups based on social interaction.

Methods
To prove their hypothesis, researchers set out to conduct a design research study. In this study, they investigated how users interact with social media as well as how they interact with different social groups.

With each participant, they had a two hour in-depth interview to attempt to ascertain how the participant interacted and viewed each social circle. One of the exercises in this study was to draw out pictures or graphs of how the participant viewed certain aspects of their life. The goal was to have users visually map out important areas of their life. By looking at these maps, the researchers were able to find similarities between participants.

Another exercise they performed was through participant evaluation of certain scenarios and storyboards.  These scenarios had to do with boundaries and limitations of interactions. Participants were asked what they liked and disliked about each scenario. They were also asked what they might change.

Results
The result of the first study showed that participants preferred two different types of map. The first being a "social meme" type map. This map looked like some sort of graph. The center node of the graph contained the person and had different social circles and activities drawn off the connected nodes. The next time of map was the timeline map. Users drew this map with daily activities in order of occurrence.

As far as communication practices went, researchers found that the participants often used different types of communication media for different social circles. A user might use a certain set of communication channels (facebook, texting, etc) to interact with one of their social circles, while using a completely different communication channel set for another group.

Based on all these results, the researchers also enumerated various design suggestions for creating new social media and allowing for better social interaction with technology.

Discussion
Aspects of this paper area already being used frequently in social media. One large example was Google's been launch of Google+. Google+ allows users to place their "friends" into social circles according to where they fit. Then, users can choose what circle they want to share information with. This is very much in line with what the researchers of this paper were suggesting.

Paper Reading #29: Usable Gestures for Blind People, Understanding Preference and Performance

Shaun K. Kane and Jacob O. Wobbrock represented the Information School of the University of Washington.
Richard E. Ladner represented the Computer Science & Engineering school of the University of Washington.

This paper was presented at CHI 2011.

Summary


Hypothesis
In this papers, researchers attempted to show how blind people interact with gestures on a touch screen device as well as which gestures they prefer.

Methods


This research was broken into two separate studies.

For the first study, the blind participants were asked to invent their own gestures that could be performed on a tablet PC. 20 Participants were recruited for this experiment: 10 people were blind and 10 had sight.  The sight group was used as a baseline to compare against.
The participants, once oriented with the screen, were asked to perform to gestures to complete a list of tasks. Some of these tasks: undo, move up, delete, etc etc.

Each participant created two gestures per command.

The next study was focused on gesture performance. The blind participants and sighted participants were asked to perform the same gestures as a way to ascertain which gestures were easier or more effective for the blind to perform.

They were also asked to rate the easiness of the following gestures: tap, flick, multi-touch, shape, symbol.

Results
For the first study, they found that most of the gestures created by the blind group were more metaphorical and abstract than the sighted group. The blind group also used landmarks or the assumption of a given physical space. For example, they made the "control-v" finger tap movement for the past command.

They also found that the blind group preferred landmarks or performing gestures close to the screen.

In the second study, the blind group was found to prefer multi-touch gestures much more than the sighted group. They also slightly preferred the flick.

Discussion
This is definitely related to the previous reading that we did this week. However, the important distinction is that a blind group was asked to create and rate gestures. The previous research indicated that the preferred and most efficient gesture was the bezel mark. The Blind group also preferred bezel gestures so perhaps maybe in future mobile input, bezel gestures will be used more for both blind and sighted users.

Paper Reading #28: Experimental Analysis of Touch-Screen Gesture Designs in Mobile Environments

Andrew Bragdon is a PhD computer science student at Brown University.
Eugene Nelson also represents Brown University.
Yang Li is a senior researcher at Google Research.
Ken Hinckley is a principal researcher at Microsoft Research.

This paper was presented at CHI 2011.

Summary


Hypothesis
The researchers presented the hypothesis that there are certain ways to control mobile devices through gesture techniques and soft buttons that allow for easier interaction in different ways.

Methods
Researchers tested various different types of input methods to control the phone.

First, they looked at hard button initiated gestures. This means that a physical button mounted on the hardware would signal when a gesture could be drawn on the screen. So the user would have to first push the hard button before they could start the gesture.

Bezel gestures happen when the user starts the gesture off the screen and bring their finger over to the main touch area. This allows the phone to know when to start allowing gesture input.

To allow for a "performance baseline", the researchers also used soft buttons for testing purposes.

They also defined several different types of gestures: paths versus marks.

For testing which gesture types worked better in different environments, they looked at the accuracy and time of the gesture during various motor activity (sitting and walking) and distraction level (no distraction, moderate situational awareness, attention-saturating task).

15 participants completed this study.

Results
Researchers found that soft buttons and gestures worked about the same when the user was looking at the phone. However, when the user wasn't looking at the phone, bezel gestures were the easiest and most accurate.

When distracted, bezel gestures again performed better than soft buttons. Researchers stated that this was encouraging because it means that gestures can actually improve performance.

Discussion

As I mentioned last week, these kinds of studies are important. These kinds of tests ensure that gesture space (or a given input method) is actually truly better than what is currently used by the mainstream groups. Not only does this research prove that gestures can perform better than soft buttons, but it also helped show which gestures would work better.

Paper Reading #27: Sensing Cognitive Multitasking for a Brain-Based Adaptive User Interface

Erin Reacy Solovey, Francine Lalooses, Krysta Chauncey, Douglas Weaver, Margarita Parasi, Matthias Scheutz and Rober J.K. Jacob represent the students and faculty of Tufts University in the Computer Science department.
Angelo Sassaroli and Sergio Fantini represent the Biomedical Engineering department of Tufts University.
Paul Schermerhorn represents Indiana University and their Cognitive Science school.
Audrey Girouard represents Queen's University and their School of Computing.

This paper was presented at CHI 2011.

Summary


Hypothesis
In this paper, researchers set out to prove that it was possible to detect multitasking contexts and to react to these changes.

Methods
The researchers first labelled three different scenarios that describe the brain's ability to multitask. First is branching. Branching is a state where the primary goal still requires attention but the secondary goal must be attended to as well, thus the user must keep certain information regarding the primary goal active in their mind. The second type is dual-task. This is where the user must make quick changes between the primary task and the secondary task regularly. The final type is delay. Delay occurs where a secondary task is ignored when the user is working on the primary task.

Researchers main goal was to determine whether or not they could actually detect when a user's mind was in a certain context.

To test this, they created an experimental task which required a human and a robot to work together over a given task. The robot simulated a Mars rover on an exploration mission. The rover would give status updates about a newly found rock or a new location change. The participant would then have to respond to the robot with some command. There were three separate test runs that each tested a different multitasking mode.

They also conducted a proof-of-concept user interface test where the participant was given a task and based on their multitasking mode, the robot would have different behaviors and interact with the human differently.

Results


Through analyzing the first experiment, researchers were able to extrapolate the way hemoglobin levels worked to determine the three various multitasking states. Below shows the three graphs of the multitasking states:

Since these levels are different, that means that multi-tasking modes can indeed be discovered and the multitasking mode can be predicted in the users mind.

Results

Again, a very fascinating paper on Brain-Computer interfaces. There are many potential uses for a system like this. One system that I'd find very useful is an application that could help keep a user focus during tasks that require concentration. For example, writing a paper or coding. So, if I was writing some code and needed to focus, this BCI would detect I'm in the delay multi-tasking mode and would therefore block out certain notifications that my computer currently sends me (new emails, new facebook notifications etc).

Paper Reading #25: TwitInfo. Aggregating and Visualizing Microblogs for Event Exploration

Adam Marcus, Michael S. Bernstein, Osama Badar, David R. Karger, Samuel Madden and Rober C. Miller all represent student or faculty of MIT CSAIL.

This paper was presented at CHI 2011.

Summary
Hypothesis
Researchers proposed that through Twitter timelines, events could be identified and visualized.

Methods
To complete this study, researchers used Twitter to search through various tweets about a given subject. The TwitInfo algorithm can begin to track certain keywords and when they're used together. It then begins to build a list of tweets that have those keywords mentioned. After these tweets are gathered together, the algorithm builds a list of peaks and determines what words are being used most often in the peaks. These peaks are then classified as events.

The system is also capable of detecting positive and negative sentiment over a given topic over time.

All of this data can be then be organized and displayed in order to better visualize results.

In order to test the user interface of TwitInfo, researchers recruited 12 participants to perform certain tasks to locate an event.

Results


Tested over time, TwitInfo correctly organized most events properly. It was used frequently to analyze the events of a given soccer match. For every match, it's easy to see when the goals were scored due to the peaks in the events.

In the participant study of TwitInfo, participants were quick to reconstruct events when given information from TwitInfo, in many cases, in less than 5 minutes.

Discussion

TwitInfo struck me initially as a very useful, concrete research project. It is a classic example of using humans to complete a given task and then aggregate the data. Essentially, humans, without even knowing it are classifying events and when they occur. This makes TwitInfo's job easier. All the algorithm has to do is collect and analyze the data.

Paper Reading #26: Embodiment in Brain-Computer Interaction

Kenton O'Hara, Abigail Sellen, and Richard Harper are all researchers at Microsoft Research at Cambridge.

This paper was presented at CHI 2011.

Summary


Hypothesis
The hypothesis of the paper was that people react to Brain-Computer Interfaces in many different ways.

Methods
The Brain-Computer Interface (BCI) used in this research experiment is a game known as MindFlex (pictured). The "game board" has a fan underneath the construction which levitates a small ball into the air. The player controls the device via a wearable sensor on the forehead. When the player concentrates, the fan speeds up which causes the ball to rise. If the player relaxes, the fan slows down and the ball is lowered. As the game board rotates, the player must overcome obstacles by lowering or raising the ball by concentrating.

In order to test their hypothesis and to study the use of BCI interfaces in the "real world", researchers recruited 4 different groups of people. For each group, one person was in charge of getting the people for their group. The four groups were decently diverse. One group, for example, was a cohabiting couple. Another group was a family of four. These groups were all asked to play the game over the course of a week and to record themselves playing the game. The video footage was then analyzed and trends were discussed.

Results


The researchers gathered several different interesting trends.

First, they found that people attempted to concentrate by changing their body orientation. For example, one player held her breath and leaned close towards the ball. They also discovered that players developed strategies to cause the ball to lower through relaxation by looking away from the game board.

They also analyzed how people tried to visualize their concentration. For example, many of the groups who played talked about making the ball rise by thinking about making the ball rise and making the ball lower by thinking about it falling. In the game however, both of these thoughts require "concentration" so the ball would rise.

Another trend they discovered was spectator verbalizations throughout the game. Spectators played a roll in the game by either trying to make the current player mess up or by encouraging them to concentrate.

Discussion

Personally, I found this paper extremely fascinating. BCI, in my opinion, is the future of input devices. When they become more accurate and faster, the entire need for keyboards or controllers or other input devices can be completely eliminated.

This paper, while studying embodiment as it relates to BCI, gives some clues for future BCI work, in my opinion. In the thought visualization portion of the paper where participant tried to visualize lowering the ball, this can tell researchers what kind of "thought" is natural for a given action. More studies like this one can begin to map the natural "thoughts" for Brain-Computer interfaces.

Paper Reading #24: Gesture Avatar, A Technique for Operating Mobile User Interfaces Using Gestures

Hao Lu is a computer science graduate student at the University of Washington.
Yang Li is a senior researcher at Google Research

This paper was presented at CHI 2011.

Summary
Hypothesis
The researcher's goal was to create a system that allows for the easy interaction with mobile interfaces.

Methods
To test their hypothesis, the authors set out to create a system called the Gesture Avatar system. To them, one of the main problems of current mobile interfaces is the small size of many interface elements. For example, small buttons that are hard to use or tiny links on a webpage that are hard to click.

To counter this problem, the Gesture Avatar system allows the user to draw a shape or a letter near the object the user wants to activate and the system let's the user control the object from this larger "gesture avatar".
For example, a slider might be hard to activate while on the move or it might just be too small. A user could draw a box close to the slider they want to control and then slide that box like the slider.
Another example is for selecting small texts or clicking small links on a webpage. By drawing the first letter of the link they're trying to activate, the user can then click on the letter and the system attempts to figure out what link is meant by the user.

To test their system, researchers used a Motorola Droid running the Android OS. They found 12 participants to test the system. They used two different tests. The first was a letter selection test. A small amount of letters would show up on the screen and the user would have to select the highlighted letter by drawing the letter gesture and select it. The next test was simply a target selection. They also had some participants walk and use the device and some were asked to walk on a treadmill and use the device.

Results


They used three different types of criteria to judge the Avatar Gesture system: time performance, error rates, and subjective preferences.

For all letter sizes, Gesture Avatar's performance time was about constant. It was significantly faster than an alternate system (Shift) on the 10 px letters however.

Error rates also remained low and about constant over the three target size tests. The error rates are SIGNIFICANTLY lower than the alternate Shift system.

In the subjective preferences, 10 out of 12 participants preferred Gesture Avatar over Shift.

Discussion
The Gesture Avatar system is certainly a novel system of controlling a device. One of the main problems with mobile interaction is the limited screen space of a device. Screen size cannot be increased without making the device larger. Thus, for cell phones, making a larger screen is not really a viable option when it comes to fostering better user interaction.

One problem that might arise from the system is the fact that they only tested the system using two hands. Often times, when I'm in a hurry walking down the street, I only used one hand on my mobile device. I would have liked to see what would happen if someone just used their thumb to draw the gestures as opposed to a whole new hand.

Paper Reading #23: User-Defined Motion Gestures for Mobile Interaction

Jaime Ruiz is a PhD student of HCI at the University of Waterloo
Yang Li is a senior researcher scientist at Google.
Edward Lank is an assistant professor of computer science at the University of Waterloo.

This paper was presented at CHI 2011

Summary


Hypothesis
Researchers set out to prove that certain motion gesture sets are more natural than others in mobile interaction.

Methods
In this research experiment, researchers essentially allowed participants to create the gestures to be investigated. The authors created a list of tasks of several different types. The tasks would either fall into the "action" or "navigation" category. These categories were then subdivided into smaller tasks.

Participants were given the full list of tasks to be completed as well as a smartphone. They were instructed to complete the simple task (like pretend to answer a call, or go to the homescreen) with the idea that the phone would know how to execute the task.

The phone they were using was equipped with specially designed software to recognize and keep track of gestures performed by the participant.

After the test, the user was asked to comment on the gestures they used and whether they were easy to perform or good match for it's use.

Results
Through the test, researchers discovered that many users preferred and performed the same gesture.

The authors found several trends in the description of motion gestures. They found that many of the gestures mimic normal use. For example, 17 out of 20 participants answered the phone by putting it to their ear. The gestures also use real-world metaphors.

Through all these results, the authors were able to analyze the various created gestures and create a taxonomy of these natural gestures.

Discussion

Research like this, in my opinion, should occur more often. By giving participants free-range of their actions, they can come up with whatever input or gestures are natural to them. After analyzing these gestures, we can figure out which are natural.

By creating more studies like this, we can find out what types of input are preferred in a system. For example, do most people prefer to execute gesture XYZ to access the menu or do they prefer to use gesture ABC instead.

Paper Reading #22: Mid-air Pan-and-Zoom on Wall-sized Displays

Mathieu Nancel is a PhD student of HCI at the University of Paris
Julie Wagner is a postgraduate research student at the University of Paris.
Emmanuel Pietriga is a full-time research assistant at the University of Paris.
Olivier Chapuis is a research scientist at University of Paris.
Wendy Mackay is the research director at INRIA. 


This paper was presented at CHI 2011.


Summary
Hypothesis
In this paper, researchers experimented with various ways to control a wall-sized display. They had several hypotheses regarding which ways would be preferred. They believed that two handed gestures will be faster and easier to use than one-handed gestures. They also believed that users will prefer circular gestures. They predicted that small gestures would be preferred.

Methods
In the project, researchers tested various different types of gestures. First, they used uni-manual (one handed input) and bi-manual (two handed input). Each of these manual gestures could be used in conjunction with 1D control, 2D surface, and 3D free space.
The 1D control was control in one plane using a linear scrollwheel or a circular scrollwheel.
The 2D surface was control using a PDA or smartphone.
The 3D free space allowed gestures to be used.
With the bi-manual input, one hand was used to show the focus of the gesture, i.e. where to focus the zoom or pan, etc.

In the experiment, the researchers tested three factors: Handedness (one handed, two handed), Gesture (circular, linear), and guidance (1D, 2D, 3D).

Each participant was given a target in a high zoom scale. They had to zoom out to display all targets and they had to pan the targets so they were all visible on the screen.

Results
In most cases, they found that two hands (bimanual gestures) were faster and had shorter movement times
They also found that 1D path control was much faster than the other input types.
Linear control was also found to be faster than circular.

In the qualitative results, participants seemed to prefer the one handed gestures. They also preferred the linear gestures to circular gestures. They found that circular gestures on a surface were too hard to do.

Discussion
The paper itself was fairly interesting although it was slightly difficult to identify with since I've never personally used a large wall-sized display. I wasn't able to really agree or disagree with their findings since I've never had to use gestures on such a large display.

This kind of research could potentially become very helpful in the future when wall-sized displays become more widely used, though. Another user qualitative might be helpful in the future as well. By letting users design their own gestures, they could choose the most natural gestures for the large wall.

Paper Reading #21: Human Model Evaluation in Interactive Supervised Learning

Rebecca Fiebrink is an assistant professor at Princeton in the school of Computer Science but also is associated with the music department.
Perry R. Cook is a professor emeritus of Princeton in the school of Computer Science and the Department of Music.
Daniel Trueman is an associate professor of music at Princeton University.

This paper was presented at CHI 2011.

Summary


Hypothesis
The researchers main hypothesis was that while using a machine learning system, humans could evaluate and adjust the machine learning generated model to better train the system.

Methods
To test their hypothesis, researchers created a system called The Wekinator. The Wekinator is a software system that allows human users to train a given machine learning given certain gesture or real time input. Based on the human input, the system attempts to figure out the best course of action for the human input. If the system is incorrect or it isn't certain enough, the human can tell the system what the correct action is and the human can train the system by performing the gesture or input over more iterations.

To test Wekinator and discover how humans used it, they created three different studies.

In Study A, researchers had members of the Music Composition department (PhD students and a faculty member) discuss Wekinator and allowed them to use it over a given amount of time. When they were done using it, the participants sat down with the researchers and talked about how they used Wekinator and they offered various suggestions. Most of the participants used Wekinator to create new sounds and new instruments through the gesture system.

In Study B, students ranging from 1st year to 4th year were tasked with creating two different types of interfaces for the machine learning system. The first was a simple interaction where based on an input (like a gesture or a movement on a trackpad), a certain sound would be produced. The next type was a "continuously controlled" instrument that would make different noises based on how the input is being entered.

In Study C, researchers worked with a cellist to build a system that could respond to the movements of a cello bow and report the movements of the bow correctly.

Results
Researchers found that in all cases, participants often time focused on iterative model-rebuilding by continuing to re-train the system and fixing the models as they needed to be adjusted with given input.

In all of the tests, direct evaluation was used more than cross-validation. Essentially, directly evaluating the model and modifying it was preferred over the cross-validation model.

The researchers also noted that cross-validation and direct evaluation was actual feedback to the participants so that they could know whether or not the system was properly interpreting their input.


Discussion

The usage of a system like Wekinator could have some very far-reaching benefits. At the moment, machine learning seems pretty bulky and hard to do if you're not an AI programmer or scientist. But Wekinator seems to make the task of machine learning easier to understand and to put into practice. In the second study, a lot of the participants didn't know much about machine learning until they were briefed on the subject. However, they were still able to put Wekinator into practice to create new types of musical instruments.

Wekinator (and systems like it) could open the door on many different input tasks. Gesture and motion control can be better fined tuned by quick training. Since multiple people might perform a gesture differently, using Wekinator, a system's model can quickly and efficiently be "tweaked" to fit a given person. This makes analog input (like speech recognition or gestures) more reliable and more accessible to a large group of people and technologies.

Paper Reading #20: The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only ANOVA Procedures

Since I completed Paper Reading #16, I'm choosing this blog as my blog to skip.

Paper Reading #19: Reflexivity in Digital Anthropology

Jennifer A. Rode is an Assistant Professor at Drexel University. She teaches in the school of Information.

This paper was presented at CHI 2011.

Summary


Hypothesis
Rode's main hypothesis and idea in this paper is to discuss the fact that ethnography are present and useful in the HCI field. She also discusses various types and aspects related to ethnography.

Methods
Through her paper, Rode talks about the needs and uses for what she calls "digital ethnography." A digital ethnography is simply an ethnography that is focused on technology in some fashion. One example that Rode gives in the paper is that of a researcher visiting super-churches and learning how each one employs technology.

To prove her points and to visit the various aspects of digital ethnography, Rode breaks down the various components. First, she covers the styles of ethnographic writing. Then she moves on to discussing ethnographic conventions before she looks at the framing ethnographic practices.

Results
First are the three styles of ethnographic writing.
The realist is interested in accurately displaying the subject. Rode writes that "the realist account strives for authenticity." The realist also goes to great lengths in order to show the reader the native's point of view.
The next kind of ethnographic writing is the confessional. In this kind of study, the writer comes up with a kind of theory on the subject, discusses it, then applies the theory to the ethnography and addresses it.
The final kind of ethnographic writing is the impressionistic. This is simply an attempt to "give an impression" of the subject. It is an attempt to capture the entire feel of the subject much like how the impressionist painters captured their subjects in a painting.

Next, she discussed the ethnographic conventions. The first discussed is the rapport that the ethnographer must build with their subject. If a researcher can't interact well with their subject, the subject is less likely to reveal information or to openly discuss a topic.
The next discussed was the participant-observation. Rode described this as simply "deep hanging out." By being around and observing the participant in their "native" environment, one can gain information.
The final ethnographic convention discussed was simply the use of theory. By coming up with theories on the subject, researchers can focus questions for focus their research.

Discussion

This paper was a good description of how ethnography applies to the digital world. This paper will be very helpful when we begin to work on the design project related to our class ethnography projects. It will help point out things to look for and ways to approach problems.

Paper Reading #18: Biofeedback Game Design, Using Direct and Indirect Physiological Control to Enhance Game Interaction

Lennart E. Nacke has a PhD in game development and is a assistant professor at the University of Ontario Institute of Technology.
Michael Kalyn is a graduate student of computer science at the University of Saskatchewan.
Calvin Lough is a student of computer science at the University of Saskatchewan.
Dr. Regan L. Mandryk is an assistant professor at the Univeristy of Saskatchewan.

This paper was presented at CHI 2011.

Summary
Hypothesis
The researchers' hypothesis in this paper was that a computer game could be controlled via both direct and indirect physiological inputs.

Methods
In order to test their hypothesis, they created a simple side-scrolling platforming game. In order to test the physiological aspects of the systems, they used 6 different sensors.

Gaze interaction: They tracked the pupil of the user's eyes
Electromyography: Detects the electrical activation of the subject's muscles.
Electrodermal activity: Detects galvanic skin response. Measures excitment.
Electrocardiography: senses the heart activity.
Respiration sensor: measured the breathing rate of the subject
Temperature sensor: directly controlled temperature sensor

To test the physiological inputs, they came up with two different sets of inputs as well as a control test.
In the control test, the game was simply played with an Xbox 360 controller.

In the study, they had 10 participants play all three tests: the two different sets of inputs and the controlled test.

Results


In the results, the researchers discovered that both input sets with the physiological inputs rated higher in fun than the control test. Participants also said that the physiological tests were more novel.

They also expressed favorite sensors. Participants expressed the fact that they preferred the physiological input sensors that they could directly control. The EKG sensor and GSR were the least favorite sensors since they were indirect.


Discussion

The usage of physiological input was proved to make games more fun than normal. In my opinion, this is pretty appealing. One of the main issues with controlling computer games is the input problem. By giving a wider ranger of inputs (besides just joysticks and buttons), it makes the game more interesting to play.

One thing that I found was interesting is the fact that the "novelty" of the system was so highly rated. I'd like to see if the novelty lasted over time. For example, the Nintendo Wii, when it first came out, was highly popular due to it's novelty. However, it's recently lost a lot of players over time.

Paper Reading #17: Privacy Risks Emerging from the Adoption of Innocuous Wearable Sensors in the Mobile Environment

Andrew Raij is an associate professor of electrical engineering at University of South Florida
Animikh Ghosh is a junior research associate at SETLabs.
Santosh Kumar is an associate professor at the University of Memphis
Mani Srivastava is an associate professor of electrical engineering at UCLA.

This paper was presented at CHI 2011.

Summary


Hypothesis
Researchers set out to prove, in this paper, that a users personal privacy can be violated by inferring data from mobile sensors.

Methods
Researchers developed a framework that allowed them to gain information from sensors and infer other data. By receiving raw data from a sensor, the information can be reconstructed to decipher behaviors and discover contexts.
An example they gave related to a woman who was jogging and has a seizure. Through sensor movement data, they could infer she was running and then that she was having seizure. They could also discover information about the context well (such as where she was jogging as well as time).

Due to this information being able to be inferred, researchers believed that this could lead to privacy threats (financial, psychological, and physical).

To confirm their thoughts, researchers set out to test three goals: discover how much people care about privacy concerns, use the framework to analyze behaviors, discover how identifications of the data effects concert.

To conduct this experiment, they used two groups. The first group had "no personal stake". They did not wear an AutoSense sensor. They were simply asked a questionnaire. The second group had a sensor and were also asked the same questionnaire.
The sensor group was asked to wear the AutoSense sensor during their awake hours for 3 days.

Results
At the end of the experiment, they were able to use data collected by AutoSense to analyze the sensor group's three days.

In the questionnaire, they were able to discover how concerned people were about given factors. The sensor group was much more concerned following the test. People said that they'd be concerned about people knowing their location at a given time.

The participants also expressed concern about who the results were shared with. They expressed much more concern if their identity was shared with the data.

Discussion

While parts of this paper were generally hard to follow, it was still an interesting concept. Many of our mobile devices are capable of sensing movement and audio. If our mobile devices started collecting information through the day about our movements, certain things might be extrapolated and thus might raise some privacy concerns.

One of the large problems in the coming age of technology is making sure this kind of sensor data does not get compromised or leak out. Especially if systems that analyze this data are created. This could become an entirely new branch of cybercrime.

Paper Reading #16: Classroom-Based Assistive Technology, Collective Use of Interactive Visual Schedules by Students with Autism

Meg Cramer is a graduate student at UC Irvine study Informatics.
Sen H. Hirano is a PhD student at UC Irvine also studying Informatics.
Monica Tentori is an assistant professor in Computer Science at the Universidad Autónoma de Baja California.
Michael T. Yeganyan is a researcher at UC Irvine.
Gillian R. Hayes is an assistant professor in Informatics at UC Irvine.

This paper was presented at CHI 2011.

Summary


Hypothesis
The researchers hypothesis was that, through their vSked interactive and collabortive system, they could assist and augment the teaching of students with autism.

Methods
The vSked system consists of portable touch devices for each student as well as a large touch display at the front of the classroom. This large display is connected to by the smaller portable devices. The smaller devices report the students progress through the day and their progression through a set of systems to the screen.

The vSked systems allows for the student to select a reward they want to work towards as they answer questions. The system can ask the student two different types of questions: either multiple choice or "voting" questions (where there is no right answer).

For the multiple choice questions, an icon for each answer is displayed for the student to tap. If the student gets it incorrect, the correct icon will wiggle on the screen.

After a student has been progressing through their questions, the teachers can award tokens that students can use towards their chosen reward.

To test the success of the system, the researchers used a combination of observation and interviews with the teachers and aids.

Results
They found that their system garnered independence as students were excited to work on the device. The students became more focused on the task at hand without needing a teacher to assist them in staying focused.
Students also stayed motivated due to the reward system based on a students correct answers.

The system also allowed for consistency through the student's educated in many different areas.
For example, the system allowed the students to keep track of their schedule and look at it whenever they needed to without being able to change it. They were also able to see changes that a teacher would make on the big screen, on their local smaller screen.

Finally, they found that the system built community. With each student having their own screen, sometimes students would want to see how another student was using the device. Students would also keep track of their peer's progress through the large screen at the front of the class room.

Discussion

Overall, the paper was a well documented, easy to understand experiment. This kind of technology used in the classroom, especially with students with autism, can be extremely powerful.

One interesting note that I'd like to know about in the future of this system is how it affected students in the long-term. Did students who used this system progress faster in their education than students who didn't use this system? Or did these students end up relying too heavily on the devices and thus this system hurt their education?
These are a few questions I'd like to learn about in the future.

Paper Reading #15: Madgets

Malte Weiss is a PhD student at RWTH Aachen University.
Florian Schwarz is a diploma thesis student at RWTH Aachen University.
Simon Jakubowski is a student assistant at RWTH Aachen University.
Jan Borchers is a professor of computer science at RWTH Aachen University.

This paper was presented at UIST 2010.

Summary


Hypothesis
The researchers attempted to prove the concept of a surface that would allow for interaction with dynamic physical interfaces on the surface through electromagnets.

Methods
The hardware set-up of the device is composed of a display panel, IR emitter and detector, electromagnet system, and fiberoptic cable.

With these system, physical devices (called Madgets in the paper) can be placed on the surface and either be manipulated by the user or by the system itself. Through this kind of system, dynamic interfaces can be created and recognized by the device.

Through complex calculations, the researchers solved various problems such as actuation, heat prevention, tracking, and more.

After setting up their system, they began to explore the possibilities of a system.

Since this paper was written as a proof-of-concept paper they did not bring in participants to interact with the surface.

Results
After the researchers created and designed their system, they discovered several interesting ways to interact with the device.

The first was the creation of general-purpose widgets. These are widgets like buttons, sliders and knobs. One example they gave was that a user could put a slider widget on a table while watching a video and control where the video was being played from. Then the device would move the slider's knob based on time left in the video.

The next was the usage of the dimension of height. Using the magnet surface, they could create buttons that would rest above the surface and could be depressed and activated.

They then explored force feedback. By using the electromagnets, knobs can be given "perceived friction" and can resist slightly when the user attempts to use them.

They even experimented with more novel madget ideas such as creating a motor through the interaction between the surface magnets and the magnets in the physical device. Also created was a madget that provided audio feedback through a bicycle bell.

Discussion


One difficult thing about this paper was the fact that they presented no user study whatsoever. I'd be very interested to see either a study on how users interacted with a given interface or how the users created their own interface with the use of Madgets to complete a given task.

Physical interaction is generally much more intuitive, natural, and more precise than interaction with touch devices. Therefore, a system like what was presented in this paper could be come very handy especially when one needs to make precise inputs.

Another beneficial use of Madgets would be the creation of custom interfaces based on user preference. If a user is trying to control something (like maybe a sound system) they might want a certain knob in one place and another slider somewhere else based on their usage. Madgets would allow this kind of custom usage.

Paper Reading #14: TeslaTouch, Electrovibration for Touch Surfaces

Olivier Bau is a research scientist at Disney Research. He has received a PhD in computer science. with a focus on Human-Computer Interaction.
Ivan Poupyrev is a senior research scientist at Disney Research.
Ali Israr is a research scientist at Disney Research. He has received a PhD in mechanical engineering.
Chris Harrison is a research scientist at Disney Research.

This paper was presented at UIST 2010.

Summary


Hypothesis


The researchers proposed a touch screen device with new tactile feedback so that a user can feel the object on the surface.

Methods


In order to complete their test, researchers designed what they called the TeslaTouch apparatus. By running a very slight current through an insulator, they could simulate a given surface based on the wave type of the current.
The entire system consisted of the surface, combined with the projector, and IR emitter and detector.

To better equip the device to work with humans, they experimented with waves of different properties and sizes and had users classify their view of the wave.

They then experimented with "psychophysics" of the device. They measured the detection threshold of the electrovibrations and the JND (Just-noticeable-differences) of the electrovibrations. By testing for these thresholds, they could determine which electrovibrations were noticeable and which were detected.

Finally, they discussed various types of interactions that TeslaTouch could be used with. These include simulations, GUI widgets, and various other rubbing interactions.

Results


Through their human factors test, they discovered the various hertz and voltage combinations that users would classify as sticky/waxy, bumpy/smooth, pleasant/unpleasant, friction/vibration. This test would aid in the future development of the device.

With the psychophysics test, they were also able to determine the thresholds previously mentioned

Discussion


Feedback is one of the most lacking features of touch screen devices. With this kind of simple tactile feedback, touch screen displays could become much more usable and create an even deeper layer of interaction.

Typing on a virtual keyboard would feel much more natural and would allow a user to rely on tactile feed as opposed to only visual feedback.

One thing the researchers could have included in this paper was a larger field study of this TeslaTouch apparatus. I would be very interested to see how normal people react to the device and to understand what kind of applications they could visualize.

Paper Reading #13: Combining Multiple Depth Cameras and Projectors for Interactions On, Above, and Between Surfaces

Andrew D. Wilson is a senior researcher at Microsoft Research who also co-authored another paper that has been read for this class.
Hrvoje Benko is also a researcher at Microsoft Research who focuses on Human-Computer Interaction

This paper was presented at UIST 2010.

Summary


Hypothesis


The hypothesis in this paper is that it's possible to use multiple depth cameras and projectors to interact with an entire physical room space.

Methods


To prove that their hypothesis is feasible, the researcherts implemented various different components using the depth camera and multiple projectors.

The first component they discussed was their simulated interactive surfaces. They wanted any surface (like a table or a wall) to be come interactive by projecting data and objects on that surface. The surface could be interacted with through movements captured by the depth cameras.

The next component was interactions between the users body and the surface. One of these interactions consisted of the user touching one object on one screen then touching a location on the other surface. Completing this action would cause the selected object to be moved across the two locations. Another interaction they added was allowing users to pick up objects from a surface. By picking up an object, a orange "ball" would appear on the users hand and allow them to transfer the object to another surface.

The final component they added was spatial menus. A user can hold his hand over the menu for a few seconds to activate it. The menu is then projected on to his or her hand.

While there have been systems similar to LightSpace, none have combined all the features that the researchers discussed including the usage of multiple depth cameras.

Their full test consisted of allowing users to interact with the LightSpace prototype at a demo event.

Results


They found that there were multiple occassions that LightSpace might fail. One being too many users in the space causing a slowdown or interactions to not be handled well. Also, some interactions were found to fail due to a user accidentally covering their hand or body with their head.

Some users even developed new ways to interact with the LightSpace system.


Discussion


Systems like LightSpace, in my opinion, are part of the future. Being able to interact with objects in a full 3D space is something like what you see in a science fiction movie. However, I feel like there are many advances the researchers could add to make the system even better.

One is a better tracking system. They mentioned in the paper that they left out 3D hand tracking. There are new algorithms being released which allows for easy quick 3D hand tracking. These algorithms could be added in to allow easier and more efficient tracking for even better results.

I think for a system like this to catch on, camera and projection techniques will also have to improve. When a user gets in the way of the camera or projection, the interaction between the user and system is disrupted.

Paper Reading #12: Enabling Beyond-Surface Interactions for Interactive Surface with an Invisible Projection

Li-Wei Chan, Hsiang-Tao, Hui-Shan Kao, Ju-Chun Ko, Home-Ru Lin, Mike Y. Chen are graduate students at the National Taiwan University.
Jane Hsu is a computer science professor at the National Taiwan University.
Yi-Ping Hung is also a professor at the National Taiwan University.

This paper was presented at UIST 2010.

Summary


Hypothesis
The researchers set out to prove that interactions can occur beyond the surface of a touch display. By using infrared project, users can use other devices to interact with the scene.

Methods
In order to facilitate the hypothesis, researchers had to create their own custom table design. The table makes use of multiple IR cameras as well as color project layer coupled with a IR projection layer. The color project projects the image onto the color projection which is seen by the human eye. However, the IR projection layer also has data which allows other devices to interact with it.
By providing this IR projection layer, mobile devices can easily determine their orientation to the table.

The researchers also provided three various ways to interact with the table.
The first was the i-m-Lamp. The lamp has a pico projector and an IR camera attached to it. When the lamp is pointed at the screen, the IR camera detects where it is looking and it figures out what to overlay over the color screen.
The i-m-flashlight is very similar to the i-m-lamp but it was used to provide a more dynamic way (as opposed to the more static lamp).
The final interaction method was the i-m-view. The view was a tablet with a camera that could detect what it was looking at and display a 3D projection of a given map.

Results


Through initial results, they found that the users used the lamp like static object (as predicted) and use the flashlight to quickly select object and display relevant information.
The flashlight was a much more dynamic object for the participants.
The view was interesting from the participants perspective but lacked some features when the user would try and look at the 3D buildings a certain way (the view could no longer detect what it was looking at since it couldn't see the table).

Discussion


The system that was created by the researchers could easily find a place in the field of augmented reality in my opinion. In fact, the i-m-view was really an experiment in augmented reality.

I think an interesting path to take this kind of technology would be to use mobile phones to interact with the large table, as well.
I envision a board game (or something similar) where players can share the same view but the mobile device could provide a unique view to the player based on where they're looking on the board.
This kind of technology has many different applications as well. For example, imagine looking at a large phone directory on your table surface. By hovering your phone over the table, it could detect what name you were looking at and present a "Call Number?" message.

Paper Reading #11: Multitoe, High-Precision Interaction with Back-Projected Floors Based on High-Resolution Multi-Touch Input

Thomas Augsten is currently a graduate student working on his masters in IT Systems at the University of Potsdam.
Konstantin Kaefer is currently also a student at the University of Potsdam
René Meusel is a student at the University of Potsdam.
Caroline Fetzer is a Human-Computer interaction student at the University of Potsdam
Dorian Kanitz is a student at the University of Potsdam.
Thomas Stoff is a student at the University of Potsdam.
Torsten Becker is a graduate student at Potsdam and also holds a B.S. in IT Systems engineering.
Christian Holz is a PhD student at Potsdam and has a research interest in Human-Computer Interaction.
Patrick Baudisch is a professor of Computer Science at the University of Potsdam and the head of the Human-Computer Interaction research department.

This paper was presented at UIST 2010

Summary


Hypothesis
The researchers set out to prove that meaningful interaction through floor based input and output can occur.

Methods
The actual floor system that they constructed was called a FTIR (frustrated total internal reflection) device. This was created to allow for images to be projected onto the floor and to be able to handle touch input from a user's shoes.
In order to figure out the most natural ways to design the system, the researchers carried out multiple experiments to see how people would naturally interact with floor user interface objects.
The first was an experiment to see "how not to activate a button". Participants were asked to activate a fake button on the floor and to not activate another button. This test allowed the researchers to see what the most preferred way to interact with floor buttons is.
Another experiment was carried out by having participants step on the FTIR and to choose what buttons on the floor should be highlighted. This allowed the researchers to discover the conceptual model behind a foot press created by the user.
Then to facilitate precise input, participants were asked to choose their preferred "hotspot" which allowed them to select precise objects.

Various other features were also added to the FTIR including being able to identify users based on shoe prints. They also were able to do more complex recognition such as identifying a walking motion.

Results
In many of the tests they performed, participants often had their own way of doing things. In the how not to activate a button test, most preferred to activate the button by tapping and walking over the button to not activate. However, there were many other actions that were used.
In a similar fashion, the hotspot test had many users use unique hotspots. Eventually the designers decided to just allow users to set their own hotspot.

Researchers also successfully implemented the various other previously discussed features such as tapping vs. walking and user identification based on analysis of the sole processing.

Discussion

This paper was organized as more of a documentation of the process to create a system that recognizes foot input. This organization made the paper more interesting and enjoyable in my opinion.

The important distinction that needs to be made with this interesting piece of technology is the fact that foot based touch screens are more of a novel technology that can fit into a proper niche.
It's not meant to be a substitute for a computer or iPhone, for example.

I can easily see this type of technology employed in future homes or offices or even restaurants. The smart home could recognize it's owner by their footprint and their steps and complete tasks based on the users location. For example, when you get home from work in the afternoon, your house knows that you're headed to the kitchen by your walking and turns on the light. Or if you're hosting a party, a foot gesture could bring up a menu which would allow you to put on new music.

Book Reading: Gang Leader for a Day

I never really ever thought that a sociology book could be interesting. That is until I read Gang Leader for a Day.

One of the interesting things I enjoyed about the book was that it was a real, visceral study of the projects and what goes on in gangs. I feel one of the big mistakes that sociologists make is the sanitation of the data. What I mean by that is that, even if the study is on a group of people, the sociologists just focus on the data without looking at the implications behind the data.

For example, we see studies all the time about the unemployment or homeless rates. But to us, those are just figures. They don't apply to us. We can't imagine them or picture what they actually mean. I think Venkatesh realized that. As opposed to just asking questions and collecting statistics, he builds relationships and explores what it means to live in the projects and to live by being involved with the gangs. If sociologists want to study society, what better way is there than to engulf yourself within a given community?

Towards the end of the book, you find that the projects that all those people rely on are being torn down. I can't help but think that if the president had gone and lived among those people like Sudhir, some other alternative would have been reached. Instead, the president was probably delivered a nice clean report that detailed the small amount of population and the large gang activity.

You can't study society with data alone. To truly study society, one must be engage, interact, and learn about the life of one in the society.

Paper Reading #10: Sensing Foot Gestures from the Pocket

Sensing Foot Gestures from the Pocket

Jeremy Scott is currently a graduate student at MIT but previously earned his bachelor of science at the University of Toronto.

David Dearman is a PhD student at the University of Toronto and is focusing on HCI

Koji Yatani is also currently a PhD student at the University of Toronto with an interest in HCI.

Khai N. Truong is an associate professor at the University of Toronto involved in HCI.

This paper was presented at UIST 2010.

Summary

Hypothesis

The main hypothesis of this paper was that foot gestures could provide a means of eyes-free input with the gestures being interpreted from a pocket by a cell phone. 

Methods

The researchers attempted to prove their hypothesis but completing two studies.

In the first study, they had participants make various foot gesture that were recorded and studied by a complex camera system. The four gestures they tested were the dorsiflexion, plantar flexion, heel rotation, and toe rotation. They had participants try to select a target by making the specified gesture. 

In the next study, they used a mobile device and it's accelerometer to detect foot gestures. Each participant wore three mobile devices in the front, the side, and the back. The user would initiate a foot gesture by "double tapping" their foot before using the gesture. The two gestures they focused on was the heel rotation and the plantar rotation. 

Results

In the first study, they found that smaller angles were generally more accurate in the target selection. They also found that the participants preferred the heel rotation as it was more comfortable. 

In the second study, they found out that the phone in the side pocket, in general, gave much higher success rates. Over all gestures, they found that the side pocket placement of the iPhone gave an accuracy rate of 85.7%. 

Based on this high accuracy, they concluded that using foot gestures was indeed a viable input for a eyes-free device. They also began to prepare for future studies using foot gestures.

Discussion

While this is mainly a reiteration of a point they discussed in the paper, having a foot sensor that could control your phone would be very helpful. For example, if you're sitting in class or standing around having a conversation and you have an incoming call, it'd be very helpful to automatically forward the call to voicemail by making a discreet foot gesture. 

A furthering of this study might be to remove the phone from the detection system. Perhaps create a small unobtrusive sensor that the user could put in his shoes that could interface with the phone. That way, one could get even higher success rates (the sensor would be right at the foot as opposed on the hip like in the study). Having the phone act as a sensor is very interesting but a simple, wireless detection sensor wouldn't be obtrusive or complicated either. 

Paper Reading #9: Jogging over a Distance Between Europe and Australia

Jogging over a Distance between Europe and Australia. 

Florian Mueller is currently a visiting scholar at Stanford University. However, he's previously worked at the University of Melbourne and Microsoft Research in Asia.

Frank Vetere is a senior lecture specializing in Human-Computer Interaction at the University of Melbourne.

Martin R. Gibbs is a lecturer at the University of Melbourne. 

Darren Edge is a Microsoft Research Asia researcher who focuses on Human-Computer interaction.

Stefan Agamonlis is currently an assistant director of the Rebecca D. Considine Research Institute but previously he worked as the chief executive and research director of Distance Lab in Scotland.

Jennifer G. Sheridan is currently the director of User Experience at Big Dog Interactive.

This paper was presented at UIST 2010

Summary

Hypothesis

The main idea behind this paper was that they wanted to see if an activity like "social jogging" would enhance the activities experience and to see if a "technologically augmented social exertion activity" would be possible or worth the design.

Methods

The system that the researchers created was composed of a small headset that participants would wear that was attached to a small cellphone and a heart rate monitor. While a participant would run, they would be able to talk to their running partner over the phone through the mic. However, they also added an additional social experience by the addition of the heart rate monitor. Before racing, participants were asked to set their target heart rate. The closeness of the person's voice to the other person was tied to their heart rate. For example, if partner A was currently at half their target heart rate and partner B was at their target heart rate, partner A would sound like he was behind to partner B and partner B would sound like he was ahead to partner A. This gave the participants a sense of who was "ahead or behind."

There were three different main design elements the researchers attempted to include: Communication Integration (allowing the joggers to talk while running despite location), Virtual mapping (allows the joggers to tell who is in front and who is behind), and Effort Comprehension (due to the heart rate monitor contributing to the virtual mapping, one can tell how they're personally doing).

Results

To test this system, they had 17 participants go on runs that used the Jogging over a Distance system. In the paper they reported on 14 different runs. The participant's locations varied from Australia to Germany. All participants also knew each other before running as well.

Most of the feedback was very positive. Participants enjoyed being able to talk to their partner while jogging despite being far away. They also enjoyed the fact that the virtual spatial mapping was tied to target heart rate. They cited the fact that normally they can't run with one of their partners due to having different target heart rates and thus running at different paces never worked.

Discussion

Exertion "games" like Jogging over a Distance are, in my opinion, extremely interesting. For me personally, jogging can sometimes be a pain. Running on the treadmill can often be boring and running by oneself outside isn't also that fun either. You often times don't go as far as you usually can if you run by yourself. These types of exertion games make it easier to forget about the work associated with running. 

I think they did a great job proving that their system worked by letting the participants freely use the system. Through this they got some very constructive feedback and it let the participants use the device how they personally wanted to use it. 

Paper Reading #8: Gesture Search, A Tool for Fast Mobile Data Access

 Gesture Search, A Tool for Fast Mobile Data Access


This paper was presented at UIST 2010.

Yang Li is a Senior Google Researcher with a PhD in Computer Science from the Chinese Academy of Sciences.

Summary


Hypothesis
The main hypothesis discussed in this paper was that this Gesture Search platform would provide an easy way for users to look-up contacts, applications, etc by using simple gestures. The Gesture Search application was built where users could directly draw a gesture to the screen and the gesture engine would attempt to locate relevant items from the user's phone.

Methods
In order to be flexible and efficient, the researcher implemented several helpful modules to aid in searching.

The first was to recognize multiple possible meanings of a gesture rather than selecting just one. For example, if the user writes something that could either be an "H" or an "A" both alternatives are considered.

Second, the system would recognize frequently searched-for terms and would appear higher up in the search results.

They also added a system to figure out whether a gesture was an actual gesture or simple a UI Event system.

In order to test out there complete system, they sent it to various Android users in the office and asked them to use the application. As a user would use the app, the app would log and report information back to a homebase where the usage data would be analyzed. At the end of the test, users were asked to complete a survey on the app usage.

Results


The core group they focused on was the group of people who used the app at least once a month and have used at more than once a week.

They found that users primarily used the app to find contacts in their phone device.

They also found that short gestures indeed helped find an item in their phone faster. In fact, 61% of 5,497 queries found the correct item. Many queries were extremely short even with large datasets.

With the survey, user support was generally solid. Again, most users reported using the app for contact search for the most part. Not many used it for searching and opening applications

Discussion


This kind of gesture system seems like it would be naturally at home on touch screen devices. It's surprising how long a system like this has taken to develop onto modern devices.
Palm Pilot's from the late 90s and early 2000s had gesture search abilities (through their "graffiti" system).
Perhaps the reason why this kind of gesture search has become so strange is our continued love affair with the QWERTY keyboard. We've begun to almost completely associate input on technology devices with a QWERTY keyboards. I believe that's why gesture search is sometimes hard to pick up and not so commonly used.

I feel as if the researchers did a good job getting their point across, though. It seemed that when users got used to using this application, it let them quickly find what they were looking for.

A possible future development for this kind of application would be to include it natively in the OS as opposed to just creating an application that must be used.

Paper Reading #7: Performance Optimizations of Virtual Keyboards for Stroke-Based Text Entry on a Touch-Based Tabletop

Performance Optimizations of Virtual Keyboards for Stroke-Based Text Entry on a Touch-Based Tabletop

Jochen Rick is an assistant professor at Saarland University in the department of Education Technology. 

This paper was presented at UIST 2010.

Summary

In this paper, the writer discussed the need for an alternate input in touch-based devices. Currently, touch based keyboards are widely used in many applications. The writer presents a comparison of the various stroke-based input methods. He presents many different stroke-based keyboard layouts that have been used over time. 

Past researchers have used many different models in an attempt to allow users to type with maximum accuracy and efficiency. 

The main purpose of this paper is to attempt to discover the most efficient stroke-based keyboards and to present various ways to measure the layouts. 

In the study, they had eight adults connect several different nodes together through a stroke on the touch device. Through this study, they were able to ascertain what strokes were easier to make and used most often. Through these kinds of studies, more efficient keyboard models can be made. 

Discussion

While this paper determined many different great ways to figure out the efficiency of the stroke-based input, I feel like many other touch screen inputs are missing from consideration. While at the moment, none come to mind, I feel like there are many other ways to do touch based input on a touch screen besides tap-based and stroke-based. 

And also, like Dvorak, it must be considered whether these complex text input methods are worth learning how to use. If it's a small gain (of around 12% efficiency) it may not be used learning and adopting. 

That being said, I felt like the study of optimal text input was very interesting and I'd personally enjoy attempting to learn such a system.

Paper Reading #6: TurKit, human computation algorithms on mechanical turk

TurKit, human computation algorithms on mechanical turk


Greg Little, Lydia B. Chilton, Max Goldman, Robert C. Miller

Greg Little is a student and researcher at the CSAIL lab at MIT.
Lydia B. Chilton is a graduate student at the University of Washington who also interned at Microsoft Research.
Max Goldman is a graduate student and researcher at the CSAIL lab at MIT.
Robert C. Miller is an associate professor in the Computer Science department at MIT.

Summary


In this paper, the researchers presented a new interface called "TurKit" that interacted with an existing tool known as MTurk. These tools allow for the consideration of human computation in systems. Human computation is a great way to allow computations that a computer might now be able to do. For example it might be difficult for a programmed system to recognize what's in a photo. However, using TurKit, you can design systems that allow the humans to essentially fill in the blanks.

The TurKit researchers specifically added a new API extension to MTurk, the idea of "crash-and-rerun" programming and an online interface for TurKIt.

TurKit allows for instructions computed in a human computation to be saved for later. This also allows for synchronization across various human computation inputs. In the API extension they added the ability for various to be used across human computation tasks (also know as human intelligence tasks or HITs). The extension also adds some "fake" multithreading abilities.

They researchers seemed to have multiple problems they hoped to address with TurKit. First, they wanted to give developers a better interface to use with MTurk. Second, by cutting down on costly operations and storing data, it's easier to save data for later or to save on computation costs.

A few ways they used to test TurKit were through iterative writing and blurry text recognition. Several outside groups also tested the use of TurKit in various ways such as psychophysics anaylsis.

Discussion


The usage of TurKit could be extremely beneficial in very many areas. For example, data labeling is a huge problem in many different environments. For example, searching for images can be difficult be the user might know what's in the picture but they can't search by tags normally. Adding in a labeling system would be beneficial to organization without a doubt.

Also mentioned in the article is the benefits of iterative writing. This is a system that would make Wikipedia much more accessible. Currently, editing Wikipedia is an extremely daunting task but if TurKit was used, articles could write on articles that need more information or fill in needed information for another article.

I think the researchers definitely fixed the problems they discussed and they presented a viable product that definitely has potential for many future products.