s
Projects
Thanks to Marcin Ignac for pointing out that the below project grid is not working properly on OS X Safari. The long horizontal line of items is not intended ;). Please use another browser to view this until i get it fixed in the next 1-2 days. Happy Holidays!
EAZ AR/MR Workshop

Watch on Vimeo.

Report: FACT HCI Augmented Reality Workshop, Success.

In early November 2005, a forty minute educational workshop experience for 300 pupils age 10-11 was given for the Liverpool FOCUS EAZ (Education Action Zone) for schools in the Fazakerley and Walton areas. The technology introduced was addressed at various times as "Augmented Reality," "Camera Vision," "Gestural Interface," and "Illuminating Lamp." Pupils were invited to step into a silhouette-driven environment and interact with six demonstration programs in a game play mode. Additionally, a five minute slideshow was given by the team (Karen Hickling, Josh Nimoy, Marta Ruperez) giving brief backgrounds on FACT (Foundation for Art and Creative Technology) and HCI (Human Computer Interaction group at John Moores University). In preparation for the workshop, Josh had authored (or massaged previously authored) software into an automatic shuffle of six abstract games. Some of the games were more multiplayer than others. Those which were not multiplayer presented limited resource for interactive satisfaction - making great exercises in collaboration, teamwork, and turn-taking for the pupils. The workshop was given a total of ten times. This essay reports on things we learned about the workshop and the HCI as we refined the workshop's structure.

In the planning stages of the workshop, Josh was initially unsure that the camera interface would be appropriate for the age group but proposed it to a few people at FACT. Artist, Carlos "Caen" Botto came back with a strongly positive response, saying, "This kind of activity is very appropriate. The most important needs at this age is that the system have a very evident reactivity (think that at this age the abstract thinking is in early formation), letting the children experience a sensation of control in a very direct way. The other important need is for high physical activity. I think this proposal is right in both aspects. The group size is not a problem. It is important not only to interact, but also to see others interacting with the system. A big group can be broken into three shifts, When one group is interacting, the other groups can observe. On the pedagogic side, I think the educative aspects are centred in psychomotricity, creativity development, cooperation and intuitive problem solving, et cetera." Just before software development, a local toy store was visited in curiosity about which products were targeted for this age range - in hopes to keep the experience from being boring, embarrassing, or too complex for the students. Similar conclusions were reached as Caen's mentions of creative development, psychomotricity, and high physical activity. These children were going to be more energetic than us!

The order of the workshop was originally planned to begin with a 35 minutes of play, with a five minute presentation at the end. The rational for this was so the students would be engaged first, generating questions in their heads. In retrospective analysis, there had been too much preparation to entertain than was actually needed. We soon realised in implementation that workshops lasted for unpredictable durations, sometimes cutting the presentation completely off from the experience. Since the presentation was a relatively important component, it was moved ahead to precede the play session. This change turned out to be beneficial as well. Ideas were put into the heads of the students pre-play, so the play sessions could be more than just play. It was informed play. This was not the only response to chaos. Besides the duration of workshops varying, it was also hard to position ten children in the projection -- it would completely block the projection and although the students were having fun, we wanted them to experience the software in a way that would allow them to comprehend the interactivities. We began to break the groups into three or four, and segmenting each software program into 1-2 minute turns, calling out each group and keeping time. Before we bothered to do this, the turn taking emerged naturally from the group behaviours. It was just faster to impose this early on for punctuality.

We gave very little instruction, and just let the students do what they did. Each group's behaviour evolved in similar ways. As the students discovered the systems, one student would back up so far that the projection was blocked completely by his (it was usually male) body, preventing the rest of the children from interacting with the systems. In the more self-governed clusters, this would result in that person being yelled at by the spectators to "move forward." On the second day, a clear barrier was drawn so that no one could back up too far. This architectural restriction was somewhat of a solution for the "block all" personality type - although this impulse was still observed in students within the remaining space. Students would also pound on the wall where the image was projected, treating the virtual objects as if they were buttons, or as if there were sensors in the walls. The only reason we introduced a rule against the wall pounding was in respect to the workshop happening just in the next room. Other than disturbance, there didn't seem to be any problem with pounding the wall. While on the subject of violence, it was interesting to note the playful violence between boys. Often, when in the shadow space, boys would pretend-fight, as in karate or boxing. This was probably not just due to the high energy levels of the activity, but the expectations that come along with any game-play mentality. The boys felt as though they were "inside the videogame" and were acting accordingly - creating the same scenes they had seen in Street Fighter and Mortal Kombat. Violence was one kind of cheap laugh among the several categories of cheap laughs discovered by the students while in play. On the first day, there was no teacher accompanying the groups. This caused them to be less obedient. However, even if on the second day, there was at least one teacher governing the students, the "play violence" persisted with unusual strength.

A special learning group came in - a smaller group with a wide range of conditions. In this case, the most immediate software worked (the sparkling star trails). The initial concept presentation was cut very short and more time was given to play. This group reacted more or less the same as the other classes of children. It is entertaining to note the democratising power of Augmented Reality: everyone seems to act like a ten-year-old child when they try it out. By the time we got to the last group, students already knew what to expect. Rumours were being passed between friends during breaks and lunchtime. The technology turns out to be not as new and shocking as predicted. Each group was asked to raise hands if someone had an "EyeToy" at home. Three to ten children would always raise their hands, completely aware of what was about to come. On the other hand, slides showing Tom Cruise in Minority Report doing gestures in front of his pre-crime computer-cave went virtually unrecognised. These children were too young to be allowed into the cinema to view this movie with that rating, despite its being responsible for disseminating the idea of AR so widely. The popular question asked was "how does this work?" The next most popular question was "Is this an EyeToy?" Josh's favourite question came from a special learning student: "Are you from America?"

Previously, an emerged personality type was mentioned - the boy who blocked the entire screen. Other such personality types reoccurred during the workshops. The "geek" type would get bored of the demonstration and sit in the back of the room with Josh and the computers, talking about more advanced topics like programming (example question: "Did you do code this in HTML using Notepad?"). The non-participant would sit in the chair, or on the floor and refuse to get up and join the play, even after being prodded by Karen or the teacher. Conversely, and more frequent was the over-active participant -- a child who did not have enough patience to wait for his/her group's turn, would have trouble leaving the game space when the turn was over, and would be found sticking arms and legs into the projector from the perimeters during other groups' turns. Somewhat related to this over-active participant was the shouting director, trying to verbally control who ever was in the ring - telling them to try different positions or interactions. For the most part, children listened to the directors. In the end, we fully realise that it really did not matter what software was running. The most entertaining part of the experience was students being allowed (for once) to let loose in a projection-obstructing frenzy.

Teachers sitting and watching the experience seemed overwhelmed. While only a few of them stepped in to try the interface, everyone had something positive to say. "It crosses the whole curriculum" said one teacher, in reference to the joining of arts and sciences that is the augmented reality field. "You have got a mixed group of people co-operating and working together. Boys mixing with girls and children who do not know each other" said another teacher, amazed at how the experience seemed to break down barriers between different kinds of students.

slow fade grid

slow fade grid

screen capture of a pen-trail

screen capture of a pen-trail

stars

stars

stars

stars

stars

stars

stars

stars

squishy satsumas

squishy satsumas

slow fade grid

slow fade grid

stars

stars

Josh coding on laptop

Josh coding on laptop

exploding watermelons

exploding watermelons

scribble pen

scribble pen

more scribble pen

more scribble pen

scribble pen again

scribble pen again

multi scribble

multi scribble

fade grid

fade grid

squashy satsumas

squashy satsumas

exploding melons

exploding melons

BallDroppings

Balldroppings was one night of idle programming that blew up unexpectedly into a web phenomenon. I learned that simplicity is elegant, and C++ is wonderful for low-latency sound+image. I also learned about addiction and glucose metabolism rate highs. Although I do not accredit myself for having originated the idea of interactive lines with bouncing balls, there exists a small following in the online gaming community that gives me such credit, particularly when accusing one another of having copied me in their recent developments. BallDroppings has also been re-implemented in other languages by random people, referencing the name "BallDroppings." All this activity is very surprising to me. It is also a clear example of the great power resulting from refraining to mark intellectual property. A lot of people mistook BallDroppings to be my graduate thesis. I don't try to correct this misunderstanding.

Re: Shadow

Watch on Vimeo.

Passers by encounter a display made from finely combed sand, reflecting expressive "shadows" of their bodies. This artificial shadow system intends to ask questions of designer intervention with nature, and aesthetically explores commonly seen polygonal body form outlines in a real-time context.

professor: Christian Moeller

Technological Diagram of Installation

Technological Diagram of Installation

Interactive Buckyballs

The Zerowave Buckyball projection was an interactive environment that allowed users to interact with squishy balls by using their shadow gestures. This piece was commissioned by Victoria Vesna and James Gimzewski. The Zerowave Buckyball Projection is legally represented by and is property of UCLA. For more documentation of this project (video, images, more interactive toys by Josh Nimoy), visit the Zerowave website (2002). The buckyball projection inevitably became the central art piece in Vesna and Gimzewski's art exhibit long after I had moved to New York. The year-long exhibit was entitled "Nano", commissioned by the Los Angeles County Museum of Art's "LACMA Lab". Screenshots of my software art ended up hanging on banners and billboards across the Los Angeles area. The Los Angeles Times Newspaper responded in their review of the exhibit by asking "Where is the artist?"

Commissioned by Victoria Vesna and James Gimzewski. The Zerowave Buckyball Projection is legally represented by and is property of UCLA.

Josh Nimoy is pushing a buckyball inward by holding his hands in the air and moving them forward.

Josh Nimoy is pushing a buckyball inward by holding his hands in the air and moving them forward.