s
Projects
LG Advanced Learning

My part in this project was inspired by a piece "pinch" which i had been emailing around at the time. I worked with Jake Seargeant, 3D artists, and director Carl Erik Rinsch at Digital Domain to create the light effects for this commercial about little robots who have a party while the owner is away. I was able to write several OpenGL applications in C++ which allowed Jake to tweak parameters and render the frames. Giving Jake this amount of control made my life more convenient and gave us more creative options.

pinch

Pinch is a hypnotic and inspiring aesthetics exploration toy that allows the player to easily generate many kinds of abstract shapes. A virtual sheet is made from free-floating physics masses. Click in different places to apply forces making the masses move towards that area of the screen, hence "pinching" the sheet. The resulting forms look organic, architectural, and sometimes erie. Pinch is highly customizable and exports to OBJ format to be imported by 3D software. Change the texture by replacing texture.png. On the mac, this is inside the app package contents under Contents/Resources/media. In the same folder, change the color pngs to customize the interface.

screenshot of textured and filling the screen

screenshot of textured and filling the screen

created and rendered in Maya with ambient occlusion by Gabe Dunne

created and rendered in Maya with ambient occlusion by Gabe Dunne

screenshot of solid fill with graph enabled

screenshot of solid fill with graph enabled

created and rendered by Dunne

created and rendered by Dunne

screenshot of green texture with graph and info enabled

screenshot of green texture with graph and info enabled

screenshot of OBJ imported into Blender by Ryan Alexander

screenshot of OBJ imported into Blender by Ryan Alexander

created and rendered by Dunne. Can you find the swan?

created and rendered by Dunne. Can you find the swan?

screenshot of architecture with pedals imported by Ryan into Blender

screenshot of architecture with pedals imported by Ryan into Blender

screenshot of twisted texture

screenshot of twisted texture

Creative Commons License
Pinch by Joshua Nimoy is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.

Gatorade - Inside Crosby

For this rich scene of what's going on inside an athlete's head, Mark Kudsi had me writing C++ code to generate slowly growing neurons into a stretched screen 'topiary'. An adhoc rendering cluster was used to quickly render random seeded versions as we evolved the style into the fantasy vision displays.

jttoolkit to OpenFrameworks

Jttoolkit is a C++ system for Processing artists. Actually, OpenFrameworks is probably going to do a much better job at this than me, so let's all get on that boat now. And for Gabe, it was really more about the cleanly managed dependencies. Cygwin and MacPorts have been absolutely a headache for us, and for students. So I wrote this tutorial guiding you how to migrate your jttoolkit apps into openFrameworks. Be free! Be free!

HP Paulo Coelho

For Paulo, finger trails of personal photos dissipate like horse hair underwater. My C++ JImage object is born - something that allows pixel addressing that can also update its display list cached textured unit rect. I later based a hair style on this aesthetic. It was a pleasure to collaborate on this spot because everything was so warm and natural.

screenshot of mouse toy stretching a woman's back

screenshot of mouse toy stretching a woman's back

Modest Mouse Dashboard

This music video turned out to be a unique family effort, rewarding in the end. A couple of the island shots feature the pop celeb's face rendered by our custom after effects plugin, Pixel F. Meanwhile indoors, Gabriel takes a simple springy spider web I had set up and activates it artistically using pointer bugs and interpolation error. The JChain addiction is born. Special l33t props go out to Gabe as well for his MEL fish fin wave behavior!

Bud Select - Just a Game

For this superbowl spot, I had another amazing collaborative coding experience with Gabe Dunne. The approach was different in high level regards. Our generative renders were being used as concept for other artists to flesh out into several shots and variations. We were also coasting on a sufficient body of pre-written code that pulling up old building blocks and combining them experimentally was at our fingertips. Because of path smoothing politics residing from previous projects, we start the JPath object, featuring smooth(int); This class will change the future. At least one project into the future.

wind graph

wind graph

shuffling cards interface

shuffling cards interface

voice volume indicators

voice volume indicators

GRL Graffiti Brush

I shared a hotel with knights of Graffiti Research Lab (GRL) et. al. in Barcelona whilst showing at offf 2007. At one point, I coded this brush that turns a few mouse drags into a randomly generated bubbly blingy style. The brush code was then massaged into L.A.S.E.R. Tag. Here is the stand-alone version for your enjoyment and algorithmic plagiarism. I should quickly note for interested jttoolkit participants that this begins my attempted merging/joining/cooperating between OpenFrameworks and jttoolkit in hopes that they could eventually be one in the same ACU strain. Thanks for hot Spanish summer nights, Bennett, Evan, James, Theo, and Zach! An intersection of bodies and code had been long overdue. I certainly got offf.

Key Controls:

MOUSE DRAG - draws one line.
ESC - quit.
SPACE - clears the drawing area and generates a new brush.
M - show/hide the mouse
L - easter egg.
F - toggle fullscreen
C - choose 3 colors (mac: get out of fullscreen mode first)

Purple green red purple - how many ohms is that?

Purple green red purple - how many ohms is that?

Purple green red purple - how many ohms is that?

Purple green red purple - how many ohms is that?

Nike 'One'

Motion Theory's workflow of the Nike 'One' commercials did not just benefit from an applet programmed by one of the team members. This time, the core workflow became a team of four visual programmers risking carpel tunnel to generate a diversity of floating engineering graphics using every trick in the book. Although the production process was organic and artistic, the team collaborated well - subclassing a common object oriented super class, using common (custom) rendering frameworks, establishing file format protocols for shared data, and using versioning systems. The result was complex yet delicate and tasteful swarm of diagrams and math floating around the heads of thinkers - interacting with the physical, emotional, and narrative surroundings. A demo applet was also published in Processing.org exhibitions. Processing Artists:

HP Pharrell

had the great honor of doing (more) algorithmic particle artistry with Motion Theory to produce this stunning new HP commercial, for their "The computer is personal again" campaign. Spinning spools of typographic smoke, shaking the pixels off shoes, and of course, a swarm of gratuitous abstract cool stuff -- these composite effects were animated primarily in Processing code with reinforcement C++ coding when needed. Thanks to Gabriel Dunne for doing some of the satellites. Unlike Nike 'One', this was quite a chromatic job. I have since gained a personal relationship with the colors magenta (FF00FF) and lime (00FF00). Special thanks to Mark Kudsi and Mathew Cullen, whose talents were absolutely essential to this wonderful project. This is what baby looks like.

EAZ AR/MR Workshop

Watch on Vimeo.

Report: FACT HCI Augmented Reality Workshop, Success.

In early November 2005, a forty minute educational workshop experience for 300 pupils age 10-11 was given for the Liverpool FOCUS EAZ (Education Action Zone) for schools in the Fazakerley and Walton areas. The technology introduced was addressed at various times as "Augmented Reality," "Camera Vision," "Gestural Interface," and "Illuminating Lamp." Pupils were invited to step into a silhouette-driven environment and interact with six demonstration programs in a game play mode. Additionally, a five minute slideshow was given by the team (Karen Hickling, Josh Nimoy, Marta Ruperez) giving brief backgrounds on FACT (Foundation for Art and Creative Technology) and HCI (Human Computer Interaction group at John Moores University). In preparation for the workshop, Josh had authored (or massaged previously authored) software into an automatic shuffle of six abstract games. Some of the games were more multiplayer than others. Those which were not multiplayer presented limited resource for interactive satisfaction - making great exercises in collaboration, teamwork, and turn-taking for the pupils. The workshop was given a total of ten times. This essay reports on things we learned about the workshop and the HCI as we refined the workshop's structure.

In the planning stages of the workshop, Josh was initially unsure that the camera interface would be appropriate for the age group but proposed it to a few people at FACT. Artist, Carlos "Caen" Botto came back with a strongly positive response, saying, "This kind of activity is very appropriate. The most important needs at this age is that the system have a very evident reactivity (think that at this age the abstract thinking is in early formation), letting the children experience a sensation of control in a very direct way. The other important need is for high physical activity. I think this proposal is right in both aspects. The group size is not a problem. It is important not only to interact, but also to see others interacting with the system. A big group can be broken into three shifts, When one group is interacting, the other groups can observe. On the pedagogic side, I think the educative aspects are centred in psychomotricity, creativity development, cooperation and intuitive problem solving, et cetera." Just before software development, a local toy store was visited in curiosity about which products were targeted for this age range - in hopes to keep the experience from being boring, embarrassing, or too complex for the students. Similar conclusions were reached as Caen's mentions of creative development, psychomotricity, and high physical activity. These children were going to be more energetic than us!

The order of the workshop was originally planned to begin with a 35 minutes of play, with a five minute presentation at the end. The rational for this was so the students would be engaged first, generating questions in their heads. In retrospective analysis, there had been too much preparation to entertain than was actually needed. We soon realised in implementation that workshops lasted for unpredictable durations, sometimes cutting the presentation completely off from the experience. Since the presentation was a relatively important component, it was moved ahead to precede the play session. This change turned out to be beneficial as well. Ideas were put into the heads of the students pre-play, so the play sessions could be more than just play. It was informed play. This was not the only response to chaos. Besides the duration of workshops varying, it was also hard to position ten children in the projection -- it would completely block the projection and although the students were having fun, we wanted them to experience the software in a way that would allow them to comprehend the interactivities. We began to break the groups into three or four, and segmenting each software program into 1-2 minute turns, calling out each group and keeping time. Before we bothered to do this, the turn taking emerged naturally from the group behaviours. It was just faster to impose this early on for punctuality.

We gave very little instruction, and just let the students do what they did. Each group's behaviour evolved in similar ways. As the students discovered the systems, one student would back up so far that the projection was blocked completely by his (it was usually male) body, preventing the rest of the children from interacting with the systems. In the more self-governed clusters, this would result in that person being yelled at by the spectators to "move forward." On the second day, a clear barrier was drawn so that no one could back up too far. This architectural restriction was somewhat of a solution for the "block all" personality type - although this impulse was still observed in students within the remaining space. Students would also pound on the wall where the image was projected, treating the virtual objects as if they were buttons, or as if there were sensors in the walls. The only reason we introduced a rule against the wall pounding was in respect to the workshop happening just in the next room. Other than disturbance, there didn't seem to be any problem with pounding the wall. While on the subject of violence, it was interesting to note the playful violence between boys. Often, when in the shadow space, boys would pretend-fight, as in karate or boxing. This was probably not just due to the high energy levels of the activity, but the expectations that come along with any game-play mentality. The boys felt as though they were "inside the videogame" and were acting accordingly - creating the same scenes they had seen in Street Fighter and Mortal Kombat. Violence was one kind of cheap laugh among the several categories of cheap laughs discovered by the students while in play. On the first day, there was no teacher accompanying the groups. This caused them to be less obedient. However, even if on the second day, there was at least one teacher governing the students, the "play violence" persisted with unusual strength.

A special learning group came in - a smaller group with a wide range of conditions. In this case, the most immediate software worked (the sparkling star trails). The initial concept presentation was cut very short and more time was given to play. This group reacted more or less the same as the other classes of children. It is entertaining to note the democratising power of Augmented Reality: everyone seems to act like a ten-year-old child when they try it out. By the time we got to the last group, students already knew what to expect. Rumours were being passed between friends during breaks and lunchtime. The technology turns out to be not as new and shocking as predicted. Each group was asked to raise hands if someone had an "EyeToy" at home. Three to ten children would always raise their hands, completely aware of what was about to come. On the other hand, slides showing Tom Cruise in Minority Report doing gestures in front of his pre-crime computer-cave went virtually unrecognised. These children were too young to be allowed into the cinema to view this movie with that rating, despite its being responsible for disseminating the idea of AR so widely. The popular question asked was "how does this work?" The next most popular question was "Is this an EyeToy?" Josh's favourite question came from a special learning student: "Are you from America?"

Previously, an emerged personality type was mentioned - the boy who blocked the entire screen. Other such personality types reoccurred during the workshops. The "geek" type would get bored of the demonstration and sit in the back of the room with Josh and the computers, talking about more advanced topics like programming (example question: "Did you do code this in HTML using Notepad?"). The non-participant would sit in the chair, or on the floor and refuse to get up and join the play, even after being prodded by Karen or the teacher. Conversely, and more frequent was the over-active participant -- a child who did not have enough patience to wait for his/her group's turn, would have trouble leaving the game space when the turn was over, and would be found sticking arms and legs into the projector from the perimeters during other groups' turns. Somewhat related to this over-active participant was the shouting director, trying to verbally control who ever was in the ring - telling them to try different positions or interactions. For the most part, children listened to the directors. In the end, we fully realise that it really did not matter what software was running. The most entertaining part of the experience was students being allowed (for once) to let loose in a projection-obstructing frenzy.

Teachers sitting and watching the experience seemed overwhelmed. While only a few of them stepped in to try the interface, everyone had something positive to say. "It crosses the whole curriculum" said one teacher, in reference to the joining of arts and sciences that is the augmented reality field. "You have got a mixed group of people co-operating and working together. Boys mixing with girls and children who do not know each other" said another teacher, amazed at how the experience seemed to break down barriers between different kinds of students.

slow fade grid

slow fade grid

screen capture of a pen-trail

screen capture of a pen-trail

stars

stars

stars

stars

stars

stars

stars

stars

squishy satsumas

squishy satsumas

slow fade grid

slow fade grid

stars

stars

Josh coding on laptop

Josh coding on laptop

exploding watermelons

exploding watermelons

scribble pen

scribble pen

more scribble pen

more scribble pen

scribble pen again

scribble pen again

multi scribble

multi scribble

fade grid

fade grid

squashy satsumas

squashy satsumas

exploding melons

exploding melons