CLOUDS Interactive Documentary

James George and Jonathan Minard co-directed an interactive documentary on the people in my scene. I was happy to be one of the interviewees and be one of the commissioned code artists. Here was the beta teaser on kickstarter:

I was responsible for implementing two of the open source visual systems: BallDroppings and Neurons. Being a part of this project was a very high honor for me and I only regret not being more available to involve myself during its production.

More info about CLOUDS on Creative Applications

and a good interview with them.

I'm posting this long overdue portfolio entry in celebration of CLOUD's screening at MoMa. In the tradition of these portfolio entries which share a small bit of the process, and indulging in the open-ness of the project, I'll describe my experience contributing.

We met up at a medium sized hackerspace in the Tenderloin district of San Francisco which was associated with the oooShiny tribe.  James, John, and a few others had already set up shop there when I arrived from Silicon Valley. A thin layer of ephemeral user network cables blanketed over the tables and floors, crawling at times like vines up the walls. USB, power, video, hot swapping in realtime to the crowd's murmur. It was a lot of macbooks, tripods, and hoodies. Exploratorium toys strewn about the space as they endearingly are year round. A dusty, cracked Processing textbook lay wedged between a copy of Curtis Roads' Microsound and a half-consumed makerbot filament case. I brought my cat Hexadecimal and let him poke around in all the boxes of miscellanea. People were walking around taking care of fifty different things. The space was teeming with life. The faint smell of smoke and curry wafting in through the victorian windows.

I joined in at the table with matching laptop and hoodie. James got me up to speed with getting all the git submodules. The platform was already organized and ready to go, all I had to do was subclass someone's handy superclass and fill it with my artwork. It wasn't just that ofxUI was already installed – Reza was in the house. The project was very deeply OF. I was also pleased to meet Patricio Gonzalez Vivo for the first time. Gmunk paid us a visit. I believe Grey Area Foundation was resident down the hall in the same space at the time. I think the whole project is a miracle.

Teaching Openframeworks

Video guiding the Processing coder through the mental transformation needed to start using OpenFrameworks to create equivalent work. This class was taught with Syed Reza Ali. We will post more episodes every time we teach this GAFFTA class. So far, I have shaved my face with a straight razor, did a claymation about pointers, and took apart a McDonalds cheeseburger. I hope it makes learning programming and its dry concepts a bit wetter. Special thanks to oooShiny for advising on the materials and teaching approach.


Watch on Vimeo.

This synaesthetic, interactive musical experience provides six original modes in which the player may produce music. Kick off your shoes and get lost in a world of delicious sounding abstract geometry.

Turn up your sound volume.
Click and drag to explore.
Press bubbles to change scenes.
Press Escape to quit.

Android and iOS versions coming soon.

Advanced configuration is possible by editing the contents of the bundled settings.yml. By changing the values of that file, you can control the window properties, startup behavior, user interface detail, and OSC network parameters. When set up on multiple machines, Bandwidth's 'grid' mode will broadcast OSC messages and most other modes will recieve.

Made possible by The Public. UK

Special Thanks:

The OpenFrameworks project,
Graham Peet
Joss Widdowson
Rebecca Shostak
the oooShiny group
Sandwell Arts Trust
Fran McHugh

Creative Commons License
Bandwidth by Josh Nimoy is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.


Marc Nimoy Extraordinaire was redoing the website for the UCLA Center for Intercultural Performance (CIP) and he had embedded a small animated gif that had a marquee of text announces and fun blur transitioning. I offered to redo it in Flash 8, and give him a lot of customization features on the html side so the thing could be dynamically driven and lower bandwidth. After pulling a happy all-nighter at Beit T'Shuvah, TextFlasher was born. Thank goodness for cool flash filters. Although I know and love as3, I chose flash 8 because at the time I write this, that's installed by 98.9% of internet users. If you look in the example files, you'll see a large associative array of values and heavy commenting.

GRL Graffiti Brush

I shared a hotel with knights of Graffiti Research Lab (GRL) et. al. in Barcelona whilst showing at offf 2007. At one point, I coded this brush that turns a few mouse drags into a randomly generated bubbly blingy style. The brush code was then massaged into L.A.S.E.R. Tag. Here is the stand-alone version for your enjoyment and algorithmic plagiarism. I should quickly note for interested jttoolkit participants that this begins my attempted merging/joining/cooperating between OpenFrameworks and jttoolkit in hopes that they could eventually be one in the same ACU strain. Thanks for hot Spanish summer nights, Bennett, Evan, James, Theo, and Zach! An intersection of bodies and code had been long overdue. I certainly got offf.

Key Controls:

MOUSE DRAG - draws one line.
ESC - quit.
SPACE - clears the drawing area and generates a new brush.
M - show/hide the mouse
L - easter egg.
F - toggle fullscreen
C - choose 3 colors (mac: get out of fullscreen mode first)

Purple green red purple - how many ohms is that?

Purple green red purple - how many ohms is that?

Purple green red purple - how many ohms is that?

Purple green red purple - how many ohms is that?

Disappeared In America Interactive Timeline

Interactive Flash-based visualization of news items pulled from a mysql database for an art-activism group. I developed this in preparation for a show at FACT, Liverpool having just made friends with project leader Naeem Mohaiemen. The project was supposed to be open source, but I can't figure out where I left the source.


I redid the website for the human computer interaction organisation in Liverpool England that was commissioning me to do a couple pieces at FACT. I used an open source flavour of flash - mtasc and swfmill with a lot of xml and a makefile. The source was made available on the site. We used the blog engine, b2evolution, to manage the content, and the flash file just pulled the content from the mysql database and presented it this multi-windowed way. Now that I look back on it, Drupal would probably have been more appropriate. The user interface was a bunch of UIComponent windows that popped up inside the browser. Small green indicators and trails were used to point out where the user had clicked and dragged the mouse in an artistic overlay. I subclassed the window component to add a resize control at the bottom right corner.


This series of seven interactive musical compositions is intended for serious play and wonderment about tools. Each piece is an abstract visual sound scoring language capable of playing itself back as the user gives input. The series is several attempted embodiments of my conceptual response to a recent HCI conclusion, as written in CWC 2004:

"Icon interpretation is inherently ambiguous because the relationship between icon and function is not determined by a set of well-defined syntactic and phonological rules."

Icon==Function is a science fiction, showing a hypothetical dystopia / utopia. What would an interface be like if its iconography were determined by a well-defined ruleset?

The software art was conceptualised by JT in a series of long-winded email discussions with David England and curator Michael Connor. The sketches were then interpreted by Josh, Matthew Phillips, and David Lu in a number of programming languages. The code is hosted by Sourceforge.

hexball coded by jt using lingo

interactive piece

gears coded by jt using p5

interactive piece

nesting circles, coded by jt using p5

interactive piece

triangle music coded by jt using p5

interactive piece

bouncing boxes coded by jt using p5

interactive piece

bouncing boxes coded by Matthew Phillips using as2

interactive piece

colored squares coded by David Lu (aka Forkinsocket) using as2

interactive piece

Each piece in the series is a simplified programming language used to instruct the computer's functional output. A whimsical code editor is exposed to the user as a visual toy that interacts with a mouse and keyboard. No other labeling or indirect tagging is used to mark the mini-programs written with each mini-language. The languages are made of simple geometries - capable of representing themselves as their own icon. The function's code can be seen as its own graphical icon. The well-defined syntax of iconography in each representation is precisely the instruction language controlling the sound output. When a user browses a small set of thumbnails looking for a state to load and play, the user is not seeing a textual name given to the work, nor a proxy graphic that attempts to explain the function by some cultural mapping. Instead, the user sees the code, itself. In the name "Icon==Function," a double equality operator is taken from modern text-based programming to communication that the icon and the function are the same as one another. If a single equality sign were used, it would signify undesirably that the function in these pieces are somehow being assigned to the icon, and that the icon is a transparent address for the function. The art series aims to collapse the two ideas into one just for the sake of both conceptual argument and exploratory scientific research.

Just Language Enough

In computer programming, the word "function" is used to describe a collection of instructions that perform a coherent task. A function hides the code inside of it from attention, and allows its code to be invoked only by calling the function's name. In creating the seven visual languages in this series, my biggest aim was to produce the beginnings of language-ness from my interactive geometries. The aim of adding the modern features of 'real' programming code (such as memory and logical flow of control) was somewhat irrelevant. In most cases, a simple linear list of instructions sufficed in order to make the artistic point, and to provide the software user with an experience that was close enough to programming that they would gain some sense of authorship with their constructions. A stronger focus was also placed on making the experience playful in a game-like way, rather than providing the user with an industrially competent solution.

Usability of the Pie are unfamiliar components that interface designers hope to channel their users to embrace through play or necessity. This is the part of the experience that I am calling the exploration.

In the pieces of Icon==Function, the interfaces I am recycling come from CAD and desktop graphics software applications like a paint program or paste-up application. The colored square game is a slight variation of a pixel editor. The triangle music interface is a modified vector-line drawing tool. "Gear Sound" and "Nested Circles" both rely on circle-drawing tools. Although HexBall heavily resembles a video game, the most basic interface is a pixel editor -- and so on. That is the intuition aspect. The exploration aspect is what happens to those graphics creation interfaces that result in something more than simply the circle that was drawn, or the pixel that was toggled. No one originally expects music to come from their rectangle dragging tool, nor would they expect the element they drew to bounce around on the screen and obey gravity after they drew it. However, this expectation should also become intuitiho are already familiar with painting and graphics applications. Even if draggable rectangles exist as part of widespread commercial GUI OSes as a 2D spatial "multi-select" for file icons, it is questionable what percentage of the user population considers that an integral part of their computing lifestyles and would recognize it in a new program, or know to check for it. My mother (age 59, health care manager) has had considerable difficulty getting into each piece as she must be told to actually drag the mouse instead of just clicking it. After being reminded to drag, she spends most of her time struggling with the touch pad on my laptop, and eventually concludes that the piece is visually beautiful, without delving deeply into trying new combinations. It is interesting for me to see her interact with the pieces because she doesn't seem to understand the language, nor is she searching for one. On the other hand, my brother (musician in his early 20s) is better familiar not only with graphics programs, but video games and software development as well. He has witnessed my software interface experiments since the mid-1990s and already knows that I expect him to explore, given little or no instruction. My brother will start by doing "the sensor dance." The sensor dance is what happens when a human is trying all the different entry points into the interaction that he or she can think of, in hopes to get any sort of response out of the unrecognized interface. While an ambiguous physical installation will cause a person to wave their arms in the air, move close and far, shout, clap, and touch, a desktop computer based work (restricted to a mouse and keyboard) evokes a different kind of sensor dance. The person will click around on everything, dragging the mouse in different places and pressing different keys. In my own observations, people have also been known to re-launch the application to see if it does something new or to check to see if it's broken. My brother is quick to engage in a neactive language, creating new instances and enjoying the outcomes as his own achievements. Since he is a musician, he's familiar with the idea of embracing a system and recombining its parts to produce a new expression. One could say that my interfaces are more intuitive for him. He is familiar on more levels than just see-and-remember. I think these outer, more anthropological scopes of understanding usability could make the problem clearer than simply analyzing the language that translates between icon and function, or icon-system and function-system.

Artist and coder: Josh Nimoy, coder: David Lu, coder: Matthew Phillips, Commission from FACT (www.fact.co.uk) and JMU HCI (www.hci-fun.org.uk) curated by Michael Connor & Marta Ruperez. HCI research headed by David England.

kalmac camera processing

A series of 7 camera view treatments that I probably did at ITP. When I went to Fabrica, I shared my source code with Joel Gethin Lewis and he quickly understood it. Shortly after, Joel went on to work with United Visual Artists. To interact with this piece, move the mouse to tumble the 3D form being generated from the webcam image.


An open source Java desktop application for authoring PIC chip firmware in a restricted form of Java. I got to know Eclipse pretty well during this time, and had a lot of fun seeing the application look native even if it was cross platform. I discovered how little programming one needs to do with all the wonderful libraries that are available.

Exquisite Corpse Paint

Creative Exquisite-Corpse Viewer, using content generated by a hundred ITP students in collaboration with my Applications group during the preceding week. Solution: a cyclical self-drawing, line-recycling, and motion-mimicking strip of communal art.

All of NYU ITP graduating class of 2004

historical plugin media http://cdn.jtn.im/itp/icm6/icm6_small.dcr

  • Page 1 of 2
  • Page 1 of 2