s
Projects
Alchemy Hack

"Forced collaboration" w. Andrey "Zed" Zaikin: Cheat sheet produced by reverse engineering an Android game found on Google Play. App is disassembled and analyzed into a visualization of all game possibilities.

Permission from game author not obtained.

http://www.arthackday.net/events/lethal-software

This was a piece begun in 2011 and finished then published at Art Hack Day in San Francisco, GAFFTA on December 13-15, 2012. Alchemy was my favorite game on Android. It asks the user to combine basic elements to produce more complex ones, beginning with earth, wind, fire, and water. Eventually, you can produce celebrities, individuals, nations, and abstract concepts. I saw cheat sheets online, but noticed none of them were exhaustive, and were probably being collected from individual trial and error. Knowing there would probably be a small percentage of connections not yet discovered, I set out to decompile the app and see what people had missed. Note, this version of the alchemy game is several versions of the software into the past. It's not a visualization of the one you can get from the Google Play store. This project is also not an attempt to steal intellectual property from its rightful owner. If you enjoy this game as much as I do, I encourage you to purchase a copy, and play it normally ;)

For Art Hack Day, I produced a moving sculpture that highlights a node and moves the network around it to give the audience glimpses of a single idea, and its neighbors.

Initially, the android APK was decompiled with android-apktool, and the Java classes were disassembled into SMALI code. Here is a screenshot of my python script, using complicated regular expressions to fish the game content from the code.

Download the dot file to visualize the associations yourself, but I did auto-generate some graphs which I thought were fascinating.

Here is a close up of the above image.

Here is another visualization method of the same dot file.

SVG script of the above image

Here is the python script that picks out the game rules.

The final cheat sheet is many pages long, and shows the great sense of humor that the author has. I'm thinking that perhaps, he did not intend anyone besides himself to make such connections. I feel like I'm seeing some philosophical secrets not meant for my eyes.

Download the PDF of the cheat sheet.

Sometimes, this is how I like to play video games.

Sky Paintings

I produced this series of code-paintings during my early recovery. They are about sublime serendipity, and the universality of biological syntax. Each render takes about as long as a human would need to actually paint the same thing. Many fractal scenes were generated, but only a few were hand-picked for final presentation. These 2400 DPI prints, coded in Java/Processing, implement an iterative function. The slowly shifting transforms result in paint-strokes. On exhibition and for sale in the "Art by Code" show, at the Public Works, Roll Up Gallery at 161 Erie St., San Francisco. Additional prints of various sizes are also available directly from the artist.

Sky #6 installed at Roll Up Gallery next to beautiful shader works by Inigo Quilez

Sky #6 installed at Roll Up Gallery next to beautiful shader works by Inigo Quilez

Sky #6

Sky #6

Sky #7

Sky #7

Sky #9

Sky #9

Sky #10

Sky #10

Sky #13

Sky #13

Critical Path

I worked with an "indie" hollywood studio in collaboration with Jody Zellen in designing and building a web-based UI to present their interview videos of the top industry game designers on HTML5 canvas and video elements. I used ProcessingJS. The interface featured a line-network of tags shared between the videos. Each video was a sprite that had its own transformation matrix. Triangle collision was also employed in order to do mouse picking with non-rectangular 2D polygons. The javascript code was extremely object oriented, using static "class members" in addition to instance scoped members. The company, Artifact Studios, ran way over their revision allowance, and continued to expect work out of me long after the contract's duration had ended. And that was the last time I worked in Hollywood.

Bandwidth

Watch on Vimeo.

This synaesthetic, interactive musical experience provides six original modes in which the player may produce music. Kick off your shoes and get lost in a world of delicious sounding abstract geometry.

INSTRUCTIONS
Turn up your sound volume.
Click and drag to explore.
Press bubbles to change scenes.
Press Escape to quit.

Android and iOS versions coming soon.

Advanced configuration is possible by editing the contents of the bundled settings.yml. By changing the values of that file, you can control the window properties, startup behavior, user interface detail, and OSC network parameters. When set up on multiple machines, Bandwidth's 'grid' mode will broadcast OSC messages and most other modes will recieve.

Made possible by The Public. UK

Special Thanks:

The OpenFrameworks project,
Graham Peet
Joss Widdowson
Rebecca Shostak
the oooShiny group
Sandwell Arts Trust
Fran McHugh


Creative Commons License
Bandwidth by Josh Nimoy is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

Circumnavigating the RealAudio

My parents are good jews. My mother and father both read torah and haftorah on a regular basis during Shabbat services at their conservative synagogue. That's why they love visiting their favorite site "Navigating the Bible" at bible.ort.org to get the weekly torah portion already color coded and even pre-recorded by a cantor. And while they make the material available online for free, the audio files are in an outdated Real Audio format. At the time I write this, the real media plugin only has 23% penetration on the web, and is a proprietary format that is not backward compatible. My parents were not able to figure out how to play the audio files. So after giving a try myself, I saw that the only thing that would play those .ra files on their mac was VLC and ffmpeg. First I tried writing my parents a 7-step email explaining how to use VLC to transcode the .ra files into .mp3 files. Then at some point, I decided to sit down and write them a [password protected] proxy through which they could visit bible.ort.org. It would allow them to click on the audio links and have them appear as mp3s in their browser. I did this in PHP by using the simplehtmldom library to parse the tags and by installing ffmpeg on the server. I found the html dom parser to be a lot of fun, and I was really happy that installing ffmpeg on my server was as easy as apt-get. This only worked because that particular site used basic HTML. It inspires me about the possibility of using a web proxy as a kind of commentary. This idea parallels biblical commentaries, like the commentaries of Rashi which even have their own rashi-script. Maybe if people like the idea of solving the real audio epidemic this way, we should set up a dedicated media cleaner proxy site so people can visit any url they want and have the media magically converted for them this way (and call the service Rashi).

Imagined Overtures Album Cover

The album art for the Los Angeles Electric 8's new release took a lot of love from the community. The circular element is a visualization of the entire album's contents. Treated as one track, the sound data was analyzed using a fast fourier transform in OpenFrameworks and stored as white space separated ASCII floats. This spectrum data was then plotted along a spiral, with color changes indicating the different pieces. The spiral was then repeated as part of the collaborative silk screening process. The typographic layouts were designed by Rebecca Shostak, using the typeface "Arual" by Curtis Mack. We used 5 Print Gocco screens in 2 days. The printing was done at Wild Magnolia Design in Culver City, then assembled at Pony House in San Pedro. I love the convenience of the gocco, but I'm interested in somehow replacing those yellow bulbs with something less wasteful, like possibly augmenting a camera's flash component to produce the same UV wavelength. If anyone out there knows more of the scientific details of the Gocco bulb, I'd be happy to do the circuit bending. It's beautiful to see a generated visualization printed in metallic gold and white ink. I hope more of my art projects in the future can be this collaborative.

OpenFrameworks screenshot

OpenFrameworks screenshot

metallic gold, brick, and a bit of mossy green

metallic gold, brick, and a bit of mossy green

albums signed and numbered by artist

albums signed and numbered by artist

re-inking allowed much chromatic variation

re-inking allowed much chromatic variation

Becca, Josh, and Phillip

Becca, Josh, and Phillip

Josh and Marc

Josh and Marc

Josh and Marc

Josh and Marc

Assembled by Marc and Becky Nimoy

Assembled by Marc and Becky Nimoy

Felix Salazar

Felix Salazar

AutoTrader

Another project with Gmunk, I did an AutoTrader spot at Black Swan that featured a generative neuron network. I got to write a voxel volume exporter for closed geometry (using even-odd ray casting), and more importantly, I started exporting my art to OBJ and FBX so that it could be rendered in a 3d app. It was amazing to see my work with glass refraction. I also got to implement mouse-selection for the nucleii, and discovered a novel way to generate a circle with even segments along a straight axis using no trig functions (more on that later). Black Swan was an amazing new company to work at because it was a dream-team made from all the beloved members of our previous projects. I hope they do really really well. Proper respect to Matt Winkel, Nick Losq, Jake Sargeant, Chris Clyne, and Jacob Glaser.

ShyB: The Run

This was a triple music video for hip hop artist Shy B. The videos string together to form a short film. I did more traditional CG chores than I'm used to for this project. Normally, I'm hired to take care of the generative elements that need a code artist. This time, I did the 2D/3D rotos, matched shapes in Maya, painted photoshop masks, and did final compositing with the help of Becca Shostak. Shy's handwriting was added to the space, the words moving and reacting to the performer. In one shot, we replaced a license plate. In another shot, we added graffiti to the side of a moving truck.

Verizon FiOS

CHARLEX requested different kinds of wormhole animations for a TV commercial about Verizon FiOS. Video elements would be placed inside the wormhole as it progressed towards the O in FiOS. As I worked remotely, my on-site counterpart was Fabian Tejada who learned to tweak the app and produce renders from it. Fabian was wonderful to work with.

Tron Legacy

I spent a half year writing software art to generate special effects for Tron Legacy, working at Digital Domain with Bradley "GMUNK" Munkowitz, Jake Sargeant, and David "dlew" Lewandowski. This page has taken a long time to be published because I've had to await clearance. A lot of my team's work was done using Adobe software and Cinema 4D. The rest of it got written in C++ using OpenFrameworks and wxWidgets, the way I've always done it with this team ;) Uniquely however, Digital Domain's CG artists were able to port my apps over to Houdini for further evolution and better rendering than OpenGL could ever provide. Special thanks to Andy King for showing me that what seasoned CG artists do at DD is actually not so far off from what's going on in the Processing community.

In addition to visual effects, I was asked to record myself using a unix terminal doing technologically feasible things. I took extra care in babysitting the elements through to final composite to ensure that the content would not be artistically altered beyond that feasibility. I take representing digital culture in film very seriously in lieu of having grown up in a world of very badly researched user interface greeble. I cringed during the part in Hackers (1995) when a screen saver with extruded "equations" is used to signify that the hacker has reached some sort of neural flow or ambiguous destination. I cringed for Swordfish and Jurassic Park as well. I cheered when Trinity in The Matrix used nmap and ssh (and so did you). Then I cringed again when I saw that inevitably, Hollywood had decided that nmap was the thing to use for all its hacker scenes (see Bourne Ultimatum, Die Hard 4, Girl with Dragon Tattoo, The Listening, 13: Game of Death, Battle Royale, Broken Saints, and on and on). In Tron, the hacker was not supposed to be snooping around on a network; he was supposed to kill a process. So we went with posix kill and also had him pipe ps into grep. I also ended up using emacs eshell to make the terminal more l33t. The team was delighted to see my emacs performance -- splitting the editor into nested panes and running different modes. I was tickled that I got emacs into a block buster movie. I actually do use emacs irl, and although I do not subscribe to alt.religion.emacs, I think that's all incredibly relevant to the world of Tron.

HexVirus is a spherical map of the globe that features vector outlines of the continents. These continent vectors are slowly eaten away by a more hexagonal representation. Algorithmically, this is a path stepping function which looks ahead for the closest matching 60-degree turns. The HexVirus globe was used in the executive board meeting scene, and also inside the grid as a visual aid in CLU's maniacal plan presentation. In the board room interface, the globe element is surrounded by the lovely work of my team.

The scoreboard was the first element I worked on. I created a line-generator that produced bursts of lines which turned at adjustable angles. The line generator had "radial mode" which arranged the geometry in concentric circle form. This line generator was used to generate generic elements and layers of style in different things, and is a GMUNK favorite. At this point, I found myself moving to multisampled FBOs because the non-antialiased polygons were just too ugly to work with, and we needed to make film-resolution renders. In fact, this is the highst res I've ever seen my apps render.

Fireworks, mmmm. I started with a regular physics simulation where a particle has an upward force applied at birth, sending it upward while gravity pulls it back down resulting in a parabola. I then added particle-children, followed by various artistic styles, including what our team has called "egyptian" across several jobs -- which is a side-stepping behavior. We were trying to create fireworks that looked enough like real fireworks but had interesting techno-aesthetic. As a homage to the original Tron character Bit, we used icosahedrons, dodecahedrons, and similar. I was disappointed that Bit isn't in this one. After doing this simulation, I've grown more aware of how often fireworks are used in movies.

For the portal climax, the TronLines app was used, but also apps like "Twist" from our team's previous jobs. Once the look was mocked up by gmunk, a houdini artist recreated the rig for deeper control.

I wrote a particle renderer that could make the head holograms slurp in and out of the data discs. Special thanks to Keith Pasko for CLUing me in about using exponential functions to create a sliding-gooey sort of delay.

When fixing Quorra, there was an element in the DNA interface called the Quorra Heart which looked like a lava lamp. I generated an isosurface from a perlin-noise volume, using the marching cubes function found in the Geometric Tools WildMagic API, a truly wonderful lib for coding biodigital jazz, among other jazzes. The isosurface was then drawn along different axes, including concentric spheres. The app was mesmerizing to stare at.

After this project, I was fed up enough with wxWidgets and Carbon that I was ready to author my own OpenGL based UI. The most important thing I could use is a floating-point slider. I also got irritated with the way the Carbon sliders would not slide all the way to the minimum and maximum values. It totally messed with my zen thing. Also, after a job like this, it's clear that a member of the Processing community working within a CG community is greatly restricted by the differences of realtime graphics rendering engines, and that probably messes with an art director's zen thing.

"TRON: Legacy" © Disney Enterprises, Inc. All Rights Reserved.

LG Optimus Launch Conference

I worked with yU+Co on this spot featuring a globe of dots connecting up. I was very happy to see how creatively they composited the renders - using my simulation in ways I never expected. The system was easy to write since it resembled older work. It was great to collaborate online with yU+Co for the first time. I appreciate how tech-savvy everyone was, and I think that made the pipeline pleasant. I'm also grateful that everyone understood and was sympathetic to me being at Beit T'Shuvah.

IBM Data Baby

This project required a lot of research demo programs. The job holds the new record for most code artists (8) hired on one MTh job. Our apps began receiving animated curves from maya, we introduced a new speed-optimized OBJ sequence file format, and we continued to accumulate maya export scripts. At the request of director Kaan Atilla, I managed to write a bunch of C++ After Effects plugins with names like [FishBall, Stripes, SchizoPath, MeshSpikes, CurveConnector], but in the end I settled back into OpenFrameworks and wxWidgets because when you compete with an Adobe app for internal resources, the Adobe app wins. I'm also disappointed in Adobe's quality of documentation and examples. I was put in a 'lead code artist' position and I feel like I handled myself better this time. We learned a whole lot! Shout outs to new algo-collaborators Jeremy Rotsztain and Tim Stutts.

Buick - Behind The Beauty

Stillshot from the TV commercial

Stillshot from the TV commercial

For this project, we tried a handful of intelligent particle techniques. One of the approaches was to have me write a renderable simulator in OpenFrameworks that could fuse our specific mix of generative diagrams, numbers, shapes, and line-art into a magical fantasy breeze. The app was able to work with Motion Theory's special rotoscope and compositing pipeline, and this was really the first time I started using wxWidgets there. Because of that, the app featured enough user interface that a non-programmer could be productive with it. Four Adobe-savvy designers were able to run my app and art direct their own shots.