How to Dance with a Tree: Visualizing Fractals With Dance

Here’s a fun project that my friend Upasana and I put together some weekends ago. It’s a visual exploration of fractals through dance, a piece of generative art that’s part performance and part mathematical exploration.

The two ingredients that went into creating this were the Microsoft Kinect sensor, which lets your computer track how your body moves, and Processing, a programming language that lets you create interactive visuals with code. Put the two together, and you can use your body to control virtual shapes and objects.

The idea for this project came about while I was walking home from work late October, idly watching the recently bare tree branches swaying in the wind. And for some reason that made me wonder, what would it be like to be a tree for an evening? Imagine lifting your arms, and a tree waves its branches.

And then I remembered reading about fractals in Daniel Shiffman’s book Nature of Code. Fractals are those wonderfully intricate structures that look the same as you keep zooming in to them. Benoit B. Mandelbrot was one of the earliest explorers of the fractal world. He coined the word fractal to mean a kind of geometric shape whose parts resemble “a reduced-size copy of the whole.”  (Some fractal humor: What does the B in Benoit B. Mandelbrot stand for? Benoit B. Mandelbrot.)

At the heart of being a fractal is self-similarity, the idea that each piece appears similar to the whole. Think of how a coastline on a map appears similarly wrinkly across different levels of zoom. The same could be said of the jagged terrain of a mountain.

Or picture the ever branching paths that lightning follows as it travels down to the Earth.

Or the nested arrangement of leaves within a fern.

Or the buds in a head of Romanesco broccoli. Each bud contains smaller buds upon it, arranged in the same spiraling pattern.

From coastlines to broccoli, and lightning to trees, many of nature’s patterns are better described by fractals than by the usual cast of shapes like lines, circles, and triangles. (In the real world, objects can only be roughly fractal, at some level of zoom the repetition will end. But in mathematics, the self-similarity of fractals continues forever.)

While shapes like circles and triangles are the essence of simplicity, fractals, on the other hand, capture a kind of organic complexity in their infinite fuzziness. There’s a world of difference between a triangle and a mountain.

Which brings me back to the tree. You can think of a tree as being fractal-like. Here’s a sketch of a tree.

But look closely, and each branch is kind of a “reduced-size copy of the whole”, to use Mandelbrot’s words.

For example, take a look at this picture.

You might think you’re looking at an entire tree, but actually, it’s a picture of a branch sticking out sideways. (I was lying down and looking up when I took the picture.) The branch looks like a smaller-scale tree, and it’s this self-similarity that we’re trying to capture.

In his book, Daniel Shiffman implements a simple algorithm in Processing to draw such fractal trees.

I slightly tweaked his code so that the angle at which the branches form is controlled by the position of my mouse. So by moving my mouse around, I could animate the tree. You can play with this for yourself.

I tried adding in some leaves to make the tree a little prettier.

Here’s an interactive version of that for you to play with.

The next step was to incorporate the Kinect sensor bar, so you can control this tree with your body. To do this, I used the Simple-OpenNI library (you can read more about how this works in my earlier post on Kinect hacking). My friend Upasana Roy volunteered to play the role of dancer and puppeteer (video at the top of this post).

And voila, by waving her hands about, she animates the tree. The angle of her hands (measured from her hip) controls the branching angles in the fractal. The physical space that her body inhabits is mapped onto an abstract mathematical space (in this case, the space of all fractals you get by varying the branching angle of this tree).

We stopped there, but there are endless other virtual shapes, objects, and spaces that you might be interested in exploring.

I asked Upasana how she felt as she started to play with this program.

What did you experience when you first started using it?

“It was frustrating at first. A lot of times my moves didn’t get any response from the fractal, and the few times it did the changes were jerky and sudden. Even when I moved my arm in one smooth motion, the fractal tree would spasm so I didn’t know what was going on.”

What changed as you spent more time using it?

“I realized that when I dance I use all the space around me with my arms. But in this case the fractal responded to motion in some parts of space more than others. For example, moving my arms up and down made the fractals change much more than moving them side-to-side.”

How did you go about choreographing the dance? What was challenging about it and what was fun about it?

“After spending some time I could figure out a relationship between my arm positions and the fractal design. So then I could move it in a predictable way. After that it was about playing around with the fractal patterns and the music – that was the fun part. The hard part was keeping my arms up the entire time!”

“I remember feeling a bit like a hi-tech puppeteer, because I was moving my hands around in the air but you weren’t seeing THAT movement, but a transformed version of it. That felt pretty cool, to control the tree with abstract hand movements in the air.”

 

References

Here’s my Processing code for the visualization, based on modifying code by Max Rheiner (who wrote the OpenNI library that lets you access the Kinect in Processing) and Dan Shiffman (who wrote the code for the recursive tree).

Fractals Chapter in Dan Shiffman’s great book Nature of Code

Making Things See by Greg Borenstein is a very handy guide to Kinect hacking with Processing

A previous post by me on using the Kinect to make a dance video

And a big thanks to Upasana Roy for being willing to dance till she couldn’t keep her arms up any more.