Alex's notes

Creative coding tools

The world is interesting because objects in the world relate to each other in complex ways. Both art and simulation try to mimic this complexity and therefore it can be useful to represent complex interaction in media art.

I have found two important aspects to modeling complex interactions with code. Firstly, interesting behavior arises when you have a system with feedback loops. By this I mean: the state of the simulation should depend on the previous state of the simulation. In contrast, a system that lacks feedback loops will usually have its entire state defined as a closed form function of time (e.g. y = sin(t)). Even if this function of time becomes very complicated (e.g. lots and lots of layered sins, cosines, and other functions), it’s easy to pick out when a system is completely determined by time.

Secondly, interesting behavior usually has an associated interesting data structure. What do I mean by this? In order to model behavior worth modeling, it is usually necessary to represent the data of the simulation in a format that lends itself to the simulation. Examples: particle simulations and spatial hashes, tree simulations and tree data structures, that one type of simulation where branches seek food, etc.

Many visual programming interfaces for creative technology make it hard to create simulations of this sort. They lend themselves well to stateless, GPU-driven graphics. I’m sure there is some underlying explanation for why they always follow this pattern. However, feedback loops have to be represented explicitly via “feedback” nodes, which adds overhead to creating dynamic feedback systems. In addition, feedback data is usually limited to textures — that is, 2D arrays of colors. This lends itself well for 2D grid based simulations like Game of Life or other “video feedback” effects. But this constraint keeps you from using complex data structures that grow, shrink, and change over time. Therefore visual programming interfaces are fundamentally handicapped when it comes to expressing dynamic, rich data simulations.

Other ideas:

?
| / | | |\ \| \| | |/ | | O | | | \ | | | \|/ / | | | | \| \| o | O . | \ ---| / \ | \ \| / |\| - \ --- |--- |/|/| / \ . \|/|/ | | | - \|\ | | . /| \|/ | \|/| /|/| | | | . | -| |/ | |/ | | / \ |/ \| \| | /\|/ \| \---| - | || |/ | | \ \|/| -| ||\| | \ - | |/ |\|| | / \ \|/| |/| \| / - | - | \ | |- |/ \|/ \ / . | ./\|/ \| . | --- \ | - /| /| / / \ \|/|/ |\-- | /|/ \ / | | / - \|/ | -- \|/| - \ | / . \ |\|/ |\|/|./ / \ | | - | | |/ ---| \ | | |\|\|\| - \|/- \|/|\| | | | \ | \|| | |/ / \| . - | |/\ \| / | \ \| | - |/ / \ \ \ | / \| --- . \| ---- - | - |/ | - \ |/ / \|/ | \|/ \| --- | | | |/ | | | | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^