Computational Information Design

6 Tool

Processing is a tool for developing visually-oriented software. It was conceived of as a way to introduce programming concepts to designers, and design concepts to programmers. While it goes back further, on-screen digital graphics have in the past twenty years emerged from the laboratory and made their way into consumer applications like Macromedia Flash-based web sites or motion graphics used for film and television. Drawing on this history, particularly with regards to interactive graphics, it is possible to distill the central themes of programming this type of work and thus make it accessible to a wider audience by codifying these ideas and simplifying how they are implemented.

The tool was created in collaboration with Casey Reas, and more recently has continued to evolve with the aid of an international community of developers collaborating over the web. It is not designed as, or intended to be, the ultimate tool for Computational Information Design, however it provides a means to make the process significantly easier, and was used for nearly all of the work presented in this thesis.

6.1

design by numbers

Our research lab at the mit Media Laboratory has always had a pedagogical focus, inspired in by Professor Maeda’s assertion that, with regards to the mix of computation and design:

…I believe that these same principles must be applied much more widely, throughout mit and indeed throughout our university system in general. At least at mit, there has been for many years an awareness of the need for combining the humanities and sciences at the curriculum level. Despite the best of intentions, however, the model of training is this area remains some form of the humanities wrapped around technology, or vice-versa…It is not enough for us simply to produce a technologist who is aware of the cultural context of technology or a humanities major who can talk fluently about technology…What is needed is a true melding of the artistic sensibility with that of the engineer in a single person.

maeda, 1998

This was part of the inspiration for the development of Design By Numbers [Maeda, 2000], a simple programming language intended to teach the basics of computation to artists and designers.

The strengths of dbn were the simplicity of its distribution, use, and syntax. It was available freely on the web, either for downloading or could be run from a web page with no installation required. This is a notable shift from most programming languages, which require complicated installation and tend to be expensive to purchase.

Typical use of dbn is also far more straightforward than most languages that involve a cycle of “type, compile, run, debug” than can often be cryptic. Code is entered at the right of the window, and run by clicking a button that looks like a familiar ‘play’ button from consumer electronics.

The dbn system was used for a number of workshops and occasionally for courses at mit and elsewhere. A supplementary ‘courseware’ system was developed by Casey Reas that enabled courses to be run online, with projects automatically uploaded to the web and visible via a presentation page that could be used for critiques in a class.

As the primary maintainers of dbn after its inception, we learned a great deal from feedback from users on what was difficult (algorithms and flow of programs), what else they wanted (color, a larger screen, type, ability to distribute programs over the web), and it provided a great deal of insight into the creation of Processing, which began as a “next generation” Design By Numbers.

In Processing we wanted to move to a more robust programming language, namely Java, but at the same time, to maintain the simplicity of the dbn interface. A system of ‘modes’ was developed, the first for drawing static drawings, where the user could simply begin typing code without the need to learn functions or other syntax:

The two additional modes get progressively more difficult (but powerful), each essentially designed to act as ‘training wheels’ for a person learning programming. The final mode is straight Java programming, with a handful of small syntax simplifications, so the skills of learning Processing are also transferable to Java or even C++ because of their similarity.

6.2

pedagogy

Casey Reas describes Processing as part of the shift from command line to GUI finally reaching programming instruction:

Graphical user interfaces became mainstream nearly twenty years ago, but programming fundamentals are still primarily taught through the command line interface. Classes proceed from outputting text to the screen, to GUI, to computer graphics (if at all). It is possible to teach programming in a way that moves graphics and concepts of interaction closer to the surface. The “Hello World” program can be replaced with drawing a line, thus shift the focus of computing from ASCII to images and engaging people with visual and spatial inclinations.

casey reas [via fishwick, in production]

For designers, to make Processing more visually engaging from the outset, the idea is to first make things happen, and use that to drive curiosity into learning how it works, and how to do more. Through this alternate approach, our hope is to engage people who might be likely to leave a course on Computer Science “fundamentals” after the second week as they become overburdened in technique and see no clear path to the type of work they’re actually pursuing.

For programmers, many have simply not developed graphical software (outside of simply GUI interfaces). This aspect of programming is almost always missing from Computer Science coursework (save for courses specifically in graphics and animation), and the ‘getting started’ portion of requires a few pages of code before something visual can be presented on-screen.

6.3

advanced frameworks

Another ancestor to Processing are the many software toolkits built at the Media Laboratory. Its immediate predecessor, called acu, was a C++ and OpenGL-based toolkit that was used to build nearly all the screen-based projects developed in the Aesthetics and Computation Group between 1999 and 2002. Prior to that were similar implementations, called acWorlds and acJava developed by the same authors. Even earlier, similar projects were underway in the vlw, the earliest known being Bob Sabiston’s “BadWindows.”

These frameworks (acu in particular) provided a basic structure for interactive graphics research in dynamic typography [Cho, 1999], visual programming systems [Schiffman, 2001], audio-visual environments [Levin, 2000], and methods for information visualization [Fry, 2000]. At the time this work required high-end workstations, but can be reproduced today on nearly any recently purchased pc. This shift has blurred the lines between the requirements for advanced research, and that of sketching and prototyping.

6.4

sketching & prototyping

The Java programming language enabled web-based distribution of mini-applications, called applets, that had nearly the full functionality of a desktop-based piece of software. The software compiles to a cross-platform byte code, which can be run without modification across any platform implementing the Java Virtual Machine (i.e. Windows, Mac OS, or Linux).

In addition to the ease of distribution, Java tends to make a better prototyping language than C++, both for its simplified syntax (the language designers begin with C++ and removed features that caused common problems) and built-in exception handling, meaning that program errors will be reported, rather than crashing the program, or worse, taking the operating system along with it. These two features lead to Java’s usefulness as a basis for the “sketching” environment sought in Processing.

Lacking in Java is the fact that two or three pages of standard code are often used across dynamic graphics projects, simply to handle setup of the program and handling basic mouse interaction. By making the assumption that Processing was intended for interactive graphics applications, this functionality was built into the base program, so the user could begin creating programs, rather than focussing on infrastructure.

6.5

api

Also lacking in Java were its graphic capabilities. The first most commonly used version of Java, version 1.1, had extremely limited graphics support. Subsequent versions (1.2, 1.3, and so on) addressed the problem, but in their attempt to provide an api (application programming interface, a set of features or functions) that could do everything, the most basic functionality (i.e. drawing a thick line on the screen) would now require several lines of code and significant overhead. Worse, these later versions of Java have not received the same wide distribution as the initial 1.1 in part because of legal battles between Sun (proprietor of Java) and Microsoft (accused of violating their contract).

To address graphics for older versions of Java, a new graphics engine was created. For the api issue, this graphics model was designed to be extremely compact, so as to simplify drawing tasks. Because of the more specific audience (people building interactive graphics), this api could be made more compact, which makes it easier for programmers to memorize nearly its entire feature set.

The imaging model also borrowed a concept developed by Tom White for the acu framework, where 2d and 3d graphics could be mixed in the same drawing surface. Most applications are 2d, but this provides a way to mix the two without requiring the developer to learn a 3d programming api, many of which tend to be difficult and counter-intuitive because of the algebra involved. With the added ability to directly manipulate pixels, the set of features provides much flexibility for designers and developers.

6.7

community

Perhaps the greatest strength of Processing is that of its community, an international group of users who started as designers or programmers and found the language. In the past year, the number of people to sign up and download the software has jumped from just over 1,000 (in June, 2003) to more than 11,000 (as of April, 2004). Thousands have signed up and use an online discussion area, where new users can get help from more advanced. Users are encouraged to post their latest work and share the code for their sketches as part of a worldwide studio.

About two dozen members of this community are actively involved in developing new, advanced work using Processing, which drives further interest because of the number of people who follow their work. Other members are actively involved in developing libraries to be plugged into the Processing software, enabling advanced sound synthesis or input devices like drawing tablets. What began as a two person project has fostered many such smaller projects that are included as part of the same support structure.

6.8

elevating practice & critique

An implicit goal of Processing is to remove the mystery from programming. People often consider themselves math-averse and therefore incapable of programming, or that programmed works with movement and behaviors that seem complicated are works of complicated mastery. While this is no doubt the case for many works, it is a minority, and the body of example programs included with Processing, most of them roughly a single page of code demonstrate many of the algorithms commonly used for motion and interaction.

By making the programming concepts accessible to a wider audience and removing some of the mystery, it places greater emphasis on the creation of meaningful works, rather than technical flourish by those “in the know.” The shift from technique to critique is a necessity for moving the field forward. The ability to evaluate is central to the contention of John Maeda’s professors in Japan:

…my teachers in told me that if I were to do what I did, I would make a lot of money and be the only one doing it, and that I would never know it if I were any good or not.

john maeda [resnick, 2000]

It is our hope that the active community of developers producing and sharing works will help elevate the practice, and contribute to making that which is technically “easy” equally as easy for anyone, and that which is “hard” be learnable.

The entire set of functions (the api) for Processing, designed to be terse vocabulary that provide only the most relevant features used by the greatest majority of users.