Aaron Koblin rocks...

Aaron Koblin rocks…

 

Beautiful Data: The Stories Behind Elegant Data SolutionsBuilding Radiohead’s House of Cards

By Toby Segaran, Jeff Hammerbacher
Excerpted from Beautiful Data: The Stories Behind Elegant Data Solutions (O’Reilly)
(this extract by Aaron Koblin with Valdean Klump)

Dateline: October 15, 2009
More Insight articles


This is the story of how the Grammy-nominated music video for Radiohead’s “House of Cards” was created entirely with data. Before you read this, you should watch the video. The definitive source for the video is the project’s Google Code page. On that site, you’ll also find several other resources, including samples of the data we used to build the video, a Flash application that lets you view the data in 3-D, some code you can use to create your own visualizations, and a making-of video. Definitely check it out.

How It All Started
In September 2007, I received an email from James Frost asking me if I’d be interested in doing a music video based on data. James is a very talented music video director who has done work for Coldplay, Norah Jones, Pearl Jam, and loads of other popular artists. He had seen my Flight Patterns project (see below), which used air traffic GPS data to visualize commercial flight patterns and density, and wanted to meet up to talk about doing a visualization for a music video.



Still image from “Flight Patterns” (2005).


A couple of months later, James, his producer Justin Glorieux, and I met up in LA for coffee and we tossed some ideas around. I showed them some of the projects I’d been working on and some technologies I thought would make nice visualizations. We discussed a couple of possibilities that involved Processing, a programming language widely used for data visualization. This direction eventually turned into the “Rest My Chemistry” video for the band Interpol, which came out in March of 2008. If you’ve never used Processing, I highly recommend you visit processing.org and check it out. As far as I’m concerned, it is the best programming language for artists, designers, or anyone interested in dynamic data visualization.

The other possibility we discussed was visualizing laser sensor data. I first encountered this technology while working on a project for the Center for Embedded Network Sensing (CENS) at UCLA. CENS was using lasers to detect how light shines through forest canopies, and I was struck by the inherent beauty in the rendered images. James agreed after seeing some examples, and he was impressed by the concept of using lasers to create a piece of film. He said: “You mean you’re shooting video without cameras? You’re shooting video without video?” He immediately saw an opportunity to do something that hadn’t been done before. Not too long afterward, he approached Radiohead with the concept.

Hopefully, you’ll find the story of how this video was made to be an inspiration for your own work. I’ll talk first about the equipment we used to capture the data. After that, I’ll talk about the data itself, the video shoot, and the post-processing of the data. Finally, we’ll take a look at the visualization code I provided for the Google Code site and discuss how you can play with it yourself.

The Data Capture Equipment
The video for “House of Cards” wouldn’t have been possible without some sophisticated data capture equipment. When you watch the video, you’ll notice that there’s a variety of scenes, from static suburban landscapes to dynamic point clouds of vocalist Thom Yorke singing. In order to get both the close-ups of Thom and the landscapes, we needed to use two different kinds of equipment: the Velodyne Lidar and the Geometric Informatics visualization system.

Velodyne Lidar

Velodyne is a company located just south of San Jose, California, that’s run by two guys who compete in robot combat events like Battle Bots and Robot Wars in their spare time. The company produces loudspeakers, stereo equipment, and (naturally) powerful laser scanning devices, including the HDL-64E Lidar we used to capture the landscape and party scenes in “House of Cards.” The HDL-64E’s real claim to fame is that it was used successfully by several of the 2007 DARPA Urban Challenge vehicles, including the winning team, to achieve environment and terrain vision. In some cases, it was these vehicles’ only vision system.

Velodyne’s HDL-64E Lidar is a scanner with 64 laser emitters and 64 laser detectors. It spins in a circle, gathering data 360 degrees horizontally and 26.8 degrees vertically at a rate of over one million data points per second, which approximates to about 5 megabytes of raw data per second. By default, the Lidar rotates at 600 RPM (10 Hz), though this can be adjusted between 300 and 900 RPM by sending a text command through the system’s computer serial port interface. We used the highest setting, 900 RPM, for maximum resolution when scanning the static landscapes.

The range of the Lidar varies based on the reflectivity of the environment. Pavement, for example, has a 50-meter range, while cars and foliage (which are more reflective) have a 120-meter range. The minimum range is 3 feet; anything closer, and the light reflects back into the detector too quickly for the device to measure it.

The emitter-detector pairs are divided into two 32-laser banks, as you can see in the diagram below. The upper bank is directed at the higher half of the elevation angles; in other words, it scans the top half of the Lidar’s vertical field of view. The lower bank, conversely, scans the lower half of the elevation angles.



The Velodyne LIDAR (image courtesy of Velodyne, Inc.).


Because the upper bank is normally directed at higher elevations, and therefore at objects farther away from the Lidar, the distance traveled by the optical pulses is larger than the lower bank. Therefore, to obtain good resolution at longer distances, the lasers in the upper bank are triggered three times for every one trigger of the lower bank. You can see an example of how this affected our data in the still image from the video, shown below.



A still image of the party scene, shot with the Velodyne Lidar; notice the higher resolution at the top of this image, which was caused by the faster trigger rate of the lasers on the upper bank.


The Lidar obtains a point of data by emitting a pulse of light (aka a laser beam) and then measuring the amount of light that comes back. As the unit runs, each of the 64 emitters releases an optical pulse that is five nanoseconds in duration. This pulse is then focused using a lens and is directed by mirrors out into the environment. When the light strikes something in the environment, a portion of the light reflects back toward the Lidar. This return light passes through the laser-receiving lens and a UV sunlight filter, which limits the amount of light introduced by the sun; without it, natural sunlight would decrease the system’s sensitivity and create a lot of noise in the data.

After the return light has gone through the sunlight filter, the receiving lens focuses the return light on a photodetector called an Avalanche Photodiode (APD), which generates an output signal relative to the strength of the received light. The output signal from the APD is amplified and then converted from analog to digital. This data is then sent to a digital signal processor, which determines the time of the signal return. The strength and return time of the pulse creates one unit of data. As I said earlier, the HDL-64E model creates over one million data points every second, which is over 5 megabytes of raw data per second.

After the data is created, the sensor outputs it to the user through a standard 100BaseT Ethernet port. Data is continuously streamed out of this port at a frame rate equal to the rotation rate (600 RPM would produce a 10 Hz frame rate). Included in these Ethernet data packets are the distance, intensity, and angle data for each emitter-detector pair. The data is then captured using an Ethernet packet capture program and, in our case, saved to a hard drive.

Geometric Informatics

The Lidar is an incredible tool for visualizing outdoor scenery. It detects a point about the size of a nickel, depending on distance, which works fine for large spaces, but for the contours and details of a person’s face, it’s not good enough. For the close-ups of Thom Yorke singing, we needed something else. We needed something with finer vision.

While thinking about how to do the close-up shots, I happened to think back to an outfit called Geometric Informatics that I discovered at the 2005 SIGGRAPH conference. (Aside: if you have a cool data visualization technology, please go to every trade show possible on the chance that I may be attending… thank you.) It had a booth at the conference and a demo of its system, which it calls GeoVideo.

GeoVideo is a real-time motion capture system that is particularly suited for capturing the geometry of a person’s face. It is significantly better than the Lidar system at close-ups, capable of discerning data points at 0.2 millimeters as opposed to 2 centimeters. With it we were able to capture the fine details of Thom Yorke singing. The point cloud data you see at the opening of the video was captured with GeoVideo.

If you think the drawing below looks a bit simple, that’s because the device is not much to look at. The system looks like a beige box roughly a foot on either side with two lenses on it. One lens projects a field of light onto the subject in front of the box, while the other lens captures the data. The light field consists of a grid of 600,000 triangles, which, in effect, forms an instant contour map projected onto the subject in front of the sensor. The sensor then reads each triangle point as a point of data, which is then outputted raw to a computer at 54 megabytes per second. The sensor can capture 180 frames per second.


The Geometric Informatics system (image courtesy of Geometric Informatics).

The advantage of the GeoVideo’s method of projecting a light field onto the subject is that whatever is in front of the sensor isn’t required to have a grid physically drawn onto it, wear a motion capture suit, or sit in front of a green screen with reference marks. The light projection creates an instant, portable reference map. It’s incredibly easy.

The GeoVideo system is also capable of texture mapping, meaning that it can not only capture the data points, but also the textures between those data points. Combined together, this results in an eerily accurate 3-D representation of an object or a person’s face. For the “House of Cards” video, we decided to forgo the textures and use only the data points. And even these we heavily downsampled. The result was the digital point cloud of Thom Yorke you see in the opening scene of the video. Rather than an exact likeness of him, he appears to be a digital avatar or soul—at least, that’s how I see it. Having seen both versions of the data—with textures and without—I can say that the version we used without textures is much more interesting. With textures, he looked a bit like a character from a video game. Sometimes taking away data makes the visualization more beautiful.

The Advantages of Two Data Capture Systems
It’s worth noting that, after working on this video, I became a big fan of using more than one data capture system on the same project. Mixing technologies can have a multiplier effect on creativity. We could have used just the Lidar or just the GeoVideo system to shoot the entire video, but I’m glad we didn’t. It may have made the project more complex, but it allowed us to be more creative. It also gave us a lot more flexibility in regard to what we could capture.

There are a couple lessons too that you can draw from my experience of finding equipment. The first is that when it comes to finding equipment for data visualization, look everywhere. There are exciting sensing technologies being developed constantly that have never been used artistically. If you’re about to embark on a visualization project, do some research online, at trade shows, or at your local university. Find out if there are new ways to capture your data that you hadn’t considered before. A different piece of equipment might add a theme to your work or reveal data you didn’t see previously. Always be looking for visualization techniques that will surprise people.

The second conclusion is a warning: if you only use one of piece of equipment, your work may be seen as just a demo of that piece of equipment. If we had only used the Lidar to create the “House of Cards” video, I have a slight suspicion that the video might have become “the Lidar video.” By using both the GeoVideo system and the Lidar, the final product couldn’t be slapped with a product label. No single tool defined the work. The mixture of two data capture systems made the story behind the video more interesting.

The Data
Before I talk about the shoot, I want to show you a sample of the data. It’s really very simple. The following are three points I pulled arbitrarily from the data of Thom Yorke singing.

They’re in the file 2067.csv, which is the 1,067th frame in the video. Because each second is 30 frames and the first frame is 1001.csv, these data points can be seen at around 0:36 in the video. You can find this datafile on the Google Code site, along with the other frames:

70.05, 162.48, -79.32, 122
70.23, 165.26, -78.82, 112
70.46, 168.00, -77.55, 95

The data are in the format x, y, z, intensity. All of the data we captured was eventually translated into this format.

The x, y, z values are relative distance measurements. The GeoVideo system, like the Lidar, has a 0, 0, 0 point upon which it bases all other points. What 70.46 means, therefore, is that the point is 70.46 units along the x-axis away from the 0 point. You can scale these numbers however you want. The intensity range is from 0 (0% white) to 256 (100% white).

You’ll find 2,000 frames’ worth of Thom singing on the Google Code site, comprising just over a minute from the video. The audio is available as well. We also included two static landscapes’ worth of data: the city and the cul-de-sac. They are in the HoC_DataApplications_ v1.0.zip archive that includes the viewer program.

The data you see on the site is in the same format as the data we delivered to the postprocessing studio, with one minor difference. The studio wanted RGB values for each point, so we repeated the last value twice—in effect, using the intensity field as the color channel.

Capturing the Data, aka “The Shoot”
We recorded the points data for “House of Cards” in May 2008 over a weekend in Palm Beach County, Florida. Radiohead was on tour there at the time, and they like to shoot music videos while they’re on tour to reduce their travel. For a behind-the-scenes look at what the production was like, check out the “Making Of” video at the Google Code site I mentioned earier.

The Outdoor Lidar Shoot

The first thing we did on arrival in Florida was set the Lidar up on the back of an old van the production crew had rented. We used the van to capture the static landscape data you see in the video, such as the city and the cul-de-sac.

Unlike the DARPA Urban Challenge vehicles, we did not put the Lidar on top of the vehicle. Instead, we tilted it 90 degrees and mounted it to the back of the van. This meant that the lasers would sweep the environment vertically. If picturing this is confusing, think of a lighthouse tipped on its side and sticking off the back of the vehicle, like a tail pipe. This meant that the lasers rotated from the street to the sky and back again. This happened 900 times per minute.

We did it this way because it gave us a very high-resolution scan of the area. And in fact, during post-processing, we isolated only one laser out of the 64, because all of them were effectively scanning the same thing. As the van was moved forward, then, the laser scanned a unique part of the environment with each revolution.

The landscape below was captured with this technique. Our van drove right down the middle of the street. Do you see how the lines on the street are perpendicular to the street itself? That’s because the Lidar was hanging off the back and facing downward. You may notice as well that there are curved lines on the side of the apartment towers. This was caused by the movement of the van coupled with the rotation and angle of the Lidar.


Data captured by one laser from the Velodyne Lidar.

The shoot went very smoothly. The production team had scouted the locations, so we simply drove to each scene and scanned them in order. When we reached an area we wanted to scan, we would slow the van down to around 10 mph and the driver would try to achieve as steady a speed as possible. Then we’d start recording.

Unlike a camera, the Lidar doesn’t start and stop. Instead, when it’s on, it’s always rotating and always outputting data. So, we didn’t have to turn it on and off, we just had to know when to start recording the data. When the moment came, our assistant director Larry Zience would shout “roll computer” as a signal to Rick Yoder, the Velodyne field engineer, to start collecting points. (This was mildly funny to some of the crew, because normally a director says “roll camera.”) Rick would then hit a key on his laptop and the Lidar data would begin outputting to his hard drive. When Larry said “cut,” we stopped recording. Rick later sent me a note about what it was like to work with a film crew:

One of the things that stood out in my mind was how hard it was to break a photographer’s habit of finding the right perspective for a shot. Conventional camera directors are not accustomed to building a model from scans and then manipulating the camera’s perspective in post processing. They would say, “we want to dolly the camera from here to there and then move upward about 20 feet while panning about 45 degrees.” We would then say, “Great! For now we’ll just stick the scanner here in the middle of the scene and you can do that later in post.”

The illustration below shows another landscape. Notice how the power lines appear jagged? There’s a simple reason for that: it’s because the van was bumping up and down due to the uneven road and the natural bounce of the vehicle. Typically, these “errors” would be compensated for with gyroscopes, accelerometers, and other fancy pieces of equipment. In our case, we wanted the errors. Not only was it cheaper and easier to process, but (in my eyes, at least) it made the data more interesting. Perfection is an admirable goal, but not always the most creative.


Another landscape image captured by the Velodyne Lidar.

The Indoor Lidar Shoot

We also used the Lidar indoors on a film set. It was used to capture the party scenes at 3:30 and 3:55 in the video. Unlike the landscape scenes, we used all 64 of the Lidar lasers’ data for this part of the shoot rather than just one. That’s because the party scenes are dynamic—the points change with every rotation of the Lidar—which means they change with each frame of the video. Therefore, you see the people in the scene moving. For this part of the shoot, we used the normal horizontal orientation for the Lidar, which is the reason the data appears in horizontal lines.

To create the party scene, we recruited some film students from a nearby school. Some of the students got very done up, thinking they would be in a Radiohead video and this was their time to shine; little did they realize, though, that all we really wanted was the form of their bodies. Sorry about that, guys!

If you count the horizontal lines in the figure below, you’ll find that there are 64. And notice also how the top half of the image appears brighter? That’s because the 32 lasers at the top of the Lidar trigger faster than the bottom. As I noted earlier, the Lidar is built this way because it normally scans large terrain spaces and requires a higher resolution for elevations approaching the horizon.


A still from the party scene, captured by the Velodyne Lidar; there are 64 lines of data, or one for each of the Lidar’s lasers.

Unfortunately, the resolution for the Lidar is very low, about 2 centimeters per point. That’s why the figures look so hazy. To me, this added to the meaning of the video. Parties are often populated by people you don’t know very well, and the visualization reflects this sense of alienation. However, the low resolution of the Lidar wasn’t going to suffice for the close-ups on Thom Yorke. For this, we used the GeoVideo system.

The Indoor GeoVideo Shoot

The point clouds of Thom Yorke, his “lover” (played by actress Lauren Maher, who you first see at 1:05), and a couple of other scenes (such as the hand at 3:50) were all captured with Geometric Informatics’ GeoVideo system.

The GeoVideo system is capable of an astonishing level of realism. If you watch the demo videos on Geometric Informatics’ website, you’ll notice that it achieves a much higher quality image than the point clouds in our video. Its visualizations also don’t suffer from the interference and errors that appear in our video.

The reason our video is lower quality is that we made it this way deliberately. James Frost, the director, didn’t want a perfect visual avatar of Thom Yorke; he wanted a fragile, evanescent vision of him (see below). When watching the opening scene, to me this implies that this is not Thom Yorke the man, but something closer to the singer’s soul. We are seeing the ghost in the machine.


A still image from the video of Radiohead singer Thom Yorke.

The low quality of the data and the frequent errors in the visualization also make it appear as if acquiring the data was difficult. This apparent difficulty enhances the story. A clearer image would not have conveyed the meaning we wanted.

The interference in the data was not done in post-processing; rather, it was created on set. The production company brought a number of props with them to break up the data, including little bits of mirror glued to a sheet of plexiglass, feathers that were dropped in front of the scanner, and running water that was poured on a piece of plexiglass in front of Thom. The mirrors ultimately worked the best to disrupt the data in a nonorganic way; the feathers didn’t do a very good job of interrupting the data, and water absorbed the light, creating only empty points in the data set.

Including both the GeoVideo and Lidar portions, the interior shoot took about 10 hours. For all of Thom’s scenes, we were careful to back up the data on multiple hard drives for fear that if we lost it, we might not be able to shoot it again.

Processing the Data

After all the data was captured, the processing work started. The first thing we did was send the raw Lidar data to 510 Systems, an engineering company in Berkeley, California that has a lot of experience processing this type of data. The company assigned the project to its in-house Lidar data guru, Pierre-Yves Droz. He’s an expert at turning raw Lidar data into usable formats.

Pierre did two things for us after receiving the data, which we mailed to him on DVDs. First, for the landscape scenes, he isolated a single laser out of the 64 and created a data set of just that laser’s points. Second, he converted all of the raw Lidar data, including the dynamic party scene data, into individual data points consisting ofx, y, z, and intensity.

To convert the raw data, Pierre needed to know the precise position and orientation of each of the Lidar’s laser emitter and detector pairs. This calibration information is provided by Velodyne, and the parameters are unique to each Lidar unit. Pierre also used the speed of the van to help calculate how far the Lidar moved in the real world as it rotated. All told, we gave 510 approximately 4 gigabytes of raw data, which turned into almost 50 gigabytes of processed data in text .obj format.

Post-Processing the Data
Once we had the processed data, we took it to The Syndicate, a visual effects house in Santa Monica, California. It rendered the scenes in 2-D and added the particle flow effects you see in the video starting around the one-minute mark.

Brandon Davis, The Syndicate’s particle specialist, worked on the project. He sent me an email describing why the project was unusual:

From the start, the Radiohead project had very unusual possibilities from a visualization standpoint. With an animated data set, you get a strange paradox: view-dependent data that can be viewed independently. It really is a “second sight,” being able to take what one sees and view it from different perspectives, revealing the gaps in that original sight. This opened the doors for some truly unique imagery.

He goes on to describe how he tackled the vaporization effect that you’ll notice throughout the video:

The client wanted to degenerate the data set over time as if the points were blowing away in a virtual wind. From the start we were looking at two distinct types of data—static Lidar point clouds of environments and dynamic animated point clouds of the singer Thom Yorke, the latter of which we knew would be the most challenging to manipulate. A static data set is relatively easy to manipulate because all you need to do is displace the points over time, so we knew it wouldn’t be too difficult to selectively trigger portions of the data set to be affected by a velocity field, creating the effect of buildings and trees vaporizing away with noise-based turbulence. And as we predicted, producing that effect on the static data sets was just a matter of moving bounding forms through the point cloud and doing a clipping test on a frame-by-frame basis. If a point was inside a bounding object on a specific frame, it was passed on to a particle system where it was free to move through a noise field, otherwise it remained static. This was very interactive and easy to control and generally fast to iterate.

The dynamic point cloud of Thom singing was another matter, however. Brandon continues:

The dynamic point cloud was the biggest challenge on the project from my perspective. While you can try to apply the same methodology from the static data set, a dynamic data set creates a singular problem: it refreshes every frame! As soon as you release a point and pass it on to a particle system, the original source point pops right back into a new location, essentially re-spawning every frame. So while I could designate the points at the tip of his nose to release and blow away, every subsequent frame there was a new point to replace the one that blew away.

Eventually, The Syndicate figured out a simple way to add the decay effect to the dynamic point cloud using a 2-D mask: it added the mask with a layer of particles blowing away on top of the 3-D point cloud. This meant that there wasn’t a perfect one-for-one particle decay like in the static landscapes, but I think the differences are imperceptible.

When I asked James, the director, why he added the particle decay effect in the first place, he said:

“House of Cards” is essentially a love song, and as with love or friendships, in life there are points where relationships break down. They don’t just stop; there is a series of events, a catalyst to start the demise or decline. For the video I wanted to explore this theme on a larger scale. It seems to me that in life we are very insular to what’s going on in the world around us. The idea is that infrastructure can start to collapse on a large scale around us, but because we don’t feel the effect of it, we as humans have reached a point where we don’t care. But eventually that event will have a ripple effect and will finally reach us on a personal level. What’s happening now with the financial institutions is a perfect example of this; a bank collapses, but unless your money is in that bank you don’t think twice about it. Then your bank collapses.

When all of the post-processing was finished, the clips were edited together by Nicholas Wayman Harris at Union Editorial. At last, the video was complete.

Launching the Video

The “House of Cards” video was the first music video to be premiered by Google. It launched on July 11, 2008. The Google site includes some of the video’s data, so that you may create your own visualizations, as well as a 3-D data visualization tool. Google’s Creative Lab developed the site.

The visualization tool was written in Flash by myself and my friend Aaron Meyers. It allows the viewer to rotate the point cloud in real time while the video is playing. To me, this is where the data becomes truly beautiful. The Flash application allows you to look at parts of the video from any angle you want in real time, something traditional video recording will never allow. You may even turn Thom Yorke’s face so that it faces away from you, effectively holding his face as a mask up to yours and allowing you to look through his eyes. This effect is very powerful, in my opinion. It makes the music video tangible in a way I doubt many people have experienced before.

We also released some of the data itself—making it open source—along with a video creation tool written in the Processing programming language. We then encouraged people to download the data and create their own videos.

I want to share the source code for the video creation tool to show you how easy it is to create your own version of the video in Processing. This is the code that outputs frames of Thom Yorke singing:

import processing.opengl.*; int frameCounter = 1; //Declare a variable to store which frame we're dealing with
void setup( ){ //Here we set up the program

  size(1024,768, OPENGL); //This is the render size. We’ll use OpenGL to draw as fast as possible

//frameRate(30); //Uncomment to watch the animation at 30 frames per second.

strokeWeight(1); //Draw lines at a width of 1, for now.

}

void draw( ){ //Here we state the things we’re going to do every frame

background(0); //We’ll use a black background

translate(width/2, height/2); //The data has 0,0,0 at the center and we want to
//draw that point at the center of our screen

translate(-150,-150); //Let’s adjust our center slightly

scale(2); //Let’s draw things bigger

//rotateY(frameCounter/50.0f); //If uncommented, this makes the data rotate over time

//rotateY(mouseX/150.0); //If uncommented, this uses the mouse’s horizontal
//location to adjust the rotation

String[] raw = loadStrings(frameCounter+”.csv”); //Here we load the current frame
//data into an array

for(int i = 0; i       //raw data

String[] thisLine = split(raw[i],’,’); //For each line we’re going separate
//each parameter

float x = float(thisLine[0]); //Now we make a decimal variable for each parameter
float y = float(thisLine[1]);
float z = float(thisLine[2]);
int intensity = int(thisLine[3]);

stroke(intensity*1.1,intensity*1.6,200,255); //We set the color of each point to
//correspond to the data’s intensity value

line(x,y,z,x+1,y+1,z+1); //Here we draw a little line for each point; this
//is much faster than a more complex object and
//we’ll be drawing a lot of them

}

frameCounter++; //Add one to the frame variable to keep track of what frame we’re
//currently on.

if(frameCounter>2101){ //If we get to the end of the data we’ll exit the
//program

exit( );

println(“done”);

}

//saveFrame(“renderedFrames/”+frameCounter+”.tga”); //This would be a way to save out
//a frame
//*remember you’re saving files to your harddrive!*
}

It’s not as beautiful as the data (this isn’t Beautiful Code, after all), but it works great. As written, the code allows you to watch Thom Yorke sing the song head on, but with a couple of modifications, you can customize the experience. Here are two examples of modifications that are commented out in the previous code. The first is:

rotateY(frameCounter/50.0f);

Uncommenting this line at the beginning of the draw function causes Thom’s face to turn around the y-axis as the frames increase.

The second modification is:

rotateY(mouseX/150.0);

Uncommenting this line at the beginning of the draw function allows you to make the rotation a function of the mouse. You may now move Thom’s face as the frames are outputted. I’m sure you can think of other things to modify; many people have done things I hadn’t even considered, which is exactly what I hoped for. Once you’ve rendered out all the frames (by uncommenting the last line), you can put the frames together into a video with a program like QuickTime Pro, Final Cut, or After Effects. Some of the videos created by other people are really impressive. Check them out at the “House of Cards” YouTube group.

It really is quite easy. All it takes is some beautiful data with which to get started.

Conclusion
While writing this chapter, I came up with seven thoughts that you may find useful for capturing and presenting your own data in a beautiful way:

  1. Looking at something ordinary in a new way can make it extraordinary.
    You don’t need to scan the moon, a tropical island, or a fashion model to obtain data that looks beautiful. Looking at common objects in a new way can have the same effect. In the case of “House of Cards,” we scanned a person’s face and some common suburban architecture. By looking at these very common things in a new way and with new visualization techniques, we made them interesting.
  2. Tell a story.
    Obviously it helps when you have an amazing song to work with, but do what you can to tell a story with your data. Just showing it to people isn’t as cool as giving it some added meaning.
  3. Using multiple visualization techniques is more interesting than using only one.
    As I mentioned previously, the “House of Cards” video was made stronger by using multiple technologies like the Velodyne Lidar, the GeoVideo system, and particle decay post-processing effects. If all we’d done was visualize raw Lidar data, it wouldn’t have been as interesting.
  4. Think about the data, not the real world.
    When we added bits of mirror to sheets of plexiglass and moved them in front of Thom’s face as he was singing, we weren’t thinking how it would look on a video camera. We were thinking about what it would do to the data. The data is the product. When you look at something you want to visualize, think about the data you will get from it.In other words, try making a bizarre reality, and then sensing that reality. This will make your story also weird and bizarre. Music videos often portray weirdness. Ask yourself, how can you manipulate your data to make it stranger, more interesting, different?
  5. You don’t have to use all the data.
    The GeoVideo system we used to scan Thom Yorke gave us much more data than we wanted. By downsampling it heavily, we produced a more interesting point cloud. We didn’t want a photograph.
  6. Set your data free.
    By allowing other people to have the data and create their own versions of the “House of Cards” video, we allowed each person to create the version he or she thought was the most beautiful. Everyone is always going to have an opinion as to what looks the best. Let people indulge that opinion. It’ll be gratifying for you when you see their work and, you never know, someone out there might do something unexpected. That’s a good thing. Over 100,000 people have already downloaded the data from the Google Code site and created some great videos.
  7. Work with Radiohead.
    I’m being a bit tongue-in-cheek here, but there’s no doubt that we were very lucky to work with one of the world’s most innovative bands. And it wasn’t just them—it was the whole crew. This video was made possible only with the cooperation of some incredibly talented people, including James Frost, Velodyne, Geometric Informatics, 510 Systems, and The Syndicate. Partner up with people who are more talented than you are, and your project will benefit enormously.