ABOUT

This project characterizes the temporal flow of a video sequence as a pattern of equilibria and disequilibria. Equilibrium occurs when a frame in some sense resembles immediately preceding frames. Disequilibrium occurs when a frame contains substantially new information relative to immediately preceding frames in a temporal image sequence. The informational novelty of a frame is computed with reference to their shape or contour properties, the visual arrangement of edges and corners. Corners and edges are pixels with reasonably high brightness gradients in one or both spatial coordinate directions. This image illustrates this idea by displaying a movie frame alongside its contour representation.

The degree of equilibrium or disequilibrium of a frame is the quantity of information contained in its contour representation relative to immediately preceding frames. The precise computation uses a measure known as the Relative Entropy or Kullback-Leibler Divergence.

The temporal organization of a sequence can be graphed as a curve where the abscissa represents the time, measured in frames, and the ordinate represents the disequilibrium of the corresponding frame. The entropic envelope of a video sequence between frames A and B is the portion of the relative entropy curve between those frames.

The following image shows the entropic curve for a sequence from the film Alphaville (Jean-Luc Godard, 1965).

The curve has a relatively high value at a moment when there is a cut, since cuts can be viewed as moments where the contour structure of the new image differs sharply from that of immediately preceding images. The entropic curve is, in general, a reasonable cut detection algorithm. Cut detection is not, however, the main aim of this project.

The curve represents what could be called the internal melodic line of a video sequence. Even within a continuous shot, there are dynamic patterns of equilibria and disequilibria. Consider for instance a sequence from Alphaville.

The curve rises whenever certain elements enter the frame with disrupt the stability of a composition. For instance, the arrival of detective Lemmy Caution in the hotel in Alphaville involves a moving camera shot that follows the main character in a fairly stable composition, but this stability is punctuated by vertical architectural elements that introduce carefully patterned moments of disequilibrium. The sequence exemplifies how the entropic envelope captures the temporal rhythm of a video sequence. It identifies and represents rising and falling patterns of equilibria and disequilibria.

Compare the entropic envelope for the above scenes from Alphaville with this one from Hitchcock’s Rope (1948)

The entropic envelope in Hitchcock’s film is clearly more stable than that of Godard's work. These two examples sharply differ from the case of Stan Brakhage’s 1963 experimental film Mothlight.

That film was made without a camera, by collecting and collaging insect wings, blades of grass, and other bits of organic matter. The result is visually far more radical than either Alphaville or Rope, as is evident from the shape of the entropic envelope from this sequence.

The following videos illustrate the entropic envelope of the three films being compared here. The red line in the center indicates the point in the curve that represents the current frame.

1. Alphaville

2. Mothlight

3. Rope

The videos shown in the introduction page for this website are made by performing a Discrete Fourier Transform on successive segments of the entropic curve.

BACKGROUND

The concept of the entropic envelope was influenced by the phenomenological tradition, particularly the work of Edmund Husserl and Bernard Stiegler. A film, like a musical piece, is an essentially temporal object. The aim of this project is to provide one possible formalization of the concept of a temporal object.

To apprehend a temporal object demands a threefold experience of attention, retention, and anticipation. The viewer’s attention to the present moment is intertwined with her anticipation of the immediately upcoming future and her retention of the immediate past. Husserl proposed that the consciousness of the present sound in a musical sequence, for instance, is entwined with the retention not only of the immediately preceding note but also of previous notes as well as the anticipation of the future notes. The influence of any specific tone on the present fades gradually over time, like the tail of a comet.

The entropic envelope model conceptualizes the notion of a temporal object as an organized pattern of equilibria and disequilibria. We can think of the entropic curve, very loosely, as a representation of the experience of an ideal spectator who pays attention only to the pattern of changing contour structures in a video sequence, and who anticipates that every coming frame will resemble recent frames. This ideal spectator makes an assumption of continuity, and is ready to register moments of disequilibrium, in which the new frame is informationally rich relative to its predecessors in the sequence.

_____________