Monday, November 4, 2013
Sunday, November 3, 2013
NOTAS LEV MANOVICH. "THE LANGUAGE OF NEW MEDIA"
Linz, Austria, 1995. I am at Ars Electronica, the
world’s most prestigious annual computer art festival. This year it drops the
“Computer Graphics” category, replacing it with the new “net art” category,
signaling a new stage in the evolution of modern culture and media. The
computer, which since the early 60s was used as a production tool, has become a
universal media machine: a tool used not only for production, but also for
storage, distribution and playback. The World Wide Web crystallized this new condition;
on the level of language, it was recognized already around 1990 when the term
“digital media” came to be used along with “computer graphics.” At the same
time, along with existing cultural forms, during the 1990s computers came to
host an array of new forms: Web sites and computer games, hypermedia CD-ROMs
and interactive installations—in short, “new media.” And if in 1985 I had to
write a long computer program in a specialized computer language just to put a
picture of a shaded cube on a computer screen, ten years later I can choose
from a number of inexpensive, menu-based 3D software tools which run on
ordinary PCs and which come with numerous ready-made 3D models, including
detailed human figures and heads.
(31)
What had also come by 1995 was Internet—the most
material and visible sign of globalization. And, by the end of the decade, it
has also become clear that the gradual computerization of culture will
eventually transform all of it. So, to invoke the old Marxist model of base and
superstructure, if the economic base of modern society from the 1950s onward
started to shift toward a service and information economy, becoming by the
1970s a so-called “post-industrial society” (Daniel Bell), and then later a
“network society” (Manual Castells), by the 1990s
the superstructure started to feel the full impact of
this change.4 If the “post- modernism” of the 1980s was the first, preliminary
echo of this shift still to
come—still weak, still possible to ignore—the 1990s’
rapid transformation of culture into e-culture, of computers into universal
culture carriers, of media into new media, demanded that we rethink our
categories and models.
(32)
I use it (NEW MEDIA concept) as an umbrella term to
refer to a number of various conventions used by designers of new media objects
to organize data and structure user’s experience.
In this book I analyze the language of new media by
placing it within the history of modern visual and media cultures. What are the
ways in which new media relies on older cultural forms and languages and what
are the ways in which it breaks with them? What is unique about how new media
objects create the illusion of reality, address the viewer, and represent space
and time? How do conventions and techniques of old media—such as the
rectangular frame, mobile viewpoint and montage—operate in new media? If we are
to construct an archeology which will connect new computer-based techniques of
media creation with previous techniques of representation and simulation, where
should we locate the essential historical breaks?
(34)
Myst and Doom, Jurassic Park and Titanic, to the works
of international new media artists and collectives such as ART+COM, antirom,
jodi.org, George Legrady, Olga Lialina, Jeffrey Shaw, and Tamas Waliczky.
The computarization of culture
TOPICS OF THE BOOK
· the parallels between cinema history and the history
of new media;
· the identity of digital cinema;
· the relations between the language of multimedia and
nineteenth century pre-
cinematic cultural forms;
· the functions of screen, mobile camera and montage in
new media as
compared to cinema;
· the historical ties between new media and avant-garde
film.
"digital materialism." Rather than imposing
some a priori theory from above, I build a theory of new media from the
ground up. I scrutinize the principles of computer hardware and software, and
the operations involved in creating cultural objects on a computer, in order to
uncover a new cultural logic at work.
(35)
The section “Principles of New Media” describes four
key trends which, in my view, are shaping the development of new media over
time: modularity, automation, variability and transcoding.
Just as avant-garde filmmakers throughout cinema's
existence offered alternatives to its particular narrative audio-visual regime,
the task of avant-garde new media artists today is to offer alternatives to the
existing language of computer media. This can be better accomplished if we have
a theory of how "mainstream" language is structured now and how it is
evolving over time.
(36)
LANGUAGE, OBJECT AND REPRESENTATION
I felt justified in using
the word “language” to signal the different focus of this work: the emergent
conventions, the recurrent design patterns, and the key forms of new media. I
considered using the words “aesthetics” and “poetics” instead of “language,”
eventually deciding against them. Aesthetics implies a set of oppositions which
I would like to avoid— between art and mass culture, between the beautiful and
the ugly, between the valuable and the unimportant. Poetics also brings with it
undesirable connotations.
In addition, I hope to activate connotations which
accompanied the use of the word “object” by the Russian avant-garde artists of
the 1920s. Russian Constructivists and Productivists referred to their
creations as objects (“vesh,” “construktsia,” “predmet”) rather than works of
art. Like their Bauhaus counterparts, they wanted to take on the roles of
industrial designers, graphic designers, architects, clothing designers and so
on, rather than remain fine artists producing one-of-a kind works for museums
or private collections. The word pointed toward the model of industrial mass production
rather than the traditional artist’s studio, and it implied the ideals of
rational organization of labor and engineering efficiency which artists wanted
to bring into their own work.
(39)
Today, those few who are able to resist the temptation
to immediately create an “interactive CD-ROM,” or to make a feature length
“digital film,” and instead are able to focus on determining the new media
equivalent of a shot, a sentence, a word, or even a letter, are rewarded with
amazing findings.
(40)
In this book I suggest that not only individual new
media objects, but also the interfaces, both of an operating system and of
commonly used software applications, also act as representations. That is, by
organizing data in particular ways and by making it possible to access it in
particular ways, they privilege particular models of the world and of the human
subject. For instance, the two key
ways to organize computer data commonly used today — a
hierarchical file system (Graphical User Interface from 1984 Macintosh onward)
and a “flat,” non- hierarchical network of hyperlinks (1990s World Wide Web) —
represent the world in two fundamentally different and, in fact, opposing ways.
Hierarchical file system assumes that the world can be reduced to a logical and
hierarchical order, where every object has a distinct and well defined place.
I define screen
as a rectangular surface which frames a virtual world and which exists within
the physical world of a viewer without completely blocking her visual field.
Simulation refers to technologies which aim to completely
“immerse” the viewer within the virtual universe: Baroque Jesuit churches,
nineteenth century panorama, twentieth century movie theaters.
Representation
— control (“Cultural Interfaces”
section). Here I oppose an image as a representation of an illusionary
fictional universe and an image as a simulation of a control panel
Representation
— action representational
technologies used to enable action, i.e. to allow the viewer to manipulate
reality through representations (maps, architectural drawings, x-ray,
telepresence). I refer to images produced by later technologies as
image-instruments.
Representational technologies allow for creation of
traditional aesthetic objects, i.e. something which is fixed in space or time
and which refers to some referent(s) outside itself. By foregrounding the
importance of person-to- person telecommunication, and “tele”-cultural forms in
general which do not produce any objects, new media forces us to reconsider the
traditional equation between culture and objects.
(41)
Simulation refers to various computer methods for
modeling other aspects of reality beyond its visual appearance: movement of
physical objects, shape changes over time in natural phenomena (water surface,
smoke), motivations, behavior, speech and language comprehension in human
beings.
WHAT’S NEW MEDIA?
Distribution
Production
Exhibition
Shall we accept this definition? If we want to understand
the effects of computerization on culture as a whole, I think it is too
limiting. There is no reason to privilege computer in the role of media
exhibition and distribution machine over a computer used as a tool for media
production or as a media storage device. All have the same potential to change
existing cultural languages. And all have the same potential to leave culture
as it is.
(43)
new media represents a convergence of two separate historical
trajectories: computing and media technologies. Both begin in the 1830's with
Babbage's Analytical Engine and Daguerre's daguerreotype.
The synthesis of these two histories? The translation of all
existing media into numerical data accessible for computers. The result is new
media: graphics, moving images, sounds, shapes, spaces and text which become
computable, i.e. simply another set of computer data.
ather than focusing on familiar categories such as
interactivity or hypermedia, I suggest a different list. This list reduces all
principles of new media to five: numerical representation, modularity,
automation, variability and cultural transcoding.
(44)
Around 1800, J.M. Jacquard invented a loom which was
automatically controlled by punched paper cards. The loom was used to weave
intricate figurative images, including Jacquard's portrait.
(45)
We should not be surprised that both trajectories — the
development of modern media, and the development of computers — begin around
the same time. Both media machines and computing machines were absolutely
necessary for the functioning of modern mass societies. The ability to
disseminate the same texts, images and sounds to millions of citizens thus
assuring that they will have the same ideological beliefs was as essential as
the ability to keep track of their birth records, employment records, medical records,
and police records. Photography, film, the offset printing press, radio and
television made the former possible while computers made possible the latter.
Mass media and data processing are the complimentary technologies of a modern
mass society; they appear together and develop side by side, making this
society possible.
(46)
Universal Turing Machine
the development of a suitable storage medium and a method
for coding data represent important parts of both cinema and computer
pre-histories.
in the same year, 1936, the two trajectories came even
closer together. Starting this year, and continuing into the Second World War,
German engineer Konrad Zuse had been building a computer in the living room of
his parents' apartment in Berlin. Zuse's computer was the first working digital
computer. One of his innovations was program control by punched tape. The tape
Zuse used was
actually discarded 35 mm movie film.15
(47)
The pretense of modern media to create simulation of
sensible reality is similarly canceled; media is reduced to its original
condition as information carrier, nothing else, nothing more. In a
technological remake of the Oedipal complex, a son murders his father. The
iconic code of cinema is discarded in favor of the more efficient binary one.
Cinema becomes a slave to the computer.
(48)
The two separate historical trajectories finally meet. Media
and computer — Daguerre's daguerreotype and Babbage's Analytical Engine, the
Lumière Cinématographie and Hollerith's tabulator — merge into one. All
existing media are translated into numerical data accessible for the computers.
The result: graphics, moving images, sounds, shapes, spaces and text become
computable, i.e. simply another set of computer data. In short, media becomes
new media.
No longer just a calculator, a control mechanism or a
communication device, a computer becomes a media processor.
No longer just an Analytical Engine, suitable only to crunch
numbers, the computer became Jacqurd's loom — a media synthesizer and
manipulator.
PRINCIPLES OF NEW MEDIA
1. Numerical Representation
All new media objects, whether they are created from scratch
on computers or converted from analog media sources, are composed of digital
code; they are numerical representations.
media becomes programmable.
As I will show below, new media follows, or actually, runs
ahead of a quite a different logic of post-industrial society — that of
individual customization, rather that of mass standardization.
(52)
2. Modularity
This principle can be called "fractal structure of new
media.” Just as a fractal has the same structure on different scales, a new
media object has the same modular structure throughout. Media elements, be it
images, sounds, shapes, or behaviors, are represented as collections of
discrete samples (pixels, polygons, voxels, characters, scripts). These
elements are assembled into larger-scale objects but they continue to maintain
their separate identity. The objects themselves can be combined into even larger
objects -- again, without losing their independence. For example, a multimedia
"movie" authored in popular Macromedia Director software may consist
from hundreds of still images, QuickTime movies, and sounds which are all
stored separately and are loaded at run time. Because all elements are stored
independently, they can be modified at any time without having to change
Director movie itself. These movies can be assembled into a larger
"movie," and so on.
3. Automation
Numerical coding of media (principle 1) and modular
structure of a media object (principle 2) allow to automate many operations
involved in media creation, manipulation and access. Thus human intentionally
can be removed from the
creative process, at least in part.19
(52 )
computer games are highly codified and rule- based
By the end of the twentieth century, the problem became no
longer how to
create a new media object such as an image; the new problem
was how to find the object which already exists somewhere. That is, if you want
a particular image, chances are it is already exists -- but it may be easier to
create one from scratch when to find the existing one. Beginning in the
nineteenth century, modern society developed technologies which automated media
creation: a photo camera, a film camera, a tape recorder, a video recorder,
etc. These technologies allowed us, over the course of one hundred and fifty
years, to accumulate an unprecedented amount of media materials: photo
archives, film libraries, audio archives...This led to the next stage in media
evolution: the need for new technologies to store, organize and efficiently
access these media materials. These new technologies are all computer-based:
media databases; hypermedia and other ways of organizing media material such
the hierarchical file system itself; text management software; programs for
content-based search and retrieval. Thus automation of media access is the next
logical stage of the process which was already put into motion when a first
photograph was taken. The emergence of new media coincides with this second
stage of a media society, now concerned as
much with accessing and re-using existing media as with
creating new one.2
(55)
4. Variability
55
A new media object is not something fixed once and for all
but can exist in different, potentially infinite, versions. This is another
consequence of numerical coding of media (principle 1) and modular structure of
a media object (principle 2). Other terms which are often used in relation to
new media and which would be appropriate instead of “variable” is “mutable” and
“liquid.”
New media, in contrast, is characterized by variability.
Instead of identical copies a new media object typically gives rise to many
different versions. And rather being created completely by a human author,
these versions are often in part automatically assembled by a computer.
(56)
in a computer age database comes to function as a cultural
form of its own. It offers a particular model of the world and of the human
experience. It also affects how the user conceives of data which it contains.
the principle of variability is a useful in allowing us to
connect many important characteristics of new media which on first sight may
appear unrelated. In particular, such popular new media structures as branching
(or menu) interactivity and hypermedia can be seen as particular instances of
variability principle (4.4 and 4.5, respectively). In the case of branching
interactivity, the user plays an active role in determining the order in which
the already generated elements are accessed. This is the simplest kind of
interactivity; more complex kinds are also possible where both the elements and
the structure of the whole object are either modified or generated on the fly
in response to user's interaction with a program. We can refer to such
implementations as open interactivity to distinguish them from the closed
interactivity which uses fixed elements arranged in a fixed branching
structure. Open interactivity can be implemented using a variety of approaches,
including procedural and object- oriented computer programming, AI, AL, and
neural networks.
Hypermedia, the other popular structure of new media, can
also be seen as a particular case of the more general principle of variability.
According to the definition by Halacz and Swartz, hypermedia systems “provide
their users with the ability to create, manipulate and/or examine a network of
information-
containing nodes interconnected by relational links.”29
Since in new media the individual media elements (images, pages of text, etc.)
always retain their
individual identity (the principle of modularity), they can
be "wired" together into more than one object. Hyperlinking is a
particular way to achieve this wiring
(59)
In programming, there is clear separation between algorithms
and data. An algorithm specifies the sequence of steps to be performed on any
data, just as a hypermedia structure specifies a set of navigation paths (i.e.,
connections between the nodes) which potentially can be applied to any set of
media objects.
(60)
More generally, every hypertext reader gets her own version
of the
complete text by selecting a particular path through it.
Similarly, every user of an interactive installation gets her own version of
the work. And so on. In this way new media technology acts as the most perfect
realization of the utopia of an ideal society composed from unique individuals.
New media objects assure users that their choices — and therefore, their
underlying thoughts and desires — are unique, rather than pre-programmed and
shared with others. As though trying to compensate for their earlier role in
making us all the same, today descendants of the Jacqurd's loom, the Hollerith
tabulator and Zuse's cinema-computer are now working to convince us that we are
all unique.
(61)
While I deduced the principle of variability from more basic
principles of new media — numerical representation (1) and modularity of
information (2) — it can also be seen as a consequence of computer’s way of to
represent data and model the world itself: as variables rather than constants.
As new media theorist
61
and architect Marcos Novak notes, a computer — and computer
culture in its
wake — substitute every constant by a variable.33
(62)
Cinema is not the only media technology which, emerging
towards the end of the nineteenth century, employed a discrete representation.
If cinema sampled time, fax transmission of images, starting in 1907, sampled a
2D space; even earlier, first television experiments (Carey, 1875; Nipkow,
1884) already involved
sampling of both time and space.36 However, reaching mass
popularity much earlier than these other technologies, cinema is the first to
make the principle of a discrete representation of the visual a public
knowledge.
Before computer multimedia became commonplace around 1990,
filmmakers were already combining moving images, sound and text (be it
intertitles of the silent era or the title sequences of the later period) for a
whole century. Cinema thus was the original modern "multimedia." We
can also much earlier examples of multiple-media displays, such as Medieval
illuminated manuscripts which combined text, graphics and representational
images.
For example, once a film is digitized and loaded in the
computer memory, any frame can be accessed with equal ease. Therefore, if
cinema sampled time but still preserved its linear ordering (subsequent moments
of time become subsequent frames), new media abandons this
"human-centered" representation altogether — in order to put
represented time fully under human control. Time is mapped onto two-dimensional
space, where it can be managed, analyzed and manipulated more easily.
(67)
sed in relation to computer-based media, the concept of
interactivity is a tautology. Modern human-computer interface (HCI) is by its
very definition interactive. In contrast to earlier interfaces such as batch
processing, modern HCI allows the user to control the computer in real-time by
manipulating information displayed on the screen. Once an object is represented
in a computer, it automatically becomes interactive. Therefore, to call
computer media interactive is meaningless -- it simply means stating the most
basic fact about computers.
Rather than evoking this concept by itself, in this book I
use a number of other concepts, such as menu-based interactivity, salability,
simulation, image- interface, and image-instrument, to describe different kinds
of interactive structures and operations. The already used distinction between
“closed” and “open” interactivity is just one example of this approach.
(71)
Also like Blade Runner, GUI vision also came to influence
many other areas of culture. This influence ranges from purely graphical (for
instance, use of GUI elements by print and TV designers) to more conceptual. In
the 1990s, as the Internet progressively grew in popularity, the role of a
digital computer shifted
from being a particular technology (a calculator, a symbol
processor, an image manipulator, etc.) to being a filter to all culture, a form
through which all kinds of cultural and artistic production is being mediated.
As a window of a Web browser comes to replace cinema and television screen, a
wall in art gallery, a library and a book, all at once, the new situation
manifest itself: all culture, past and present, is being filtered through a
computer, with its particular human-computer
interface.
(76)
The second section “The Screen and the User” discusses the
key element of the modern interface — the computer screen. As in the first
section, I am interested in analyzing continuities between a computer interface
and older cultural forms, languages and conventions. The section positions the
computer screen within a longer historical tradition and it traces different
stages in the development of this tradition: the static illusionistic image of
Renaissance painting; the moving image of film screen, the real-time image of
radar and television; and real-time interactive image of a computer screen.
(79)
The term human-computer interface (HCI) describes the ways
in which the user interacts with a computer. HCI includes physical input and
output devices such a monitor, a keyboard, and a mouse. It also consists of
metaphors used to conceptualize the organization of computer data. For
instance, the Macintosh interface introduced by Apple in 1984 uses the metaphor
of files and folders arranged on a desktop. Finally, HCI also includes ways of
manipulating this data, i.e. a grammar of meaningful actions which the user can
perform on it. The example of actions provided by modern HCI are copy, rename
and delete file; list the contents of a directory; start and stop a computer
program; set computer’s date and time.
The term HCI was coined when computer was mostly used as a
tool for work. However, during the 1990s, the identity of computer has changed.
In the beginning of the decade, a computer was still largely thought of as a
simulation of a typewriter, a paintbrush or a drafting ruler -- in other words,
as a tool used to produce cultural content which, once created, will be stored
and distributed in its appropriate media: printed page, film, photographic
print, electronic recording. By the end of the decade, as Internet use became
commonplace, the computer's public image was no longer that of tool but also
that a universal media machine, used not only to author, but also to store,
distribute and access all media.
universal
media machine
(80)
Cinema, the printed word and human-computer interface: each
of these traditions has developed its own unique ways of how information is
organized, how it is presented to the user, how space and time are correlated
with each other, how human experience is being structured in the process of
accessing information. Pages of text and a table of contents; 3D spaces framed
by a rectangular frame which can be navigated using a mobile point of view;
hierarchical menus, variables, parameters, copy/paste and search/replace
operations -- these and other elements of these three traditions are shaping
cultural interfaces today. Cinema, the printed word and HCI: they are the three
main reservoirs of metaphors and strategies for organizing information which
feed cultural interfaces.
(82)
Subscribe to:
Comments (Atom)