Application Software for Electronic Literature 
|
D. Fox Harrell
R esearcher, writer and artist, D. Fox Harrell is Associate Professor of Digital Media in the Comparative Media Studies Program and Computer Science and Artificial Intelligence Laboratory at MIT. Exploring the relationship between imaginative cognition and computation, Harrell's work focuses on the development of computer-media-narrative and authoring software that use elements of interactivity, social critique, cross-cultural narrative; cognitive semantics; gaming; and the social aspects of user-interface design. His seminal GRIOT System utilizes code to create/generate interactive and significant "polymorphic" poems -- such as The Girl with Skin of Haints and Seraphs and Walking Blues Changes Undersea. GRIOT (named for West African storytellers who often incorporate improvisation in their performances) uses a combination of knowledge engineering, interactivity, cultural identity, and Joseph Goguen's mathematical approach to meaning representation called algebraic semiotics. Harrell has also worked with Kenny Chow to create a "new form of concrete polymorphic poetry inspired by Japanese renku poetry, iconicity of Chinese character forms, and generative models from contemporary art." [1] D. Fox Harrell's work continues to provide a basis for interactive and generative multimedia systems. In his book chapter for Second Person [2] Harrell has stated that his "longer-term project involves using this technical and theoretical framework as a basis for creating further computational narrative artworks where in addition to textual input, users can interact with graphical or gamelike interfaces. This user interaction will still drive the generation of new metaphors and concepts, but along with text will also result in blends of graphical and/or audio media." Indeed, his recent projects such as Living Liberia Fabric, an interactive narrative peace memorial affiliated with the Truth and Reconciliation of Liberia, have fulfilled this early objective. Fox Harrell's work has been published, performed and exhibited internationally, including the University of Toronto Press, MIT Press, Elsevier, Springer-Verlag, CTheory, The Journal of Research into New Media Technologies, The Fibreculture Journal, and the Electronic Literature Organization, and he is the recipient of the National Science Foundation CAREER Award for his project "Computing for Advanced Identity Representation." His book, Phantasmal Media: An Approach to Imagination, Computation, and Expression was published by MIT Press in 2013.
1.D. Fox Harrell and Kenny K. N. Chow Fox Harrell: The GRIOT System T he GRIOT system was invented by Fox Harrell, and it comprises technical support for implementing narrative and other forms of computational discourse with the following characteristics: generative content, semantics-based interaction, reconfigurable discourse structure, and strong cognitive and socio-cultural grounding. Strong cognitive and socio-cultural grounding here implies that meaning is considered to be contextual, dynamic, and embodied. The formalizations used derive from cognitive linguistics theories with such notions of meaning. Furthermore, the notion of narrative here is not biased toward one particular cultural model. Using semantically based media elements as a foundation, an author can implement a range of culturally specific or experimental narrative structures. The GRIOT Architecture The following describes the GRIOT architecture as used in initial text-based experiments with narratively structured poetry. User input, in the form of keywords, is used to select the conceptual space network from a set of ontologies, called "theme domains," that each contain sets of axioms about a particular theme. These axioms consist of binary relations between sorted constants. This conceptual space network, called an "input diagram," consists of a generic space, two input spaces, and mappings from the generic space to each of the input spaces. The input diagram is passed as input to the ALLOY conceptual blending algorithm. ALLOY is the core component of GRIOT that is responsible for generating new content. An "output diagram," consisting of a blended conceptual space and morphisms from the input spaces to the blended space, is output by ALLOY. Concepts are combined according to principles that produce "optimal" blends. Typically this optimality results in "common sense" blends, but for particular poetic effects different, "dis-optimal" criteria can be utilized. "Phrase templates," granular fragments of poetry organized by narrative clause type, are combined with the output of ALLOY (converted to natural language by mappings called "grammar morphisms") to result in poems that differ not only in how the phrases are selected and configured, but in the meaning being expressed by the blended concepts. The phrases are said to be "instantiated" when they are combined with the natural language representations of the blends by replacing "wildcards" in the text. These wildcards are tokens representing where generated output can be incorporated; they also contain variables that specify how they are to be replaced, e.g. constraining the choice of theme domains, or selecting the lexical form to be mapped to by the grammar morphism. These templates are selected according to an automaton called a "Narrative Structure Machine," which also structures the reading of user input.
Multimedia Semantics
The server consists of the following components:
1) Semantic annotation This extends elements of the GRIOT system with an algorithm to judge fitness (matching) of images to be composed and a layout structure for multimedia images in addition to the standard discourse structuring rules. The client consists of the following components:
5) Graphical assets Graphical assets are actual image data files. These assets are described on the server side using semantic annotation. (the relationship between the metadata and image data files is indicated by a dotted line in the Multimedia Semantics Figure above) This annotation, using XML that is parsed and processed to produce LISP data structures, describes the visual, structural, and conceptual content of images. Fox Harrell's statement on The Griot System originally appeared on the website of the Digital Media Program, School of Literature, Communications, and Culture Georgia Institute of Technology |
Authoring Software Home Index
Writers and Artists
Mark Bernstein
J. R. Carpenter
M.D. Coverley
Dene Grigar
Deena Larsen
Nick Montfort
Stuart Moulthrop
Andrew Plotkin
Sonya Rapoport:
Silvia Stoyanova and Ben Johnston
Stephanie Strickland
|
