>







Editor:
Judy Malloy


Classic Authoring
Resources

Book Reviews
Authoring Systems and Interfaces


Documentation:
Conferences and Exhibitions

The End(s) of Electronic Literature:
The 2015 Electronic Literature Organization Conference,
Bergen, Norway, August 4-7, 2015

Hold the Light:
The 2014 Electronic Literature Organization Conference,
Milwaukee, WI, June 2014

Dene Grigar
Documenting Curation
as Critical Practice

Pathfinders: 25 Years
of Experimental Literary Art, MLA2014 Convention, Chicago

Chercher le texte: the 2013 Conference of the Electronic Literature Organization Brings Electronic Literature to the Public in Paris, September 23-28

E-Poetry 2013, Kingston University, London in June; Program Features Presentations, Exhibitions, Performances, and a Pedagogy Colloquium

With a Theme of "Avenues of Access", MLA2013 Includes an Exhibition of Electronic Literature and over 60 Digital Humanities Panels

Remediating the Social, Edinburgh, November 1-3, 2012

ELO2012,
Media Art
Exhibition to be Held
at WVU, June 20-23

Critical Code Studies
Working Group 2012
:
Reading Code in Context

Belgrade Resonate Festival
March 16-17, 2012

2012 MLA Convention to Feature Elit Panels and Exhibition

Dene Grigar, Lori Emerson,
and Kathi Inman Berens Impact Report for the
Electronic Literature Exhibit at MLA2012

Dangerous Readings
Explores Frontiers
of New Narrative

Elit Well Represented
at ISEA2011

Electronic Literature Organization Moves to MIT



Caitlin Fisher
Andromeda: augmented reality poem
Authoring System: SnapDragonAR


Detail: Caitlin Fisher: Andromeda augmented reality poem


C aitlin Fisher holds a Canada Research Chair in Digital Culture in the Department of Film at York University, Toronto. A co-founder of York's Future Cinema Lab, her research investigates the future of narrative through explorations of interactive storytelling and interactive cinema in Augmented Reality environments.

Her work is poetic, exploratory, interesting, and innovative, currently combining the development of authoring software with evocative literary constructs. She completed one of Canada's first born-digital hypertextual dissertations in 2000, and her hypermedia novella, These Waves of Girls, won the International Electronic Literature Award for Fiction in 2001. Most recently, her augmented reality poem, Andromeda, was co-awarded the 2008 International Cuidad de Vinaròs Prize for Electronic Literature in the digital poetry category.

Caitlin Fischer is the writer/director for Chez Moi, a part of Queerstory, a locative app tour of the political, cultural and social history of Toronto’s queer community.

Other projects include:

Wallace Edwards Illustrations - Immersive Worlds. Investiagtions into immersive, creative storyworlds.

and

Breaking the Chains. AR Experience in partnership with the Harriet Tubman Institute at York University, Toronto, ON, 2012.

In her statement, Caitlin Fisher talks about the development of the Snapdragon authoring environment in her AR Lab at York University, the creation of Andromeda with Snapdragon, and the creation of the subsequent performative version, Andromeda2.

More information can be found in her homepage at http://www.yorku.ca/caitlin/




Caitlin Fisher: Andromeda: augmented reality poem


Andromeda

A ndromeda is an augmented reality journey poem about stars, loss and women named Isabel, enabled by a unique software solution and a custom marker library. Augmented reality overlays digital imagery on physical objects, and, in this piece, the power of robust, multiple, simultaneous fiducial recognition has been made easy to work with through the development of a new expressive tool for the creation of simple, 2D augmented reality pieces: SnapDragonAR software, created in my AR Lab at York University, Toronto. SnapDragon is a unique authoring environment and a wonderful medium for poetic expression.

Andromeda uses a found pop-up book, overlaid with augmented reality markers and the poem is brought to life when a reader, using a camera attached to a computer, unlocks the textual, video and audio elements associated with the markers -- the basic idea being that the camera "sees" these symbols being explored and overlays digital content. The resulting poem can be viewed on the computer screen or through a head-mounted display (probably the coolest way to see it; there is something uncanny about holding paper in our hands and watching it come to life when the piece is mediated via a computer screen where we are used to seeing visual trickery, the effect isn't quite as magical. But I digress.)

Andromeda is the first fully realized poem written using the software, but is part of a larger suite of poems, tabletop theatre, web-viewable and immersive augmented reality fictions I'm building, and software development is proceeding iteratively with the creation of these new pieces as we troubleshoot and new features appear on the wish list. In the case of Andromeda, when it came time to submit the Vinaròs Digital Literature contest, I hadn't quite worked out some kinks with the audio; multiple marker recognition was working perfectly, but once the sound associated with a particular marker started to play, it would loop endlessly, making it more like a choral poem. I did have a work-around in mind: making the video clips longer and soundless at the end so they would keep playing but there would be no sound to drown out the sound associated with subsequent markers detected by the camera. But when I compared the two versions of Andromeda that resulted, I actually found that I had grown to like the layered audio effect and I kept it in. Still, you can only have so many pieces like that, so the first new feature added after Andromeda was finished was sound activated through proximity detection; in the current version of SnapDragon, the closer the marker is to the camera, the louder the sound. Markers detected further away are silent or whisper.

Andromeda2

A subsequent version of the poem involved performance and took advantage of the flexibility of fiducials (they can be printed on a regular printer at any size) to explode the first version of the poem outside the confines of the pop-up book. Markers were printed on large sheets of paper that could be shuffled, held, played with and combined and recombined to create new poetic possibilities. There were 41 lexias that made up the content of the poem (each with a granularity of roughly a stanza, 30 seconds of video or spoken word etc.). I held up some markers and taped others to the floor and walls and then walked "through" the poem with an industrial point-grey camera and projected the resulting poem, inhabited in a unique way, on the wall for the audience to experience. In this sense Andromeda2 was a magic mirror AR installation; the audience saw me interacting with the paper symbols and, at the same time, they saw me, via projector, interacting with the same pieces of paper, only on screen they were able to see me alongside still and moving images at multiple scales, stars and fish and roadside diners, and see scrolling textual elements, too:


and he takes her to the room at the top of those inn stairs
amidst stares
all in one breath it's late, dark outside
they undress, get in bed
and finally she's quiet, through with shouldering the weeping
and they are near that
tentative, anxiety-washed pinprick of sleep
the bed cold, hard, the sheets worried,
in their heads already driving the car through to Wales
they've left the hotel, better yet, they've set all clocks back three days and the boat docks in England and his mother is alive, saved as a consequence of the joyful news of a marriage, or else she flies on the back of a bird and brings back presents from the dead. As They sleep.


How does it work?

Computer vision techniques provide a low cost solution for working with this medium. Black and white markers -- sometimes called fiducials -- are used too mark coordinates in a real scene. Think of the markers as a library of symbols that the camera can read. There are many marker systems available but SnapDragon uses the Mfd-5 marker library, a particularly robust tracking system that tracks well even in low light that was developed by our collaborator Dr. Mark Fiala. It is being used exclusively by our lab to create new tools for writers of electronic literature, artists and designers. The software itself was created by Andrew Roth and Andrei Rotenstein, under the direction of Caitlin Fisher and in collaboration Dr. Mark Fiala.

SnapDragon is a stand-alone application built as a plug-in to Max/MSP (but you don't need to have Max/MSP to run it). We can't offer you the Max code, but we create custom interfaces depending on the project or installation we're working on. SnapDragon has the most popular of these features. The full feature version allows you to scale the video in real time, move the video right off the marker or pull video from online sources - enabling you to contextualize your elit wiith current weather reports or news headlines. If you would like to make an augmented reality poem of your own, SnapDragonAR is available for free trial and purchase here: www.futurestories.ca/snapdragon (don't ask why it's not just open source - I wish it could be, but it's a long story. We do have other AR software that is open source). Want to learn more about the software and see some updated documentation? try here: http://www.yorku.ca/caitlin/futurestories/snapdragon










content | code | process
Home







Index


Mark Amerika

Bill Bly

Jay Bushman

J. R. Carpenter
__ The Broadside of a Yarn
__ Entre Ville
__ Chronicles of Pookie and JR
__ STRUTS

M.D. Coverley
__ Egypt: The Book of
Going Forth by Day

__ Tin Towns

Caitlin Fisher

Chris Funkhouser

Dene Grigar
__ 24-Hr. Micro-Elit
__ Fallow Field

Fox Harrell

William Harris

Megan Heyward
__of day, of night
__The Secret Language of Desire

Adriene Jenik

Antoinette LaFarge

Deena Larsen
__ Marble Springs 3.0
__ The Pines at Walden Pond

Judy Malloy

Mark C. Marino

María Mencía

Nick Montfort
__Nick Montfort and
Stephanie Strickland
Sea and Spar Between

Judd Morrissey

Karen O'Rourke

Regina Pinto

Andrew Plotkin
__ Hoist Sail for the Heliopause and Home
__ The Dreamhold

Aaron Reed

Scott Rettberg

Silvia Stoyanova and Ben Johnston
The Zibaldone Hypertext Research Platform

Stephanie Strickland
__Nick Montfort and Stephanie Strickland
Sea and Spar Between

Eugenio Tisselli

Dan Waber




Interviews

Mark Bernstein
__Interview with Mark Bernstein

Sonya Rapoport:
__Interview with
Sonya Rapoport

Stuart Moulthrop
__ Interview with
Stuart Moulthrop