Deakin University
Browse

File(s) under permanent embargo

Mechanisms for multimodality : taking fiction to another dimension

conference contribution
posted on 2007-01-01, 00:00 authored by K Glass, Shaun BangayShaun Bangay, B Alcock
We present methods for automatically constructing representations of fiction books in a range of modalities: audibly, graphically and as 3D virtual environments. The correspondence between the sequential ordering of events against the order of events presented in the text is used to correctly resolve the dynamic interactions for each representation. Synthesised audio created from the fiction text is used to calibrate the base time-line against which the other forms of media are correctly aligned. The audio stream is based on speech synthesis using the text of the book, and is enhanced using distinct voices for the different characters in a book. Sound effects are included automatically. The graphical representation represents the text (as subtitles), identifies active characters and provides visual feedback of the content of the story. Dynamic virtual environments conform to the constraints implied by the story, and are used as a source of further visual content. These representations are all aligned to a common time-line, and combined using sequencing facilities to provide a multimodal version of the original text.

History

Event

International conference on computer graphics, virtual reality, visualisation and interaction in Africa (5th : 2007 : Grahamstown, South Africa)

Pagination

135 - 144

Publisher

Association for Computing Machinery

Location

Grahamstown, South Africa

Place of publication

New York, N.Y.

Start date

2007-10-29

End date

2007-10-31

ISBN-13

9781595939067

Language

eng

Publication classification

E1.1 Full written paper - refereed

Copyright notice

2007, Association for Computing Machinery

Editor/Contributor(s)

S Spencer

Title of proceedings

Afrigraph '07 : proceedings of the 5th international conference on computer graphics, virtual reality, visualisation and interaction in Africa

Usage metrics

    Research Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC