The 1999 application, eArthur, was designed as a "think piece", investigating what would be possible in terms of furthering our educational mission, given the new digital technologies then (and still) looming on the horizon. Generally speaking, the technologies that were exploited were

- Multiple audio tracks, defined in the ATSC standard,
- Multiple caption tracks, EIA-708,
- Multiple video streams, As in a "multicast" scenario,
- Non synchronous playback, i.e. Pause - Play - Fwd - Rew via either PVR or VOD,
- Web style Interactivity, ala DASE or something similar.

Given this feature set, a variety of enhancements were developed the reinforce the educational mission of Arthur In Brief. These were:

- A character based "guide": a character who instructs users how to access various functions of the application,
- Multiple language and purpose caption tracks including English, Spanish, simplified English,
- Multiple language audio tracks, English and Spanish, which used in conjunction with the caption tracks allow for teaching either English or Spanish,
- A dynamic glossary function linked from the caption tracks,
- A separate video track containing a signed ASL version of the show,
- A variety of games, including a hugely popular camera based application which allow viewers to insert themselves into the program and dress themselves as an Arthur character.

This demonstration was extremely successful in raising consciousness about what would be possible with the next generation of television technologies, but it was just that, a demo, written in Director designed to simulate rather than to actually utilize these technologies. As part of our ongoing LEC research into interactive television we decided to attempt to implement eArthur in a manner that was actually deployable.

Enter the AFI

The AFI Enhanced TV Workshop pairs producers with technology companies experienced with enhanced television to create working prototypes of next generation of TV programming. This seemed like an excellent way to build out a test version of eArthur and to determine if it was possible to deploy eArthur in the real world, so we applied and some months later were accepted.

Our Partners

Obviously a real world deployment of eArthur would require the latest in high powered set top boxes and new media technology. For platform providers we were paired with Digeo and iVAST. Digeo's set top box product, Media Center, incorporates a Flash based environment coupled with a PVR which seemed like the optimal platform for this trial. iVASTs product is an end to end MPEG-4 system, which also contained the necessary functionality to deploy eArthur.

On the production end we were joined by Emmy award winning production house Beyond-Z and LA video compression heavies Metro Encoding.

The Production Process

Because there were two platforms, we pursued two parallel production courses. The flash based Digeo implementation would be handled by Beyond-z while the MPEG-4 version would be handled by iVAST. The first task was to redesign parts of the application which had proved problematic in the initial deployment and would be common to both platforms.

Design Issues

The first problems we addressed were interface issues revealed in consumer testing. The original interface to the enhanced content consisted of a round watch graphic which users navigated around to select a given enhancement. Unfortunately, in an ITV environment in which the user has a remote control with four arrows and a select button, navigating this interface was both difficult and confusing. This interface was replaced by a more simple column arrangement.



Another problem was the lack of a consistent navigation between the show and the enhancements. This was solved by creating two icons, one featuring Nora the guide character which consistently led to the enhancements within the show, and one featuring Arthur himself which consistently led back to the show.

Technology Issues

The next task was to determine which functionality was supported by the various platforms. It should be noted that we were trying to duplicate a think piece. The original eArthur was based on predicted or specified, but not implemented, functionality of next generation systems. This is obviously quite different from a normal production environment where technical parameters are known and therefore drive content design.

The best parallel which can be drawn to this situation would be NASA trying to build the Millennium Falcon. The specs are known, there are even blueprints, but exactly how one builds a warp drive is still to be determined. Accordingly there were a number of situations in which the original demo specified functionality which could not be realistically duplicated.

For the basic animation sections of the enhancements there were little problems. Flash 5 compliant versions of the animations were created for the Digeo platform, while MPEG-4 versions were created with the iVAST MPEG-4 authoring tool for the iVAST platform. However technical challenges began to appear in three areas integrating the various enhance caption functions, video overlay and the camera.

The original eArthur application featured highlight text for read along and glossary functions. This was based on an interpretation of the specified functionality contained within the EIA-708 caption standard.



In a number of places in the original demo, the video stops and the guide character appears on top of the image, creating a situation where moving video is played with a transparency layer on top of paused video.

The camera feature was the killer ap of the original demo. Although this functionality was theoretically supported by both platforms, there was difficulty in implementation

Implementation 1 - Digeo

In spite of best efforts on the part of Digeo, and Digeo's parent company Vulcan, we were never able to obtain actual Media Center hardware to develop applications on. It should be noted however that the conduct of our Digeo contacts bordered on the heroic. In an industry too often typified by apathetic customer support, Digeo/Vulcan went so far as to assign a senior programmer to the project who was tasked to make the hardware and software work with our project.

The Digeo platform handled captioning by simply passing through standard 608 captions, which for our purposes, wasn't adequate. However a much more interesting possibility presented itself in it's integrated news ticker. Our plan was to utilize this functionality to present the enhanced caption track in addition to the standard 608 captions. However this would have required either passing caption info from the MPEG stream to the ticker app or drawing the caption information from a separate source and then synchronizing it with the video.

The Digeo platform specified an extremely advanced graphics engine for rendering Flash content. However while it could render Flash on top of video, rendering video on top of video with a transparency channel proved to be to much to ask of it. In all fairness, this type of functionality in this context, or really in any context outside of a high end video post facility, borders on the extreme if not the bizarre. There were two suggested fixes to this, swapping a graphic for the background video layer upon pausing or creating a flash version of the video for the overlay.

The Digeo platform specifies a camera for use in video telephony. We had hoped that this would be a slam dunk for this project, but unfortunately it was not. Implementing the camera for use as a frame grabber in our application was not possible. This was due to the fact that our application required flash compliant files, and files generated from the camera would need to be transcoded requiring a great deal of additional engineering.

Implementation 2 - iVAST

Captioning in the iVAST platform was considerably simpler, as was multiple audio tracks and video. MPEG-4 has built in support for multiple audio, video and caption tracks, so much of this functionality worked "out of the box". How ever while we were able to synchronize glossary words with sections of the show, we were unable to highlight them within the caption tracks.



The issue of video on top of video was also problematic within the iVAST platform. While technically it could support it, the level of CPU power required to do it properly bordered on the insane (think Cray). In this case we used the approach of swapping a graphic for the background video layer upon pausing.

Camera functionality, while also possible within the MPEG-4 spec proved problematic on the iVAST platform as well. However this was a case in which the creative team provided for us by the AFI saved the day. The original implementation by iVAST featured an icon function where the user could choose an Arthur character to represent them in the Arthur world.

Extending on this a bit further, we thought what if the user could create an Arthur style character that looked like them? The end result was a character construction kit which allows users to create a character which resembles them by allowing them to choose head type, complexion, hair style and hair color. We had planned to include eye color, but then discovered that Arthur characters don't have eye color. Oooops...



However the piece d' resistance of the application was the video puzzle built for us by iVAST. The puzzle application consisted of a 3 by 3 grid with a moving video picture split into 9 pieces. Users have to try to assemble the video picture into the correct order which is much harder than it looks. This was both an extremely entertaining addition to the eArthur games and an impressive technical achievement.

Conclusions 1 - The AFI, Our Partners

First and foremost the AFI ETV workshop was an incredible experience. Not only was our team excellent, but we were afforded the opportunity to meet, and talk shop, with many of the major players in next generation television. We recommend it for anyone wanting to develop an interactive television application.

We were extremely pleased with the application in its finished state. iVAST is a very good tool, particularly since at the time of this project it was at a very early stage of development. MPEG-4 is a very good environment for developing advanced television applications, with the only downside being the large amount of computing power required. This problem is likely to be solved by Moore's law in rather short order.

Digeo, even with significant delays in deployment, with roll out expected on the Charter system in mid 2003, is still poised as the leader in the next generation of ITV platforms. A developer program is expected to appear around the same time and we (and many others) can't wait to get our hands on their hardware and software.

Additionally, mention should be made of the teams at Beyond-z and Metro Encoding, both of whom performed well above the call of duty. Of particular note should be the fact that all the teams made significant contributions in areas that were not necessarily their specialty, making this venture a truly collaborative enterprise.

Conclusions 2 - What is the State of the Art?

While many of the problems we encountered were the result of trying to duplicate the functionality in what was a deliberately a "kitchen sink" demo (as in everything but), there were still some basic issues that need to be addressed to enable truly advanced functionality.

In the course of our research as both AFI participants and members of the LEC, we have become convinced that a platform capable of truly advanced functionality will require tight integration with either a PVR or VOD system. This will require two additions to standard ITV functionality. The first would be development of a standardized set of commands for controlling basic PVR and VOD functions such as play pause etc, and a methodology for accessing and referencing time code in the content housed on these systems.

The second requirement would be a standardized metadata scheme which describes scene information and possibly enhancement data. These functions could be implemented as additional fields within existing EPG schemes, pointing an advanced set top box to the location of the metadata and enhancements. There are currently standardization efforts around this by among others, the cable end consumer electronics industries.

Another element vital to the creation of truly advanced ITV applications is a full featured declarative environment. Because our bias is towards implementation of ITV smaller operations, notably public television stations, we feel that a declarative environment, such as HTML or XHTML offers a much greater possibility for development by these entities. Full plug in support, to allow for extension of the capability of the unit, would also be necessary to deploy more advanced applications such as eArthur.

This declarative environment would require a fairly powerful set top box, but these devices are beginning to emerge from the consumer electronics and computer industries as "home media centers" or "Digital Media Libraries". Additionally, a simple development environment would allow for a wide variety of smaller operations, outside of the major broadcasting concerns, to participate in this development as well, creating a wide range of new content, and accordingly demand.

Finally, it is our belief that this environment be open source which would once again allow for development by entities outside of the major broadcasting concerns.

Original eArthur application produced by WGBH Interactive under a grant from the Corporation for Public Broadcasting. © 1999 WGBH Educational Foundation. All rights reserved.

Modified eArthur application produced by WGBH Interactive in cooperation with the AFI Enhanced television workshop. © 2002 WGBH Educational Foundation. All rights reserved.

Arthur is produced by WGBH Boston and CINAR. © 2002 WGBH Educational Foundation. All rights reserved.

All characters and underlying materials (including artwork) copyright by Marc Brown. Arthur, D.W., and the other Marc Brown characters are trademarks of Marc Brown.

Watch Arthur on PBS KIDS.

Revised Monday, 14-Apr-2003 15:04:02 CDT - h

© 2000 - 2003 Local Enhancement Collaborative & .