> Semantic Web Use Cases and Case
Studies 
Software has found its way into the classroom in many different guises, from the interactive whiteboards used by teachers to the productivity applications and courseware used by students. But some types of pedagogy are a harder fit for technology than others. Contemporary Dance is taught with heavy dependence on kinaesthetic and experiential learning (learning by doing and reflecting). Each dance student acquires performance and choreography skills through practice, while strengthening their skills to assess personal and group performance by keeping a reflective diary. Although video recording technology is used for performance capture and review, no specific software tools exist to filter or analyse the extensive video collections created for each dance project, and as such they are underutilised. This document describes how the Semantic Web can assist dancers, by tagging their video collections such that specific segments of choreography from across a dance project’s lifecycle may be easily found, compared, and cited in support of a dancer’s reflective diary entries.
The work described herein took place as part of a major research project funded under the UK’s Technology Enhanced Learning Programme, ending in 2011. The Ensemble Project explored the potential role of Semantic Web technologies in diverse areas of higher and early professional education. The team worked closely with a range of educators in many settings, using workshops and rapid prototyping to develop Semantic Web sites supporting and evolving their teaching practice. Settings were carefully chosen to provide a variety of radically different pedagogical approaches; examples being Archaeology at a long-established research-leaning university and Contemporary Dance at a modern skills-orientated university. The team treated the broad scope and timescale of the project as license to change the game, finding new and unusual ways of using semantically linked data to meet the needs of each setting.
SIMILE Exhibit, an open source framework originating at MIT’s Computer Science and Artificial Intelligence Laboratory, eventually became Ensemble’s primary tool of choice for building Semantic Web pages. Exhibit provides a lightweight client-side JavaScript framework that lowers the barrier to entry for Semantic Web sites by allowing data owners, without programming experience, to transform their data into rich Semantic Web applications featuring sophisticated visualisation components, such as tables, timelines and maps. The team found Exhibit’s simple lightweight approach fitted well with the rapid prototyping methods they had adopted. Although initially the Ensemble team developed regular Exhibit pages, eventually they began to compliment and extend the standard Exhibit codebase with their own new components.
The video component was one such innovation, and its initial spark came about in one of the most unlikely of settings: an exchange with two Contemporary Dance students during a brainstorming session. Contemporary Dance and the Semantic Web may seem like odd bedfellows, yet a key requirement within the discipline is the ability to analyse how a project has evolved during its lifecycle, in terms of its choreography, the personal development of individual dance students, and their role in a team. Students keep reflective diaries that form a central part of their assessment; they employ formal vocabularies describing movement and style, supplemented with informal vocabularies specific to a given choreography. By tracking these vocabularies across the lifecycle of a project the Ensemble team had planned to create a Semantic Web resource useful to the reflective process, but an alternative possibility opened up when it was suggested that this semantic data might somehow be mapped to the video recordings taken of every rehearsal, team discussion and performance.
In Contemporary Dance the choreography evolves as part of a process of experimentation and rehearsal. Recordings are kept of every rehearsal, group discussion and performance, with video sometimes shot from multiple angles. These recordings are useful in an immediate sense because they allow dancers to analyse current choreography and feed the results back into the creative process. But, more crucially, the recordings are also cited in each dancer’s reflective diary as evidence of their personal development as a performer. But with each project generating considerable quantities of video footage, students often have trouble locating the clips that support their arguments; and even when they can locate the required clips, they have limited ways of submitting them as part of their assessment (the more enthusiastic students attached DVDs to their work).
The Ensemble team found that the dancers used standard vocabularies, such as Laban Movement Analysis (LMA), to describe generic actions; these formal terms were coupled with a choreography-specific vocabulary to describe particular novel dance movements or sections within a specific choreography. The Ensemble team posited that these terms could be applied to the video footage, creating semantic data that would aid the dancers to, for example, locate all the clips from a given rehearsal week or featuring a particular type of movement.
The dancers began to upload their videos to the Ensemble project web site, while the Ensemble team coded a web tool to allow dancers to quickly tag sections of each video with semantic data. But it soon became apparent even one dance project contained more video footage than the project server could comfortably hold, so a choice was made to switch video hosting to YouTube, using the privacy settings to hide videos from public search. YouTube not only provided a stable video hosting platform, but its popularity and ubiquity meant at some point in the future students might be able to upload their own personal clips (captured directly from their phones) to their YouTube channels, and cite them alongside the standard dance studio footage.
Figure 1: Adding semantic data to video with the Resource/Clip Manager
The tool developed by Ensemble took the form of a web application split into two main parts. The first part, the Resource/Clip Manager, allowed YouTube videos to be imported and semantic tagging applied – where “imported” means referenced by their YouTube identifier. Once imported, each video could be dragged and dropped into a video player, viewed, and semantic data entered using a form interface. As the video played, multiple clips (references to the original video using start and end timings) could be created, and clip-specific semantic data entered.
The dance students and their tutors decided for themselves what types of data needed to be collected for videos and clips, and the Ensemble team constructed an appropriate user interface. The semantic data the students and tutors settled on for each video contained a mixture of the usual metadata (project title, recording date,…) combined with the performance type (rehearsal, performance,…) and venue. Each clip inherited all this information by means of its link to its source video, but further extended it with dance vocabulary terms and spatial zones (the areas of the performance space being used).
Two dance students were selected to take the role of administrators, in charge of the clipping and tagging process. Thanks to their efforts, a collection of fully described video resources and clips slowly built up. This data was then ready to be used in the second part of the tool: the Reflection Manager.
Figure 2: Editing a new reflective diary entry, while comparing two clips
The Reflection Manager was where the majority of users would spend their time; it was intended to replace the paper based reflective diaries. Each diary entry consisted of a title and a main text body, accompanied by supporting video and clip citations.
Whereas previously a dancer would have to scour through countless hours of recordings to find the specific moments supporting their analysis, now the power of Semantic Web faceted browsing (courtesy of SIMILE Exhibit and the data the two admin students had diligently applied to each video) meant that navigating the videos and clips to home in on a particular segment was a matter of just a few clicks.
Two video players were included on the page to allow videos or video clips to be viewed side by side; the dancers were particularly keen on having this comparison feature. At the click of a button, videos or clips could be added to the citation list of the current reflective diary entry. Once the student was happy with their diary entry, the application could create a link to a standalone page featuring the entry text, all the video citations, and one or more video players. This standalone link may be sent to the dance tutor to be considered as part of the dancer’s assessment, or shared with other web users.
In 2011 a group of students volunteered to try out the new system, submitting their reflective diaries only through the online system, rather than on paper. The Ensemble team spoke to them afterwards to gauge their reaction to their new Semantic Web application.
While software technologies have long played a role in the classroom, it was clear software has struggled to make an impact in the dance studio. So the students welcomed their new software tool, particularly the ability to easily locate videos and compare them side-by-side. They were particularly interested in better ways to bring their own video into the system, and making the system more collaborative. Some of the students admitted to using their smart phones to capture ideas when outside the studio, and wanted a way to share their personal clips with fellow performers in their dance team. They also wanted to enhance the online reflective diary to permit team members to add feedback to an entry, extending the personal record to capture some of the interactions and ideas within their dance team as a whole.
One of the issues that did emerge from the administrators was the laborious process of tagging up each video and clip manually. Other semantic technologies dealing with video have resorted to using natural language processing on the subtitle track to extract semantic information programmatically, but software cannot currently identify choreography from the free movement of several dancers within a video; even if it could, the Contemporary Dance students would still need to train the system to correctly identify project specific choreography. So there seemed no easy way to introduce automation into the tagging process. After some discussion with the volunteers it was suggested a more Web 2.0/crowd sourced solution might be worth exploring. Instead of a formal process of dividing and tagging each video in the Resource Manager before it could be used in the Reflection Manager, every student using the Reflection Manager could be given the videos with minimal tagging (from the YouTube metadata) and ad hoc tag and clip the videos themselves. The students seems to consider this idea worthy of testing, not least because they wanted the freedom to segment the videos into clips as seemed appropriate for themselves and their dance team, rather than have a clip structure imposed upon them by the administrators.
This was only part of the work the Ensemble team did with the Semantic Web and video; another offshoot involved integrating video as a first class citizen in the SIMILE Exhibit framework, by developing an animated facet that allowed page contents to automatically update based upon their semantic relationship to the current segment of a video. But in essence this work was just another variation of the power of applying semantic tagging to video.
Each dance project generates a large quantity of separate video recordings, but because these recordings are about the development of a single piece of choreography, the collection as a whole contains within it strong semantic connections. The Semantic Web, and specifically faceted browsing, was able to expose these connections to dance students in a way that was easy to view and manipulate. In doing so, the Semantic Web has afforded an opportunity to greatly enhance a key part of the pedagogy in Contemporary Dance, namely the reflective accounts of personal development each dancer is required to keep as part of their course assessment.
The Ensemble project ended in October 2011, but the team has already found some follow on funding to take the dance application and make it generic, meaning it can be configured to collect semantic data on any topic (not specifically Contemporary Dance). In the summer of 2012 a student researcher at Liverpool John Moore’s University’s Faculty of Education, Community and Leisure used the Resource Manager part of the tool to collate and describe a semantic collection of videos for use in teacher training, and it has been suggested the Reflection Manager might be used with student teachers as a way of capturing their engagement with the video resources. Meanwhile the Contemporary Dance tutors are keen to find further substantial funding to continue to explore the potential of their new tool.
© Copyright 2012, LJMU.