Video Data and Linguistic Corpora

Computer screen showing sign language videos with a timeline of the words in the video.

ASL Motion-Capture Corpus of American Sign Language

Linguistically annotated corpus of video and motion-capture recordings of American Sign Language...

Video of a signer asking a comprehension question.

Stimuli and Questions for Evaluating Animations

Tools for evaluating perception and understanding of facial expressions in ASL animations or videos...

Cyberglove in front of computer screen showing the hand shape.

Accessible Cyberglove Calibration Protocol

Protocol for accurately and efficiently calibrating motion-capture gloves with deaf participants...

Computer screen showing sign language videos with a timeline of the words in the video.

ASL Motion-Capture Corpus: First Release

The ASL Motion-Capture Corpus is the result of a multi-year project to collect, annotate, and analyze an ASL motion-capture corpus of multi-sentential discourse. At this time, we are ready to release to the research community the first sub-portion of our corpus that has been checked for quality. The corpus consists of unscripted, single-signer, multi-sentence ASL passages that were the result of various prompting strategies that were designed to encourage signers to use pronominal spatial reference yet minimize the use of classifier predicates. The annotation of the corpus includes glosses for each sign, an English translation of each passage, and details about the establishment and use of pronominal spatial reference points in space. Using this data, we are seeking computational models of the referential use of signing space and of spatially inflected verb forms for use in American Sign Language (ASL) animations, which have accessibility applications for deaf users.

How to Obtain the Files

Please send email to matt (dot) huenerfauth (at) rit.edu to inquire about accessing the corpus.

Examples of the Data

Examples of excerpts of the data contained in the corpus may be available by request. Please send email to matt (dot) huenerfauth (at) rit.edu to request access.

What format of files do we release?

The corpus consists of four types of files, for each story that we have recorded.

  • Gloss text file, with start and end keyframe number for each gloss
  • Gloss text file for the non-dominant hand (only available for some videos), with start and end keyframe number for each gloss
  • Referents list text file that identifies the entities that have been established in the signing space during this story
  • English translation text file
  • BVH file - motion capture data in a commonly distributed file format
  • FBX file - motion capture data in MotionBuilder format, as originally recorded at our lab
  • Videos in MOV format: front, side, face views

How many stories and signers are included in this release?

This first release of the corpus consists of data collected from 3 signers, a total of 98 stories. Each story is generally 30 seconds to 4 minutes in length.

Citations and More Information

If you make use of this corpus, please cite the following publication:

Pengfei Lu, Matt Huenerfauth. 2012. "CUNY American Sign Language Motion-Capture Corpus: First Release." Proceedings of the 5th Workshop on the Representation and Processing of Sign Languages: Interactions between Corpus and Lexicon, The 8th International Conference on Language Resources and Evaluation (LREC 2012), Istanbul, Turkey.

Funding Support

This material is based upon work supported in part by the National Science Foundation under award number 0746556.

Video of a signer asking a comprehension question.

Experimental Stimuli and Questions for Evaluating Facial Expressions in Animations of American Sign Language

We have developed a collection of stimuli (with accompanying comprehension questions and subjective-evaluation questions) that can be used to evaluate the perception and understanding of facial expressions in ASL animations or videos. The stimuli have been designed as part of our laboratory's on-going research on synthesizing ASL facial expressions such as Topic, Negation, Yes/No Questions, WH-questions, and RH-questions.

How to Obtain the Files

Please send email to matt.huenerfauth at rit.edu to inquire about accessing the corpus.

Examples of the Data

Examples of excerpts of the data contained in the corpus may be available by request. Please send email to matt.huenerfauth at rit.edu to request access.

Funding Support

This material is based upon work supported in part by the National Science Foundation under award number 1065013.

What format of files do we release?

The corpus consists of four types of files, for each story that we have recorded.

  • Tab-delimited TXT text file with the glosses for each stimulus, English translation of the stimulus (when the facial expression is conveyed), English translation of the stimulus (for the alternative interpretation of the stimulus when the facial expression is not conveyed), a list of four comprehension for each stimulus, and answers to each comprehension question.
  • HTML files containing multiple choice Likert-scale questions to evaluate each animation.
  • MOV files: Videos of a male signer performing each stimulus passage.
  • MOV files: Videos of the same male signer performing comprehension questions (four per stimulus passage).
  • MOV files: Videos of a female signer performing each of the comprehension questions. (It may be desirable for a different signer to ask the questions.)
  • MOV files: Videos of a male signer and a female signer performing a brief introductory message that could be used at the beginning of a study to convey instructions for participants.
  • Comma-Separated TXT text files containing MPEG4 Facial Action Parameter values for the male signer performing each of the stimuli passages.

How many stories and signers are included in this collection?

This collection consists of 48 stimulus passages, performed by a male signer. Each stimulus is accompanied by four comprehension questions. Each comprehension question is performed by both a male signer (the same one performing the stimulus passage) and a female signer.

Citations and More Information

If you make use of this collection, please cite the following publication:

Matt Huenerfauth, Hernisa Kacorri. 2014. "Release of Experimental Stimuli and Questions for Evaluating Facial Expressions in Animations of American Sign Language." Proceedings of the 6th Workshop on the Representation and Processing of Sign Languages: Beyond the Manual Channel, The 9th International Conference on Language Resources and Evaluation (LREC 2014), Reykjavik, Iceland.

Cyberglove in front of computer screen showing the hand shape.

Cyberglove Calibration Protocol

We have created a website demonstrating how we calibrate the cybergloves for experiments at the lab, using a protocol we developed. It is discussed in a paper presented at the ASSETS 2009 conference and in a journal article in TACCESS in 2010.

This protocol is designed to be an efficient approach to producing a high-quality human calibration of the gloves, and it has been designed to be accessible to deaf research participants.

The accuracy of the calibrations produced using this protocol has been experimentally measured and shown to be more accurate than the standard automatic calibration software that accompanies the gloves.

Website with Videos for Calibration Process

We have created a website with videos and other materials for researchers who wish to replicate our calibration process.

Citations and More Information

If you make use of this collection, please cite the following publication:

Matt Huenerfauth and Pengfei Lu. 2010. "Accurate and Accessible Motion-Capture Glove Calibration for Sign Language Data Collection." ACM Transactions on Accessible Compututing. Volume 3, Issue 1, Article 2 (September 2010), 32 pages. DOI=10.1145/1838562.1838564 http://doi.acm.org/10.1145/1838562.1838564

Want to participate?