Transfer of Motor Skills mediated through Wearable Technology by means of Haptic Nonverbal Communication

Abstract

This dissertation proposes a haptic nonverbal communication environment, supported by wearable technology, aiming to assist people in transferring motor skills to others. Current challenges in transmitting motor skills from experts to learners include the inherent complexity of certain movements and the difficulties of explaining verbally complex movements, which adds comprehension difficulties for learners. Additionally, traditional motor-skill lessons barely use technology, not for a lack of enthusiasm from experts, but for a lack of adequate devices designed for the task of transmitting motor-skills. The basic concepts and components of nonverbal communication are discussed, with specific emphasis on haptic nonverbal communication (HNVC). Based upon a thorough review of the literature, a taxonomy has been derived that organizes related work that has focused on HNVC assisted by technology during the last five years. The taxonomy provides insights on current gaps in knowledge of utilizing haptic technology for nonverbal communication purposes. The taxonomy?s main categories consider the intent of using HNVC. Applications aiming to convey meaning are classified under interpretive HNVC; those aiming to either identify of promote certain sentiments or moods in users are classified under affective HNVC, whereas applications aiming to produce an action as a result are classified under active HNVC. Each category has sub-categories to further refine the classification. Findings include which technologies are preferred for each category, which haptic approach is common, and future research directions based on each category. Research directions set the foundation for the conceptual proposal of an environment aimed to support transferring of motor skills. The proposed environment achieves human-human interaction mediated by wearable devices through the sense of touch. The environment is a conceptual proposal that comprises elements that are essential to achieve a technology-enhanced interaction in conventional motor-skill instruction settings. The main components of the proposed environment are wearable actuators and sensors, an autonomous movement classifier, a tactile encoder, and a database of reference movements. The interaction flow between the expert and learners is mediated through wearable devices and the autonomous classifier. The flow begins when the expert performs a movement, which the wearable sensors deliver to the autonomous classifier. The autonomous classifier identifies which movement it is, and sends, through the tactile encoder, a tactile-instruction to the learners? wearable actuators. Learners identify the instructions, and perform the movement, delivered to the autonomous classifier through their wearable sensors. The classifier identifies the movement, and provides a quality measurement, comparing the received movement against a reference database. The database contains movements performed by experts, which are the basis to determine movement quality. A step-by-step methodology on how to instantiate the environment is also proposed. As a first target skill, leader-follower dance was chosen. The decision is based on the haptic communication occurring between leader and follower in this dance style, providing a relevant scenario to test a HNVC mediated through wearable devices. Initial user research with experts on leader-follower dance guided the instantiation of our proposed environment, also reflecting on experts? approval of our approach. Three experiments were undertaken to instantiate and validate three essential components of the environment: tactile communication between learners and wearable devices, experts? interaction with wearable sensors to gather movement data, and an autonomous software component capable of classifying signals into movement categories, and providing quantitative quality measurements. The experiments show that the proposed environment has acceptance potential from both experts and learners, as well as competitive results on classifying movement signals. Our research advances knowledge in the following areas: First, in human-computer interaction, by providing alternative interactions through the sense of touch and nonverbal communication. Second, in motor skill transfer assisted by technology, by introducing haptic wearable technologies into an environment that enhances conventional instruction settings, and providing tools for both experts and learners. Third, we advance the application of classification techniques from signals obtained by wearable sensors