情報技術を用いて人と寄り添う社会を作る

Global Communication

グローバル社会で一番の課題となる【言語の違い】を情報技術を駆使してクリアする。
「何処でも」「誰とでも」対等かつ円滑にコミュニケーションが取れるICTツールの開発を目指しています。

Although global teams have a high potential for innovation and creativity, they are confronted with various difficulties such as coordination problems, lower trust and cohesion, as well as unhealthy subgroup dynamics. Language barrier is one of the major factors that cause these problems. In this project, we develop technologies to overcome the language barrier, and understand how technologies affect multilingual communication, teamwork and trust.

Title: Beyond Lingua Franca: System-Facilitated Language Switching Diversifies Participation in Multiparty Multilingual Communication
Authors: Mei-Ling Chen, Naomi Yamashita, Hao-Chuan Wang
Abstract: When multiple When multiple non-native speakers (NNSs) who share the same native language join a group discussion with native speakers (NSs) of the common language used in the discussion, they sometimes switch back and forth between common language and their native language to reach common ground. However, such code-switching makes others feel excluded and thus not considered appropriate during formal meetings. To offer NNSs more flexibility to code-switch in a group discussion while minimizing the cost of excluding others, we introduced a language support tool that automatically detects between two pre-defined spoken languages, and then transcribe as well as translate them into another language (common language or NNS’s native language). In a within-subject study involving 19 quads (two Japanese and two Chinese) in a collocated setting, participants were asked to perform a series of decision-making tasks with and without the tool. Results showed that the language support tool encouraged diverse use of language during a meeting, resulting in more equal participation from each group member. Although the perceived quality of collaboration became lower, it also elicited helping behaviors among the NNS pairs.

Title: Why Did They Do That? Exploring Attribution Mismatches Between Native and Non-Native Speakers Using Videoconferencing
Authors: Helen Ai He, Naomi Yamashita, Ari Hautasaari, Xun Cao, Elaine M. Huang
Abstract: The meaning we attribute to another’s actions significantly impact our subsequent behaviors and interactions towards that person. Distributed teams often combine native speakers (NS) and non-native speakers (NNS) and are particularly prone to making attribution errors. Language difficulties place NNS under a higher cognitive load, potentially leading NS to make inaccurate attributions of NNS. We conducted an exploratory laboratory study to investigate the attributions NS and NNS form about each other in multiparty videoconferencing. Our findings revealed significant mismatches in NS’ attributions of NNS behavior, but no significant mismatch in NNS’ attributions of NS behavior. Due to cognitive overload stemming from language challenges, NNS were only able to engage in “compromised” impression management during the task. Yet, NS were relatively unaware of how profoundly language difficulties impacted NNS’ behaviors. Our findings identify opportunities for technology support for NS-NNS interactions, particularly with regards to impression construction and impression management.

Title: Improving multilingual collaboration by displaying how non-native speakers use automated transcripts and bilingual dictionaries
Authors: Ge Gao, Naomi Yamashita, Ari Hautasaari, Susan Fussell
Abstract: Conversational grounding, or establishing mutual knowledge that messages have been understood as intended, can be difficult to achieve when some conversational participants are using a non-native language. These difficulties in grounding can be challenging for native speakers to detect. In this paper, we examine the value of signaling potential grounding problems to native speakers (NS) by displaying how non-native speakers (NNS) use automated transcripts and bilingual dictionaries. We conducted a laboratory experiment in which NS and NNS of English collaborated via audio conferencing on a map navigation task. Triads of one NS guider, one NS follower, and one NNS follower performed the task using one of three awareness displays: (a) a no awareness display that showed only the automated transcripts, (b) a general awareness display that showed whether each follower was reading the automated transcripts and/or translating a word; or (c) a detailed awareness display that showed which line of the transcripts a follower was reading and/or which words he/she was translating. NS guiders and NNS followers collaborated most successfully with the detailed awareness display, while NS guiders and NS followers performed equally across conditions. Our findings suggest several ways to improve systems to support multilingual collaboration.

Title: Effects of Machine Translation on Collaborative Work
Authors: Naomi Yamashita, Toru Ishida
Even though multilingual communities that use machine translation to overcome language barriers are increasing, we still lack a complete understanding of how machine translation affects communication. In this study, eight pairs from three different language communities?China, Korea, and Japan?worked on referential tasks in their shared second language (English) and in their native languages using a machine translation embedded chat system. Drawing upon prior research, we predicted differences in conversational efficiency and content, and in the shortening of referring expressions over trials. Quantitative results combined with interview data show that lexical entrainment was disrupted in machine translation-mediated communication because echoing is disrupted by asymmetries in machine translations. In addition, the process of shortening referring expressions is also disrupted because the translations do not translate the same terms consistently throughout the conversation. To support natural referring behavior in machine translation-mediated communication, we need to resolve asymmetries and inconsistencies caused by machine translations.

Mental Healthcare

家庭内介護(医療行為ではなく、平穏な日常生活を送ること)を改善することにより、患者の症状悪化を予防し早期回復を助けるICTツールを開発することを目指しています。
2017年までの研究成果をもとに家庭内介護支援アプリ「みまもメイト」を開発し、現在、NPO法人COMHBOにて会員限定で一般公開しています。

The demand for informal caregiving is increasing at a rapid speed, which is becoming an ever more important concern for our society. Such changes are increasing the importance of informal home care. This project aims to design a technology that helps family caregivers improve their care and communication with their care recipients at home. We specifically focus on helping family caregivers cope better with a family member suffering from mental illness such as depression because they typically seclude themselves at home, which makes informal care particularly critical.

Title: How Information Sharing about Care Recipients by Family Caregivers Impacts Family Communication
Authors: Naomi Yamashita, Hideaki Kuzuoka, Takashi Kudo, Keiji Hirata, Eiji Aramaki,Kazuki Hattori
Abstract: Previous research has shown that tracking technologies have the potential to help family caregivers optimize their coping strategies and improve their relationships with care recipients. In this paper, we explore how sharing the tracked data (i.e., caregiving journals and patient’s conditions) with other family caregivers affects home care and family communication. Although previous works suggested that family caregivers may benefit from reading the records of others, sharing patients’ private information might fuel negative feelings of surveillance and violation of trust for care recipients. To address this research question, we added a sharing feature to the previously developed tracking tool and deployed it for six weeks in the homes of 15 family caregivers who were caring for a depressed family member. Our findings show how the sharing feature attracted the attention of care recipients and helped the family caregivers discuss sensitive issues with care recipients.

Title: Changing Moods: How Manual Tracking by Family Caregivers Improves Caring and Family Communication
Authors: Naomi Yamashita1, Hideaki Kuzuoka,Keiji Hirata,Takashi Kudo,Eiji Aramaki,Kazuki Hattori
Abstract: Previous research on healthcare technologies has shown how health tracking promotes desired behavior changes and effective health management. However, little is known about how the family caregivers’ use of tracking technologies impacts the patient-caregiver relationship in the home. In this paper, we explore how health-tracking technologies could be designed to support family caregivers cope better with a depressed family member. Based on an interview study, we designed a simple tracking tool called Family Mood and Care Tracker (FMCT) and deployed it for six weeks in the homes of 14 family caregivers who were caring for a depressed family member. FMCT is a tracking tool designed specifically for family caregivers to record their caregiving activities and patient’s conditions. Our findings demonstrate how caregivers used it to better understand the illness and cope with depressed family members. We also show how our tool improves family communication, despite the initial concerns about patient-caregiver conflicts.

Title: Identifying support opportunities for foreign students: Disentangling language and non-language problems among a unique population
Authors: Jack Jamieson, Naomi Yamashita, Jeffrey Boase
Abstract: This study investigates how foreign students address language-related and other problems as a means of identifying opportunities to support them with language and social technologies. We identify support opportunities by dis-tinguishing between different types of problems ? e.g. whether they are lan-guage-related and whether they involve essential activities in their lives or at school ? and the extent to which support already exists. Our unique sample of 15 foreign graduate students who live in Japan but study in English helped us disentangle problems relating to language skills versus those relat-ing to other challenges. We examine these issues using a multi-method ap-proach where students used a mobile app to record experiences and interac-tions over five weeks, and then discussed this data during an in-depth inter-view. We use our results to identify specific support opportunities that can be addressed through the development of social and language technologies.

Title: Understanding the Conflicting Demands of Family Caregivers Caring for Depressed Family Members
Authors: Naomi Yamashita, Hideaki Kuzuoka, Keiji Hirata, Takashi Kudo
Abstract: Depression is one of the most common disabilities in developed countries. Despite its often devastating impact on families, scant research has focused on how to facilitate the well-being of family caregivers. The aim of this paper is to uncover the challenges faced by family caregivers and support their well-being with the use of technologies. To understand the emotional and social burden of caregivers and how they handle their stress, we conducted in-depth interviews with 15 individuals who have cared for a depressed family member. Our findings reveal the multifaceted dilemma of caring for a depressed family member as well as the various strategies engaged in by caregivers to improve their own situations. Based on our findings, we suggest design implications for healthcare technologies to improve the wellness of caregivers who are looking after depressed family members.

Co-investigator Projects

私たちは多様な文脈・場・環境に合ったコミュニケーション支援技術の試作と評価を繰り返すことにより、
互いの気遣いを生むコミュニケーションをデザインするための基礎理論を構築することを目指します。

●科研費 基盤研究(B)「連鎖的相互行為を支援する超時空間遠隔対話システムの研究開発」(研究代表者:葛岡英明),研究分担者,2017-2019.
メディアを介した遠隔対話において連鎖的な相互行為を支援することを目的とします。
空間を圧縮した撮影・表示形態である全周映像を利用した通信端末と、予期的アウェアネス提示と遅延提示という時間操作を導入した、超時空間遠隔対話環境を提案します。

代表的な研究成果:

  • Zhengqing Li, Shio Miyafuji, Toshiki Sato, Hideki Koike, Naomi Yamashita, Hideaki Kuzuoka , “How Display Shapes Affect 360-Degree Panoramic Video Communication,” Proceedings of ACM Conference on Designing Interactive Systems (DIS’18), to appear.
  • Christian Licoppe, Paul Luff, Christian Heath, Hideaki Kuzuoka, Naomi Yamashita, Sylvaine Tuncer, ” Showing Objects: holding and manipulating artefacts in video-mediated collaborative settings,” Proceedings of ACM Conference on Human Factors in Computing Systems (CHI’17), pp. 5295-5306.

●科研費 基盤研究(B)「ソーシャルテレプレゼンスのためのロボットエンハンスドディスプレイ」(研究代表者:中西英之),研究分担者,2014-2017.
日本の街中において実際に実店舗へのロボットの導入を行いながら、世界最高レベルの接客能力を有するロボットの実現を目指します。

代表的な研究成果:

  • Hideyuki Nakanishi, Kazuaki Tanaka, Ryoji Kato, Xing Geng, Naomi Yamashita, “Robotic Table and Bench Enhance Mirror Type Social Telepresence,” Proceedings of ACM Conference on Designing Interactive Systems (DIS’17), pp.779-790.
  • Kazuaki Tanaka, Naomi Yamashita, Hideyuki Nakanishi, Hiroshi Ishiguro, “Teleoperated or Autonomous?: How to Produce a Robot Operator’s Pseudo Presence in HRI,” Proceedings of ACM Conference on Human Robot Interaction (HRI’16), pp.133-140.
  • 田中一晶,山下直美, 中西英之, 石黒浩, “自律・遠隔操作の曖昧化によるロボット操作者との対話感覚の創出,” 情報処理学会論文誌,Vol.57, No.4, pp.1108-1115, 2016.

●過去の主要なプロジェクト
「自他の作業空間を区別なく動き回れる実写ベースの遠隔ビデオ協調作業支援の研究」(研究代表者:平田圭二)

遠隔地間で動き回る必要性のある遠隔医療や遠隔教育といった場において、実対面環境に近い円滑で質の高い協調作業を実現することを目指します。私たちは、遠隔地間において視聴覚情報の対称性が保証される(つまり、遠隔ユーザが対等に互いの周辺視聴覚情報を共有できる)システム「t-Room」を開発しました。これを用いて複数の統制実験を行い、対称性が遠隔協調作業に及ぼす影響などについて調査しました。

代表的な研究成果:

  • Paul Luff, Naomi Yamashita, Hideaki Kuzuoka, Christian Heath, ” Flexible Ecologies And Incongruent Locations,” Proceedings of ACM Conference on Human Factors in Computing Systems (CHI’15), pp. 877-886. (Honorable Mention Award )
  • Naomi Yamashita, Hideaki Kuzuoka, Keiji Hirata, Shigemi Aoyagi, Yoshinari Shirai “Supporting Fluid Tabletop Collaboration across Distances,” Proceedings of ACM Conference on Human Factors in Computing Systems (CHI’11), pp. 2827-2836, 2011.
  • Naomi Yamashita, Katsuhiko Kaji, Hideaki Kuzuoka, Keiji Hirata, ” Improving Visibility of Remote Gestures in Distributed Tabletop Collaboration,” Proceedings of ACM Conference on Computer Supported Cooperative Work (CSCW’11), pp. 95-104, 2011.
  • 山下直美, 梶克彦, 葛岡英明,平田圭二,青柳滋己, “遠隔ユーザのジェスチャの可視性を向上させる手法の提案と評価,”情報処理学会論文誌, Vol.52, No.1, 2011. (情報処理学会2011年度 論文賞受賞 )
  • Naomi Yamashita, Keiji Hirata, Shigemi Aoyagi, Hideaki Kuzuoka, Yasunori Harada “Impact of Seating Positions on Group Video Communication,” Proceedings of ACM Conference on Computer Supported Collaborative Work (CSCW’08), pp. 177-186, 2008.