-
Sensors (Basel, Switzerland) Nov 2023This paper proposes, analyzes, and evaluates a deep learning architecture based on transformers for generating sign language motion from sign phonemes (represented using...
This paper proposes, analyzes, and evaluates a deep learning architecture based on transformers for generating sign language motion from sign phonemes (represented using HamNoSys: a notation system developed at the University of Hamburg). The sign phonemes provide information about sign characteristics like hand configuration, localization, or movements. The use of sign phonemes is crucial for generating sign motion with a high level of details (including finger extensions and flexions). The transformer-based approach also includes a stop detection module for predicting the end of the generation process. Both aspects, motion generation and stop detection, are evaluated in detail. For motion generation, the dynamic time warping distance is used to compute the similarity between two landmarks sequences (ground truth and generated). The stop detection module is evaluated considering detection accuracy and ROC (receiver operating characteristic) curves. The paper proposes and evaluates several strategies to obtain the system configuration with the best performance. These strategies include different padding strategies, interpolation approaches, and data augmentation techniques. The best configuration of a fully automatic system obtains an average DTW distance per frame of 0.1057 and an area under the ROC curve (AUC) higher than 0.94.
Topics: Humans; Algorithms; Sign Language; Motion; Movement; Hand
PubMed: 38067738
DOI: 10.3390/s23239365 -
Frontiers in Psychology 2023Modern society depends on numerical information, which must be communicated accurately and effectively. Numerical communication is accomplished in different... (Review)
Review
Modern society depends on numerical information, which must be communicated accurately and effectively. Numerical communication is accomplished in different modalities-speech, writing, sign, gesture, graphs, and in naturally occurring settings it almost always involves more than one modality at once. Yet the modalities of numerical communication are often studied in isolation. Here we argue that, to understand and improve numerical communication, we must take seriously this multimodality. We first discuss each modality on its own terms, identifying their commonalities and differences. We then argue that numerical communication is shaped critically by interactions among modalities. We boil down these interactions to four types: one modality can the message of another; it can attention to content from another modality (e.g., using a gesture to guide attention to a relevant aspect of a graph); it can another modality (e.g., verbally explaining the meaning of an axis in a graph); and it can a modality (e.g., framing an upwards-oriented trend as a bad outcome). We conclude by discussing how a focus on multimodality raises entirely new research questions about numerical communication.
PubMed: 37564312
DOI: 10.3389/fpsyg.2023.1130777 -
Data in Brief Aug 2023Deaf and hard-of-hearing individuals use sign language as a means of communication. However, those around them, especially family members like the children of deaf...
Deaf and hard-of-hearing individuals use sign language as a means of communication. However, those around them, especially family members like the children of deaf adults, may face communication challenges if they are unfamiliar with sign language. This issue has prompted numerous researchers to conduct studies on sign language translation and recognition. However, there is currently no publicly available dataset specifically for Malaysian sign language. This article introduces an image dataset of the Malaysian Sign Language (MySL) hand gestures used in everyday situations. The dataset, named MyWSL2023, comprises 3,500 images of ten static Malaysian sign language words collected from five participants (two males and three females) aged between 20 and 21 years old. The data collection took place indoors under normal lighting conditions. The MyWSL2023 dataset, which has been made freely accessible to all researchers, serves as a valuable resource for not only investigating and developing automated systems for hearing-impaired and deaf individuals but also gesture and sign language recognition using vision-based methods. The dataset can be accessed for free at https://data.mendeley.com/datasets/zvk55p7ktd.
PubMed: 37600131
DOI: 10.1016/j.dib.2023.109338 -
Acta Neurologica Taiwanica Jun 2023A 20-month-old female, not immunized with Bacillus Calmette-Guérin (BCG) vaccine, was admitted due to a four-day history of fever and cough. In the past three months,...
A 20-month-old female, not immunized with Bacillus Calmette-Guérin (BCG) vaccine, was admitted due to a four-day history of fever and cough. In the past three months, she presented respiratory infections, weight loss and enlarged cervical lymph nodes. On day two of admission, she displayed drowsiness and positive Romberg's sign; cerebrospinal fluid (CSF) workout revealed 107/ul cells, low glucose and high protein levels. Ceftriaxone and acyclovir were initiated, and she was transferred to our tertiary hospital. Brain magnetic resonance imaging showed punctiform focal areas of restricted diffusion in left capsular lenticular region suggestive of vasculitis secondary to infection. Tuberculin skin test and interferon-gamma release assay were positive. She started tuberculostatic therapy, but two days later she presented tonic-clonic seizures and impaired consciousness. Cerebral computed tomography (CT) revealed tetrahydrocephalus (Figure 1), needing external ventricular derivation. She had a slow clinical improvement, requiring several neurosurgical interventions and developing a syndrome of inappropriate antidiuretic secretion alternating with cerebral salt wasting. Positive results for Mycobacterium tuberculosis were obtained by CSF culture and by polymerase chain reaction in CSF, bronchoalveolar lavage and gastric aspirate specimens. Repeated brain CT showed a large-vessel vasculitis with basal meningeal enhancement, typical of central nervous system (CNS) tuberculosis (Figure 2). She completed one month of corticosteroids and maintained antituberculosis treatment. At two years of age, she has spastic paraparesis and no language skills. Portugal had 1836 cases of tuberculosis (17.8 per 100000) in 2016 and was considered a low-incidence country; consequently, BCG vaccination is not universal (1). We present a severe case of CNS tuberculosis with intracranial hypertension, vasculitis and hyponatremia, associated with poorer outcomes (2). A high index of suspicion allowed prompt start of antituberculosis treatment. Diagnosis was corroborated by microbiological positivity and a typical triad in neuroimaging (hydrocephalus, vasculitis and basal meningeal enhancement) (3), which we wish to emphasize.
Topics: Humans; Female; Infant; BCG Vaccine; Tuberculosis, Central Nervous System; Tuberculosis; Neuroimaging; Antitubercular Agents; Vasculitis; Tuberculosis, Meningeal
PubMed: 37198514
DOI: No ID Found -
Animals : An Open Access Journal From... Nov 2023Adult chimpanzees Tatu and Loulis lived at the Fauna Foundation sanctuary. They had acquired signs of American Sign Language (ASL) while young and continued to use them...
Adult chimpanzees Tatu and Loulis lived at the Fauna Foundation sanctuary. They had acquired signs of American Sign Language (ASL) while young and continued to use them as adults. Caregivers with proficiency in ASL maintained daily sign language records during interactions and passive observation. Sign checklists were records of daily vocabulary use. Sign logs were records of signed interactions with caregivers and other chimpanzees. This study reports sign use from eight years of these records. Tatu and Loulis used a majority of their base vocabularies consistently over the study period. They used signs that they had acquired decades earlier and new signs. Their utterances served a variety of communicative functions, including responses, conversational devices, requests, and descriptions. They signed to caregivers, other chimpanzees, including those who did not use signs, and to themselves privately. This indicates the importance of a stimulating and interactive environment to understand the scope of ape communication and, in particular, their use of sign language.
PubMed: 38003104
DOI: 10.3390/ani13223486 -
Sensors (Basel, Switzerland) Jul 2023This article is devoted to solving the problem of converting sign language into a consistent text with intonation markup for subsequent voice synthesis of sign phrases...
This article is devoted to solving the problem of converting sign language into a consistent text with intonation markup for subsequent voice synthesis of sign phrases by speech with intonation. The paper proposes an improved method of continuous recognition of sign language, the results of which are transmitted to a natural language processor based on analyzers of morphology, syntax, and semantics of the Kazakh language, including morphological inflection and the construction of an intonation model of simple sentences. This approach has significant practical and social significance, as it can lead to the development of technologies that will help people with disabilities to communicate and improve their quality of life. As a result of the cross-validation of the model, we obtained an average test accuracy of 0.97 and an average val_accuracy of 0.90 for model evaluation. We also identified 20 sentence structures of the Kazakh language with their intonational model.
Topics: Humans; Speech; Sign Language; Quality of Life; Speech Perception; Language
PubMed: 37514679
DOI: 10.3390/s23146383 -
Frontiers in Psychology 2023Medical interpreters experience emotional burdens from the complex demands at work. Because communication access is a social determinant of health, protecting and...
INTRODUCTION
Medical interpreters experience emotional burdens from the complex demands at work. Because communication access is a social determinant of health, protecting and promoting the health of medical interpreters is critical for ensuring equitable access to care for language-minority patients. The purpose of this study was to pilot a condensed 8-h program based on Mindful Practice in Medicine addressing the contributors to distress and psychosocial stressors faced by medical sign and spoken language interpreters.
METHODS
Using a single-arm embedded QUAN(qual) mixed-methods pilot study design, weekly in-person 1-h sessions for 8 weeks involved formal and informal contemplative practice, didactic delivery of the week's theme (mindfulness, noticing, teamwork, suffering, professionalism, uncertainty, compassion, and resilience), and mindful inquiry exercises (narrative medicine, appreciative interviews, and insight dialog). Quantitative well-being outcomes (mean±SEM) were gathered via survey at pre-, post-, and 1-month post-intervention time points, compared with available norms, and evaluated for differences within subjects. Voluntary feedback about the workshop series was solicited post-intervention via a free text survey item and individual exit interviews. A thematic framework was established by way of qualitative description.
RESULTS
Seventeen medical interpreters (46.2 ± 3.1 years old; 16 women/1 man; 8 White/9 Hispanic or Latino) participated. Overall scores for teamwork ( ≤ 0.027), coping ( ≤ 0.006), and resilience ( ≤ 0.045) increased from pre- to post-intervention and pre- to 1-month post-intervention. Non-judging as a mindfulness component increased from pre- to post-intervention ( = 0.014). Compassion satisfaction ( = 0.021) and burnout ( = 0.030) as components of professional quality of life demonstrated slightly delayed effects, improving from pre- to 1-month post-intervention. Themes such as , and are related to the overarching topic areas of intervention logistics and content. Integration of the findings accentuated the positive .
DISCUSSION
The results of this research demonstrate that mindful practice can serve as an effective resource for medical interpreters when coping with work-related stressors. Future iterations of the mindful practice intervention will further aspire to address linguistic and cultural diversity in the study population for broader representation and subsequent generalization.
PubMed: 37954177
DOI: 10.3389/fpsyg.2023.1171993 -
Sensors (Basel, Switzerland) Oct 2023The analysis and recognition of sign languages are currently active fields of research focused on sign recognition. Various approaches differ in terms of analysis... (Review)
Review
The analysis and recognition of sign languages are currently active fields of research focused on sign recognition. Various approaches differ in terms of analysis methods and the devices used for sign acquisition. Traditional methods rely on video analysis or spatial positioning data calculated using motion capture tools. In contrast to these conventional recognition and classification approaches, electromyogram (EMG) signals, which measure muscle electrical activity, offer potential technology for detecting gestures. These EMG-based approaches have recently gained attention due to their advantages. This prompted us to conduct a comprehensive study on the methods, approaches, and projects utilizing EMG sensors for sign language handshape recognition. In this paper, we provided an overview of the sign language recognition field through a literature review, with the objective of offering an in-depth review of the most significant techniques. These techniques were categorized in this article based on their respective methodologies. The survey discussed the progress and challenges in sign language recognition systems based on surface electromyography (sEMG) signals. These systems have shown promise but face issues like sEMG data variability and sensor placement. Multiple sensors enhance reliability and accuracy. Machine learning, including deep learning, is used to address these challenges. Common classifiers in sEMG-based sign language recognition include SVM, ANN, CNN, KNN, HMM, and LSTM. While SVM and ANN are widely used, random forest and KNN have shown better performance in some cases. A multilayer perceptron neural network achieved perfect accuracy in one study. CNN, often paired with LSTM, ranks as the third most popular classifier and can achieve exceptional accuracy, reaching up to 99.6% when utilizing both EMG and IMU data. LSTM is highly regarded for handling sequential dependencies in EMG signals, making it a critical component of sign language recognition systems. In summary, the survey highlights the prevalence of SVM and ANN classifiers but also suggests the effectiveness of alternative classifiers like random forests and KNNs. LSTM emerges as the most suitable algorithm for capturing sequential dependencies and improving gesture recognition in EMG-based sign language recognition systems.
Topics: Humans; Sign Language; Reproducibility of Results; Pattern Recognition, Automated; Neural Networks, Computer; Algorithms; Electromyography; Gestures
PubMed: 37837173
DOI: 10.3390/s23198343 -
Children (Basel, Switzerland) Dec 2023Hearing loss is the most common sensory deficit and one of the most common congenital abnormalities. The estimated prevalence of moderate and severe hearing loss in a... (Review)
Review
Hearing loss is the most common sensory deficit and one of the most common congenital abnormalities. The estimated prevalence of moderate and severe hearing loss in a normal newborn is 0.1-0.3%, while the prevalence is 2-4% in newborns admitted to the newborn intensive care unit. Therefore, early detection and prompt treatment are of utmost importance in preventing the unwanted sequel of hearing loss on normal language development. The problem of congenital deafness is today addressed on the one hand with hearing screening at birth, on the other with the early (at around 3 months of age) application of hearing aids or, in case of lack of benefit, by the cochlear implant. Molecular genetics, antibody tests for some viruses, and diagnostic imaging have largely contributed to an effective etiological classification. A correct diagnosis and timely fitting of hearing aids or cochlear implants is useful for deaf children. The association between congenital deafness and "mutism", with all the consequences on/the consideration that deaf mutes have had since ancient times, not only from a social point of view but also from a legislative point of view, continued until the end of the nineteenth century, with the development on one side of new methods for the rehabilitation of language and on the other of sign language. But we need to get to the last decades of the last century to have, on the one hand, the diffusion of "universal newborn hearing screening", the discovery of the genetic causes of over half of congenital deafness, and on the other hand the cochlear implants that have allowed thousands of children born deaf the development of normal speech. Below, we will analyze the evolution of the problem between deafness and deaf-mutism over the centuries, with particular attention to the nineteenth century.
PubMed: 38255364
DOI: 10.3390/children11010051