-
Nature Aug 2023Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech...
Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech into text or sound. Early demonstrations, although promising, have not yet achieved accuracies sufficiently high for communication of unconstrained sentences from a large vocabulary. Here we demonstrate a speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant-who can no longer speak intelligibly owing to amyotrophic lateral sclerosis-achieved a 9.1% word error rate on a 50-word vocabulary (2.7 times fewer errors than the previous state-of-the-art speech BCI) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant's attempted speech was decoded at 62 words per minute, which is 3.4 times as fast as the previous record and begins to approach the speed of natural conversation (160 words per minute). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for restoring rapid communication to people with paralysis who can no longer speak.
Topics: Humans; Amyotrophic Lateral Sclerosis; Brain-Computer Interfaces; Cerebral Cortex; Microelectrodes; Paralysis; Speech; Vocabulary; Neural Prostheses
PubMed: 37612500
DOI: 10.1038/s41586-023-06377-x -
Nature Aug 2023Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive. Here we use...
Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive. Here we use high-density surface recordings of the speech cortex in a clinical-trial participant with severe limb and vocal paralysis to achieve high-performance real-time decoding across three complementary speech-related output modalities: text, speech audio and facial-avatar animation. We trained and evaluated deep-learning models using neural data collected as the participant attempted to silently speak sentences. For text, we demonstrate accurate and rapid large-vocabulary decoding with a median rate of 78 words per minute and median word error rate of 25%. For speech audio, we demonstrate intelligible and rapid speech synthesis and personalization to the participant's pre-injury voice. For facial-avatar animation, we demonstrate the control of virtual orofacial movements for speech and non-speech communicative gestures. The decoders reached high performance with less than two weeks of training. Our findings introduce a multimodal speech-neuroprosthetic approach that has substantial promise to restore full, embodied communication to people living with severe paralysis.
Topics: Humans; Cerebral Cortex; Clinical Trials as Topic; Communication; Deep Learning; Face; Gestures; Movement; Neural Prostheses; Paralysis; Speech; Vocabulary; Voice
PubMed: 37612505
DOI: 10.1038/s41586-023-06443-4 -
Cognitive Science Oct 2023A central concern of the cognitive science of language since its origins has been the concept of the linguistic system. Recent approaches to the system concept in... (Review)
Review
A central concern of the cognitive science of language since its origins has been the concept of the linguistic system. Recent approaches to the system concept in language point to the exceedingly complex relations that hold between many kinds of interdependent systems, but it can be difficult to know how to proceed when "everything is connected." This paper offers a framework for tackling that challenge by identifying *scale* as a conceptual mooring for the interdisciplinary study of language systems. The paper begins by defining the scale concept-simply, the possibility for a measure to be larger or smaller in different instances of a system, such as a phonemic inventory, a word's frequency value in a corpus, or a speaker population. We review sites of scale difference in and across linguistic subsystems, drawing on findings from linguistic typology, grammatical description, morphosyntactic theory, psycholinguistics, computational corpus work, and social network demography. We consider possible explanations for scaling differences and constraints in language. We then turn to the question of *dependencies between* sites of scale difference in language, reviewing four sample domains of scale dependency: in phonological systems, across levels of grammatical structure (Menzerath's Law), in corpora (Zipf's Law and related issues), and in speaker population size. Finally, we consider the implications of the review, including the utility of a scale framework for generating new questions and inspiring methodological innovations and interdisciplinary collaborations in cognitive-scientific research on language.
Topics: Humans; Language; Linguistics; Psycholinguistics
PubMed: 37823747
DOI: 10.1111/cogs.13341 -
Trends in Cognitive Sciences Jul 2023A deeply heterogeneous set of ideological cohorts have shaped the course of history. From anarchists and authoritarians to Zionists and Zapatistas, the expansive... (Review)
Review
A deeply heterogeneous set of ideological cohorts have shaped the course of history. From anarchists and authoritarians to Zionists and Zapatistas, the expansive alphabet of politics demands an equally expansive psychological vocabulary to describe political belief systems. We propose that constructing such a vocabulary is best facilitated by decentering familiar models that emphasize psychological differences between leftists and rightists. Synthesizing recent developments in the fields of personality, political science, and psychopathology, we characterize individual variation in politics as high-dimensional, heterarchical, intrapersonally eclectic, and contextually shaped and activated. Developing a data-driven taxonomic model of political-psychological phenomena will help create a foundational base of knowledge within political psychology that is more rigorous, more replicable, and certainly richer to investigate.
Topics: Humans; Politics; Personality; Psychological Theory
PubMed: 37080806
DOI: 10.1016/j.tics.2023.03.012 -
Annals of the New York Academy of... Jul 2023Biomolecular communication demands that interactions between parts of a molecular system act as scaffolds for message transmission. It also requires an organized system... (Review)
Review
Biomolecular communication demands that interactions between parts of a molecular system act as scaffolds for message transmission. It also requires an organized system of signs-a communicative agency-for creating and transmitting meaning. The emergence of agency, the capacity to act in a given context and generate end-directed behaviors, has baffled evolutionary biologists for centuries. Here, I explore its emergence with knowledge grounded in over two decades of evolutionary genomic and bioinformatic exploration. Biphasic processes of growth and diversification exist that generate hierarchy and modularity in biological systems at widely ranging time scales. Similarly, a biphasic process exists in communication that constructs a message before it can be transmitted for interpretation. Transmission dissipates matter-energy and information and involves computation. Agency emerges when molecular machinery generates hierarchical layers of vocabularies in an entangled communication network clustered around the universal Turing machine of the ribosome. Computations canalize biological systems to perform biological functions in a dissipative quest to structure long-lived occurrents. This occurs within the confines of a "triangle of persistence" that maximizes invariance with trade-offs between economy, flexibility, and robustness. Thus, learning from previous historical and circumstantial experiences unifies modules in a hierarchy that expands the agency of systems.
Topics: Humans; Cognition; Computational Biology; Biological Evolution
PubMed: 37219369
DOI: 10.1111/nyas.15005 -
Child Development 2023This meta-analysis synthesizes research on media use in early childhood (0-6 years), word-learning, and vocabulary size. Multi-level analyses included 266 effect sizes... (Meta-Analysis)
Meta-Analysis Review
This meta-analysis synthesizes research on media use in early childhood (0-6 years), word-learning, and vocabulary size. Multi-level analyses included 266 effect sizes from 63 studies (N = 11,413) published between 1988-2022. Among samples with information about race/ethnicity (51%) and sex/gender (73%), most were majority White/Non-Hispanic and between 40%-60% female. Analyses revealed a small overall positive relation between screen media exposure and vocabulary (r = .23). Experimental studies yielded a small-to-medium effect (r = .30), with stronger effects for e-books than TV/video or games/apps, and non-significant effects for video chat. In correlational studies, there was no overall association between vocabulary size and naturalistic media exposure (r = .07), with the exception of naturalistic exposure to educational media (r = .17).
Topics: Child; Child, Preschool; Female; Humans; Male; Learning; Verbal Learning; Vocabulary; Infant
PubMed: 37042116
DOI: 10.1111/cdev.13927 -
IEEE Transactions on Pattern Analysis... Jun 2024As the most fundamental scene understanding tasks, object detection and segmentation have made tremendous progress in deep learning era. Due to the expensive manual...
As the most fundamental scene understanding tasks, object detection and segmentation have made tremendous progress in deep learning era. Due to the expensive manual labeling cost, the annotated categories in existing datasets are often small-scale and pre-defined, i.e., state-of-the-art fully-supervised detectors and segmentors fail to generalize beyond the closed vocabulary. To resolve this limitation, in the last few years, the community has witnessed an increasing attention toward Open-Vocabulary Detection (OVD) and Segmentation (OVS). By "open-vocabulary", we mean that the models can classify objects beyond pre-defined categories. In this survey, we provide a comprehensive review on recent developments of OVD and OVS. A taxonomy is first developed to organize different tasks and methodologies. We find that the permission and usage of weak supervision signals can well discriminate different methodologies, including: visual-semantic space mapping, novel visual feature synthesis, region-aware training, pseudo-labeling, knowledge distillation, and transfer learning. The proposed taxonomy is universal across different tasks, covering object detection, semantic/instance/panoptic segmentation, 3D and video understanding. The main design principles, key challenges, development routes, methodology strengths, and weaknesses are thoroughly analyzed. In addition, we benchmark each task along with the vital components of each method in appendix and updated online at awesome-ovd-ovs. Finally, several promising directions are provided and discussed to stimulate future research.
PubMed: 38875096
DOI: 10.1109/TPAMI.2024.3413013 -
Journal of Speech, Language, and... Sep 2023We sought to examine second grade teachers' word use throughout the school day to identify the amount and type of teacher vocabulary use across content areas as well as...
PURPOSE
We sought to examine second grade teachers' word use throughout the school day to identify the amount and type of teacher vocabulary use across content areas as well as to examine relationships between this teacher talk and student language and literacy achievement.
METHOD
Second grade teachers ( = 64) and a random sample of half of their students ( = 619) participated. Teachers recorded instruction during the school day throughout the year, and students were assessed on vocabulary, grammar, and reading measures in the fall and spring.
RESULTS
Findings reveal second grade students hear thousands of words spoken by the teacher each hour of the school day, including more than a thousand different words per hour on average. The large majority of words were the most common words in the English language. On average, there were few academic or curriculum vocabulary words used, but this varied widely between teachers. The proportion of academic words used by teachers during the school day significantly predicted students' end-of-year vocabulary. Teachers who used more academic words had students with higher vocabulary achievement at the end of the school year. There were no other significant relationships between teachers' language and student achievement.
CONCLUSIONS
This correlational evidence adds to the existing knowledge of the importance of academic language to student school outcomes and provides implications for further research in the area of academic language at the early elementary level.
Topics: Humans; Vocabulary; Literacy; Language; Students; Linguistics; School Teachers
PubMed: 37541302
DOI: 10.1044/2023_JSLHR-22-00605 -
International Journal of Psychology :... Aug 2023Current research on emotion knowledge and competence emphasises the role of language. Emotion vocabulary is one of the indicators of emotion knowledge that can be...
Current research on emotion knowledge and competence emphasises the role of language. Emotion vocabulary is one of the indicators of emotion knowledge that can be objectively measured; however, the metric properties of the scores obtained in tests and tasks to measure it have seldom been adequate. In this study we designed and validated a Spanish emotion vocabulary test (MOVE) employing a corpus approach to construct cloze multiple-choice items, administered the test to a Spanish-speaking sample from two countries, Spain and Argentina, and analysed structural validity of the test items with the Rasch model measurement approach. Eighty-eight items showed adequate fit. Overall, a substantial percentage of variance was explained by a latent variable. Reliability indexes at the test, item, and person level were also adequate. As a vocabulary test, the MOVE can be used in psychological and neurological investigation, as well as in language learning research.
Topics: Humans; Vocabulary; Reproducibility of Results; Language; Emotions; Spain; Psychometrics
PubMed: 36950973
DOI: 10.1002/ijop.12903 -
Journal of Autism and Developmental... May 2024This study sought to understand the wide variability in vocabulary development among autistic children by testing potential social and linguistic correlates of...
This study sought to understand the wide variability in vocabulary development among autistic children by testing potential social and linguistic correlates of vocabulary size. The correlation between overlapping vocalization (i.e., an aspect of social interaction relevant to accessing input for vocabulary acquisition) and phonological memory (i.e., retaining linguistic sound sequences) with vocabulary size were examined in 22 autistic children (3 to 11 years old) engaged in a structured nonword repetition task. Overlapping vocalization and phonological memory were correlated with vocabulary size. Overlapping vocalization remained a significant predictor of receptive and expressive vocabulary size when controlling phonological memory and nonverbal cognition. Both social and linguistic factors were associated with receptive and expressive vocabulary size in autistic children engaged in a structured task.
PubMed: 38796794
DOI: 10.1007/s10803-024-06396-1