-
Nature Aug 2023Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive. Here we use...
Speech neuroprostheses have the potential to restore communication to people living with paralysis, but naturalistic speed and expressivity are elusive. Here we use high-density surface recordings of the speech cortex in a clinical-trial participant with severe limb and vocal paralysis to achieve high-performance real-time decoding across three complementary speech-related output modalities: text, speech audio and facial-avatar animation. We trained and evaluated deep-learning models using neural data collected as the participant attempted to silently speak sentences. For text, we demonstrate accurate and rapid large-vocabulary decoding with a median rate of 78 words per minute and median word error rate of 25%. For speech audio, we demonstrate intelligible and rapid speech synthesis and personalization to the participant's pre-injury voice. For facial-avatar animation, we demonstrate the control of virtual orofacial movements for speech and non-speech communicative gestures. The decoders reached high performance with less than two weeks of training. Our findings introduce a multimodal speech-neuroprosthetic approach that has substantial promise to restore full, embodied communication to people living with severe paralysis.
Topics: Humans; Cerebral Cortex; Clinical Trials as Topic; Communication; Deep Learning; Face; Gestures; Movement; Neural Prostheses; Paralysis; Speech; Vocabulary; Voice
PubMed: 37612505
DOI: 10.1038/s41586-023-06443-4 -
The New England Journal of Medicine Jul 2021Technology to restore the ability to communicate in paralyzed persons who cannot speak has the potential to improve autonomy and quality of life. An approach that...
BACKGROUND
Technology to restore the ability to communicate in paralyzed persons who cannot speak has the potential to improve autonomy and quality of life. An approach that decodes words and sentences directly from the cerebral cortical activity of such patients may represent an advancement over existing methods for assisted communication.
METHODS
We implanted a subdural, high-density, multielectrode array over the area of the sensorimotor cortex that controls speech in a person with anarthria (the loss of the ability to articulate speech) and spastic quadriparesis caused by a brain-stem stroke. Over the course of 48 sessions, we recorded 22 hours of cortical activity while the participant attempted to say individual words from a vocabulary set of 50 words. We used deep-learning algorithms to create computational models for the detection and classification of words from patterns in the recorded cortical activity. We applied these computational models, as well as a natural-language model that yielded next-word probabilities given the preceding words in a sequence, to decode full sentences as the participant attempted to say them.
RESULTS
We decoded sentences from the participant's cortical activity in real time at a median rate of 15.2 words per minute, with a median word error rate of 25.6%. In post hoc analyses, we detected 98% of the attempts by the participant to produce individual words, and we classified words with 47.1% accuracy using cortical signals that were stable throughout the 81-week study period.
CONCLUSIONS
In a person with anarthria and spastic quadriparesis caused by a brain-stem stroke, words and sentences were decoded directly from cortical activity during attempted speech with the use of deep-learning models and a natural-language model. (Funded by Facebook and others; ClinicalTrials.gov number, NCT03698149.).
Topics: Adult; Brain Stem Infarctions; Brain-Computer Interfaces; Deep Learning; Dysarthria; Electrocorticography; Electrodes, Implanted; Humans; Male; Natural Language Processing; Neural Prostheses; Quadriplegia; Sensorimotor Cortex; Speech
PubMed: 34260835
DOI: 10.1056/NEJMoa2027540 -
Hearing Research May 2014Methods to control neural activity by light have been introduced to the field of neuroscience. During the last decade, several techniques have been established,... (Review)
Review
Methods to control neural activity by light have been introduced to the field of neuroscience. During the last decade, several techniques have been established, including optogenetics, thermogenetics, and infrared neural stimulation. The techniques allow investigators to turn-on or turn-off neural activity. This review is an attempt to show the importance of the techniques for the auditory field and provide insight in the similarities, overlap, and differences of the techniques. Discussing the mechanism of each of the techniques will shed light on the abilities and challenges for each of the techniques. The field has been grown tremendously and a review cannot be complete. However, efforts are made to summarize the important points and to refer the reader to excellent papers and reviews to specific topics. This article is part of a Special Issue entitled
. Topics: Animals; Auditory Pathways; Gene Expression Regulation; Humans; Infrared Rays; Light Signal Transduction; Neural Prostheses; Neurons; Optics and Photonics; Optogenetics; Photic Stimulation; Photons; Prosthesis Design; Rhodopsins, Microbial; Temperature; Transient Receptor Potential Channels
PubMed: 24709273
DOI: 10.1016/j.heares.2014.03.008 -
Advanced Materials (Deerfield Beach,... Mar 2014Recent advances in nanotechnology have generated wide interest in applying nanomaterials for neural prostheses. An ideal neural interface should create seamless... (Review)
Review
Recent advances in nanotechnology have generated wide interest in applying nanomaterials for neural prostheses. An ideal neural interface should create seamless integration into the nervous system and performs reliably for long periods of time. As a result, many nanoscale materials not originally developed for neural interfaces become attractive candidates to detect neural signals and stimulate neurons. In this comprehensive review, an overview of state-of-the-art microelectrode technologies provided fi rst, with focus on the material properties of these microdevices. The advancements in electro active nanomaterials are then reviewed, including conducting polymers, carbon nanotubes, graphene, silicon nanowires, and hybrid organic-inorganic nanomaterials, for neural recording, stimulation, and growth. Finally, technical and scientific challenges are discussed regarding biocompatibility, mechanical mismatch, and electrical properties faced by these nanomaterials for the development of long-lasting functional neural interfaces.
Topics: Animals; Biocompatible Materials; Humans; Microelectrodes; Nanostructures; Neural Prostheses; Neurons
PubMed: 24677434
DOI: 10.1002/adma.201304496 -
Journal of Neural Engineering Dec 2009The success of modern neural prostheses is dependent on a complex interplay between the devices' hardware and software and the dynamic environment in which the devices... (Review)
Review
The success of modern neural prostheses is dependent on a complex interplay between the devices' hardware and software and the dynamic environment in which the devices operate: the patient's body or 'wetware'. Over 120 000 severe/profoundly deaf individuals presently receive information enabling auditory awareness and speech perception from cochlear implants. The cochlear implant therefore provides a useful case study for a review of the complex interactions between hardware, software and wetware, and of the important role of the dynamic nature of wetware. In the case of neural prostheses, the most critical component of that wetware is the central nervous system. This paper will examine the evidence of changes in the central auditory system that contribute to changes in performance with a cochlear implant, and discuss how these changes relate to electrophysiological and functional imaging studies in humans. The relationship between the human data and evidence from animals of the remarkable capacity for plastic change of the central auditory system, even into adulthood, will then be examined. Finally, we will discuss the role of brain plasticity in neural prostheses in general.
Topics: Animals; Auditory Pathways; Brain; Cochlea; Cochlear Implants; Humans; Neuronal Plasticity; Software
PubMed: 19850976
DOI: 10.1088/1741-2560/6/6/065008 -
Nature Aug 2023Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech...
Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech into text or sound. Early demonstrations, although promising, have not yet achieved accuracies sufficiently high for communication of unconstrained sentences from a large vocabulary. Here we demonstrate a speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant-who can no longer speak intelligibly owing to amyotrophic lateral sclerosis-achieved a 9.1% word error rate on a 50-word vocabulary (2.7 times fewer errors than the previous state-of-the-art speech BCI) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant's attempted speech was decoded at 62 words per minute, which is 3.4 times as fast as the previous record and begins to approach the speed of natural conversation (160 words per minute). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for restoring rapid communication to people with paralysis who can no longer speak.
Topics: Humans; Amyotrophic Lateral Sclerosis; Brain-Computer Interfaces; Cerebral Cortex; Microelectrodes; Paralysis; Speech; Vocabulary; Neural Prostheses
PubMed: 37612500
DOI: 10.1038/s41586-023-06377-x -
Advanced Materials (Deerfield Beach,... Dec 2015Neural-interfacing devices are an artificial mechanism for restoring or supplementing the function of the nervous system, lost as a result of injury or disease.... (Review)
Review
Neural-interfacing devices are an artificial mechanism for restoring or supplementing the function of the nervous system, lost as a result of injury or disease. Conducting polymers (CPs) are gaining significant attention due to their capacity to meet the performance criteria of a number of neuronal therapies including recording and stimulating neural activity, the regeneration of neural tissue and the delivery of bioactive molecules for mediating device-tissue interactions. CPs form a flexible platform technology that enables the development of tailored materials for a range of neuronal diagnostic and treatment therapies. In this review, the application of CPs for neural prostheses and other neural interfacing devices is discussed, with a specific focus on neural recording, neural stimulation, neural regeneration, and therapeutic drug delivery.
Topics: Animals; Biocompatible Materials; Drug Carriers; Electrodes; Gels; Humans; Neural Prostheses; Neurons; Polymers; Regeneration
PubMed: 26414302
DOI: 10.1002/adma.201501810 -
AJNR. American Journal of Neuroradiology Jan 2021Analogous to hearing restoration via cochlear implants, vestibular function could be restored via vestibular implants that electrically stimulate vestibular nerve... (Clinical Trial)
Clinical Trial
Analogous to hearing restoration via cochlear implants, vestibular function could be restored via vestibular implants that electrically stimulate vestibular nerve branches to encode head motion. This study presents the technical feasibility and first imaging results of CT for vestibular implants in 8 participants of the first-in-human Multichannel Vestibular Implant Early Feasibility Study. Imaging characteristics of 8 participants (3 men, 5 women; median age, 59.5 years; range, 51-66 years) implanted with a Multichannel Vestibular Implant System who underwent a postimplantation multislice CT ( = 2) or flat panel CT ( = 6) are reported. The device comprises 9 platinum electrodes inserted into the ampullae of the 3 semicircular canals and 1 reference electrode inserted in the common crus. Electrode insertion site, positions, length and angle of insertion, and number of artifacts were assessed. Individual electrode contacts were barely discernible in the 2 participants imaged using multislice CT. Electrode and osseous structures were detectable but blurred so that only 12 of the 18 stimulating electrode contacts could be individually identified. Flat panel CT could identify all 10 electrode contacts in all 6 participants. The median reference electrode insertion depth angle was 9° (range, -57.5° to 45°), and the median reference electrode insertion length was 42 mm (range, -21-66 mm). Flat panel CT of vestibular implants produces higher-resolution images with fewer artifacts than multidetector row CT, allowing visualization of individual electrode contacts and quantification of their locations relative to vestibular semicircular canals and ampullae. As multichannel vestibular implant imaging improves, so will our understanding of the relationship between electrode placement and vestibular performance.
Topics: Aged; Artifacts; Female; Humans; Image Processing, Computer-Assisted; Male; Middle Aged; Neural Prostheses; Tomography, X-Ray Computed; Vestibular Nerve; Vestibule, Labyrinth
PubMed: 33361382
DOI: 10.3174/ajnr.A6991 -
Critical Reviews in Biomedical... 2016The ideal neuroprosthetic interface permits high-quality neural recording and stimulation of the nervous system while reliably providing clinical benefits over chronic... (Review)
Review
The ideal neuroprosthetic interface permits high-quality neural recording and stimulation of the nervous system while reliably providing clinical benefits over chronic periods. Although current technologies have made notable strides in this direction, significant improvements must be made to better achieve these design goals and satisfy clinical needs. This article provides an overview of the state of neuroprosthetic interfaces, starting with the design and placement of these interfaces before exploring the stimulation and recording platforms yielded from contemporary research. Finally, we outline emerging research trends in an effort to explore the potential next generation of neuroprosthetic interfaces.
Topics: Biomedical Research; Brain-Computer Interfaces; Electrodes, Implanted; Humans; Neural Prostheses; Prosthesis Design; Prosthesis Implantation
PubMed: 27652455
DOI: 10.1615/CritRevBiomedEng.2016017198 -
Neuron Apr 2015The field of invasive brain-machine interfaces (BMIs) is typically associated with neuroprosthetic applications aiming to recover loss of motor function. However, BMIs... (Review)
Review
The field of invasive brain-machine interfaces (BMIs) is typically associated with neuroprosthetic applications aiming to recover loss of motor function. However, BMIs also represent a powerful tool to address fundamental questions in neuroscience. The observed subjects of BMI experiments can also be considered as indirect observers of their own neurophysiological activity, and the relationship between observed neurons and (artificial) behavior can be genuinely causal rather than indirectly correlative. These two characteristics defy the classical object-observer duality, making BMIs particularly appealing for investigating how information is encoded and decoded by neural circuits in real time, how this coding changes with physiological learning and plasticity, and how it is altered in pathological conditions. Within neuroengineering, BMI is like a tree that opens its branches into many traditional engineering fields, but also extends deep roots into basic neuroscience beyond neuroprosthetics.
Topics: Animals; Brain; Brain-Computer Interfaces; Humans; Neural Prostheses; Neurons
PubMed: 25856486
DOI: 10.1016/j.neuron.2015.03.036