-
Psychonomic Bulletin & Review Apr 2024Across the millennia, and across a range of disciplines, there has been a widespread desire to connect, or translate between, the senses in a manner that is meaningful,... (Review)
Review
Across the millennia, and across a range of disciplines, there has been a widespread desire to connect, or translate between, the senses in a manner that is meaningful, rather than arbitrary. Early examples were often inspired by the vivid, yet mostly idiosyncratic, crossmodal matches expressed by synaesthetes, often exploited for aesthetic purposes by writers, artists, and composers. A separate approach comes from those academic commentators who have attempted to translate between structurally similar dimensions of perceptual experience (such as pitch and colour). However, neither approach has succeeded in delivering consensually agreed crossmodal matches. As such, an alternative approach to sensory translation is needed. In this narrative historical review, focusing on the translation between audition and vision, we attempt to shed light on the topic by addressing the following three questions: (1) How is the topic of sensory translation related to synaesthesia, multisensory integration, and crossmodal associations? (2) Are there common processing mechanisms across the senses that can help to guarantee the success of sensory translation, or, rather, is mapping among the senses mediated by allegedly universal (e.g., amodal) stimulus dimensions? (3) Is the term 'translation' in the context of cross-sensory mappings used metaphorically or literally? Given the general mechanisms and concepts discussed throughout the review, the answers we come to regarding the nature of audio-visual translation are likely to apply to the translation between other perhaps less-frequently studied modality pairings as well.
Topics: Humans; Visual Perception; Auditory Perception; Synesthesia; History, 20th Century
PubMed: 37803233
DOI: 10.3758/s13423-023-02343-w -
Proceedings. Biological Sciences Oct 2023Synaesthesia is a sensory phenomenon where external stimuli, such as sounds or letters, trigger additional sensations (e.g. colours). Synaesthesia aggregates in families...
Synaesthesia is a sensory phenomenon where external stimuli, such as sounds or letters, trigger additional sensations (e.g. colours). Synaesthesia aggregates in families but its heritability is unknown. The phenomenon is more common in people on the autism spectrum compared with the general population and associated with higher autistic traits. Using classical twin design, we assessed the heritability of individual differences in self-reported synaesthesia and the genetic and environmental contributions to their association with autistic traits within a population twin cohort ( = 4262, age = 18 years). We estimated individual differences in synaesthesia to be heritable and influenced by environmental factors not shared between twins. The association between individual differences in synaesthesia and autistic traits was estimated to be predominantly under genetic influence and seemed to be mainly driven by non-social autistic traits (repetitive behaviours, restricted interests and attention to detail). Our study suggests that the link between synaesthesia and autism might reside in shared genetic causes, related to non-social autistic traits such as alterations in perception. Future studies building on these findings may attempt to identify specific groups of genes that influence both autism, synaesthesia and perception.
Topics: Humans; Adolescent; Autistic Disorder; Sensation; Self Report; Autism Spectrum Disorder
PubMed: 37876199
DOI: 10.1098/rspb.2023.1888 -
Behavior Research Methods Dec 2023Synesthesia is a phenomenon where sensory stimuli or cognitive concepts elicit additional perceptual experiences. For instance, in a commonly studied type of...
Synesthesia is a phenomenon where sensory stimuli or cognitive concepts elicit additional perceptual experiences. For instance, in a commonly studied type of synesthesia, stimuli such as words written in black font elicit experiences of other colors, e.g., red. In order to objectively verify synesthesia, participants are asked to choose colors for repeatedly presented stimuli and the consistency of their choices is evaluated (consistency test). Previously, there has been no publicly available and easy-to-use tool for analyzing consistency test results. Here, the R package synr is introduced, which provides an efficient interface for exploring consistency test data and applying common procedures for analyzing them. Importantly, synr also implements a novel method enabling identification of participants whose scores cannot be interpreted, e.g., who only give black or red color responses. To this end, density-based spatial clustering of applications with noise (DBSCAN) is applied in conjunction with a measure of spread in 3D space. An application of synr with pre-existing openly accessible data illustrating how synr is used in practice is presented. Also included is a comparison of synr's data validation procedure and human ratings, which found that synr had high correspondence with human ratings and outperformed human raters in situations where human raters were easily mislead. Challenges for widespread adoption of synr as well as suggestions for using synr within the field of synesthesia and other areas of psychological research are discussed.
Topics: Humans; Synesthesia; Perceptual Disorders; Color Perception
PubMed: 36357762
DOI: 10.3758/s13428-022-02007-y -
Frontiers in Psychology 2023Much research has focused on the modeling of the near-death experience (NDE) by classical and atypical psychedelics; however, to date, no study has reported on the...
INTRODUCTION
Much research has focused on the modeling of the near-death experience (NDE) by classical and atypical psychedelics; however, to date, no study has reported on the relationship between the NDE and the experience induced by the highly potent, endogenous psychedelic drug 5-Methoxy-DMT (5MeO-DMT). This article presents a case study of an individual who is popularly documented to have had a profound near-death experience while in a coma caused by bacterial meningoencephalitis. Additionally, the individual also subsequently underwent an experience with 5MeO-DMT.
METHODS
A semi-structured interview was conducted with the subject concerning his experiences with both the NDE and 5MeO-DMT. A basic thematic analysis was performed on both the original text describing the NDE as well as the interview itself, which mainly focused on the subject's experience with 5MeO-DMT. This analysis was organized to identify both the similar and different emergent themes between the two states, with a particular emphasis on the subject's perceptions of the similarities and differences between the experiences.
RESULTS
There is a very high level of comparability between the original NDE and psychedelic experiences in general, including shared characteristics such as entering other worlds, meeting menacing or benevolent entities, experiencing synesthesia, perinatal regression, and lucid dreamlike properties. Much comparability was also identified with the 5MeO-DMT experience, in particular the major mystical experiential domains, such as ego dissolution, but especially transcendence of time and space. However, there were also a few unique themes (life review, the deceased, and the threshold) that emerged in the NDE that were not present in the 5MeO-DMT experience or other psychedelic experience studies, suggesting that these themes may be more unique to the NDE.
DISCUSSION
Despite such similarities, the participant asserted that his NDE and psychedelic experiences were not similar enough to be attributed to endogenous psychedelics. In this study, we discussed several mechanisms that could potentially account for the NDE, including lucid dreams and perinatal regression. However, the study also explored the possibility that the unique etiology of the participant's NDE, bacterial meningoencephalitis affecting the neocortex, may have triggered similar downstream neural activity as that initiated by psychedelic agents through pyramidal neuronal activation. This hypothesis is presented with appropriate caveats and acknowledged as speculative.
PubMed: 37457069
DOI: 10.3389/fpsyg.2023.1083361 -
PNAS Nexus Mar 2024Recent artificial intelligence (AI) tools have demonstrated the ability to produce outputs traditionally considered creative. One such system is text-to-image generative...
Recent artificial intelligence (AI) tools have demonstrated the ability to produce outputs traditionally considered creative. One such system is text-to-image generative AI (e.g. Midjourney, Stable Diffusion, DALL-E), which automates humans' artistic execution to generate digital artworks. Utilizing a dataset of over 4 million artworks from more than 50,000 unique users, our research shows that over time, text-to-image AI significantly enhances human creative productivity by 25% and increases the value as measured by the likelihood of receiving a favorite per view by 50%. While peak artwork Content Novelty, defined as focal subject matter and relations, increases over time, average Content Novelty declines, suggesting an expanding but inefficient idea space. Additionally, there is a consistent reduction in both peak and average Visual Novelty, captured by pixel-level stylistic elements. Importantly, AI-assisted artists who can successfully explore more novel ideas, regardless of their prior originality, may produce artworks that their peers evaluate more favorably. Lastly, AI adoption decreased value capture (favorites earned) concentration among adopters. The results suggest that ideation and filtering are likely necessary skills in the text-to-image process, thus giving rise to "generative synesthesia"-the harmonious blending of human exploration and AI exploitation to discover new creative workflows.
PubMed: 38444602
DOI: 10.1093/pnasnexus/pgae052 -
Neuroscience of Consciousness 2024Predictive processing theories state that our subjective experience of reality is shaped by a balance of expectations based on previous knowledge about the world (i.e....
Predictive processing theories state that our subjective experience of reality is shaped by a balance of expectations based on previous knowledge about the world (i.e. priors) and confidence in sensory input from the environment. Divergent experiences (e.g. hallucinations and synaesthesia) are likely to occur when there is an imbalance between one's reliance on priors and sensory input. In a novel theoretical model, inspired by both predictive processing and psychological principles, we propose that predictable divergent experiences are associated with natural or environmentally induced prior/sensory imbalances: inappropriately strong or inflexible (i.e. maladaptive) high-level priors (beliefs) combined with low sensory confidence can result in reality discrimination issues, a characteristic of psychosis; maladaptive low-level priors (sensory expectations) combined with high sensory confidence can result in atypical sensory sensitivities and persistent divergent percepts, a characteristic of synaesthesia. Crucially, we propose that whether different divergent experiences manifest with dominantly sensory (e.g. hallucinations) or nonsensory characteristics (e.g. delusions) depends on mental imagery ability, which is a spectrum from aphantasia (absent or weak imagery) to hyperphantasia (extremely vivid imagery). We theorize that imagery is critically involved in shaping the sensory richness of divergent perceptual experience. In sum, to predict a range of divergent perceptual experiences in both clinical and general populations, three factors must be accounted for: a maladaptive use of priors, individual level of confidence in sensory input, and mental imagery ability. These ideas can be expressed formally using nonparametric regression modeling. We provide evidence for our theory from previous work and deliver predictions for future research.
PubMed: 38348335
DOI: 10.1093/nc/niae006 -
Scientific Reports Jul 2023Reading acquisition is enabled by deep changes in the brain's visual system and language areas, and in the links subtending their collaboration. Disruption of those...
Reading acquisition is enabled by deep changes in the brain's visual system and language areas, and in the links subtending their collaboration. Disruption of those plastic processes commonly results in developmental dyslexia. However, atypical development of reading mechanisms may occasionally result in ticker-tape synesthesia (TTS), a condition described by Francis Galton in 1883 wherein individuals "see mentally in print every word that is uttered (…) as from a long imaginary strip of paper". While reading is the bottom-up translation of letters into speech, TTS may be viewed as its opposite, the top-down translation of speech into internally visualized letters. In a series of functional MRI experiments, we studied MK, a man with TTS. We showed that a set of left-hemispheric areas were more active in MK than in controls during the perception of normal than reversed speech, including frontoparietal areas involved in speech processing, and the Visual Word Form Area, an occipitotemporal region subtending orthography. Those areas were identical to those involved in reading, supporting the construal of TTS as upended reading. Using dynamic causal modeling, we further showed that, parallel to reading, TTS induced by spoken words and pseudowords relied on top-down flow of information along distinct lexical and phonological routes, involving the middle temporal and supramarginal gyri, respectively. Future studies of TTS should shed new light on the neurodevelopmental mechanisms of reading acquisition, their variability and their disorders.
Topics: Male; Humans; Synesthesia; Brain Mapping; Reading; Language; Magnetic Resonance Imaging; Neuroimaging; Dyslexia
PubMed: 37500762
DOI: 10.1038/s41598-023-39276-2 -
Cortex; a Journal Devoted To the Study... Nov 2023As first described by Francis Galton, some persons perceive vividly and automatically in their mind's eye the written form of words that they are hearing. This...
As first described by Francis Galton, some persons perceive vividly and automatically in their mind's eye the written form of words that they are hearing. This phenomenon, labeled ticker-tape synesthesia (TTS), is thought to reflect an abnormally strong influence of speech processing in language areas on to orthographic representations in the visual cortex. Considering the relevance of TTS for the study of reading acquisition, we looked for objective behavioral advantages or impairments in 22 synesthetes, as compared to 22 matched control participants. In three auditory tasks relying on orthographic working memory (letters counting, backward spelling, and letter shape decision), we predicted and observed better performance in synesthetes than in controls. In two visual tasks (lexical decision and letter decision) with a concurrent auditory stimulation, we predicted that synesthetes should suffer from a larger interference by irrelevant speech than controls, but eventually found no difference between the groups. Those results, which we discuss in relation to orthographic processing, mental imagery, and working memory, promote TTS from pure subjectivity to an experimentally measurable phenomenon.
Topics: Humans; Synesthesia; Perceptual Disorders; Memory, Short-Term; Acoustic Stimulation; Imagery, Psychotherapy; Color Perception
PubMed: 37832491
DOI: 10.1016/j.cortex.2023.08.011 -
Psychological Research Mar 2024Synaesthesia has been conceptualised as a joining of sensory experiences. Taking a holistic, embodied perspective, we investigate in this paper the role of action and...
Synaesthesia has been conceptualised as a joining of sensory experiences. Taking a holistic, embodied perspective, we investigate in this paper the role of action and emotion, testing hypotheses related to (1) changes to action-related qualities of a musical stimulus affect the resulting synaesthetic experience; (2) a comparable relationship exists between music, sensorimotor and emotional responses in synaesthetes and the general population; and (3) sensorimotor responses are more strongly associated with synaesthesia than emotion. 29 synaesthetes and 33 non-synaesthetes listened to 12 musical excerpts performed on a musical instrument they had first-hand experience playing, an instrument never played before, and a deadpan performance generated by notation software, i.e., a performance without expression. They evaluated the intensity of their experience of the music using a list of dimensions that relate to sensorimotor, emotional or synaesthetic sensations. Results demonstrated that the intensity of listeners' responses was most strongly influenced by whether or not music is performed by a human, more so than familiarity with a particular instrument. Furthermore, our findings reveal a shared relationship between emotional and sensorimotor responses among both synaesthetes and non-synaesthetes. Yet it was sensorimotor intensity that was shown to be fundamentally associated with the intensity of the synaesthetic response. Overall, the research argues for, and gives first evidence of a key role of action in shaping the experiences of music-colour synaesthesia.
Topics: Humans; Color Perception; Color; Music; Photic Stimulation; Synesthesia; Emotions
PubMed: 37453940
DOI: 10.1007/s00426-023-01856-2 -
Scientific Reports Mar 2024There are many different kinds of 'phantom' percepts but it is unknown whether they are united by common mechanisms. For example, synaesthesia (e.g., numbers evoking...
There are many different kinds of 'phantom' percepts but it is unknown whether they are united by common mechanisms. For example, synaesthesia (e.g., numbers evoking colour) and hallucinations appear conceptually and phenomenologically similar: both result in a percept that does not have an environmental correlate. Here, people with synaesthesia (n = 66) performed a conditioned hallucinations paradigm known to be sensitive to hallucination susceptibility, and we asked whether synaesthetes would show the same behavioural profile as hallucinators in this task. Repeated pairing of checkerboards with tones, and gratings with colours encourages the participant to draw on prior knowledge when asked to report on the presence of the difficult-to-detect target stimulus. Synaesthetes show increased modelled expectancies for the stimulus association across the board, resulting in a higher number of detections at all stimulus intensities. This is in contrast to the pattern observed in hallucinators, who weigh their prior beliefs more strongly than controls, giving rise to more conditioned hallucinations. Results indicate that fundamentally different perceptual processes may be at the core of these seemingly similar experiences.
Topics: Humans; Synesthesia; Perceptual Disorders; Color Perception; Hallucinations; Caffeine
PubMed: 38453946
DOI: 10.1038/s41598-024-53663-3