-
Neural Networks : the Official Journal... Jul 2023The contrast sensitivity function (CSF) is a fundamental signature of the visual system that has been measured extensively in several species. It is defined by the...
The contrast sensitivity function (CSF) is a fundamental signature of the visual system that has been measured extensively in several species. It is defined by the visibility threshold for sinusoidal gratings at all spatial frequencies. Here, we investigated the CSF in deep neural networks using the same 2AFC contrast detection paradigm as in human psychophysics. We examined 240 networks pretrained on several tasks. To obtain their corresponding CSFs, we trained a linear classifier on top of the extracted features from frozen pretrained networks. The linear classifier is exclusively trained on a contrast discrimination task with natural images. It has to find which of the two input images has higher contrast. The network's CSF is measured by detecting which one of two images contains a sinusoidal grating of varying orientation and spatial frequency. Our results demonstrate characteristics of the human CSF are manifested in deep networks both in the luminance channel (a band-limited inverted U-shaped function) and in the chromatic channels (two low-pass functions of similar properties). The exact shape of the networks' CSF appears to be task-dependent. The human CSF is better captured by networks trained on low-level visual tasks such as image-denoising or autoencoding. However, human-like CSF also emerges in mid- and high-level tasks such as edge detection and object recognition. Our analysis shows that human-like CSF appears in all architectures but at different depths of processing, some at early layers, while others in intermediate and final layers. Overall, these results suggest that (i) deep networks model the human CSF faithfully, making them suitable candidates for applications of image quality and compression, (ii) efficient/purposeful processing of the natural world drives the CSF shape, and (iii) visual representation from all levels of visual hierarchy contribute to the tuning curve of the CSF, in turn implying a function which we intuitively think of as modulated by low-level visual features may arise as a consequence of pooling from a larger set of neurons at all levels of the visual system.
Topics: Humans; Contrast Sensitivity; Visual Perception; Neurons; Neural Networks, Computer; Psychophysics; Pattern Recognition, Visual
PubMed: 37156217
DOI: 10.1016/j.neunet.2023.04.032 -
Psychonomic Bulletin & Review Oct 2020Evidence accumulation models like the diffusion model are increasingly used by researchers to identify the contributions of sensory and decisional factors to the speed... (Review)
Review
Evidence accumulation models like the diffusion model are increasingly used by researchers to identify the contributions of sensory and decisional factors to the speed and accuracy of decision-making. Drift rates, decision criteria, and nondecision times estimated from such models provide meaningful estimates of the quality of evidence in the stimulus, the bias and caution in the decision process, and the duration of nondecision processes. Recently, Dutilh et al. (Psychonomic Bulletin & Review 26, 1051-1069, 2019) carried out a large-scale, blinded validation study of decision models using the random dot motion (RDM) task. They found that the parameters of the diffusion model were generally well recovered, but there was a pervasive failure of selective influence, such that manipulations of evidence quality, decision bias, and caution also affected estimated nondecision times. This failure casts doubt on the psychometric validity of such estimates. Here we argue that the RDM task has unusual perceptual characteristics that may be better described by a model in which drift and diffusion rates increase over time rather than turn on abruptly. We reanalyze the Dutilh et al. data using models with abrupt and continuous-onset drift and diffusion rates and find that the continuous-onset model provides a better overall fit and more meaningful parameter estimates, which accord with the known psychophysical properties of the RDM task. We argue that further selective influence studies that fail to take into account the visual properties of the evidence entering the decision process are likely to be unproductive.
Topics: Decision Making; Humans; Models, Psychological; Motion Perception; Pattern Recognition, Visual; Psychophysics
PubMed: 32514800
DOI: 10.3758/s13423-020-01742-7 -
Journal of Vision Jul 2020Current computational models of visual salience accurately predict the distribution of fixations on isolated visual stimuli. It is not known, however, whether the global...
Current computational models of visual salience accurately predict the distribution of fixations on isolated visual stimuli. It is not known, however, whether the global salience of a stimulus, that is, its effectiveness in the competition for attention with other stimuli, is a function of the local salience or an independent measure. Further, do task and familiarity with the competing images influence eye movements? Here, we investigated the direction of the first saccade to characterize and analyze the global visual salience of competing stimuli. Participants freely observed pairs of images while eye movements were recorded. The pairs balanced the combinations of new and already seen images, as well as task and task-free trials. Then, we trained a logistic regression model that accurately predicted the location-left or right image-of the first fixation for each stimulus pair, accounting too for the influence of task, familiarity, and lateral bias. The coefficients of the model provided a reliable measure of global salience, which we contrasted with two distinct local salience models, GBVS and Deep Gaze. The lack of correlation of the behavioral data with the former and the small correlation with the latter indicate that global salience cannot be explained by the feature-driven local salience of images. Further, the influence of task and familiarity was rather small, and we reproduced the previously reported left-sided bias. Summarized, we showed that natural stimuli have an intrinsic global salience related to the human initial gaze direction, independent of the local salience and little influenced by task and familiarity.
Topics: Adult; Attention; Computer Simulation; Female; Fixation, Ocular; Humans; Male; Psychophysics; Saccades; Visual Perception; Young Adult
PubMed: 32720973
DOI: 10.1167/jov.20.7.27 -
Nature Communications Jul 2022The brain adapts dynamically to the changing sensory statistics of its environment. Recent research has started to delineate the neural circuitries and representations...
The brain adapts dynamically to the changing sensory statistics of its environment. Recent research has started to delineate the neural circuitries and representations that support this cross-sensory plasticity. Combining psychophysics and model-based representational fMRI and EEG we characterized how the adult human brain adapts to misaligned audiovisual signals. We show that audiovisual adaptation is associated with changes in regional BOLD-responses and fine-scale activity patterns in a widespread network from Heschl's gyrus to dorsolateral prefrontal cortices. Audiovisual recalibration relies on distinct spatial and decisional codes that are expressed with opposite gradients and time courses across the auditory processing hierarchy. Early activity patterns in auditory cortices encode sounds in a continuous space that flexibly adapts to misaligned visual inputs. Later activity patterns in frontoparietal cortices code decisional uncertainty consistent with these spatial transformations. Our findings suggest that regions within the auditory processing hierarchy multiplex spatial and decisional codes to adapt flexibly to the changing sensory statistics in the environment.
Topics: Acoustic Stimulation; Adult; Auditory Cortex; Auditory Perception; Brain Mapping; Humans; Magnetic Resonance Imaging; Photic Stimulation; Psychophysics; Visual Perception
PubMed: 35798733
DOI: 10.1038/s41467-022-31549-0 -
The Journal of the Acoustical Society... Feb 2020Negative masking (NM) is a ubiquitous finding in near-"threshold" psychophysics in which the detectability of a near-threshold signal improves when added to a copy of...
Negative masking (NM) is a ubiquitous finding in near-"threshold" psychophysics in which the detectability of a near-threshold signal improves when added to a copy of itself, i.e., a pedestal or masker. One interpretation of NM suggests that the pedestal acts as an informative cue, thereby reducing uncertainty and improving performance relative to detection in its absence. The purpose of this study was to test this hypothesis. Intensity discrimination thresholds were measured for 100-ms, 1000-Hz near-threshold tones. In the reference condition, thresholds were measured in quiet (no masker other than the pedestal). In comparison conditions, thresholds were measured in the presence of one of two additional maskers: a notched-noise masker or a random-frequency multitone masker. The additional maskers were intended to cause different amounts of uncertainty and, in turn, to differentially influence NM. The results were generally consistent with an uncertainty-based interpretation of NM: NM was found both in quiet and in notched-noise, yet it was eliminated by the multitone masker. A competing interpretation of NM based on nonlinear transduction does not account for all of the results. Profile analysis may have been a factor in performance and this suggests that NM may be attributable to, or influenced by, multiple mechanisms.
Topics: Auditory Threshold; Noise; Perceptual Masking; Psychophysics; Uncertainty
PubMed: 32113297
DOI: 10.1121/10.0000652 -
IEEE Transactions on Visualization and... Nov 2022Virtual Reality (VR) is becoming ubiquitous with the rise of consumer displays and commercial VR platforms. Such displays require low latency and high quality rendering...
Virtual Reality (VR) is becoming ubiquitous with the rise of consumer displays and commercial VR platforms. Such displays require low latency and high quality rendering of synthetic imagery with reduced compute overheads. Recent advances in neural rendering showed promise of unlocking new possibilities in 3D computer graphics via image-based representations of virtual or physical environments. Specifically, the neural radiance fields (NeRF) demonstrated that photo-realistic quality and continuous view changes of 3D scenes can be achieved without loss of view-dependent effects. While NeRF can significantly benefit rendering for VR applications, it faces unique challenges posed by high field-of-view, high resolution, and stereoscopic/egocentric viewing, typically causing low quality and high latency of the rendered images. In VR, this not only harms the interaction experience but may also cause sickness. To tackle these problems toward six-degrees-of-freedom, egocentric, and stereo NeRF in VR, we present the first gaze-contingent 3D neural representation and view synthesis method. We incorporate the human psychophysics of visual- and stereo-acuity into an egocentric neural representation of 3D scenery. We then jointly optimize the latency/performance and visual quality while mutually bridging human perception and neural scene synthesis to achieve perceptually high-quality immersive interaction. We conducted both objective analysis and subjective studies to evaluate the effectiveness of our approach. We find that our method significantly reduces latency (up to 99% time reduction compared with NeRF) without loss of high-fidelity rendering (perceptually identical to full-resolution ground truth). The presented approach may serve as the first step toward future VR/AR systems that capture, teleport, and visualize remote environments in real-time.
Topics: Humans; Computer Graphics; Virtual Reality; User-Computer Interface; Psychophysics
PubMed: 36044494
DOI: 10.1109/TVCG.2022.3203102 -
PloS One 2020The widely known hue-heat effect, a multisensory phenomenon between vision and thermal sensing, is a hypothesis based on the idea that light and colors affect perceived...
The widely known hue-heat effect, a multisensory phenomenon between vision and thermal sensing, is a hypothesis based on the idea that light and colors affect perceived temperature. However, the application of this effect has not been prevalent in our daily lives. To work towards developing more practical use of the hue-heat effect, we conducted a series of psychophysical experiments to investigate the relationship between perceived temperature and illumination in a well-controlled experimental environment. The results showed that illumination had three types of effects to change our sense of coolness/warmness: creating, eliminating, and exchanging effects. Furthermore, we confirmed the existence of two distinctive time courses for the three effects: creating effect started immediately, but the eliminating effect takes time. These findings provide us with a better understanding of the hue-heat effect and enable us to apply it in everyday life. Paired with the new technologies it can also help with energy conservation.
Topics: Color Perception; Female; Hot Temperature; Humans; Lighting; Male; Photic Stimulation; Psychophysics; Thermosensing; Young Adult
PubMed: 32776987
DOI: 10.1371/journal.pone.0236321 -
Quarterly Journal of Experimental... Apr 2023Laming has recently proposed a way to measure the accessibility (as opposed to availability) of memories via recognition testing. His measure "Accessibility" is...
Laming has recently proposed a way to measure the accessibility (as opposed to availability) of memories via recognition testing. His measure "Accessibility" is calculated by subtracting the hit rate and false alarm rate that fall at the point where the receiver-operating characteristic (ROC) curve's derivative is 1. I prove that, if one works within the framework of Unequal-Variance Signal Detection Theory (UVSDT), as Laming does, the measure "Accessibility" depends on the location of the response criterion (though always with a neutral likelihood ratio). Furthermore, I prove that the measure varies with the underlying variances of UVSDT regardless of which definition of bias (criterion or likelihood ratio) is used and, crucially, this holds even when the accuracy of discrimination performance or "sensitivity" in UVSDT is constant. As such, from the standpoint of (at least) UVSDT, it is questionable whether or to what extent the new measure of "Accessibility" actually measures the accessibility of any memory.
Topics: Humans; Signal Detection, Psychological; Recognition, Psychology; ROC Curve; Bias
PubMed: 35802038
DOI: 10.1177/17470218221113559 -
Scientific Reports Nov 2020Multi-sensory human-machine interfaces are currently challenged by the lack of effective, comfortable and affordable actuation technologies for wearable tactile displays...
Multi-sensory human-machine interfaces are currently challenged by the lack of effective, comfortable and affordable actuation technologies for wearable tactile displays of softness in virtual- or augmented-reality environments. They should provide fingertips with tactile feedback mimicking the tactual feeling perceived while touching soft objects, for applications like virtual reality-based training, tele-rehabilitation, tele-manipulation, tele-presence, etc. Displaying a virtual softness on a fingertip requires the application of quasi-static (non-vibratory) forces via a deformable surface, to control both the contact area and the indentation depth of the skin. The state of the art does not offer wearable devices that can combine simple structure, low weight, low size and electrically safe operation. As a result, wearable softness displays are still missing for real-life uses. Here, we present a technology based on fingertip-mounted small deformable chambers, which weight about 3 g and are pneumatically driven by a compact and cost-effective unit. Weighting less than 400 g, the driving unit is easily portable and can be digitally controlled to stimulate up to three fingertips independently. Psychophysical tests proved ability to generate useful perceptions, with a Just Noticeable Difference characterised by a Weber constant of 0.15. The system was made of off-the-shelf materials and components, without any special manufacturing process, and is fully disclosed, providing schematics and lists of components. This was aimed at making it easily and freely usable, so as to turn tactile displays of softness on fingertips into a technology 'at fingertips'.
Topics: Biomechanical Phenomena; Fingers; Humans; Psychophysics; Touch; Touch Perception
PubMed: 33235252
DOI: 10.1038/s41598-020-77591-0 -
Ophthalmic & Physiological Optics : the... Mar 2020Reading is vital to full participation in modern society. To millions of people suffering from macular disease that results in a central scotoma, reading is difficult... (Review)
Review
PURPOSE
Reading is vital to full participation in modern society. To millions of people suffering from macular disease that results in a central scotoma, reading is difficult and inefficient, rendering reading as the primary goal for most patients seeking low vision rehabilitation. The goals of this review paper are to summarize the dependence of reading speed on several key visual and typographical factors and the current methods or technologies for improving reading performance for people with macular disease.
IMPORTANT FINDINGS
In general, reading speed for people with macular disease depends on print size, text contrast, size of the visual span, temporal processing of letters and oculomotor control. Attempts at improving reading speed by reducing the crowding effect between letters, words or lines; or optimizing properties of typeface such as the presence of serifs or stroke-width thickness proved to be futile, with any improvement being modest at best. Currently, the most promising method to improve reading speed for people with macular disease is training, including perceptual learning or oculomotor training.
SUMMARY
The limitation on reading speed for people with macular disease is likely to be multi-factorial. Future studies should try to understand how different factors interact to limit reading speed, and whether different methods could be combined to produce a much greater benefit.
Topics: Humans; Macular Degeneration; Psychophysics; Reading; Vision Tests; Visual Acuity
PubMed: 31925832
DOI: 10.1111/opo.12664