Similarity judgments and visual working memory do not sharethe same cognitive representation

Abstract

Many cognitive models assume a set of processes operating upon a psychological representation that is derived using multidimensional scaling (MDS) of similarity judgments. While MDS-based models have been useful in exploring cognitive phenomena, the underlying representation itself should not be left unexplained. The issue of defining the appropriate cognitive representation recently came to the fore in the field of visual working memory (VWM). Most VWM models are constructed on the physical stimulus space, but Schurgin et al. (2020) influentially argued that VWM is better modeled on the psychological similarity space. To address this discrepancy, we took a Bayesian generative modeling approach, applying cognitive models to the open data of three tasks from Tomić & Bays (2024): a perceptual reproduction task, and a memory reproduction task, and a similarity comparison task. We show that the underlying representation for these tasks can be recovered with this approach, and find that there is non-uniformity in the representations with respect to the physical stimuli. Our critical result is that the representation differs between the similarity comparison and stimulus reproduction tasks. This result suggests that the psychological representation derived from similarity tasks cannot be assumed to be the representation underlying VWM. We also show that by including detailed mechanisms in our cognitive models (in this case, swap errors in the memory reproduction task) improves the recovered mental representation. We conclude by arguing that mental representations should be explained by being included in cognitive process models rather than derived from independent tasks like similarity comparisons.

View Preprint on PsyArXiv

Data and Code