Predicting Visual Memory Schemas with Variational Autoencoders

Cameron Kyle-Davidson (University of York), Adrian Bors (University of York), Karla Evans (University of York)

Abstract
Visual memory schema (VMS) maps show which regions of an image cause that image to be remembered or falsely remembered. Previous work has succeeded in generating low resolution VMS maps using convolutional neural networks. We instead approach this problem as an image-to-image translation task making use of a variational autoencoder. This approach allows us to generate higher resolution dual channel images that represent visual memory schemas, allowing us to evaluate predicted true memorability and false memorability separately. We also evaluate the relationship between VMS maps, predicted VMS maps, ground truth memorability scores, and predicted memorability scores.

DOI
10.5244/C.33.150
https://dx.doi.org/10.5244/C.33.150

Files
Paper (PDF)

BibTeX