Computer Vision for Memorability
Info
Reimplemementing and testing Khosla et al.’s MemNet[1], and beyond.
Goals
The original MemNet has proved difficult to use and access in recent years. We would like to implement it so that we can use their work for our own research. The short term goal is to simply get it working. In the long term we’d like to improve upon the architecture using pre-training, and possibly some other progress in neural network architecture that has been made since 2015. In addition we’d like to get a tool running for in-lab use. This will be implemented in PyTorch.
A decent amount of progress has been made on this project so far. The progress will be chronicled in a number of blog posts.
A release version of ResMem, the best performing model I’ve made so far (at least that isn’t too large to run on personal computers) is publicly available! The source code can be found at github. But there’s an easy-to-install version on PyPI, with a web interface coming soon. Just type into your console:
pip install resmem
Resources
1 : Khosla, Aditya, Akhil S. Raju, Antonio Torralba, and Aude Oliva. 2015.“Understanding and Predicting Image Memorability at a Large Scale.” In2015 Ieee International Conference on Computer Vision (Iccv), 2390–8. IEEE.https://doi.org/10.1109/ICCV.2015.275.1
2 : Bordelon, B., Canatar, A., & Pehlevan, C. (2020). Spectrum dependent learning curves in kernel regression and wide neural networks. ArXiv:2002.02561 [Cs, Stat]. http://arxiv.org/abs/2002.02561
3 : L. Goetschalckx, A. Andonian, A. Oliva, and P. Isola. GANalyze: Toward Visual Definitions of Cognitive Image Properties. , 2019. http://ganalyze.csail.mit.edu/