Vision-Language Models – Uppsala Vision, Language and Learning

Multimodal Deep Learning and Vision-Language Models

Our PhD student Robin Hollifeldt and PostDoc Tianru Zhang are working on multimodal deep learning and vision-language models. We are studying how combining language with images/videos can lead to better representation of events around us, making AI models more capable and intelligent.

The project is a part of the Beijer Laboratory for Artificial Intelligence Research, funded by Kjell och Märta Beijer Foundation.

Robin Hollifeldt is working in close collaboration with supervisors Asst. Prof. Ekta Vats and Prof. Thomas Schön.

Tianru Zhang is working is close collaboration with Assoc. Prof. Prashant Singh and the Sciml group at UU.

Related Publication:

Li Ju, Mayank Nautiyal, Andreas Hellander, Ekta Vats and Prashant Singh, Epistemic Uncertainty Quantification for Pre-trained VLMs via Riemannian Flow Matching, Forty-Third International Conference on Machine Learning (ICML), Accepted, 2026. arXiv preprint
Robin Hollifeldt, Tianru Zhang, Jessica Ström and Ekta Vats, Generative Text Modeling and Language-Guided Visual Attention for Deciphering Palimpsests, Transactions of the Association for Computational Linguistics (TACL), Accepted, 2026.
Mayank Nautiyal, Tianru Zhang, Li Ju, Ekta Vats and Prashant Singh, PROVE: Probabilistic Visual Grounding via Patch-Level Evidence for Vision-Language Models, the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Accepted, 2026.
Li Ju, Max Andersson, Stina Fredriksson, Edward Glöckner, Andreas Hellander, Ekta Vats and Prashant Singh, Exploiting the Asymmetric Uncertainty Structure of Pre-trained VLMs on the Unit Hypersphere. Advances in neural information processing systems 38 (2026): 121317-121335. arXiv preprint (NeurIPS poster link)

Multispectral Imaging

In collaboration with team MISHA at the Rochester Institute of Technology, Ekta Vats and her group have built a cost-effective MSI system in the lab that will enable research studies on degraded manuscripts. This pre-study was partially supported by the Kjell och Märta Beijer Foundation and our heartfelt gratitude to Prof. Thomas Schön for his encouragement and support. Together with the National Library of Sweden, we aim at potentially leading MSI-based digitisation and research on historical manuscripts in Sweden.