Page Not Found
Page not found. Your pixels are in another canvas.
Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
Juan Montesinos
About me
Page not in menu
This is a page not in th emain menu
Posts
Always inspect your data: The Grid Audio-Visual Speech Corpus
Published:
Inspecting the training data and the results is usually underrated task which can help us to understand how our models works. Today I will show how apparently bad quality samples generated for lip reading revealed issues with the The Grid Audio-Visual Speech Corpus dataset.
Visualizing Pytorch’s transformer activations
Published:
Transformers are a trendy architecture nowadays. The paralellism and possibility of working with sequences of different lenghts allowed this architecture to achieve awesome results in different fields. Today we are gonna learn how to visualize the attention probabilities when using pytorch’s official transformer modules.
Optimal Remote working
Published:
After the COVID19 pandemic, working on remote/hybrid is starting to be really common. In this post I show some tools to work from remote being data scientist, ML engineer, python developer or similar.
Speech separation with voice identity
Published:
We usually claim that audio-visual methods performs better than audio-only in blind sound source separation. We are gonna check the performance of audio-only methods by doing a simple experiment. To code a U-Net to perform source separation using identiy embeddings.
Masks in sound source separation: An ablation
Published:
In this blogpost I explain how masking works in sound source separation. It adresses binary mask and complex masks. An ablation study on their performance for the two-sources case is carried out.
Paper: Visually Guided Sound Source Separation using Cascaded Opponent Filter Network
Published:
PDF
Year: 2020
Goal: Audio-visual sound source separation
Library: MIR Eval library ported to pytorch
Published:
I’ve ported MIR eval library to PyTorch enabling a faster evaluation.
It’s is available here –> torch_mir_eval
- Only the Source separation functions has been ported, even though the project is opensource and open to grow.
Tutorial: Loading AudioVisual Content with Nvidia DALI
Published:
Tutorial: Loading AudioVisual Content with Nvidia DALI
Paper: Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos
Published:
PDF
Year: 2020
Conference: CVPR
Goal: Weakly supervised pose detection
Resume: Self-supervised Learning of Interpretable Keypoints from Unlabelled Videos
Paper: End-to-end weakly-supervised semantic alignment
Published:
PDF
Year: 2018
Conference: CVPR
Goal:Aligning two mages depicting objects of the same category
Tutorial: Audio Preprocessing
Published:
Let’s show how to preprocess audio data to be used in Deep Learning. To do so we are going to use two very standard libraries. numpy and librosa.
portfolio
Working from remote in Ecuador!
Published:
The experience of working from remote in Ecuador! 
publications
Solos: A Dataset for Audio-Visual Music Source Separation and Localization
Published in IEEE MMSP2020, 2020
Audiovisual Dataset of musicians playing different instruments. Openpose skeleton provided framewise
Recommended citation: Juan F. Montesinos, Olga Slizovskaia, Gloria Haro (2020). "Solos: A Dataset for Audio-Visual Music Source Separation and Localization" IEEE MMSP 2020 1. https://arxiv.org/pdf/2006.07931.pdf
Multi-channel U-Net for Music Source Separation
Published in IEEE MMSP 2020, 2020
Weighted losses applied to a Multi-channel U-Net
Recommended citation: Venkatesh Shenoy, Juan F. Montesinos, Gloria Haro, Emilia Gómez (2020). "Multi-channel U-Net for Music Source Separation." IEEE MMSP2020 1. https://arxiv.org/pdf/2003.10414.pdf
Estimating Individual A Cappella Voices in Music Videos with Singing Faces
Published in S&S CVPR21, 2021
Audiovisual singing voice separation
Recommended citation: Venkatesh S. Kadandale,Juan F. Montesinos, Gloria Haro (2021). "Estimating Individual A Cappella Voices in Music Videos with Singing FacesS6S CVPR21 https://sightsound.org/papers/2021/Venkatesh_Shenoy_Kadandale_Estimating_Individual_A_Cappella_Voices_in_Music_Videos_with_Singing_Faces.pdf
A cappella: Audio-visual Singing Voice Separation
Published in BMVC 2021, 2021
Graph CNN for singing voice separation
Recommended citation: Juan F. Montesinos, Venkatesh S. Kadandale, Gloria Haro (2021). "A cappella: Audio-visual Singing Voice SeparationBMVC 21 https://arxiv.org/abs/2104.09946
VocaLiST: An Audio-Visual Synchronisation Model for Lips and Voices
Published in Under review, 2022
Transformer for AV synchronization
Recommended citation: Venkatesh S. Kadandale, Juan F. Montesinos, Gloria Haro (2022). "VocaLiST: An Audio-Visual Synchronisation Model for Lips and VoicesReview https://arxiv.org/abs/2204.02090
VoViT: Low Latency Graph-based Audio-Visual Voice Separation Transformer
Published in Under review, 2022
AV Transformer for voice separation
Recommended citation: Juan F. Montesinos, Venkatesh S. Kadandale, Gloria Haro (2022). "VoViT: Low Latency Graph-based Audio-Visual Voice Separation Transformerreview https://arxiv.org/abs/2203.04099
talks
Talk 1 on Relevant Topic in Your Field
Published:
This is a description of your talk, which is a markdown files that can be all markdown-ified like any other post. Yay markdown!
Conference Proceeding talk 3 on Relevant Topic in Your Field
Published:
This is a description of your conference proceedings talk, note the different field in type. You can put anything in this field.
teaching
Calculus II
Undergraduate course, Pompeu Fabra University, DTIC, 2019
Source separation in musical videos via motion analysis
Degree's Dissertation, Pompeu Fabra University, DTIC, 2019
Repository
Resume EN: This project proposes a method for the task of audio source separation of a signal, based on the movements of the players related to that signal. The process is composed of three blocks. The first block, computes a frequential analysis of the original signal by Non- negative Matrix Factorization (NMF). The video processing block estimates the velocity signal of the movements of each player by two types of video segmentation: the first one is based on motion trajectories of the objects in the scene, while the second one, uses optical flow and Principal Component Analysis. The last processing block makes a cor- relation between the frequential information and the velocity signals, using four variation of a method based on NMF and Non-Negative Least Squares. Finally, some experiments show the efficacy of the different variants of the audio source separation method.