Dec 9, 2022 · We study various pretraining architectures and objectives within the masked autoencoding framework, motivated by the success of similar methods in natural ...

Here, we focus on a thorough comparison of multiple architectures and ob- jectives for audiovisual masked autoencoders. In contrast, those works explore ...

This repository contains the official implementation (in PyTorch) of the Contrastive Audio-Visual Masked Autoencoder (CAV-MAE) proposed in the ICLR 2023 paper.

Our audiovisual pretraining enables us to achieve state-of-the-art results in downstream, audiovisual datasets such as VGGSound and AudioSet. Moreover, we show ...

Here, we focus on a thorough comparison of multiple architectures and ob- jectives for audiovisual masked autoencoders. In contrast, those works explore ...

Supplementary Material: Audiovisual Masked Autoencoders. Mariana-Iuliana ... Tables A4, A5 and A6 ablate the effect of the masking ratio in the case of ...

This work shows that masked autoencoding can be used to train a simple Vision Transformer on images and videos, without requiring any labeled data.

AV-MAE [18] is a joint masked autoencoder for audio, visual, and joint audio/visual classification. The authors explore different encoding policies for dual- ...

Duration: 5:01
Posted: Jun 23, 2024