1st MeViS Chanllenge

Introduction

The 1st MeViS challenge will be held in conjunction with CVPR 2024 PVUW Workshop in Seattle, USA. In this edition of the workshop and challenge, we focus on referring video segmentation with motion expressions, i.e., segmenting objects in video content based on a sentence describing the motion of the objects. MeViS contains 2,006 video clips and 443k high-quality object segmentation masks, with 28,570 sentences indicating 8,171 objects in complex environments. The goal of MeViS dataset is to provide a platform that enables the development of effective language-guided video segmentation algorithms that leverage motion expressions as a primary cue for object segmentation in complex video scenes. The workshop will culminate in a round table discussion, in which speakers will debate the future of video object representations.

Leaderboard

**TABLE 1. Top 3 Leaderboard of MeViS Challenge in CVPR 2024 PVUW Workshop.**
Team Name	Team Members	Organization	Technical Report	*J&F* \| J \| F
Tapall.ai	Mingqi Gao^1,2,4, Jingnan Luo², Jinyu Yang¹, Jungong Han^3,4, Feng Zheng^1,2	¹Tapall.ai, ²Southern University of Science and Technology, ³University of Sheffield, ⁴University of Warwick	PDF Video	54.5 \| 50.5 \| 58.5
CASIA_IVA	Bin Cao^1,2,3, Yisi Zhang⁴, Xuanxu Lin², Xingjian He¹, Bo Zhao³, Jing Liu^1,2	¹Institute of Automation, Chinese Academy of Sciences, ²School of Artificial Intelligence, University of Chinese Academy of Sciences, ³Beijing Academy of Artificial Intelligence, ⁴University of Science and Technology Beijing	PDF	54.2 \| 51.0 \| 57.4
TIME	Feiyu Pan¹, Hao Fang¹, Xiankai Lu¹	¹School of Software, Shandong University	PDF	51.5 \| 46.1 \| 56.9

Dates

● 1 Feb 2024: Release the training and validation dataset, check [here].
● 1 Feb 2024: Setup the submission server on CodaLab and open the submission of the validation results.
● 8 Apr 2024: Workshop paper submission deadline.
● 12 Apr 2024: Notification to authors of workshop paper.
● 15 May 2024: Release the test dataset and open the submission of the test results.
● 25 May 2024: Challenge submission end.
● 30 May 2024: The final competition results will be announced and high-performance teams will be invited.
● 17 Jun 2024: The workshop begins.

Rules

● Extra training datasets besides MeViS are allowed, but contestants must disclose any extra datasets used.
● There is no limitations to the models, large models like SAM can be used, but contestants must report the models used.

Call for Papers

This workshop includes workshop papers, covering but not limit to the following topics:
● Semantic/panoptic segmentation for images/videos
● Video object/instance segmentation
● Efficient computation for video scene parsing
● Object tracking
● Language-guided segmentation
● Semi-supervised recognition in videos
● New metrics to evaluate the quality of video scene parsing results
● Real-world video applications, including autonomous driving, indoor robotics, visual navigation, etc.

Submission: We invite authors to submit unpublished papers (8-page CVPR format) to our workshop, to be presented at a poster session upon acceptance. All submissions will go through a double-blind review process. Accepted papers will be published in the official CVPR Workshops proceedings and the Computer Vision Foundation (CVF) Open Access archive. All contributions must be submitted (along with supplementary materials, if any) at this link.
Paper Submission Dates:
● Workshop paper submission deadline: 8 April 2024 (23:59 PST)
● Notification to authors: 12 April 2024
● Camera ready deadline: 14 April 2024

MeViS Dataset Examples

Evaluation

Online Evaluation (🔥ready now!)

● We use Region Jaccard J, Boundary F measure F, and their mean J&F as the evaluation metrics.
● For the validation sets, the expressions are released to indicate the objects that are considered in evaluation.
● The validation set online evaluation server is [here] for daily evaluation.
● The test set online evaluation server will be open during the competition period only (TBD).

MeViS Challenge Organizers


Henghui Ding Primary Organizer Fudan University	Chang Liu Primary Organizer Nanyang Technological University	Shuting He Nanyang Technological University	Xudong Jiang Nanyang Technological University	Chen Change Loy Nanyang Technological University

BibTeX

Please consider to cite MeViS if it helps your research.

@inproceedings{MeViS,
  title={{MeViS}: A Large-scale Benchmark for Video Segmentation with Motion Expressions},
  author={Ding, Henghui and Liu, Chang and He, Shuting and Jiang, Xudong and Loy, Chen Change},
  booktitle={ICCV},
  year={2023}
}

License

MeViS is licensed under a CC BY-NC-SA 4.0 License. The data of MeViS is released for non-commercial research purpose only.

CVPR 2024 Motion Expression Video Segmentation Challenge