Introduction

The 1st MeViS challenge will be held in conjunction with CVPR 2024 PVUW Workshop in Seattle, USA. In this edition of the workshop and challenge, we focus on referring video segmentation with motion expressions, i.e., segmenting objects in video content based on a sentence describing the motion of the objects. MeViS contains 2,006 video clips and 443k high-quality object segmentation masks, with 28,570 sentences indicating 8,171 objects in complex environments. The goal of MeViS dataset is to provide a platform that enables the development of effective language-guided video segmentation algorithms that leverage motion expressions as a primary cue for object segmentation in complex video scenes. The workshop will culminate in a round table discussion, in which speakers will debate the future of video object representations.

Dates

  ● 1 Feb 2024: Release the training and validation dataset, check [here].
  ● 1 Feb 2024: Setup the submission server on CodaLab and open the submission of the validation results.
  ● 8 Apr 2024: Workshop paper submission deadline.
  ● 12 Apr 2024: Notification to authors of workshop paper.
  ● 15 May 2024: Release the test dataset and open the submission of the test results.
  ● 30 May 2024: The final competition results will be announced and high-performance teams will be invited.
  ● 17 Jun 2024: The workshop begins.

Rules

  ● Extra training datasets besides MeViS are allowed, but contestants must disclose any extra datasets used.
  ● There is no limitations to the models, large models like SAM can be used, but contestants must report the models used.

Call for Papers

This workshop includes workshop papers, covering but not limit to the following topics:
  ● Semantic/panoptic segmentation for images/videos
  ● Video object/instance segmentation
  ● Efficient computation for video scene parsing
  ● Object tracking
  ● Language-guided segmentation
  ● Semi-supervised recognition in videos
  ● New metrics to evaluate the quality of video scene parsing results
  ● Real-world video applications, including autonomous driving, indoor robotics, visual navigation, etc.

Submission: We invite authors to submit unpublished papers (8-page CVPR format) to our workshop, to be presented at a poster session upon acceptance. All submissions will go through a double-blind review process. Accepted papers will be published in the official CVPR Workshops proceedings and the Computer Vision Foundation (CVF) Open Access archive. All contributions must be submitted (along with supplementary materials, if any) at this link.
Paper Submission Dates:
  ● Workshop paper submission deadline: 8 April 2024 (23:59 PST)
  ● Notification to authors: 12 April 2024
  ● Camera ready deadline: 14 April 2024

MeViS Dataset Examples

0442a954  d321dde4  02221fb0  bbe97d18 

Evaluation



  • ● We use Region Jaccard J, Boundary F measure F, and their mean J&F as the evaluation metrics.
    ● For the validation sets, the expressions are released to indicate the objects that are considered in evaluation.
    ● The validation set online evaluation server is [here] for daily evaluation.
    ● The test set online evaluation server will be open during the competition period only (TBD).

    MeViS Challenge Organizers

    Henghui Ding
    Chang Liu
    Shuting He
    Xudong Jiang
    Philip H.S. Torr
    Henghui Ding

    Primary Organizer
    Fudan University

    Chang Liu

    Primary Organizer
    Nanyang Technological University

    Shuting He

    Nanyang Technological University

    Xudong Jiang

    Nanyang Technological University

    Chen Change Loy

    Nanyang Technological University

    BibTeX

    Please consider to cite MeViS if it helps your research.
    @inproceedings{MeViS,
      title={{MeViS}: A Large-scale Benchmark for Video Segmentation with Motion Expressions},
      author={Ding, Henghui and Liu, Chang and He, Shuting and Jiang, Xudong and Loy, Chen Change},
      booktitle={ICCV},
      year={2023}
    }

    License

    Creative Commons License
    MeViS is licensed under a CC BY-NC-SA 4.0 License. The data of MeViS is released for non-commercial research purpose only.