Leaderboard

  1. To make submissions to the leaderboard, you will need to sign up first with your email address at the competition page. A verification email will be sent to that address. Please click the link in the email to activate your account.

  2. Then go to the competition page and click the Participate tab to start the registration.

  3. After that, you will receive an email to confirm your registration. The email includes a Google Form link. Please submit the Google Form to finilize your registration.

  4. We (organizers) will receive a notification after you submit the form and we will approve your participation.

  5. Then you can download the test set from the Participate → Get Data tab and start to make submissions in the Participate → Submit / View Results tab.

    It usually takes ~1 hour to finish the evaluation for each submission. You can click the Refresh status to check the evaluation status in the Participate → Submit / View Results tab.

Note

In the leaderboard, there are two official entries:

  • The entry submitted by the user urgent represents the official baseline.
  • The entry submitted by the user noisy represents the result of the original noisy input speech (without enhancement).

How to read the leaderboard?

  • Each row represents the best submission from one user (team).
  • For all evaluation phases, there are four categories of metrics for each submission: non-intrusive SE metrics, intrusive SE metrics, downstream-task-independent metrics, and downstream-task-dependent metrics. Each category includes several metrics.
    • For each metric cell, the first number represents the metric value, and the second number in parentheses represents the rank of this metric among all submissions.
    • For the final blind test phase, an additional category (Subjective SE metrics) will be added to evaluate the MOS score. See Data for more information.
  • The last column (overall ranking score) represents the final score used for ranking all submissions. It is calculated following the rules defined in the Rules tab.