Leaderboard

The blind test leaderboard has been closed since the challenge has completed. The final results can be found at https://urgent-challenge.com/competitions/5#final_results.

The blind test leaderboard is now live at https://urgent-challenge.com.

  1. To make submissions to the leaderboard, you will need to sign up first with your email address. A verification email will be sent to that address. Please click the link in the email to activate your account.

  2. Then go to the competition page and click the Participate tab to start the registration.

  3. After that, you will receive an email message to confirm your registration. Please click the Google Form link in the email to submit the information of your team.

  4. We (organizers) will receive a notification after you submit the form and we will approve your participation.

  5. Then you can download the test set from the Participate → Get Data tab and start to make submissions in the Participate → Submit / View Results tab.

    It usually takes ~1 hour to finish the evaluation for each submission. You can click the Refresh status to check the evaluation status in the Participate → Submit / View Results tab.

Note

In the leaderboard, there are two official entries:

  • The entry submitted by the user urgent represents the official baseline.
  • The entry submitted by the user noisy represents the result of the original noisy input speech (without enhancement).

How to read the leaderboard?

  • Each row represents the best submission from one user (team).
  • For all evaluation phases, there are four categories of metrics for each submission: non-intrusive SE metrics, intrusive SE metrics, downstream-task-independent metrics, and downstream-task-dependent metrics. Each category includes several metrics.
    • For each metric cell, the first number represents the metric value, and the second number in parentheses represents the rank of this metric among all submissions.
    • For the final blind test phase, an additional category (Subjective SE metrics) will be added to evaluate the MOS score. See Data for more information.
  • The last column (overall ranking score) represents the final score used for ranking all submissions. It is calculated following the rules defined in the Rules tab.