Leaderboard

The validation leaderboard is now live at https://urgent-challenge.com.

  1. To make submissions to the leaderboard, you will need to sign up first with your email address. A verification email will be sent to that address. Please click the link in the email to activate your account.

  2. Then go to the competition page and click the Participate tab to start the registration.

  3. After that, you will receive an email message to confirm your registration. Please click the Google Form link in the email to submit the information of your team.

  4. We (organizers) will receive a notification after you submit the form and we will approve your participation.

  5. Finally, you can download the validation subset from the Participate → Get Data tab and start to make submissions in the Participate → Submit / View Results tab.

    It usually takes ~1 hour to finish the evaluation for each submission. You can click the Refresh status to check the evaluation status in the Participate → Submit / View Results tab.

Note

In the leaderboard, there are two official entries:

  • The entry submitted by the user urgent represents the official baseline.
  • The entry submitted by the user noisy represents the result of the original noisy input speech (without enhancement).

How to read the leaderboard?

  • Each row represents the best submission from one user (team).
  • There are four categories of metrics for each submission: non-intrusive SE metrics, intrusive SE metrics, downstream-task-independent metrics, and downstream-task-dependent metrics. Each category includes several metrics.
    • For each metric cell, the first number represents the metric value, and the second number in parentheses represents the rank of this metric among all submissions.
  • The last column (overall ranking score) represents the final score used for ranking all submissions. It is calculated following the rules defined in the Rules tab.