Leaderboard
-
To make submissions to the leaderboard, you will need to sign up first with your email address at the competition page. A verification email will be sent to that address. Please click the link in the email to activate your account.
-
Then go to the competition page and click the
Participate
tab to start the registration. -
After that, you will receive an email to confirm your registration. The email includes a Google Form link. Please submit the Google Form to finilize your registration.
-
We (organizers) will receive a notification after you submit the form and we will approve your participation.
-
Then you can download the test set from the
Participate → Get Data
tab and start to make submissions in theParticipate → Submit / View Results
tab.It usually takes ~1 hour to finish the evaluation for each submission. You can click the
Refresh status
to check the evaluation status in theParticipate → Submit / View Results
tab.
Note
In the leaderboard, there are two official entries:
- The entry submitted by the user
urgent
represents the official baseline. - The entry submitted by the user
noisy
represents the result of the original noisy input speech (without enhancement).
How to read the leaderboard?
- Each row represents the best submission from one user (team).
- For all evaluation phases, there are four categories of metrics for each submission: non-intrusive SE metrics, intrusive SE metrics, downstream-task-independent metrics, and downstream-task-dependent metrics. Each category includes several metrics.
- For each metric cell, the first number represents the metric value, and the second number in parentheses represents the rank of this metric among all submissions.
- For the final blind test phase, an additional category (
Subjective SE metrics
) will be added to evaluate the MOS score. SeeData
for more information.
- The last column (overall ranking score) represents the final score used for ranking all submissions. It is calculated following the rules defined in the
Rules
tab.