🔍 Read the full analysis: Egyptian And Nigerian English Take Their Place On The Open ASR Leaderboard on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Hugging Face has added two private Liva AI English speech evaluation splits to its Open ASR Leaderboard: 1.34 hours of Egyptian-accented speech and 1.55 hours of Nigerian-accented speech. The recordings broaden the accents represented in the leaderboard, but the announcement does not detail speaker demographics or establish how representative the small samples are.
Hugging Face has added two private English speech evaluation splits from Liva AI to its Open ASR Leaderboard, covering Egyptian- and Nigerian-accented English. The conversational recordings total 2.89 hours and broaden the accents included in speech-recognition assessments, although the announcement does not provide enough information to judge how well the samples represent speakers in either country.
The new splits are named Liva AI Conversational EG and Liva AI Conversational NG. Hugging Face says the Egyptian split contains 1.34 hours of audio and the Nigerian split contains 1.55 hours. Both consist of spontaneous conversation. Their transcriptions are punctuated and cased, and retain disfluencies, according to the announcement.
The recordings are private evaluation data, rather than datasets available to download and use as ordinary public benchmarks. Hugging Face says private test material is intended to make evaluations more trustworthy by reducing developers’ ability to tune systems directly against known test examples. The announcement does not report how the additions have affected model scores or leaderboard rankings.
The leaderboard’s Private tab will present two comparisons: US versus non-US accents, and Inner Circle versus Outer and Expanding Circles. The non-US group currently includes the UK, Australia, Canada, India, Nigeria and Egypt. The circle-based grouping draws on linguist Braj Kachru’s framework, with the US, UK, Australia and Canada in the Inner Circle, and India, Nigeria and Egypt in the Outer and Expanding grouping as currently presented.
Egyptian and Nigerian English join the Open ASR Leaderboard
Hugging Face added private Liva AI speech splits to broaden accent evaluation. The short recordings add coverage, while leaving questions about how representative the results are.
What was added
Both datasets contain spontaneous conversation with punctuated, cased transcripts that retain disfluencies.
Liva AI Conversational EG
A private evaluation split for Egyptian-accented English.
Liva AI Conversational NG
A private evaluation split for Nigerian-accented English.
New ways to compare accents
The leaderboard’s Private tab will show results across two groupings.
- Non-US includes the UK, Australia, Canada, India, Nigeria and Egypt.
- Inner Circle compared with Outer and Expanding Circles, based on Kachru’s framework.
- Hugging Face plans to separate the latter circles as more accents are added.
Why private evaluation matters
Keeping test examples private may make it harder to tune systems directly to known benchmark material.
Users can inspect recognition results for two additional English varieties.
Scores describe these specific short test sets, not all speakers in either country.
Broader Accent Coverage, Small Samples
Automatic speech recognition can perform differently across varieties of English. A single overall score may not show how well a system handles every speaker, so adding Egyptian and Nigerian English gives users another way to inspect model performance beyond the accents already represented in the leaderboard’s listed groupings.
The additions are a limited expansion, not a broad national measure. The two splits contain less than three hours of audio combined, and Hugging Face has not disclosed speaker counts or demographic and regional details. Results would describe performance on these particular test sets; they cannot, on the information released, establish how a system performs across all Egyptian or Nigerian English speakers.
Private data may make it more difficult to optimize models against the exact evaluation examples. That can make benchmark results more useful as an assessment, but the announcement does not include an independent review of the splits or evidence showing how much the new comparisons change the leaderboard’s picture of model performance.
Products Worth Considering
Bose QuietComfort Wireless Noise-Cancelling Headphones, Cypress Green
Bose QuietComfort 45 SE Wireless Noise-Cancelling Over-Ear Headphones
Sony WH-1000XM5 Premium Noise Cancelling Wireless Headphones, Midnight Blue
As an affiliate, we earn on qualifying purchases.
Earlier Private Accent Evaluations
The Liva AI collaboration extends an existing effort to add private speech data to the Open ASR Leaderboard. Hugging Face says it previously added 11 private splits from Appen and DataOceanAI, covering scripted and conversational speech across five accents. It later added a private split for Indian-accented English in collaboration with Voice Arena.
The new comparisons organize results by accent group as well as allowing users to view the added splits. The announcement credits Hume AI with the leaderboard idea and says the circle groupings use Kachru’s Three Circles framework. Hugging Face plans to separate the Outer and Expanding Circles into distinct groups as more accents are added, but has not specified when that change will happen.
““We’re excited to announce a new collaboration with Liva AI to add even more private data with two new accents: Egyptian and Nigerian.””
— Hugging Face
Speaker Details and Score Effects
The announcement does not say when the recordings were collected, how many people contributed, or how speakers were selected. It also provides no breakdown by region, age, or other characteristics that would help readers assess the samples’ range. The source does not offer an independent assessment of whether the recordings represent Egyptian and Nigerian English more broadly.
Hugging Face has not reported baseline results or explained whether the additions changed existing model rankings. The splits are described as private, and the announcement does not say whether the audio or detailed dataset documentation may later become public. The stated comparison groups also do not yet separate the Outer and Expanding Circles.
More Accents and Separate Circle Groups
Hugging Face says it intends to continue adding accents and languages and plans to split the Outer and Expanding Circles into separate leaderboard groupings when more accents are available. No date or minimum number of accents for that change was announced.
For now, readers can view the new comparisons in the leaderboard’s Private tab. Further information about speaker composition, collection methods and per-accent results would help users interpret what the scores show and how far they can be applied beyond these specific recordings.
Key Questions
Which accents are covered by the new splits?
The private splits cover Egyptian-accented English and Nigerian-accented English. They are named Liva AI Conversational EG and Liva AI Conversational NG.
How much audio do the splits contain?
The Egyptian split contains 1.34 hours and the Nigerian split 1.55 hours, for a combined total of 2.89 hours.
What kind of speech is included?
Hugging Face describes both datasets as spontaneous conversational audio. The transcripts are punctuated and cased and include disfluencies.
Can the public download the evaluation data?
No public download is described. Hugging Face identifies the recordings as private evaluation data used for leaderboard assessment.
Do the results represent all Egyptian or Nigerian English speakers?
The announcement does not establish that. It gives no speaker counts or demographic and regional breakdowns, and the two splits together contain 2.89 hours of speech. Scores should be understood as results on these specific test sets.
Primary source: Hugging Face · via ThorstenMeyerAI.com
