CLDR-18843 Remove organization Special #5116
Conversation
-Remove Organization.special and special.visibleOnFrontEnd -Remove all lines from Locales.txt referencing special organization -Remove lines from Locales.txt referencing Cldr organization where same locale was included for special, with 3 exceptions: sr_Latn, qu, to, which are needed to pass test TestCLDRLocaleCoverage.TestCldrSuperset -Remove SubmissionLocales.SPECIAL_ORG_LOCALES, no longer need to subtract from CLDR locales -Revise SubmissionLocales.HIGH_LEVEL_LOCALES, removing some special locales -Remove ChartLocaleGrowth.SpecialLocales, isSpecial -No longer need to subtract special from CLDR locales in ListCoverageLevels, ShowInconsistentAvailable, DiffLanguageGroups, ListProblemDates, TestUnits -In ShowLocaleCoverage, remove specialFlag -In VettingViewer, remove HC = high coverage for special locales -In TestStandardCodes.testTargetCoverageLevel, remove assertion for locale br
| Cldr ; yue ; modern ; T5 Cantonese | ||
|
|
||
| #Cldr Tier generated | ||
| Cldr ; sr_Latn ; modern ; Tx Serbian (auto-generated) |
There was a problem hiding this comment.
This line for "sr_Latn" is moved, not removed. It is essentially unchanged, except for the addition of a comment explaining the relation to TestCLDRLocaleCoverage.TestCldrSuperset. The same applies to "qu" and "to".
|
The strategy that resulted in these changes was essentially: (1) remove all "special" from Locales.txt; (2) remove those same locales from "Cldr" in Locales.txt, except for sr_Latn, qu, to, which are needed to pass test |
| // Note: ALL of these were found in Locales.txt under cldr. | ||
| "chr", // Cherokee | ||
| "gd", // Scottish Gaelic, Gaelic | ||
| "fo", // Faroese |
There was a problem hiding this comment.
I'm confused by this since ha, kok, pcm are TC locales.
There was a problem hiding this comment.
Is the confusing thing (1) that chr and fo were removed, or (2) that ha, kok, and pcm were not removed (or were present in the first place)?
There was a problem hiding this comment.
chr and fo were removed since they were special.
ha, kok, and pcm were not removed since they weren't special. Why they were present in the first place, I'm not sure, though I suspect they might be required for TestCldrSuperset to pass.
There was a problem hiding this comment.
There is a test that CLDR org is a superset of the TC orgs' locales. We were tracking Specials as part of that, but that was a hack so we're removing it. So locales that were only in CLDR org by virtual of being Special are being removed.
There was a problem hiding this comment.
I think the HIGH_LEVEL_LOCALES should also go away; they were intended to align with the Specials above basic, but we are no longer tracking specials. However, that can be in a different PR.
|
|
||
| Cldr ; fo ; moderate ; Faroese | ||
| Cldr ; qu ; modern ; Quechua | ||
| Cldr ; br ; moderate ; Breton |
There was a problem hiding this comment.
I thought that the CLDR list was a superset of all the locales across all of the organizations?
There was a problem hiding this comment.
The test TestCldrSuperset ensures that, if I understand it.
These are deleted from Cldr because they were "special", and there was a lot of code that relied on the special locales being subtracted from the Cldr locales. The general strategy I employed was to remove the special locales from Cldr, with the 3 exceptions mentioned
| "doi,BASIC", // CLDR locale | ||
| "nn,MODERN", // CLDR locale | ||
| "hnj,MODERN", // Maximum coverage (hmong) | ||
| "br,MODERATE", // Maximum (Breton) |
There was a problem hiding this comment.
I don't understand this change?
There was a problem hiding this comment.
br was removed from Cldr in Locales.txt because it was one of the special locales. Consequently this test failed, unless br was removed from the test.
| // Note: ALL of these were found in Locales.txt under cldr. | ||
| "chr", // Cherokee | ||
| "gd", // Scottish Gaelic, Gaelic | ||
| "fo", 8000 // Faroese |
There was a problem hiding this comment.
I think the HIGH_LEVEL_LOCALES should also go away; they were intended to align with the Specials above basic, but we are no longer tracking specials. However, that can be in a different PR.
|
Marking as draft because we don't want to merge before branch. |
|
Okay, in that case in a follow-on PR, can we move the CLDR org to be the first organization in the locales.txt file and add an explanation of what it contains in the comment at the top of the file? |
|
We should add t A354 he comment, but what I'd like to do is move to using the reformatting tool. |
|
That is, copy in https://github.com/unicode-org/cldr-staging/blob/main/docs/charts/48/tsv/coverage_goals.tsv instead (it reformats with each charts build), and do the ticket to clean out the deadwood orgs (that haven't contributed in (say) 5 years. |
-Remove Organization.special and special.visibleOnFrontEnd
-Remove all lines from Locales.txt referencing special organization
-Remove lines from Locales.txt referencing Cldr organization where same locale was included for special, with 3 exceptions: sr_Latn, qu, to, which are needed to pass test TestCLDRLocaleCoverage.TestCldrSuperset
-Remove SubmissionLocales.SPECIAL_ORG_LOCALES, no longer need to subtract from CLDR locales
-Revise SubmissionLocales.HIGH_LEVEL_LOCALES, removing some special locales
-Remove ChartLocaleGrowth.SpecialLocales, isSpecial
-No longer need to subtract special from CLDR locales in ListCoverageLevels, ShowInconsistentAvailable, DiffLanguageGroups, ListProblemDates, TestUnits
-In ShowLocaleCoverage, remove specialFlag
-In VettingViewer, remove HC = high coverage for special locales
-In TestStandardCodes.testTargetCoverageLevel, remove assertion for locale br
CLDR-18843
ALLOW_MANY_COMMITS=true