8000 Intl has Kosovo in DENYLIST · Issue #54711 · symfony/symfony · GitHub
[go: up one dir, main page]

Skip to content

Intl has Kosovo in DENYLIST #54711

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Renkas opened this issue Apr 24, 2024 · 18 comments
Open

Intl has Kosovo in DENYLIST #54711

Renkas opened this issue Apr 24, 2024 · 18 comments

Comments

@Renkas
Copy link
Renkas commented Apr 24, 2024

Symfony version(s) affected

Description

Intl component has Kosovo in DENYLIST. Why is that? We rely on this component to populate country selection but we need Kosovo to be also included

How to reproduce

Just call: Countries::getNames() and see that Kosovo is not in the result

Possible Solution

remove Kosovo from DENYLIST: https://github.com/symfony/intl/blob/510a40cbf59244af027905cdb838eb7b57447752/Data/Generator/RegionDataGenerator.php#L59

Additional Context

No response

@Hanmac
Copy link
Contributor
Hanmac commented Apr 24, 2024

Answer there: #40020 (comment)

I think the reason is that the Republic of Kosovo is not recognized as an official country by the ISO 3166 standard (https://en.wikipedia.org/wiki/List_of_ISO_3166_country_codes), which is the one we use to get the list of countries (same as most of other software projects do). That's why it's been included in the "deny list"

@Renkas
Copy link
Author
Renkas commented Apr 24, 2024

Yeah now I stumbled on that also after creating this issue. But I don't think that is a correct way to handle this issue.

I managed to write a reply in the other issue already: #40020 (comment)

The point is that in real world people need solution for that.

@llupa
Copy link
Contributor
llupa commented Jul 25, 2024

Funny, I have been watching this issue for a while. cc @javiereguiluz We have talked about this.

This year in May IIRC there was a vote that would affect this status so I have been waiting, but since nothing happened there, I would like to update the Symfony core team with this section from my correspondence with ISO themselves:

Currently, the ISO 3166 Maintenance Agency has received no information from the UN Headquarters that Kosovo has been added to the above list.

Nonetheless, the ISO 3166-1 does allow for cases where an official code may not be allocated by permitting the use of "user-assigned" codes as indicated in 8.1.3 of ISO 3166-1. If users need code elements to represent country names not included in this part of ISO 3166, the series of letters AA, QM to QZ, XA to XZ, and ZZ, and the series AAA to AAZ, QMA to QZZ, XAA to XZZ, and ZZA to ZZZ respectively. The series of numeric country code numbers (http://unstats.un.org/unsd/methods/m49/m49alpha.htm) are assigned by the UN Statistics Division who also make available the series, 900 to 999, as "user-assigned" codes.

The ISO 3166/MA has received information that the CE Commission has allocated the alpha-2 user-assigned code "XK" to represent Kosovo in the interim of being recognized by the UN as a member state.

If you are interested to see this in action by the European Commission too feel free to browse: https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Glossary:Country_codes

I understand that the original thought was to make this non-political and I am sure members do not have any prejudice against an entire entity such as Kosovo, but I believe that since there seem to be exceptions within ISO themselves and to make this truly a-political my proposal is as follows:

  • Add an option to allow user-assigned country codes
  • update documentation

Default behavior can be kept as is, disallow them initially, and for projects that do need this code, they can enable them

I would like to work on this. Any extra input needed?

@javiereguiluz
Copy link
Member
javiereguiluz commented Jul 25, 2024

Yes, I like your proposal:

  • Keep adhering strictly to the ISO 3166 list of countries/territories
  • But allow developers to add their own territories via user assigned codes

Thanks!

fabpot added a commit that referenced this issue Sep 13, 2024
This PR was squashed before being merged into the 5.4 branch.

Discussion
----------

[Intl] Update ICU data from 74.1 to 75.1

| Q             | A
| ------------- | ---
| Branch?       | 5.4
| Bug fix?      | no
| New feature?  | no
| Deprecations? |no
| Issues        | Not reported
| License       | MIT

While doing some work for [#54711](#54711) I had to track too many file changes because of latest ICU data update. I am opening this PR to simply update the data-dumps.

The changes were generated by simply running

```bash
$ php src/Symfony/Component/Intl/Resources/bin/update-data.php
```

Commits
-------

ff2b7f9 [Intl] Update ICU data from 74.1 to 75.1
@carsonbot
Copy link

Hey, thanks for your report!
There has not been a lot of activity here for a while. Is this bug still relevant? Have you managed to find a workaround?

@norkunas
Copy link
Contributor

still relevant

@carsonbot carsonbot removed the Stalled label Jan 27, 2025
@stof
Copy link
Member
stof commented Jan 27, 2025

@norkunas there has been a proposal above about a new feature that would solve that case. But nobody has stepped in to actually implement that proposal. If this is relevant for you, maybe you could contribute an implementation.

@norkunas
Copy link
Contributor

Sure probably I could but for me this still looks that it's missing some pieces:

  • Should the option with additional codes be at framework level configuration? If not, then where?
  • As the Countries api is static, how it should reach this config to not skip user assigned codes?
  • CountryValidator, form CountryType and twig's country_name should somehow then also see this;
  • Currently as I understand these codes are ignored at the generator level, then the "deny list" should be moved elsewhere? Like Symfony\Component\Intl\Countries ?

@llupa
Copy link
Contributor
llupa commented Jan 27, 2025

Back when I tried to implement my own proposal, even as a first attempt I had the same issues like you @norkunas
What is the best way to make a static data dump, behave dynamically.

Due to personal life I had to* put this in the back burner, but looking at all my emails from this issue today, I can take a jab this week again, as I have something with array unpacking that looks OK-ish.

@stof
Copy link
Member
stof commented Jan 27, 2025

The symfony/intl component should stay focused on the ISO list, for 2 reasons:

  • a static API does not have way to inject configuration
  • forcing to define the full data (equivalent to the CLDR data) for a user-assigned code might be too complex (most projects would probably need a subset of the usages only).

To me, this feature about user-assigned codes should be a feature defined at the level of the CountryType, where the label for those codes would be provided as part of the option adding the user-assigned code (alongside the code).
The Country constraint might get support for user-assigned codes as well, thanks to an option passing a list of codes that will be considered valid.

@norkunas
Copy link
Contributor

But to provide translations for each user assigned code would be a little cumbersome 🤔

@llupa
Copy link
Contributor
llupa commented Jan 27, 2025

All data is already in ICU. I will try to finish my PR by Feb 2nd and we can pick that apart 👍

@wouterj
Copy link
Member
wouterj commented Jan 27, 2025

Thanks everyone for getting this moving again! 🚀

@stof
Copy link
Member
stof commented Jan 27, 2025

@llupa the data for Kosovo is in ICU. But that's not the feature that @javiereguiluz suggested.

@llupa
Copy link
Contributor
llupa commented Jan 27, 2025

@stof I think the wording But allow developers to add their own territories via user assigned codes can be interpreted as being able to add completely custom codes too. I read it in the context of User-Assigned codes as per ISO norm. Unless @javiereguiluz cares to elaborate ?

BTW my plan is to have the PR just cover Kosovo as a first step, this topic looks deceptively easy.

@stof
Copy link
Member
stof commented Jan 27, 2025

even with a restricting of user-assigned codes per the ISO norm (which I think is a good idea to avoid compatibility issues when the ISO adds new official codes), it still means that the translations will not be in ICU.

@connorhu
Copy link
Contributor
connorhu commented Apr 3, 2025

I've been playing around with this topic (it was necessary) and my opinion is that it would be better to create a file format and method that allows the developer to extend the list runtime.
Intl::addExtra(new JsonReader('some.json'));
Advantage: no need to touch everything (twig-extra, form type etc.)
First idea: #59693 (comment)

@connorhu
Copy link
Contributor
connorhu commented Apr 3, 2025

Thats all we have to do.
This opens up the possibility for the developer to compile their own data. Later, the data can be split into a separate package so that the developer can choose not to use the standard list at all.

Subject: [PATCH] added NamesUserAssigned part
---
Index: src/Symfony/Component/Intl/Intl.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/src/Symfony/Component/Intl/Intl.php b/src/Symfony/Component/Intl/Intl.php
--- a/src/Symfony/Component/Intl/Intl.php	(revision 4802aab8ab5ae11ca5ba97a45d08454ee7e449a9)
+++ b/src/Symfony/Component/Intl/Intl.php	(date 1743692426761)
@@ -57,6 +57,8 @@
     private static string|false|null $icuVersion = false;
     private static string $icuDataVersion;
 
+    private static string $dataDirectory;
+
     /**
      * Returns whether the intl extension is installed.
      */
@@ -114,7 +116,12 @@
      */
     public static function getDataDirectory(): string
     {
-        return __DIR__.'/Resources/data';
+        return self::$dataDirectory ?? __DIR__.'/Resources/data';
+    }
+
+    public static function setDataDirectory(string $dataDirectory): void
+    {
+        self::$dataDirectory = $dataDirectory;
     }
 
     /**

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants
0