Allow use of UTF-8 catalogues in non-UTF-8 applications #2339
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… be used with another charset
fabpot
added a commit
that referenced
this pull request
Oct 7, 2011
Commits ------- 5473d3b [Translation] Allow use of UTF-8 encoded catalogues into non-UTF-8 applications deb6dea [Translation] Add failing tests to verify that UTF-8 lang files can't be used with another charset Discussion ---------- Allow use of UTF-8 catalogues in non-UTF-8 applications This is #2313 but targetting the master branch. Bug fix: yes Feature addition: ?:) Backwards compatibility break: no Symfony2 tests pass: yes Fixes the following tickets: - The problem I'm having is that, while porting an existing app, we are using UTF-8 everywhere to have a migration path ready, but the current application and DB is still in ISO-8859-1, which means translations containing accented chars are broken. Also, we didn't hit the issue yet since we don't use forms much, but I imagine we would have similar issues with core translations for the validator which are all UTF-8 encoded. Note that I explicitly suppressed this conversion in case your application is setup as UTF-8, to make sure most people are not affected by any slow down this introduces.
|
I do wonder if this should be reverted or not. I had to abandon it for another more customized solution here. The reason is that some strings aren't different enough to trigger UTF-8 or ISO-8559-15, take "ü" for example, depending on the mb_detect_encoding encoding order you pass, it will always detect the first encoding of the two, because there's not enough info in "ü" to distinguish. Then it force-decoded stuff that was already in ISO-8859-1, or failed to decode stuff that was in UTF-8 depending on the order I set. Now this might still come in handy for eastern european, arabic, russian, asian languages and the others I forget which are all way beyond the ISO-8559-1/UTF-8 compatibility range. So maybe it's ok to leave it in. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is #2313 but targetting the master branch.
Bug fix: yes
Feature addition: ?:)
Backwards compatibility break: no
Symfony2 tests pass: yes
Fixes the following tickets: -
The problem I'm having is that, while porting an existing app, we are using UTF-8 everywhere to have a migration path ready, but the current application and DB is still in ISO-8859-1, which means translations containing accented chars are broken.
Also, we didn't hit the issue yet since we don't use forms much, but I imagine we would have similar issues with core translations for the validator which are all UTF-8 encoded.
Note that I explicitly suppressed this conversion in case your application is setup as UTF-8, to make sure most people are not affected by any slow down this introduces.