Banning the "limited use" category by default in the "General Security Profile" is interesting given it overrepresents indigenous scripts.
For example, Canadian Aboriginal Syllabics is the writing system of Inuktitut, the primary language of the Canadian territory of Nunavut spoken by 63.1% of the population. It seems like they're being disenfranchised by this proposal.
Bruce Schneier wrote about it more than two decades ago:
"Unicode is too complex to ever be secure".
Systems created by people who have zero understanding of security and who cannot realize that complex systems cannot possibly ever be secure is already bad.
But to then see the same people implement "security mechanisms" for the mess they created for in the first place is just sad.
FWIW I love Unicode, where it belongs: in translation / properties files.
I've got hooks verifying that all source code files are ASCII-only.
I've got rules for my LLMs to only ever emit ASCII in source code.
I've got firewall rules blocking any domain using punnycode (aka Unicode domain names). A bit tricky to do that one.
I've got my editor configured so that any single non-visible ASCII char (except space and enter) and every single Unicode character appears in bold fluo magenta on a dark background. All zero-width chars are forced to have a width.
I wonder how people who disagree with those who believe that Unicode is too complex to ever be secure reconcile with the fact that we have something titled:
"Unicode® Technical Standard #39 Unicode Security Mechanisms"
The question is: how the heck did we end up here?
What mindset led to this clusterfuck?
Who, in his right mind, decided that it was a good thing to have Unicode chars in URLs? Certainly not security experts.
The homoglyph attacks were predicted by Bruce Schneier (and others) decades ago. Countless people got tricked and lost their live savings because URLs allow an 'a' to not be an 'a'. It's sad.
And it's of course not the only attack using Unicode: there are plenty.
It's really just all a sad joke and many of us saw it coming from day one.
Demanding that everyone use our alphabet because we're too dumb to recognize the difference between google.com and xn--ggle-55da.com is the worst kind of "security through superior firepower"
(the fact that HN shows these correctly in the EDIT window but have to dump to punycode in display is sad, tbh)
Not even "most" of Europe - almost all European languages use diacritics to some extent or another. In the context of domain names, some words are comprehensible with the diacritics stripped, but there's some loss of precision. (For instance, an "ñ" is critical if you're trying to say "happy new year" in Spanish...)
UCS-2 (i.e. Javascript, 16-bit character space) has enough codepoints for every language still spoken except Chinese (even after undoing Han Unification, which was dumb).
Unicode is China forcing variable-length character encodings on the rest of the world.
No, it's not. UTF-16 supports the same characters as UTF-32 and UTF-8 (all three support all of Unicode). UTF-16 does that by having... variable width characters, in the form of 2-UTF16 code units for some chars.
UTF-16 wastes a bunch of space (when you don't need 16 bits to mark a character), and doesn't avoid variable-width encoding. UTF-8 tries to optimize space (but is ASCII skewed) with more variance (but once you can't infer length from bytes, what's the difference?) UTF-32 is a space hog at the benefit of predictable length encoding (a full Int32 to represent the null char, or a carriage return)
Congratulations, you've discovered why UTF-16 is stupid. You pay the space penalty of two-byte characters and the complexity penalty of variable-length characters.
It's the worst of both worlds!
Seriously nobody advocates UTF-16. The smart people advocate UCS-2.
You can either have Javascript uses UTF16 because UCS2 is dead. Or you can have variable width is stupid. JavaScript supports UTF16 (otherwise; no emoji). UCS2 is also a Unicode standard - just an old one, in their own words:
> In particular, for the purposes of data exchange, UCS-2 and UTF-16 are identical formats.
The insistence that Javascript means UCS-2 not UTF-16 involves selectively reading a version of the standards document that's more than a decade obsolete.
Unicode is complex because human language is complex. If you want someone to blame, blame God:
> Go to, let us go down, and there confound their language, that they may not understand one another's speech. So the LORD scattered them abroad from thence upon the face of all the earth: and they left off to build the city. Therefore is the name of it called Babel; because the LORD did there confound the language of all the earth: and from thence did the LORD scatter them abroad upon the face of all the earth.
Perhaps attempting to build one character set for all languages is just another Babel-like act of hubris, and the security vulnerabilities are God's judgment upon us for not sticking with the mess of incompatible local encodings that preceded it.
What is the alternative? ASCII isn't it because there isn't only English out there. All the different encodings we had before weren't it either.
Having only ASCII in urls is incredible US centric. Yes it brings problems, yes they are solvable. It takes work to not have US and it's writing system to be the center of everything.
... might sound like a reasonable solution until you start trying to write about languages. It is unreasonable for a sentence like "In Japanese, the word for 'hello' is こんにちは; in Korean, it's 안녕하세요." to be unrepresentable.
For example, Canadian Aboriginal Syllabics is the writing system of Inuktitut, the primary language of the Canadian territory of Nunavut spoken by 63.1% of the population. It seems like they're being disenfranchised by this proposal.
Bruce Schneier wrote about it more than two decades ago:
"Unicode is too complex to ever be secure".
Systems created by people who have zero understanding of security and who cannot realize that complex systems cannot possibly ever be secure is already bad.
But to then see the same people implement "security mechanisms" for the mess they created for in the first place is just sad.
FWIW I love Unicode, where it belongs: in translation / properties files.
I've got hooks verifying that all source code files are ASCII-only.
I've got rules for my LLMs to only ever emit ASCII in source code.
I've got firewall rules blocking any domain using punnycode (aka Unicode domain names). A bit tricky to do that one.
I've got my editor configured so that any single non-visible ASCII char (except space and enter) and every single Unicode character appears in bold fluo magenta on a dark background. All zero-width chars are forced to have a width.
I wonder how people who disagree with those who believe that Unicode is too complex to ever be secure reconcile with the fact that we have something titled:
"Unicode® Technical Standard #39 Unicode Security Mechanisms"
The question is: how the heck did we end up here?
What mindset led to this clusterfuck?
Who, in his right mind, decided that it was a good thing to have Unicode chars in URLs? Certainly not security experts.
The homoglyph attacks were predicted by Bruce Schneier (and others) decades ago. Countless people got tricked and lost their live savings because URLs allow an 'a' to not be an 'a'. It's sad.
And it's of course not the only attack using Unicode: there are plenty.
It's really just all a sad joke and many of us saw it coming from day one.
Demanding that everyone use our alphabet because we're too dumb to recognize the difference between google.com and xn--ggle-55da.com is the worst kind of "security through superior firepower"
(the fact that HN shows these correctly in the EDIT window but have to dump to punycode in display is sad, tbh)
Unicode is China forcing variable-length character encodings on the rest of the world.
If you want to say that the facts are dumb, well, you can do that.
UTF-16 wastes a bunch of space (when you don't need 16 bits to mark a character), and doesn't avoid variable-width encoding. UTF-8 tries to optimize space (but is ASCII skewed) with more variance (but once you can't infer length from bytes, what's the difference?) UTF-32 is a space hog at the benefit of predictable length encoding (a full Int32 to represent the null char, or a carriage return)
It's the worst of both worlds!
Seriously nobody advocates UTF-16. The smart people advocate UCS-2.
> In particular, for the purposes of data exchange, UCS-2 and UTF-16 are identical formats.
https://web.archive.org/web/20030818043641/http://www.unicod...
If you don't like facts that is dumb.
> Go to, let us go down, and there confound their language, that they may not understand one another's speech. So the LORD scattered them abroad from thence upon the face of all the earth: and they left off to build the city. Therefore is the name of it called Babel; because the LORD did there confound the language of all the earth: and from thence did the LORD scatter them abroad upon the face of all the earth.
Perhaps attempting to build one character set for all languages is just another Babel-like act of hubris, and the security vulnerabilities are God's judgment upon us for not sticking with the mess of incompatible local encodings that preceded it.
Having only ASCII in urls is incredible US centric. Yes it brings problems, yes they are solvable. It takes work to not have US and it's writing system to be the center of everything.