Glossary Data & Encoding

Unicode

Unicode is the universal standard that gives every character in the world's writing systems, plus symbols and emoji, a unique number called a code point.

Last reviewed:

AdSense Placeholder
Slot: header_reference_page

In more detail

Code points are written U+0041 for A or U+1F600 for a smiling face. The code space runs from U+0000 to U+10FFFF, 1,114,112 positions in 17 planes, of which well over 150,000 are assigned, and new versions add characters roughly every year. Unicode defines the characters and their properties; encodings such as UTF-8 define how code points are stored as bytes. What a reader sees as one character can be several code points: an accent can be a separate combining mark, and many emoji are sequences.

Why it matters

The same visible text can be stored in different ways, so two strings that look identical may compare as unequal until they are normalised (NFC or NFD). String "length" depends on what is counted: code points, bytes or UTF-16 units. Look-alike and invisible characters can be abused, for example a Cyrillic а (U+0430) disguising a domain name (see Punycode) or a zero-width space hiding inside a password or filename.

Example

é can be one code point, U+00E9, or two, e followed by U+0301 (combining acute accent). The family emoji 👨‍👩‍👧 is five code points (three people joined by two zero-width joiners), 18 bytes in UTF-8 and a length of 8 in JavaScript.

Common confusion

Unicode is the character set; UTF-8, UTF-16 and UTF-32 are ways of encoding it. Unicode's first 128 code points are identical to ASCII.

AdSense Placeholder
Slot: footer_leaderboard