happypdf / utf16Encode
Function: utf16Encode()
function utf16Encode(input, byteOrderMark?): Uint16Array;Defined in: src/utils/unicode.ts:203
Encodes a string to UTF-16.
Parameters
| Parameter | Type | Default value | Description |
|---|---|---|---|
input | string | undefined | The string to be encoded. |
byteOrderMark | boolean | true | Whether or not a byte order marker (BOM) should be added to the start of the encoding. (default true) |
Returns
Uint16Array
A Uint16Array containing the UTF-16 encoding of the input string.
JavaScript strings are composed of Unicode code points. Code points are integers in the range 0 to 1,114,111 (0x10FFFF). When serializing a string, it must be encoded as a sequence of words. A word is typically 8, 16, or 32 bytes in size. As such, Unicode defines three encoding forms: UTF-8, UTF-16, and UTF-32. These encoding forms are described in the Unicode standard [1]. This function implements the UTF-16 encoding form.
In UTF-16, each code point is mapped to one or two 16-bit integers. The UTF-16 mapping logic is as follows [2]:
• If a code point is in the range U+0000..U+FFFF, then map the code point to a 16-bit integer with the most significant byte first.
• If a code point is in the range U+10000..U+10000, then map the code point to two 16-bit integers. The first integer should contain the high surrogate and the second integer should contain the low surrogate. Both surrogates should be written with the most significant byte first.
It is important to note, when iterating through the code points of a string in JavaScript, that if a character is encoded as a surrogate pair it will increase the string's length by 2 instead of 1 [4]. For example:
> 'a'.length
1
> '💩'.length
2
> '語'.length
1
> 'a💩語'.length
4The results of the above example are explained by the fact that the characters 'a' and '語' are not represented by surrogate pairs, but '💩' is.
Because of this idiosyncrasy in JavaScript's string implementation and APIs, we must "jump" an extra index after encoding a character as a surrogate pair. In practice, this means we must increment the index of our for loop by 2 if we encode a surrogate pair, and 1 in all other cases.
References: