Contains routines used to perform conversions between Unicode and System Code Page encodings.

lconvencoding.pas provides types, variable, constants, and routines used to perform conversions to and from Unicode and /or System Code Page encodings. Functions in this unit are thread-safe.

For environments where the RTL uses UTF-8 encoding, the UseSystemCPConv compiler define is enabled to include System Code Page conversions.

lconvencoding.pas is part of the LazUtils package.

Represents the behaviors for an error occurring in an encoding conversion.

TConvertEncodingErrorMode is an enumerated type with values that represent the behavior applied when an encoding conversion error is detected. TConvertEncodingErrorMode is the type used for the ConvertEncodingErrorMode variable.

Introduced in version 2.2.
Skip or ignore the encoding conversion error.
Raises an EConvertError exception with a message relevant to the conversion error.
Replace the suspect value with an error placeholder (usually the '?' character).
Return an empty string ('') for the suspect value.
Error handling behavior for encoding conversion errors.

ConvertEncodingErrorMode is a unit global TConvertEncodingErrorMode variable which controls the behavior when an error is detected in an encoding conversion routine. For example:

- UTF-8 to single byte encoding
- DBCS (Asian code pages) encoding to UTF-8
- UTF-8 to DBCS (Double Byte Character Sets)

The default value for the variable is ceemSkip, and indicates that an error is skipped or ignored when encountered in an encoding conversion.

Introduced in version 2.2. Replaces use of the ConvertEncodingErrorRaisesException variable.
Encoding name for UTF-8.
Encoding name for ANSI.
Encoding name for UTF-8 with a byte order mark.
Encoding name for UCS 2-byte Little Endian.
Encoding name for UCS 2-byte Big Endian.
Encoding name for Code Page 1250.
Encoding name for Code Page 1251.
Encoding name for Code Page 1252.
Encoding name for Code Page 1253.
Encoding name for Code Page 1254.
Encoding name for Code Page 1255.
Encoding name for Code Page 1256.
Encoding name for Code Page 1257.
Encoding name for Code Page 1258.
Encoding name for Code Page 437.
Encoding name for Code Page 850.
Encoding name for Code Page 852.
Encoding name for Code Page 866.
Encoding name for Code Page 874.
Encoding name for Code Page 932.
Encoding name for Code Page 936.
Encoding name for Code Page 949.
Encoding name for Code Page 950.
Encoding name for the Macintosh Code Page.
Encoding name for KOI8-R Code Page.
Encoding name for KOI8-U Code Page.
Encoding name for KOI8-RU Code Page.
Encoding name for ISO 8859-1 Code Page.
Encoding name for ISO 8859-2 Code Page.
Encoding name for ISO 8859-15 Code Page.
ANSI representation for the UTF-8 Byte Order Mark.
ANSI representation for the UTF-16 Big Endian Byte Order Mark.
ANSI representation for the UTF-16 Little Endian Byte Order Mark.
ANSI representation for the UTF-32 Big Endian Byte Order Mark.
ANSI representation for the UTF-32 Little Endian Byte Order Mark.
Tries to determine the encoding used for the specified value.

GuessEncoding is a String function which tries to determine the encoding used for the value specified in S. The return value contains an encoding name, like 'utf8' or 'ISO-8859-1'. It may contain an empty string ('') when S is also an empty string.

GuessEncoding checks S for various Byte Order Marks at the start of the value, including: UTF8BOM, UTF16LEBOM, and UTF16BEBOM. When present, the BOM determines the encoding used for the value.

Next, it checks for an explict '{%encoding ' marker at the start of the value. When present, the value after the marker (up to the closing '}' character) is normalized and used as the return value.

Finally, it checks for a valid UTF-8 encoding (which includes ASII characters). All characters in S are examined until a character whose UTF-8 code point is not valid is encountered.

When EncodingValid is True, EncodingAnsi is assumed. Otherwise, the default encoding for the platform is used. When the return value is EncodingUTF8, it is changed to 'ISO-8859-1'. This is done because the system may use the UTF-8 encoding, but the value in S does not. ISO 8859-1 has a full mapping to Unicode, and this prevents data loss in encoding conversions.

Encoding name detected, or a default value.
String with the content examined in the routine.
Converts the encoded value from UTF-8 to the encoding with the specified name.

The s argument contains the UTF-8-encoded value converted in the routine.

ToEncoding is a String value with the name for the target encoding. It is normalized in the routine to remove hyphen characters in the encoding name. The value corresponds to the encoding name constants defined in the unit, and is used to determine which routine is called to perform the encoding conversion.

For example, the following is a sample of the encoding names and their corresponding conversion routines:

EncodingUTF8BOM ('utf8bom')
  UTF8ToUTF8BOM
EncodingCPIso1 ('iso88591')
  UTF8ToISO_8859_1
EncodingCP1250 ('cp1250')
  UTF8ToCP1250
EncodingCP1252 ('cp1252')
  UTF8ToCP1252
EncodingCP850 ('cp850')
  UTF8ToCP850
EncodingCPKOI8R ('koi8r')
  UTF8ToKOI8R
EncodingAnsi ('ansi')
  ConvertUTF8ToAnsi

Encoded is a Boolean output parameter which indicates if a routine was found to perform the requested conversion. Its value is False when a conversion routine was not found for the target encoding in ToEncoding.

SetTargetCodePage is a Boolean argument which indicates if SetCodePage is called to apply a translated code page for the converted value. The default value for the argument is False, and omits the code page for the return value.

ConvertEncodingFromUTF8 is used in the implementation of the ConvertEncoding function.

String value after conversion to the specified encoding.
UTF-8-encoded value converted in the routine.
Encoding name for the converted value.
True if the return value contains multi-byte encoded values, False if it contains a single-byte ANSI string.
True if the code page should be set for the return value, False if a code page is not used.
Converts the specified string value to the UTF-8 encoding.

ConvertEncodingToUTF8 converts the encoding for a string value from the specified encoding to UTF-8. The s argument contains the string value converted in the routine.

FromEncoding is a String value with the name for the existing encoding is s. It is normalized in the routine to remove hyphen characters in the encoding name. The value corresponds to the encoding name constants defined in the unit, and is used to determine which routine is called to perform the encoding conversion.

For example, the following is a sample of the encoding names and their corresponding conversion routines:

EncodingUTF8BOM ('utf8bom')
  UTF8BOMToUTF8
EncodingCPIso1 ('iso88591')
  ISO_8859_1ToUTF8
EncodingCP1250 ('cp1250')
  CP1250ToUTF8
EncodingCP1252 ('cp1252')
  CP1252ToUTF8
EncodingCP850 ('cp850')
  CP850ToUTF8
EncodingCPKOI8R ('koi8r')
  KOI8RToUTF8
EncodingAnsi ('ansi') or the default text encoding for the platform
  ConvertAnsiToUTF8

Encoded is a Boolean output parameter which indicates if a routine was found to perform the requested conversion. Its value is False when a conversion routine was not found for the source encoding in FromEncoding.

ConvertEncodingToUTF8 is used in the implementation of the ConvertEncoding function.

Conversions to the UTF-8 encoding will always set the code page for the converted value to CP_UTF8 (65001). In other words, the SetTargetCodePage argument to the ConvertEncoding() function is ignored for conversions to UTF-8.
Value for the specified string after encoding conversion.
String with the encoded value converted to UTF-8.
Encoding name for the value in s.
True if the result contains a multi-byte encoded value. Always True for conversions to UTF-8.
Converts the specified value from its source encoding to a target encoding using an optional code page.

ConvertEncoding is a String function used to convert the string specified in s from its original encoding name (FromEncoding) to a target encoding (ToEncoding).

When FromEncoding is omitted, or specified as an empty string (''), UTF-8 is used as the original encoding. If the original encoding is ANSI, the default text encoding for the platform (GetDefaultTextEncoding) is used as the original encoding.

Conversely, when ToEncoding is omitted or an empty string, UTF-8 is used as the target encoding. If the target encoding is ANSI, the default text encoding for the platform (GetDefaultTextEncoding) is used as the target encoding.

No actions are performed in the routine when FromEncoding and ToEncoding have the same values.

ConvertEncoding calls ConvertEncodingFromUTF8 to convert the string value when the FromEncoding is UTF-8. ConvertEncodingToUTF8 is called when ToEncoding is UTF-8. For other encoding conversions, ConvertEncodingToUTF8 is called to both get a UTF-8 value which is then converted using ConvertEncodingFromUTF8.

SetTargetCodePage indicates whether the string with the converted encoding should set the code page for the result. The default value for the argument is False. The SetTargetCodePage argument is ignored when ToEncoding is the UTF-8 encoding ('utf8'). The code page is always set to CP_UTF8 (65001) when UTF-8 encoding is used in the result.

For platforms where libiconv is enabled, the iconvert routine is used as a fallback when an encoding conversion routine was not found.

The return value is the original string in s if a transcoding operation is not successfully performed (or not needed) for the original string value using the specified encodings.

Value for the specified string after encoding conversion.
String value converted in the routine.
Encoding name for the original string value.
Encoding name for the converted value.
True if the code page should be set for the converted value. Default is False.
Gets the name for the default text encoding used on the platform or operating system.

GetDefaultTextEncoding gets the default encoding name used for text from the DefaultTextEncoding variable (in the unit implementation), when assigned. Otherwise, platform-specific routines are used to get return value like: GetWindowsEncoding, GetUnixEncoding. The value in the EncodingUTF8 constant is used for the Darwin platforms. The return value contains the encoding used for AnsiString in the FPC RTL, and is retained in the DefaultTextEncoding variable for subsequent calls to the routine.

The name for the default text encoding used on the platform.
Gets the encoding used for a console application on the platform.

GetConsoleTextEncoding is String function which gets the encoding name used for console applications.

The return value may differ from the normal system encoding on some Windows versions, where the OEM code page is used for a console application. For more details, see: https://gitlab.com/freepascal.org/lazarus/lazarus/-/issues/20552.

For all other platforms, the value from GetDefaultTextEncoding is used in the return value. The return value is normalized (hyphen characters removed and converted to lowercase).

Name of the encoding used for text in a console application.
Converts the specified encoding name to lowercase and removes '-' characters.

Converts a value like 'UTF-8' to 'utf8', or 'ISO-8859-1' to 'iso88591'.

NormalizeEncoding is used in the implementation of conversion routines, like:

- GetDefaultTextEncoding
- GetConsoleTextEncoding
- GuessEncoding
- ConvertEncodingFromUTF8
- ConvertEncodingToUTF8
- ConvertEncoding

Contains the encoding name converted to lowercase with hyphen ('-') characters removed.
Encoding name normalized in the routine.
Specifies a function used to convert a single-byte string value to a specific multi-byte encoding.

TConvertEncodingFunction is the type used for the ConvertAnsiToUTF8 variable.

Value after converting the value to the required encoding.
Single-byte string value examined and converted in the routine.
Specifies a function used to convert a string value from UTF-8 encoding to another encoding with an optional code page.

TConvertUTF8ToEncodingFunc is the type used for the ConvertUTF8ToAnsi variable in lconvencoding.pas.

Value for the string using the code page / encoding implemented in the routine.
UTF-8-encoded string value converted in the routine.
True to set the target code page, False if the code page is not needed.
Alias for the TCharToUTF8Table type in CodepagesCommon.pas.
Specifies an Integer function type used to get a character identifier for a specific Unicode character.

TUnicodeToCharID is the type passed as an argument to conversion routines like UTF8ToSingleByte and UTF8ToDBCS.

Integer value for the specified Unicode character.
Cardinal value for the Unicode character converted in the routine.
Contains the routine used to convert an ANSI string value to UTF-8 encoding.

ConvertAnsiToUTF8 is a TConvertEncodingFunction variable with the routine used to convert a String from its ANSI-encoded value to the UTF-8 encoding. A routine which implements the TConvertEncodingFunction interface can be assigned to ConvertAnsiToUTF8 to handle conversions from the default text encoding, which may use an OEM code page, to the UTF-8 encoding.

By default, a routine is NOT assigned to ConvertAnsiToUTF8.

ConvertAnsiToUTF8 is called (when assigned) from the ConvertEncodingToUTF8 routine.

Contains the routine used to convert from UTF-8 encoding to ANSI encoding.

ConvertUTF8ToAnsi is a TConvertUTF8ToEncodingFunc variable with the routine used to convert a string value from UTF-8 encoding to ANSI encoding. A routine which implements the TConvertUTF8ToEncodingFunc interface can be assigned to ConvertUTF8ToAnsi to the conversion.

The arguments to the routine include the UTF-8-encoded string converted in the function, and a boolean argument which indicates if the code page should be set for the string in the return value. Set the SetTargetCodePage parameter to True to set the code page for the ANSI string in the return value. The default value for the argument is False.

By default, a routine is NOT assigned to ConvertUTF8ToAnsi.

ConvertUTF8ToAnsi is called (when assigned) from the ConvertEncodingFromUTF8 routine.

Removes the UTF-8 BOM from the UTF-8 encoded value.

The return value is the value in s after removing the Byte Order Mark in UTF8BOM from the start of the string. The return value is the same as the value in s when UTF8BOM is not found in the string.

No actions are performed in the routine when s is an empty string (''). The return value is also an empty string.

Value after removing the UTF-8 BOM.
UTF-8-encoded value examined in the routine.
Converts an ISO 8859-1-encoded string to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayISO_8859_1ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in ISO 8859-1 encoding.
Converts an ISO 8859-15-encoded string to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayISO_8859_15ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in ISO 8859-15 encoding.
Converts an ISO 8859-2-encoded string to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayISO_8859_2ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in ISO 8859-2 encoding.
Converts a Code Page 1250-encoded string to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1250ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1250 encoding.
Converts a Code Page 1251-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1251ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1251 encoding.
Converts a Code Page 1252-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1252ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1252 encoding.
Converts a Code Page 1253-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1253ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1253 encoding.
Converts a Code Page 1254-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1254ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1254 encoding.
Converts a Code Page 1255-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1255ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1255 encoding.
Converts a Code Page 1256-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1256ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1256 encoding.
Converts a Code Page 1257-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1257ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1257 encoding.
Converts a Code Page 1258-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP1258ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 1258 encoding.
Converts a Code Page 437-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP437ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 437 encoding.
Converts a Code Page 850-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP850ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 850 encoding.
Converts a Code Page 852-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP852ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 852 encoding.
Converts a Code Page 866-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP866ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 866 encoding.
Converts a Code Page 874-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayCP874ToUTF8.

UTF-8-encoded value for the specified string.
String with the value in Code Page 874 encoding.
Converts a KOI8-R-encoded value to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayKOI8RToUTF8.

UTF-8-encoded value for the specified string.
String with the value in the KOI8R encoding.
Deprecated in Lazarus 2.2.0. Use KOI8RToUTF8 instead.

Deprecated in Lazarus 2.2.0. Use KOI8RToUTF8 instead.

Converts a value encoded using the the Macintosh Code Page to UTF-8.

Calls SingleByteToUTF8 to convert the specified string using the character translation array in ArrayMacintoshToUTF8.

UTF-8-encoded value for the specified string.
String with the value in the Macintosh Roman encoding.
Converts a string with single-byte values to UTF-8 using a character translation array.

s is the string, in its single-byte encoding, with the character values converted in the routine.

Table is a TCharToUTF8Table type and contains an array with the multi-byte values for each of the character using the UTF-8 encoding. The array has 256 elements representing #0 and each of the decimal values in the single byte character set, and may contain 1-4 byte values as needed for the UTF-8 encoding.

SingleByteToUTF8 iterates over the character values in s, and converts each single-byte character to the multi-byte value at the corresponding ordinal position in Table. If s is an empty string (''), the return value is also an empty string.

The return value is cast to a RawByteString type, and its code page is set to CP_UTF8 (65001).

ArrayISO_8859_1ToUTF8 ArrayISO_8859_15ToUTF8 ArrayISO_8859_2ToUTF8 ArrayCP1250ToUTF8 ArrayCP1251ToUTF8 ArrayCP1252ToUTF8 ArrayCP1253ToUTF8 ArrayCP1254ToUTF8 ArrayCP1255ToUTF8 ArrayCP1255ToUTF8 ArrayCP1257ToUTF8 ArrayCP437ToUTF8 ArrayCP850ToUTF8 ArrayCP866ToUTF8 ArrayKOI8RToUTF8 ArrayKOI8UToUTF8 ArrayMacintoshToUTF8
String with the UTF-8-encoded value, or an empty string.
String with the single-byte values converted in the routine.
Table with Character to PChar mappings for the converted value.
Converts a string from UCS 2-byte LE encoding to UTF-8.

UCS2LEToUTF8 is a String function used to convert a value encoded using UCS2 LE (Little Endian) to its UTF-8 encoding. UCS2 is a fixed-length encoding where each character is represented using 2 bytes (16-bits). Byte values are stored in Least Significant (Little Endian) byte order.

UCS2LEToUTF8 iterates over the characters in the string value, and converts each character to the variable length multi-byte encoding used for characters in UTF-8. LEToN is called to convert the byte values to the byte order used for the platform. The UnicodeToUTF8SkipErrors routine in lazutf8.pas is called to handle code points which are malformed, require translation or are not used in UTF-8.

An exception is raised in UCS2LEToUTF8 if the length of the converted string is longer than 1.5 times the original string length.

The return value is cast to a RawByteString type, and SetCodePage is called to set the code page to CP_UTF8 (65001) in the result.

No actions are performed in the routine when s is an empty string (''), and the return value is an empty string.

UnicodeToUTF8SkipErrors SetCodePage LEToN
String value after conversion to UTF-8 encoding.
String value using UCS2 LE encoding.
Converts a string from UCS 2-byte BE encoding to UTF-8.

UCS2BEToUTF8 is a String function used to convert a value encoded using UCS2 BE (Big Endian) to its UTF-8 encoding. UCS2 is a fixed-length encoding where each character is represented using 2 bytes (16-bits). Byte values are stored in Most Significant (Big Endian) byte order.

UCS2BEToUTF8 iterates over the characters in the string value, and converts each character to the variable length multi-byte encoding used for characters in UTF-8. BEToN is called to convert the byte values to the byte order used for the platform. The UnicodeToUTF8SkipErrors routine in lazutf8.pas is called to handle code points which are malformed, require translation or are not used in UTF-8.

An exception is raised in UCS2BEToUTF8 if the length of the converted string is longer than 1.5 times the original string length.

The return value is cast to a RawByteString type, and SetCodePage is called to set the code page to CP_UTF8 (65001) in the result.

No actions are performed in the routine when s is an empty string (''), and the return value is an empty string.

UnicodeToUTF8SkipErrors SetCodePage BEToN
String value after conversion to UTF-8 encoding.
String value using UCS2 BE encoding.
Converts a string from UTF-8 encoding to UTF-8 with a Byte Order Mark.

UTF8ToUTF8BOM simply prepends the byte values in UTF8BOM to the value passed in the s argument. No actual encoding conversion is required. The return value contains the concatenated values.

UTF-8-encoded string value after the BOM is inserted at the beginning of the string.
UTF8-encoded string value updated in the routine.
Converts a string from UTF-8 encoding to ISO 8859-1.

UTF8ToISO_8859_1 is a RawByteString function used to convert a UTF-8-encoded string to the ISO 8859-1 (Central European) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the ISO 8859-1 code page (28591). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToISO_8859_1 to perform character conversions needed for the target encoding.

UTF8ToISO_8859_1 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to ISO 8859-1.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if the default system code page (CP_ACP) is used.
Converts a string from UTF-8 encoding to ISO 8859-15.

UTF8ToISO_8859_15 is a RawByteString function used to convert a UTF-8-encoded string to the ISO 8859-15 (Western European) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the ISO 8859-15 code page (28605). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToISO_8859_15 to perform character conversions needed for the target encoding.

UTF8ToISO_8859_15 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to ISO 8859-15.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if the default system code page (CP_ACP) is used.
Converts a string from UTF-8 encoding to ISO 8859-2.

UTF8ToISO_8859_2 is a RawByteString function used to convert a UTF-8-encoded string to the ISO 8859-2 (Eastern European) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the ISO 8859-2 code page (28592). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToISO_8859_2 to perform character conversions needed for the target encoding.

UTF8ToISO_8859_2 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to ISO 8859-2.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if the default system code page (CP_ACP) is used.
Converts a string from UTF-8 encoding to Code Page 1250.

UTF8ToCP1250 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1250 (Central European) encoding.

UTF8ToCP1250 calls the UTF8ToSingleByte routine using UnicodeToCP1250 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#129) to the target code page. The character translation table in UnicodeToCP1250 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1250 in the return value.

UTF8ToCP1250 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1250.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 1251.

UTF8ToCP1251 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1251 (Cyrillic) encoding.

UTF8ToCP1251 calls the UTF8ToSingleByte routine using UnicodeToCP1251 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#152) to the target code page. The character translation table in UnicodeToCP1251 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1251 in the return value.

UTF8ToCP1251 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1251.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 1252.

UTF8ToCP1252 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1252 (Latin 1) encoding.

UTF8ToCP1252 calls the UTF8ToSingleByte routine using UnicodeToCP1252 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#128) to the target code page. The character translation table in UnicodeToCP1252 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1252 in the return value.

UTF8ToCP1252 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1252.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 1253.

UTF8ToCP1253 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1253 (Greek) encoding.

UTF8ToCP1253 calls the UTF8ToSingleByte routine using UnicodeToCP1253 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#129) to the target code page. The character translation table in UnicodeToCP1253 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1253 in the return value.

UTF8ToCP1253 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1253.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 1254.

UTF8ToCP1254 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1254 (Turkish) encoding.

UTF8ToCP1254 calls the UTF8ToSingleByte routine using UnicodeToCP1254 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#129) to the target code page. The character translation table in UnicodeToCP1254 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1254 in the return value.

UTF8ToCP1254 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1254.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 1255.

UTF8ToCP1255 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1255 (Hebrew) encoding.

UTF8ToCP1255 calls the UTF8ToSingleByte routine using UnicodeToCP1255 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#129) to the target code page. The character translation table in UnicodeToCP1255 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1255 in the return value.

UTF8ToCP1255 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1255.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 1256.

UTF8ToCP1256 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1256 (Arabic) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the Arabic code page (1256). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToCP1256 to perform character conversions needed for the target encoding.

UTF8ToCP1256 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1256.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if the default system code page (CP_ACP) is used.
Converts a string from UTF-8 encoding to Code Page 1257.

UTF8ToCP1257 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1257 (Baltic) encoding.

UTF8ToCP1257 calls the UTF8ToSingleByte routine using UnicodeToCP1257 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#129) to the target code page. The character translation table in UnicodeToCP1257 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1257 in the return value.

UTF8ToCP1255 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1257.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 1258.

UTF8ToCP1258 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 1258 (Vietnamese) encoding.

UTF8ToCP1258 calls the UTF8ToSingleByte routine using UnicodeToCP1258 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#129) to the target code page. The character translation table in UnicodeToCP1258 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 1258 in the return value.

UTF8ToCP1258 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 1258.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 437.

UTF8ToCP437 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 437 (DOS Latin US) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the DOS Latin US code page (437). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToCP437 to perform character conversions needed for the target encoding.

UTF8ToCP437 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 437.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 850.

UTF8ToCP850 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 850 (DOS Western Europe) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the DOS Western Europe code page (850). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToCP850 to perform character conversions needed for the target encoding.

UTF8ToCP850 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 850.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 852.

UTF8ToCP852 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 852 (DOS Central Europe) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the DOS Central Europe code page (852). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToCP852 to perform character conversions needed for the target encoding.

UTF8ToCP852 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 852.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 866.

UTF8ToCP866 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 866 (DOS Cyrillic Russian) encoding.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the DOS Cyrillic Russian code page (866). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToCP866 to perform character conversions needed for the target encoding.

UTF8ToCP866 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 866.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to Code Page 874.

UTF8ToCP874 is a RawByteString function used to convert a UTF-8-encoded string to the Code Page 874 (Thai) encoding.

UTF8ToCP874 calls the UTF8ToSingleByte routine using UnicodeToCP874 to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because of a failure when translating Unicode code points (#129) to the target code page. The character translation table in UnicodeToCP874 is used instead. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 874 in the return value.

UTF8ToCP874 is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

SetCodePage
String value after conversion to Code Page 874.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to KOI8-R encoding.

UTF8ToKOI8R is a RawByteString function used to convert a UTF-8-encoded string to the KOI8-R (Russian Cyrillic 8-bit) encoding. KOI8R stands for Kod Obmena Informatsiey, 8-bit Russian.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the KOI8-R code page (20866). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToKOI8R to perform character conversions needed for the target encoding.

UTF8ToKOI8R is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

String value after conversion to KOI8-R encoding.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to KOI8-U encoding.

UTF8ToKOI8U is a RawByteString function used to convert a UTF-8-encoded string to the KOI8-U (Russian Ukranian 8-bit) encoding. KOI8R stands for Kod Obmena Informatsiey, 8-bit Ukranian.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the KOI8-U code page (21866). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToKOI8U to perform character conversions needed for the target encoding.

UTF8ToKOI8U is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

String value after conversion to KOI8-U encoding.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a string from UTF-8 encoding to KOI8-RU encoding.

UTF8ToKOI8RU is a RawByteString function used to convert a UTF-8-encoded string to the KOI8RU (Russian, Belarusian, Ukranian Cyrillic) encoding.

UTF8ToKOI8RU calls the UTF8ToSingleByte routine using UnicodeToKOI8RU to perform character conversions needed for the target encoding. System code pages are not used (when enabled) because KOI8RU does not have an official code page. The character translation table in UnicodeToKOI8RU is used to convert character values for the encoding. When SetTargetCodePage is True, SetCodePage is called (without character conversion) to set the code page to 0 in the return value.

UTF8ToKOI8RU is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

String value after conversion to KOI8-RU encoding.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a value from UTF-8 encoding to the Macintosh Code Page.

UTF8ToMacintosh is a RawByteString function used to convert a UTF-8-encoded string to the Apple Macintosh Roman code page.

For platforms where system code pages are enabled, SetCodePage is called to convert the value to the Macintosh Roman code page (10000). If SetTargetCodePage is False, SetCodePage is called again to apply the CP_ACP (ANSI) code page in the return value.

For other platforms, the UTF8ToSingleByte routine is called using UnicodeToMacintosh to perform character conversions needed for the target encoding.

UTF8ToMacintosh is used in the implementation of the ConvertEncodingFromUTF8 encoding conversion routine.

String value after conversion to Macintosh encoding.
UTF-8-encoded string value converted in the routine.
True if the target code page is set in the return value, False if a code page is not used.
Converts a UTF-8-encoded string value to a single-bye character set using a conversion function.

UTF8ToSingleByte is used in an internal routine to convert the UTF-8-encoded value in s by calling the custom conversion routine in UTF8CharConvFunc. The return value is a string with the single-byte representation for the string value. UTF8ToSingleByte is called when the UseSystemCPConv compiler flag has not been defined for the platform, and system code pages are not availabe for the purpose.

No actions are performed in the routine when s is an empty string (''), and the return value is set to an empty string.

UTF8ToSingleByte iterates over the Unicode codepoints in s, and calls the routine in UTF8CharConvFunc for each of the codepoints. If a codepoint is not handled by the UTF8CodepointToUnicode routine, the value in ConvertEncodingErrorMode is used to determine the action taken for the character value. This includes:

ceemSkip
  The character value is ignored.
ceemReplace
  Inserts a '?' character for the unknown codepoint.
ceemReturnEmpty
  Returns an empty string ('') for the conversion routine.
ceemException
  Raises an EConvertError exception with the message 'Cannot convert UTF8 to single byte'.

UTF8ToSingleByte is called from an internal conversion routine, and occurs when the following routines are executed:

- UTF8ToISO_8859_1
- UTF8ToISO_8859_2
- UTF8ToISO_8859_15
- UTF8ToCP1250
- UTF8ToCP1251
- UTF8ToCP1252
- UTF8ToCP1253
- UTF8ToCP1254
- UTF8ToCP1255
- UTF8ToCP1256
- UTF8ToCP1257
- UTF8ToCP1258
- UTF8ToCP437
- UTF8ToCP850
- UTF8ToCP852
- UTF8ToCP866
- UTF8ToCP874
- UTF8ToKOI8R
- UTF8ToKOI8U
- UTF8ToKOI8RU
- UTF8ToMacintosh

String value after conversion to an encoding using a single-byte character set.
UTF-8-encoded string value converted in the routine.
Routine used translate Unicode code points to the singe-byte character set for the encoding.
Converts a UTF-8-encoded value to UCS 2-byte Little Endian encoding.

UTF8ToUCS2LE converts the string value in s from its multi-byte variable length UTF-8 encoding to the fixed length 16-bit character set used in the UCS2 LE (Little Endian) encoding. UTF8ToUCS2LE iterates over each of the UTF-8 codepoints in s, and calls NtoLE to convert the Unicode character value to the Least Signifcant byte order used in the encoding. Unicode characters with a value $FFFF or greater are ignored in the conversion.

No actions are performed in the routine when s contains an empty string (''), and the return value is set to an empty string as well.

UTF8ToUCS2LE does not include a byte order mark (BOM) in the UCS2-encoded output.

String value after conversion to UCS2 LE encoding.
UTF-8-encoded string value converted in the routine.
Converts a UTF-8-encoded value to UCS 2-byte Big Endian encoding.

UTF8ToUCS2BE converts the string value in s from its multi-byte variable length UTF-8 encoding to the fixed length 16-bit character set used in the UCS2 BE (Big Endian) encoding. UTF8ToUCS2BE iterates over each of the UTF-8 codepoints in s, and calls NtoBE to convert the Unicode character value to the Most Signifcant byte order used in the encoding. Unicode characters with a value $FFFF or greater are ignored in the conversion.

No actions are performed in the routine when s contains an empty string (''), and the return value is set to an empty string as well.

UTF8ToUCS2BE does not include a byte order mark (BOM) in the UCS2-encoded output.

String value after conversion to UCS2 BE encoding.
UTF-8-encoded string value converted in the routine.
Converts a string value using Code Page 932 to UTF-8.

CP932ToUTF8 is defined for platforms where Asian code pages are enabled.

String value after conversion to UTF-8 encoding.
String value using code page 932.
Converts a string value using Code Page 936 to UTF-8.

CP936ToUTF8 is defined for platforms where Asian code pages are enabled.

String value after conversion to UTF-8 encoding.
String value using code page 936.
Converts a string value using Code Page 949 to UTF-8.

CP949ToUTF8 is defined for platforms where Asian code pages are enabled.

String value after conversion to UTF-8 encoding.
String value using code page 949.
Converts a string value using Code Page 950 to UTF-8.

CP950ToUTF8 is defined for platforms where Asian code pages are enabled.

String value after conversion to UTF-8 encoding.
String value using code page 950.
Converts a string from UTF-8 encoding to Code Page 932.

UTF8ToCP932 is defined for platforms where Asian code pages are enabled.

SetCodePage
String value after conversion to code page 932.
String value using UTF-8 encoding.
True to set the code page for the result, Falase to use the ANSI code page.
Converts a string from UTF-8 encoding to Code Page 936.

UTF8ToCP936 is defined for platforms where Asian code pages are enabled.

SetCodePage
String value after conversion to code page 936.
String value using UTF-8 encoding.
True to set the code page for the result, Falase to use the ANSI code page.
Converts a string from UTF-8 encoding to Code Page 949.

UTF8ToCP949 is defined for platforms where Asian code pages are enabled.

SetCodePage
String value after conversion to code page 949.
String value using UTF-8 encoding.
True to set the code page for the result, Falase to use the ANSI code page.
Converts a string from UTF-8 encoding to Code Page 950.

UTF8ToCP950 is defined for platforms where Asian code pages are enabled.

SetCodePage
String value after conversion to code page 949.
String value using UTF-8 encoding.
True to set the code page for the result, Falase to use the ANSI code page.
Converts a value from UTF-8 encoding to a Double Byte Character Set encoding.

Defined for platforms where Asian code pages are enabled. This is a common function used in all conversion routines which convert UTF-8 to an Asian code page.

String value using the double-byte character set generated in the conversion function.
String value using UTF-8 encoding.
Conversion routine used to translate characters to the double-byte encoding used in the result.
Adds supported encoding names for the platform to the specified string list.

GetSupportedEncodings is a procedure used to get the names for the supported encodings on the platform or operating system. GetSupportedEncodings adds each of the encoding names to List; it does NOT removing any existing content in the string list.

GetSupportedEncodings stores the following values in the List argument:

- 'UTF-8'
- 'UTF-8BOM'
- 'Ansi'
- 'cp1250' (EncodingCP1250)
- 'cp1251' (EncodingCP1251)
- 'cp1252' (EncodingCP1252)
- 'cp1253' (EncodingCP1253)
- 'cp1254' (EncodingCP1254)
- 'cp1255' (EncodingCP1255)
- 'cp1256' (EncodingCP1256)
- 'cp1257' (EncodingCP1257)
- 'cp1258' (EncodingCP1258)
- 'cp437' (EncodingCP437)
- 'cp850' (EncodingCP850)
- 'cp852' (EncodingCP852)
- 'cp866' (EncodingCP866)
- 'cp874' (EncodingCP874)

For platforms that support Asian code pages, the following encoding names are added to the list:

- 'cp932' (EncodingCP932)
- 'cp936' (EncodingCP936)
- 'cp950' (EncodingCP950)
- 'cp949' (EncodingCP949)

The following encoding names are added to the end of the list:

- 'ISO-8859-1'
- 'ISO-8859-2'
- 'ISO-8859-15'
- 'KOI8-R'
- 'KOI8-U'
- 'KOI8-RU'
- 'Macintosh'
- 'UCS-2LE'
- 'UCS-2BE'

String list updated with the supported encoding names.