XML Declaration and Encoding

The XML Declaration and Encoding

A document should open with an XML declaration, <?xml ... ?>, which has three pseudo-attributes:

The declaration must come first in the file (even a leading space is a fatal error), and it must tell the truth. A classic ingestion bug is a Latin-1 export that claims to be UTF-8:

A lying encoding declaration and an honest one
# The byte 0xE9 is "é" in ISO-8859-1 (Latin-1) but is not valid UTF-8 on its own
printf '<?xml version="1.0" encoding="UTF-8"?>\n<title>Caf\xe9</title>\n' > liar.xml
printf '<?xml version="1.0" encoding="ISO-8859-1"?>\n<title>Caf\xe9</title>\n' > honest.xml
xmllint --noout liar.xml
xmllint --encode UTF-8 honest.xml > fixed.xml
cat fixed.xml; file fixed.xml
Output
liar.xml:2: I/O error : Invalid bytes in character encoding
Bytes: 0xE9 0x3C 0x2F 0x74
<title>Caf
          ^
<?xml version="1.0" encoding="UTF-8"?>
<title>Café</title>
fixed.xml: XML 1.0 document, Unicode text, UTF-8 text

The honest file parses, and --encode UTF-8 rewrites it as UTF-8 with a matching declaration. Normalize every inbound feed to UTF-8 at the edge of the pipeline. (To read the declaration at all, a parser first guesses the encoding from a BOM or the bytes of <?xm, as XML 1.0's Appendix F describes.)


Document Declaration

An XML document should begin with an XML declaration, denoted by <?xml …… ?>. The three possible attributes are:
ch02-xml-declaration-example.xmlXML
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<bookslist>
</bookslist>