Xsd validation fails when encoding is different from the xsd file

[rajzik]

When I try to validate xml file with XSD document and the encoding is set in the XML document I get encoding errors in the result.

I am loading XmlDocuments from string so encoding should be ignored.

When I replace encoding with utf-8 everything works

Reproduction

https://github.com/rajzik/libxml-repro

[shunkica]

The provided test fails because XmlDocument.fromString internally encodes the input JavaScript string to UTF-8 bytes before passing them to libxml2 for parsing. However, the XML declaration specifies encoding="ISO-8859-15", causing libxml2 to decode those UTF-8 bytes as the declared ISO encoding instead. This mismatch results in the garbled attribute value 'MÃŒller' during parsing, which fails the XSD fixed value validation.

You should use the fromBuffer method for this. It is much faster as well than the fromString method

import {describe, it, expect} from "vitest";
import {
    XmlDocument,
    XsdValidator,
} from "libxml2-wasm";
import {readFileSync} from "fs";


describe("libxml2", () => {
    it("works with buffers", () => {
        const xsdDoc = XmlDocument.fromBuffer(readFileSync("xml/test.xsd"));
        const xmlDoc = XmlDocument.fromBuffer(readFileSync("xml/test.xml"));
        expect(() => {
            XsdValidator.fromDoc(xsdDoc).validate(xmlDoc);
        }).not.toThrow();
    });
});

[shunkica]

This could possibly be fixed with something like:

export function xmlReadString(
    ctxt: XmlParserCtxtPtr,
    xmlString: string,
    url: string | null,
    encoding: string | null,
    options: number,
): XmlDocPtr {
    // If encoding is specified, we need to convert the string to bytes using that encoding
    // and then let libxml2 parse it with the specified encoding
    if (encoding) {
        // Convert string to bytes using the specified encoding
        const encoder = new TextEncoder();
        // For latin1/ISO-8859-15, we need to convert each character to its byte value
        const bytes = new Uint8Array(xmlString.length);
        for (let i = 0; i < xmlString.length; i++) {
            bytes[i] = xmlString.charCodeAt(i) & 0xFF;
        }
        return withCString(
            bytes,
            (xmlBuf, len) => withStrings(
                (urlBuf, encodingBuf) => libxml2._xmlCtxtReadMemory(ctxt, xmlBuf, len, urlBuf, encodingBuf, options),
                url,
                encoding,
            ),
        );
    } else {
        return withStringUTF8(
            xmlString,
            (xmlBuf, len) => withStrings(
                (urlBuf, encodingBuf) => libxml2._xmlCtxtReadMemory(ctxt, xmlBuf, len, urlBuf, encodingBuf, options),
                url,
                encoding,
            ),
        );
    }
}

export function xmlReadMemory(
    ctxt: XmlParserCtxtPtr,
    xmlBuffer: Uint8Array,
    url: string | null,
    encoding: string | null,
    options: number,
): XmlDocPtr {
    return withCString(
        xmlBuffer,
        (xmlBuf, len) => withStrings(
            (urlBuf, encodingBuf) => libxml2._xmlCtxtReadMemory(ctxt, xmlBuf, len, urlBuf, encodingBuf, options),
            url,
            encoding,
        ),
    );
}

And using the encoding in parse:

function parse<Input>(
    parser: (
        ctxt: XmlParserCtxtPtr,
        source: Input,
        url: string | null,
        encoding: string | null,
        options: number,
    ) => XmlDocPtr,
    source: Input,
    url: string | null,
    options: ParseOptions,
): XmlDocument {
    const xmlOptions = options.option ?? ParseOption.XML_PARSE_DEFAULT;
    const ctxt = xmlNewParserCtxt();
    const errIndex = error.storage.allocate([]);
    xmlCtxtSetErrorHandler(ctxt, error.errorCollector, errIndex);
    const xml = parser(
        ctxt,
        source,
        url ?? null,
        options.encoding ?? null,
        xmlOptions,
    );
    try {
        const errDetails = error.storage.get(errIndex);
        if (errDetails.length > 0) {
            if (!xml) {
                xmlFreeDoc(xml);
            }
            throw new XmlParseError(errDetails!.map((d) => d.message).join(''), errDetails!);
        }
    } finally {
        error.storage.free(errIndex);
        xmlFreeParserCtxt(ctxt);
    }

    if (!xml) {
        // no error from libxml2, but failed to parse. Usually due to invalid input.
        throw new XmlParseError('Failed to parse XML', []);
    }
    const xmlDocument = XmlDocument.getInstance(xml);
    if (xmlOptions & ParseOption.XML_PARSE_XINCLUDE) {
        xmlDocument.processXInclude();
    }
    return xmlDocument;
}

Then a test like this would pass:

    describe('encoding support', () => {
        it('should properly handle string encoding when passed', () => {
            using xmlDoc = XmlDocument.fromString(readFileSync('test/testfiles/test.xml', 'latin1'), { encoding: 'ISO-8859-15' });
            using xsdDoc = XmlDocument.fromString(readFileSync('test/testfiles/test.xsd', 'utf-8'));
            using validator = XsdValidator.fromDoc(xsdDoc);

            expect(() => validator.validate(xmlDoc)).to.not.throw();
        });
    });

[jameslan]

For XmlDocument.fromString, it should encode the unicode string to specific encoding, so that it could be consistent with the encoding declaration inside the xml.
For XmlDocument.fromBuffer, it is already encoded bytes, so the encoding parameter must be null or same with the encoding declaration.

However, all the DOM functions and properties are unicode on JS side, we need to confirm that they can handle the encoding correctly.