libfwevt_xml_document_substitute_template_value: unable to retrieve template value: 4 from array.

[nbareil]

Hello Joachim!

I get the following fatal error when exporting logs using evtxexport (b524d6b):

Unable to export file.
libcdata_array_get_entry_by_index: invalid entry index value out of bounds.
libfwevt_xml_document_substitute_template_value: unable to retrieve template value: 4 from array.
libfwevt_xml_document_read_optional_substitution: unable to substitute template value.
libfwevt_xml_document_read_attribute: unable to read optional substitution.
libfwevt_xml_document_read_element: unable to read attribute.
libfwevt_xml_document_read_element: unable to read element.
libfwevt_xml_document_read_element: unable to read element.
libfwevt_xml_document_read_template_instance: unable to read element.
libfwevt_xml_document_read_fragment: unable to read document template instance.
libfwevt_xml_document_substitute_template_value: unable to read fragment header.
libfwevt_xml_document_read_optional_substitution: unable to substitute template value.
libfwevt_xml_document_read_element: unable to read optional substitution.
libfwevt_xml_document_read_template_instance: unable to read element.
libfwevt_xml_document_read_fragment: unable to read document template instance.
libfwevt_xml_document_read_with_template_values: unable to read fragment header.
libfwevt_xml_document_read: unable to read XML document.
libevtx_record_values_read_xml_document: unable to read binary XML document.
libevtx_io_handle_read_chunk: unable to read record values XML document.
libfdata_list_get_element_value: unable to read element data at offset: 0x00004b30.
libfdata_list_get_element_value_by_index: unable to retrieve element value.
libevtx_file_get_record_by_index: unable to retrieve record values: 20.
export_handle_export_records: unable to retrieve record: 20.
export_handle_export_file: unable to export records.

I isolated the broken record in the attached broken.evtx.gz file. This file can be opened in Windows Event Viewer, it corresponds to "The VSS service is shutting down due to shutdown event from the Service Control Manager. %1". Yet, the record is 68KB ?!?

$ evtxinfo broken.evtx
evtxinfo 20190904

Windows Event Viewer Log (EVTX) information:
        Version                         : 3.1
        Number of records               : 1
        Number of recovered records     : 111

$ evtxexport broken.evtx
evtxexport 20190904

Unable to export file.
libcdata_array_get_entry_by_index: invalid entry index value out of bounds.
libfwevt_xml_document_substitute_template_value: unable to retrieve template value: 4 from array.
libfwevt_xml_document_read_optional_substitution: unable to substitute template value.
libfwevt_xml_document_read_attribute: unable to read optional substitution.
libfwevt_xml_document_read_element: unable to read attribute.
libfwevt_xml_document_read_element: unable to read element.
libfwevt_xml_document_read_element: unable to read element.
libfwevt_xml_document_read_template_instance: unable to read element.
libfwevt_xml_document_read_fragment: unable to read document template instance.
libfwevt_xml_document_substitute_template_value: unable to read fragment header.
libfwevt_xml_document_read_optional_substitution: unable to substitute template value.
libfwevt_xml_document_read_element: unable to read optional substitution.
libfwevt_xml_document_read_template_instance: unable to read element.
libfwevt_xml_document_read_fragment: unable to read document template instance.
libfwevt_xml_document_read_with_template_values: unable to read fragment header.
libfwevt_xml_document_read: unable to read XML document.
libevtx_record_values_read_xml_document: unable to read binary XML document.
libevtx_io_handle_read_chunk: unable to read record values XML document.
libfdata_list_get_element_value: unable to read element data at offset: 0x00001200.
libfdata_list_get_element_value_by_index: unable to retrieve element value.
libevtx_file_get_record_by_index: unable to retrieve record values: 0.
export_handle_export_records: unable to retrieve record: 0.
export_handle_export_file: unable to export records.

If I use evtx_structure.py, I get the following:

$ evtx_structure.py broken.evtx
File header
  magic: ElfFile�
  oldest_chunk: 0x0
  current_chunk_number: 0x0
  next_record_number: 0x2
  header_size: 0x80
  minor_version: 0x1
  major_version: 0x3
  header_chunk_size: 0x1000
  chunk_count: 0x1
  flags: 0x0
  checksum: 0xd0ff1810
  verify: True
  dirty: False
  full: False
  Chunk
    offset: 0x1000
    magic: ElfChnk�
    file_first_record_number: 0x1
    file_last_record_number: 0x1
    log_first_record_number: 0x1
    log_last_record_number: 0x1
    header_size: 0x80
    last_record_offset: 0x200
    next_record_offset: 0x6d8
    data_checksum: 0x779c967b
    header_checksum: 0x1b3405e2
    verify: True
    templates: 1
    Record
      offset: 0x1200
      magic: 0x2a2a
      size: 0x4d8
      number: 0x1
      timestamp: 2018-07-23 09:26:38.304127
      verify: True
      RootNode(offset=0x18)
        StreamStartNode(offset=0x18)
        TemplateInstanceNode(offset=0x1c, resident=True, length=0x345)
          TemplateNode(offset=0x26)
            StreamStartNode(offset=0x3e)
            OpenStartElementNode(offset=0x42) --> Event
              AttributeNode(offset=0x65) --> xmlns
                ValueNode(offset=0x7e)
                  WstringTypeNode(offset=0x80) --> http://schemas.microsoft.com/win/2004/08/events/event
              CloseStartElementNode(offset=0xec)
              OpenStartElementNode(offset=0xed) --> System
                CloseStartElementNode(offset=0x10e)
                OpenStartElementNode(offset=0x10f) --> Provider
                  AttributeNode(offset=0x138) --> Name
                    ValueNode(offset=0x14f)
                      WstringTypeNode(offset=0x151) --> VSS
                  CloseEmptyElementNode(offset=0x159)
                OpenStartElementNode(offset=0x15a) --> EventID
                  AttributeNode(offset=0x181) --> Qualifiers
                    ConditionalSubstitutionNode(offset=0x1a4)
                  CloseStartElementNode(offset=0x1a8)
                  ConditionalSubstitutionNode(offset=0x1a9)
                  CloseElementNode(offset=0x1ad)
                OpenStartElementNode(offset=0x1ae) --> Level
                  CloseStartElementNode(offset=0x1cd)
                  ConditionalSubstitutionNode(offset=0x1ce)
                  CloseElementNode(offset=0x1d2)
                OpenStartElementNode(offset=0x1d3) --> Task
                  CloseStartElementNode(offset=0x1f0)
                  ConditionalSubstitutionNode(offset=0x1f1)
                  CloseElementNode(offset=0x1f5)
                OpenStartElementNode(offset=0x1f6) --> Keywords
                  CloseStartElementNode(offset=0x21b)
                  ConditionalSubstitutionNode(offset=0x21c)
                  CloseElementNode(offset=0x220)
                OpenStartElementNode(offset=0x221) --> TimeCreated
                  AttributeNode(offset=0x250) --> SystemTime
                    ConditionalSubstitutionNode(offset=0x273)
                  CloseEmptyElementNode(offset=0x277)
                OpenStartElementNode(offset=0x278) --> EventRecordID
                  CloseStartElementNode(offset=0x2a7)
                  ConditionalSubstitutionNode(offset=0x2a8)
                  CloseElementNode(offset=0x2ac)
                OpenStartElementNode(offset=0x2ad) --> Channel
                  CloseStartElementNode(offset=0x2d0)
                  ValueNode(offset=0x2d1)
                    WstringTypeNode(offset=0x2d3) --> Application
                  CloseElementNode(offset=0x2eb)
                OpenStartElementNode(offset=0x2ec) --> Computer
                  CloseStartElementNode(offset=0x311)
                  ValueNode(offset=0x312)
                    WstringTypeNode(offset=0x314) --> XXXX
                  CloseElementNode(offset=0x332)
                OpenStartElementNode(offset=0x333) --> Security
                  AttributeNode(offset=0x35c) --> UserID
                    ConditionalSubstitutionNode(offset=0x377)
                  CloseEmptyElementNode(offset=0x37b)
                CloseElementNode(offset=0x37c)
              ConditionalSubstitutionNode(offset=0x37d)
              CloseElementNode(offset=0x381)
            EndOfStreamNode(offset=0x382)
      Substitutions(offset=0x383)
        UnsignedByteTypeNode(offset=0x3d7) --> 4
        UnsignedByteTypeNode(offset=0x3d8) --> 0
        UnsignedWordTypeNode(offset=0x3d9) --> 0
        UnsignedWordTypeNode(offset=0x3db) --> 8225
        UnsignedWordTypeNode(offset=0x3dd) --> 0
        Hex64TypeNode(offset=0x3df) --> 0x0080000000000000
        FiletimeTypeNode(offset=0x3e7) --> 2018-07-23 09:26:38.272814
        NullTypeNode(offset=0x3ef)
        UnsignedDwordTypeNode(offset=0x3ef) --> 0
        UnsignedDwordTypeNode(offset=0x3f3) --> 0
        UnsignedQwordTypeNode(offset=0x3f7) --> 1812
        UnsignedByteTypeNode(offset=0x3ff) --> 0
        NullTypeNode(offset=0x400)
        NullTypeNode(offset=0x400)
        NullTypeNode(offset=0x400)
        NullTypeNode(offset=0x400)
        NullTypeNode(offset=0x400)
        NullTypeNode(offset=0x400)
        NullTypeNode(offset=0x400)
        BXmlTypeNode(offset=0x400) --> 
          RootNode(offset=0x400)
            StreamStartNode(offset=0x400)
            TemplateInstanceNode(offset=0x404, resident=False)
          Substitutions(offset=0x40e)
            WstringArrayTypeNode(offset=0x41e) --> <string></string>

            UnsignedDwordTypeNode(offset=0x420) --> 168
            BinaryTypeNode(offset=0x424) --> LSBDb2RlOiAgQ09SU1ZDQzAwMDAwNzU3LSBDYWxsOiAgQ09SU1ZDQzAwMDAwNzQxLSBQSUQ6ICAwMDAwMTIwMC0gVElEOiAgMDAwMDEyMTItIENNRDogIEM6XFdJTkRPV1Ncc3lzdGVtMzJcdnNzdmMuZXhlICAgLSBVc2VyOiBOYW1lOiBOVCBBVVRIT1JJVFlcU1lTVEVNLCBTSUQ6Uy0xLTUtMTgg

I wish I could help you more!

[joachimmetz]

I'll have a look when time permits