Skip to content

Limited parallel parsing #219

@Whiteknight

Description

@Whiteknight

Parsing currently cannot be multi-threaded because ISequence implementations are not multi-threaded. However, some types such as CharBufferSequence, ListSequence and decorator types (FilterSequence, MapSequence) should be able to be parallelized. Once we have a reference to the underlying data buffer, the actual position tracking and advancing can be in AsyncLocalStorage or ThreadLocalStorage.

Exactly how we use this is TBD. We can have a ParallelEach() which works like Each() but attempts several different parses from the current position and returns all of the results. We could also break the input up on delimiters or by chunk sizes, and parse each delimited section separately, in parallel.

This is very exploratory in nature.

Metadata

Metadata

Assignees

No one assigned

    Labels

    exploratoryWe might do it, pending investigationoptimizationSomething needs to be faster

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions