Skip to content

Neither ProcessPool nor ThreadPool respawn dead workers #2

@movermeyer

Description

@movermeyer

If you are processing a stream that has a chance of failure, each workers in your ThreadPool/ProcessPool will eventually hit a failure case and die.

Once all of the workers have died, processing stops.

For example, your "Retrieving web pages concurrently" only works because your ThreadPool has more workers than failures (4 workers vs 2 failures). If you reduce the number of workers to 1, it only processes the first two URLs before dying.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions