Skip to content
This repository was archived by the owner on Nov 16, 2023. It is now read-only.

Add sample configuration loaders for inMemory and ciab#106

Open
jeffmcaffer wants to merge 1 commit intoconfigRefactorfrom
jm/configLoaders
Open

Add sample configuration loaders for inMemory and ciab#106
jeffmcaffer wants to merge 1 commit intoconfigRefactorfrom
jm/configLoaders

Conversation

@jeffmcaffer
Copy link
Contributor

Sketched out implementations of two different config loaders that go with the PR from @patrick-steele-idem for refactoring the crawler factory. That PR should end up in this branch eventually.

This change takes a poor man's dependency injection approach (thanks @willbar). The basic idea is that a config loader creates effectively a dependency injection container full of services that make up the crawler (including the crawler itself). Each service is created by a factory (or whatever you want). The standard pattern is for the factory function to get a container of services previously created, an optional name (if provided it should add itself to the container) and an optional set of default settings.

There is also a philosophical change in that the provider code should have core defaults built in so the user does not have to spec any/many options. We'll have to document the available options for each provider but that's a good thing anyway.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants