feat: decouple log configuration from apifylog

[l2ysho]

Introduces a CrawleeLogger interface and BaseCrawleeLogger abstract class so users can plug in any logger (Winston, Pino, Bunyan, etc.) instead of being locked to @apify/log.

1. CrawleeLogger interface — the contract any logger must satisfy
2. BaseCrawleeLogger abstract class — implement _log() + _createChild(), get 13 methods for free
3. Configuration.loggerProvider — the injection point

How it works internally

Every module that needs logging now calls config.getLogger() instead of importing @apify/log directly. Two patterns:

Pattern	When
config.getLogger().child({ prefix })	Classes that receive config (BasicCrawler, AutoscaledPool, etc.)
Configuration.getGlobalConfig().getLogger()	Standalone functions without config access (request.ts, cookie_utils.ts, etc.)

Only two files still import @apify/log directly: log.ts (defines interface) and configuration.ts (default fallback).

• Example — Winston adapter in ~15 lines:

class WinstonAdapter extends BaseCrawleeLogger {
   constructor(private logger: winston.Logger, options?: Partial<CrawleeLoggerOptions>) {
       super(options);
   }

    protected _log(level: number, message: string, data?: Record<string, any> | null): void {
       const winstonLevel = { 1: 'error', 2: 'warn', 3: 'warn', 4: 'info', 5: 'debug', 6: 'debug' }[level] ?? 'info';
        this.logger.log(winstonLevel, message, { ...data, prefix: this.getOptions().prefix });
    }

    protected _createChild(options: Partial<CrawleeLoggerOptions>): CrawleeLogger {
        return new WinstonAdapter(this.logger.child({ prefix: options.prefix }), { ...this.getOptions(), ...options });
    }
}

[l2ysho]

ready for review, user now only need to define 2 functions and wrapper handle others.

[barjin]

Thank you @l2ysho 🙌 I only have some minor notes ⬇️

[barjin]

nit / refactor: can getLog(ger) be a static method on Request, just to make the design a bit tidier?

[barjin]

If this is exposed in the public interface, we should assume people will use this (especially if there is no other method that does this).

How about naming this logger.logWithLevel or sth similar and add a friendly comment to encourage the (power)users instead?

[l2ysho]

You mean rename just in interface and keep the internal implementation in Abstract Class?

[barjin]

I suppose you'll need to rename both the interface member and the implementing function, but yes, it's just about the name / JSDoc for me. I believe our more advanced users could find usecases for this method too, no need to hide it from them.

[l2ysho]

I am digging deep into it and I am not sure how you mean it. Internal function is implemented in apify/log, you want this change to bubble up to apify/log package?

[barjin]

I'm sorry if I'm confusing.

I'd prefer to have a more descriptive name for the internal method, as it's public. This change would have to rename both CrawleeLogger.internal and BaseCrawleeLogger.internal.

As Honza mentions in the other comment

We don't really need to keep @apify/log directly assignable to CrawlingContext.log

so the @apify/log's interface doesn't really bother me now (ok it does - for the same reason as here - but we don't need to fix it now :)).

[l2ysho]

I'm sorry if I'm confusing.

I believe I am the source of my confusion :]

But I have this example, this is one of a few usages of .internal -> basic-crawler.ts

this.log.internal(LogLevel[(options.level as 'DEBUG') ?? 'DEBUG'], message, data);

this.log is either @apify/log or custom logger provided by user. So i have to ensure that it exists on both. I cannot rename it in CrawleeLogger and BaseCrawleeLogger because I have to match @apify/log

Or I am missing something? 🤔

[barjin]

As Honza mentions in the other comment

Feel free to tweak stuff to a reasonable degree; we can afford BC breaks. We don't really need to keep @apify/log directly assignable to CrawlingContext.log. It's no problem if that's going to require some additional wrapper or even adding a new method to @apify/log.

I.e. right now, you don't have to match the @apify/log and CrawleeLogger APIs (I believe that was the original reason for these changes; we want to free ourselves from @apify/log and create our own Crawlee thing).

this.log is either @apify/log or custom logger provided by user.

imo after merging this PR, this.log should implement CrawleeLogger interface (and that's all we'll know about it). Whether it's a custom user-supplied logger or a CrawleeLogger wrapper for @apify/log should be inconsequential.

Let's keep the .internal() method for now if it's easier. I don't want to muddle your PR with minor things like this :) It's still good that we discussed this, because maybe I'm understanding the motivations behind this PR wrong.

Can we please get a third pair of eyes @janbuchar? 🙏

[barjin]

This is a semantic change - originally, we would call warningOnce on the global instance (so it would log the warning only once).

Now we're calling warningOnce on each crawler's instance separately. These do not share the state, so this will log the message n times (n == crawler count).

[B4nan]

Does it matter really? Feels all right to me.

[barjin]

It logs the same multiline message multiple times... not the end of the world, but imo unnecessary. I noticed this just because I dealt with this when implementing this not so long ago :)

If using the global log instance proves too complicated, I'm fine with this; if it's a one-line change, I'd prefer clean logs.

[l2ysho]

yop, also claude suggested this to me but I was not believe him at the moment. I can take a look, probably I can store the state and pass it to child.

[l2ysho]

fixed by static logger instance for basic-crawler

[l2ysho]

feedback from martin

Important (Should Fix)

• Fix stale lazy singleton in request.ts after config reset
• Fix exception() discarding the Error object in log.ts
• Complete decoupling — wire 7 files still using @apify/log through Configuration.getLogger()
• Standardize data parameter type in CrawleeLogger interface

Suggestions (Nice to Have)

• Add loggerProvider to Configuration JSDoc table
• Add end-to-end integration test for custom logger via Configuration
• Fix ProxyConfiguration bypassing injected config
• Add null guard for getOptions().prefix in basic-crawler.ts
• Add explanation for SOFT_FAIL mapping in Winston example

[janbuchar]

@l2ysho is it fine if I postpone my review until the conflicts are resolved and Jindra's feedback incorporated?

[l2ysho]

@l2ysho is it fine if I postpone my review until the conflicts are resolved and Jindra's feedback incorporated?

Definitelly

[l2ysho]

@janbuchar I think you can start, I addressed most critical issues from feedback. In the meantime I have still 10 minor/nice-to-have issues in stack to check