Logging

The Apify SDK is logging useful information through the logging module from Python's standard library, into the logger with the name apify.

Automatic configuration

When you create an Actor from an Apify-provided template, either in Apify Console or through the Apify CLI, you do not have to configure the logger yourself. The template already contains initialization code for the logger,which sets the logger level to DEBUG and the log formatter to ActorLogFormatter.

Manual configuration

Configuring the log level

In Python's default behavior, if you don't configure the logger otherwise, only logs with level WARNING or higher are printed out to the standard output, without any formatting. To also have logs with DEBUG and INFO level printed out, you need to call the Logger.setLevel method on the logger, with the desired minimum level as an argument.

Configuring the log formatting

By default, only the log message is printed out to the output, without any formatting. To have a nicer output, with the log level printed in color, the messages nicely aligned, and extra log fields printed out,you can use the ActorLogFormatter class from the apify.log module.

Example log configuration

To configure and test the logger, you can use this snippet:

import logging

from apify.log import ActorLogFormatter


async def main() -> None:
    handler = logging.StreamHandler()
    handler.setFormatter(ActorLogFormatter())

    apify_logger = logging.getLogger('apify')
    apify_logger.setLevel(logging.DEBUG)
    apify_logger.addHandler(handler)

This configuration will cause all levels of messages to be printed to the standard output, with some pretty formatting.

Logger usage

Here you can see how all the log levels would look like.

You can use the extra argument for all log levels, it's not specific to the warning level. When you use Logger.exception, there is no need to pass the Exception object to the log manually, it will automatiacally infer it from the current execution context and print the exception details.

import logging

from apify import Actor
from apify.log import ActorLogFormatter


async def main() -> None:
    handler = logging.StreamHandler()
    handler.setFormatter(ActorLogFormatter())

    apify_logger = logging.getLogger('apify')
    apify_logger.setLevel(logging.DEBUG)
    apify_logger.addHandler(handler)

    async with Actor:
        Actor.log.debug('This is a debug message')
        Actor.log.info('This is an info message')
        Actor.log.warning('This is a warning message', extra={'reason': 'Bad Actor!'})
        Actor.log.error('This is an error message')
        try:
            raise RuntimeError('Ouch!')
        except RuntimeError:
            Actor.log.exception('This is an exceptional message')

Result:

DEBUG This is a debug message
INFO  This is an info message
WARN  This is a warning message ({"reason": "Bad Actor!"})
ERROR This is an error message
ERROR This is an exceptional message
      Traceback (most recent call last):
        File "main.py", line 6, in <module>
          raise RuntimeError('Ouch!')
      RuntimeError: Ouch!

Redirect logs from other Actor runs

In some situations, one Actor is going to start one or more other Actors and wait for them to finish and produce some results. In such cases, you might want to redirect the logs and status messages of the started Actors runs back to the parent Actor run, so that you can see the progress of the started Actors' runs in the parent Actor's logs. This guide will show possibilities on how to do it.

Redirecting logs from Actor.call

Typical use case for log redirection is to call another Actor using the Actor.call method. This method has an optional logger argument, which is by default set to the default literal. This means that the logs of the called Actor will be automatically redirected to the parent Actor's logs with default formatting and filtering. If you set the logger argument to None, then no log redirection happens. The third option is to pass your own Logger instance with the possibility to define your own formatter, filter, and handler. Below you can see those three possible ways of log redirection when starting another Actor run through Actor.call.

import logging

from apify import Actor


async def main() -> None:
    async with Actor:
        # Default redirect logger
        await Actor.call(actor_id='some_actor_id')
        # No redirect logger
        await Actor.call(actor_id='some_actor_id', logger=None)
        # Custom redirect logger
        await Actor.call(
            actor_id='some_actor_id', logger=logging.getLogger('custom_logger')
        )

Each default redirect logger log entry will have a specific format. After the timestamp, it will contain cyan colored text that will contain the redirect information - the other actor's name and the run ID. The rest of the log message will be printed in the same manner as the parent Actor's logger is configured.

The log redirection can be deep, meaning that if the other actor also starts another actor and is redirecting logs from it, then in the top-level Actor, you can see it as well. See the following example screenshot of the Apify log console when one actor recursively starts itself (there are 2 levels of recursion in the example).

Redirecting logs from already running Actor run

In some cases, you might want to connect to an already running Actor run and redirect its logs to your current Actor run. This can be done using the ApifyClient and getting the streamed log from a specific Actor run. You can then use it as a context manager, and the log redirection will be active in the context, or you can control the log redirection manually by explicitly calling start and stop methods.

You can further decide whether you want to redirect just new logs of the ongoing Actor run, or if you also want to redirect historical logs from that Actor's run, so all logs it has produced since it was started. Both options are shown in the example code below.

import asyncio

from apify import Actor


async def main() -> None:
    async with Actor:
        # Lifecycle of redirected logs is handled by the context manager.
        async with await Actor.apify_client.run('some_actor_id').get_streamed_log(
            # Redirect all logs from the start of that run, even the logs from past.
            from_start=True
        ):
            await asyncio.sleep(5)
            # Logging will stop out of context

        # Lifecycle of redirected logs can be handled manually.
        streamed_log = await Actor.apify_client.run('some_id').get_streamed_log(
            # Do not redirect historical logs from this actor run.
            # Redirect only new logs from now on.
            from_start=False
        )
        streamed_log.start()
        await asyncio.sleep(5)
        await streamed_log.stop()