### Setup and Run Tests with Redis Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/running_tests.md Use these commands to start Redis, install dependencies, and run tests. Ensure Redis is running in detached mode before executing tests. ```bash docker compose up -d redis ``` ```bash pip install -r requirements.txt ``` ```bash env CELERY_BROKER_URL=redis://0.0.0.0:6379 DJANGO_SETTINGS_MODULE=config.settings.test pytest test_app ``` ```bash env CELERY_BROKER_URL=redis://0.0.0.0:6379 DJANGO_SETTINGS_MODULE=config.settings.test_demo_app pytest django_structlog_demo_project ``` ```bash docker compose stop redis ``` -------------------------------- ### Example Log Output for Basic Command Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/commands.md This output shows the logs generated by a basic command, including command start, execution, and finish events, with added command_name and command_id. ```bash $ python manage.py example_command bar 2023-09-13T21:10:50.084368Z [info ] command_started [django_structlog.commands] command_name=django_structlog_demo_project.users.example_command command_id=be723d34-59f5-468e-9258-24232aa4cedd 2023-09-13T21:10:50.085325Z [info ] my log [django_structlog_demo_project.users.management.commands.example_command] command_id=be723d34-59f5-468e-9258-24232aa4cedd foo=bar 2023-09-13T21:10:50.085877Z [info ] command_finished [django_structlog.commands] command_id=be723d34-59f5-468e-9258-24232aa4cedd ``` -------------------------------- ### Install django-structlog Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/getting_started.md Use pip to install the library. ```bash pip install django-structlog ``` -------------------------------- ### Clone and Install Django-Structlog Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/development.md Clone the repository and install project dependencies using pip. Ensure you have pre-commit hooks installed for development. ```bash git clone https://github.com/jrobichaud/django-structlog.git cd django-structlog pip install -r requirements.txt pre-commit install ``` -------------------------------- ### Install django-structlog with Commands Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/commands.md Install the django-structlog package with the 'commands' extra to enable command support. This also installs django-extensions. ```bash pip install django-structlog[commands] ``` -------------------------------- ### Start Demo App with RabbitMQ Broker Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/development.md Configure and start the demo application using RabbitMQ as the message broker instead of Redis by specifying additional Docker Compose files. ```bash docker compose -f ./docker-compose.yml -f ./docker-compose.amqp.yml up --build ``` -------------------------------- ### Build and Run Docker Compose Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/demo.md Use this command to build the Docker image and start the application services. Access the demo at http://127.0.0.1:8000/. ```bash docker compose up --build ``` -------------------------------- ### Django Template Example Source: https://github.com/jrobichaud/django-structlog/blob/main/django_structlog_demo_project/templates/users/user_form.html A basic Django template demonstrating form rendering and CSRF token usage. No specific setup required beyond standard Django practices. ```html {% extends "base.html" %} {% load crispy_forms_tags %} {% block title %}{{ user.username }}{% endblock %} {% block content %} {{ user.username }} =================== {% csrf_token %} {{ form|crispy }} Update {% endblock %} ``` -------------------------------- ### JSON Log Output Example Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Example of a log entry formatted as JSON, including timestamp, level, event, and custom context. ```json {"request_id": "3a8f801c-072b-4805-8f38-e1337f363ed4", "user_id": 1, "ip": "0.0.0.0", "event": "an_error_occurred", "timestamp": "2019-04-13T19:39:31.089925Z", "logger": "my_awesome_project.my_awesome_module", "level": "info", "bar": "Buz"} ``` -------------------------------- ### Example JSON Log Output Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst This is an example of log output formatted as JSON, suitable for structured logging systems. ```json {"request_id": "c53dff1d-3fc5-4257-a78a-9a567c937561", "user_id": 1, "ip": "0.0.0.0", "request": "GET /", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36", "event": "request_started", "timestamp": "2019-04-13T19:39:29.321453Z", "logger": "django_structlog.middlewares.request", "level": "info"} {"request_id": "c53dff1d-3fc5-4257-a78a-9a567c937561", "user_id": 1, "ip": "0.0.0.0", "code": 200, "event": "request_finished", "timestamp": "2019-04-13T19:39:29.345207Z", "logger": "django_structlog.middlewares.request", "level": "info"} ``` -------------------------------- ### Install django-extensions Directly Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/commands.md Alternatively, install the django-extensions package separately if you do not need other features of django-structlog. ```bash pip install django-extensions ``` -------------------------------- ### Standard Logging Example Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Demonstrates basic logging using Python's standard logging library. This is the default behavior without django-structlog. ```python import logging logger = logging.get_logger(__name__) logger.info("An error occurred") ``` ```bash An error occurred ``` -------------------------------- ### Example Flat Lines Log Output Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst This is an example of log output formatted as flat lines, commonly used for simple log file analysis. ```bash timestamp='2019-04-13T19:39:29.321453Z' level='info' event='request_started' logger='django_structlog.middlewares.request' request_id='c53dff1d-3fc5-4257-a78a-9a567c937561' user_id=1 ip='0.0.0.0' request=GET / user_agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36' timestamp='2019-04-13T19:39:29.345207Z' level='info' event='request_finished' logger='django_structlog.middlewares.request' request_id='c53dff1d-3fc5-4257-a78a-9a567c937561' user_id=1 ip='0.0.0.0' code=200 timestamp='2019-04-13T19:39:31.086155Z' level='info' event='request_started' logger='django_structlog.middlewares.request' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' request=POST /success_task user_agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36' timestamp='2019-04-13T19:39:31.089925Z' level='info' event='Enqueuing successful task' logger='django_structlog_demo_project.home.views' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' timestamp='2019-04-13T19:39:31.147590Z' level='info' event='task_enqueued' logger='django_structlog.middlewares.celery' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' child_task_id='6b11fd80-3cdf-4de5-acc2-3fd4633aa654' timestamp='2019-04-13T19:39:31.153081Z' level='info' event='This is a successful task' logger='django_structlog_demo_project.taskapp.celery' task_id='6b11fd80-3cdf-4de5-acc2-3fd4633aa654' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' result='None' timestamp='2019-04-13T19:39:31.160043Z' level='info' event='request_finished' logger='django_structlog.middlewares.request' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' code=201 timestamp='2019-04-13T19:39:31.162372Z' level='info' event='task_succeed' logger='django_structlog.middlewares.celery' task_id='6b11fd80-3cdf-4de5-acc2-3fd4633aa654' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' result='None' ``` -------------------------------- ### Install django-structlog with Celery Extra Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/upgrade_guide.md Install django-structlog with the 'celery' extra to ensure compatibility with your Celery version. ```bash django-structlog[celery]==4.0.0 ``` -------------------------------- ### Flat Lines Log Output Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/example_outputs.md This example shows the default flat-line log format. It is useful for quick human inspection of logs. ```bash timestamp='2019-04-13T19:39:29.321453Z' level='info' event='request_started' logger='django_structlog.middlewares.request' request_id='c53dff1d-3fc5-4257-a78a-9a567c937561' user_id=1 ip='0.0.0.0' request=GET / user_agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36' timestamp='2019-04-13T19:39:29.345207Z' level='info' event='request_finished' logger='django_structlog.middlewares.request' request_id='c53dff1d-3fc5-4257-a78a-9a567c937561' user_id=1 ip='0.0.0.0' code=200 timestamp='2019-04-13T19:39:31.086155Z' level='info' event='request_started' logger='django_structlog.middlewares.request' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' request=POST /success_task user_agent='Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36' timestamp='2019-04-13T19:39:31.089925Z' level='info' event='Enqueuing successful task' logger='django_structlog_demo_project.home.views' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' timestamp='2019-04-13T19:39:31.147590Z' level='info' event='task_enqueued' logger='django_structlog.middlewares.celery' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' child_task_id='6b11fd80-3cdf-4de5-acc2-3fd4633aa654' timestamp='2019-04-13T19:39:31.153081Z' level='info' event='This is a successful task' logger='django_structlog_demo_project.taskapp.celery' task_id='6b11fd80-3cdf-4de5-acc2-3fd4633aa654' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' timestamp='2019-04-13T19:39:31.160043Z' level='info' event='request_finished' logger='django_structlog.middlewares.request' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' code=201 timestamp='2019-04-13T19:39:31.162372Z' level='info' event='task_succeed' logger='django_structlog.middlewares.celery' task_id='6b11fd80-3cdf-4de5-acc2-3fd4633aa654' request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' user_id=1 ip='0.0.0.0' result='None' ``` -------------------------------- ### Standard Django Logging Example Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/index.md Demonstrates basic logging using Python's standard logging library within a Django context. This is shown for comparison with structured logging. ```python import logging logger = logging.get_logger(__name__) logger.info("An error occurred") ``` -------------------------------- ### Example Log Output for Nested Commands Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/commands.md This output demonstrates logging for nested commands, showing how parent_command_id is added to track command hierarchy. ```bash $ python manage.py example_command bar 2023-09-15T00:10:10.466616Z [info ] command_started [django_structlog.commands] command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed command_name=django_structlog_demo_project.users.example_command 2023-09-15T00:10:10.467250Z [info ] my log [django_structlog_demo_project.users.management.commands.example_command] command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed foo=bar 2023-09-15T00:10:10.468176Z [info ] command_started [django_structlog.commands] baz=2 command_id=57524ccb-a8eb-4d30-a989-4e83ffdca9c0 command_name=django_structlog_demo_project.users.example_nested_command parent_command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed 2023-09-15T00:10:10.468871Z [info ] my nested log [django_structlog_demo_project.users.management.commands.example_nested_command] command_id=57524ccb-a8eb-4d30-a989-4e83ffdca9c0 parent_command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed 2023-09-15T00:10:10.469418Z [info ] command_finished [django_structlog.commands] command_id=57524ccb-a8eb-4d30-a989-4e83ffdca9c0 parent_command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed 2023-09-15T00:10:10.469964Z [info ] my log 2 [django_structlog_demo_project.users.management.commands.example_command] command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed 2023-09-15T00:10:10.470585Z [info ] command_finished [django_structlog.commands] command_id=f2a8c9a8-5aa3-4e22-b11c-f387449a34ed ``` -------------------------------- ### JSON Log Output Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/example_outputs.md This example demonstrates the JSON log format, which is ideal for structured logging and integration with log analysis tools. ```json {"request_id": "c53dff1d-3fc5-4257-a78a-9a567c937561", "user_id": 1, "ip": "0.0.0.0", "request": "GET /", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36", "event": "request_started", "timestamp": "2019-04-13T19:39:29.321453Z", "logger": "django_structlog.middlewares.request", "level": "info"} {"request_id": "c53dff1d-3fc5-4257-a78a-9a567c937561", "user_id": 1, "ip": "0.0.0.0", "code": 200, "event": "request_finished", "timestamp": "2019-04-13T19:39:29.345207Z", "logger": "django_structlog.middlewares.request", "level": "info"} {"request_id": "3a8f801c-072b-4805-8f38-e1337f363ed4", "user_id": 1, "ip": "0.0.0.0", "request": "POST /success_task", "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36", "event": "request_started", "timestamp": "2019-04-13T19:39:31.086155Z", "logger": "django_structlog.middlewares.request", "level": "info"} {"request_id": "3a8f801c-072b-4805-8f38-e1337f363ed4", "user_id": 1, "ip": "0.0.0.0", "event": "Enqueuing successful task", "timestamp": "2019-04-13T19:39:31.089925Z", "logger": "django_structlog_demo_project.home.views", "level": "info"} {"request_id": "3a8f801c-072b-4805-8f38-e1337f363ed4", "user_id": 1, "ip": "0.0.0.0", "child_task_id": "6b11fd80-3cdf-4de5-acc2-3fd4633aa654", "event": "task_enqueued", "timestamp": "2019-04-13T19:39:31.147590Z", "logger": "django_structlog.middlewares.celery", "level": "info"} {"task_id": "6b11fd80-3cdf-4de5-acc2-3fd4633aa654", "request_id": "3a8f801c-072b-4805-8f38-e1337f363ed4", "user_id": 1, "ip": "0.0.0.0", "event": "This is a successful task", "timestamp": "2019-04-13T19:39:31.153081Z", "logger": "django_structlog_demo_project.taskapp.celery", "level": "info"} {"request_id": "3a8f801c-072b-4805-8f38-e1337f363ed4", "user_id": 1, "ip": "0.0.0.0", "code": 201, "event": "request_finished", "timestamp": "2019-04-13T19:39:31.160043Z", "logger": "django_structlog.middlewares.request", "level": "info"} {"task_id": "6b11fd80-3cdf-4de5-acc2-3fd4633aa654", "request_id": "3a8f801c-072b-4805-8f38-e1337f363ed4", "user_id": 1, "ip": "0.0.0.0", "result": "None", "event": "task_succeed", "timestamp": "2019-04-13T19:39:31.162372Z", "logger": "django_structlog.middlewares.celery", "level": "info"} ``` -------------------------------- ### Searching Logs with flat_line Format Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/index.md Provides an example command for searching logs formatted with flat_line, specifically filtering by a request ID. ```bash cat logs/flat_line.log | grep request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' ``` -------------------------------- ### Configure 4XX Status Log Level Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/configuration.md Set the logging level for 4XX HTTP status codes. This example sets it to INFO. ```python import logging DJANGO_STRUCTLOG_STATUS_4XX_LOG_LEVEL = logging.INFO ``` -------------------------------- ### Get a structlog logger Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/getting_started.md Obtain a logger instance using structlog.get_logger() for use in your application code. ```python import structlog logger = structlog.get_logger(__name__) ``` -------------------------------- ### Configure Celery Logger with Django Structlog Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/celery.md Configure Celery's logging using dictConfig to define formatters (JSON, console, key-value) and handlers for different log outputs. This setup is applied via the setup_logging signal. ```python @setup_logging.connect def receiver_setup_logging(loglevel, logfile, format, colorize, **kwargs): # pragma: no cover logging.config.dictConfig( { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.JSONRenderer(), }, "plain_console": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.dev.ConsoleRenderer(), }, "key_value": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.KeyValueRenderer(key_order=['timestamp', 'level', 'event', 'logger']), }, }, "handlers": { "console": { "class": "logging.StreamHandler", "formatter": "plain_console", }, "json_file": { "class": "logging.handlers.WatchedFileHandler", "filename": "logs/json.log", "formatter": "json_formatter", }, "flat_line_file": { "class": "logging.handlers.WatchedFileHandler", "filename": "logs/flat_line.log", "formatter": "key_value", }, }, "loggers": { "django_structlog": { "handlers": ["console", "flat_line_file", "json_file"], "level": "INFO", }, "django_structlog_demo_project": { "handlers": ["console", "flat_line_file", "json_file"], "level": "INFO", }, } } ) structlog.configure( processors=[ structlog.contextvars.merge_contextvars, structlog.stdlib.filter_by_level, structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), structlog.processors.StackInfoRenderer(), structlog.processors.format_exc_info, structlog.processors.UnicodeDecoder(), structlog.stdlib.ProcessorFormatter.wrap_for_formatter, ], logger_factory=structlog.stdlib.LoggerFactory(), cache_logger_on_first_use=True, ) ``` -------------------------------- ### Modify Task Context Before Publish Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/celery.md Connect to the `modify_context_before_task_publish` signal to alter task metadata before it's stored. This example strips down the context to keep only specific keys. ```python from django_structlog import signals from django.dispatch import receiver @receiver(signals.modify_context_before_task_publish) def receiver_modify_context_before_task_publish(sender, signal, context, task_routing_key=None, task_properties=None, **kwargs): keys_to_keep = {"request_id", "parent_task_id"} new_dict = {key_to_keep: context[key_to_keep] for key_to_keep in keys_to_keep if key_to_keep in context} context.clear() context.update(new_dict) ``` -------------------------------- ### Custom Log Filtering with Django Logging Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/how_tos.md Implement a custom Django logging filter to exclude specific log events from being recorded. This example defines an 'ExcludeEventsFilter' and configures it within the Django settings. ```python # your_project/logging/filters.py import logging class ExcludeEventsFilter(logging.Filter): def __init__(self, excluded_event_type=None): super().__init__() self.excluded_event_type = excluded_event_type def filter(self, record): if not isinstance(record.msg, dict) or self.excluded_event_type is None: return True # Include the log message if msg is not a dictionary or excluded_event_type is not provided if record.msg.get('event') in self.excluded_event_type: return False # Exclude the log message return True # Include the log message # in your settings.py LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'handlers': { 'console': { 'class': 'logging.StreamHandler', 'filters': ['exclude_request_started'] }, }, 'filters': { 'exclude_request_started': { '()': 'your_project.logging.filters.ExcludeEventsFilter', 'excluded_event_type': ['request_started'] # Example excluding request_started event }, }, 'loggers': { 'django': { 'handlers': ['console'], 'level': 'DEBUG', }, }, } ``` -------------------------------- ### Celery Worker with Task Protocol v1 Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/celery.md When using Celery's task_protocol v1, django-structlog may not be able to transfer metadata to child tasks. This example shows how to configure it. ```python app = Celery("your_celery_project", task_protocol=1) ``` -------------------------------- ### Build and Serve Documentation Locally Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/development.md Build and serve the project's documentation locally using a dedicated Docker Compose file. The documentation will be available at http://127.0.0.1:8080. ```bash docker compose -p django-structlog-docs -f docker-compose.docs.yml up --build ``` -------------------------------- ### Configure structlog and Django Logging Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/getting_started.md Set up structlog processors and Django's logging handlers and formatters for JSON, console, and key-value output. ```python import structlog LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.JSONRenderer(), }, "plain_console": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.dev.ConsoleRenderer(), }, "key_value": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.KeyValueRenderer(key_order=['timestamp', 'level', 'event', 'logger']), }, }, "handlers": { # Important notes regarding handlers. # # 1. Make sure you use handlers adapted for your project. # These handlers configurations are only examples for this library. # See python's logging.handlers: https://docs.python.org/3/library/logging.handlers.html # # 2. You might also want to use different logging configurations depending of the environment. # Different files (local.py, tests.py, production.py, ci.py, etc.) or only conditions. # See https://docs.djangoproject.com/en/dev/topics/settings/#designating-the-settings # "console": { "class": "logging.StreamHandler", "formatter": "plain_console", }, "json_file": { "class": "logging.handlers.WatchedFileHandler", "filename": "logs/json.log", "formatter": "json_formatter", }, "flat_line_file": { "class": "logging.handlers.WatchedFileHandler", "filename": "logs/flat_line.log", "formatter": "key_value", }, }, "loggers": { "django_structlog": { "handlers": ["console", "flat_line_file", "json_file"], "level": "INFO", }, # Make sure to replace the following logger's name for yours "django_structlog_demo_project": { "handlers": ["console", "flat_line_file", "json_file"], "level": "INFO", }, } } structlog.configure( processors=[ structlog.contextvars.merge_contextvars, structlog.stdlib.filter_by_level, structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), structlog.processors.StackInfoRenderer(), structlog.processors.format_exc_info, structlog.processors.UnicodeDecoder(), structlog.stdlib.ProcessorFormatter.wrap_for_formatter, ], logger_factory=structlog.stdlib.LoggerFactory(), cache_logger_on_first_use=True, ) ``` -------------------------------- ### django-structlog with flat_line Output Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/index.md Shows how to use django-structlog with the flat_line formatter for structured logging. This format is useful for easy searching in log files. ```python import structlog logger = structlog.get_logger(__name__) logger.info("an_error_occurred", bar="Buz") ``` -------------------------------- ### Configure Structlog Processors Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Set up structlog processors in Django's settings.py for custom logging behavior, including timestamping, logger names, and level addition. ```python import structlog LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.JSONRenderer(), }, "plain_console": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.dev.ConsoleRenderer(), }, "key_value": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.KeyValueRenderer(key_order=['timestamp', 'level', 'event', 'logger']), }, }, "handlers": { "console": { "class": "logging.StreamHandler", "formatter": "plain_console", }, "json_file": { "class": "logging.handlers.WatchedFileHandler", "filename": "logs/json.log", "formatter": "json_formatter", }, "flat_line_file": { "class": "logging.handlers.WatchedFileHandler", "filename": "logs/flat_line.log", "formatter": "key_value", }, }, "loggers": { "django_structlog": { "handlers": ["console", "flat_line_file", "json_file"], "level": "INFO", }, "django_structlog_demo_project": { "handlers": ["console", "flat_line_file", "json_file"], "level": "INFO", }, } } structlog.configure( processors=[ structlog.contextvars.merge_contextvars, structlog.stdlib.filter_by_level, structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), structlog.processors.StackInfoRenderer(), structlog.processors.format_exc_info, structlog.processors.UnicodeDecoder(), structlog.stdlib.ProcessorFormatter.wrap_for_formatter, ], logger_factory=structlog.stdlib.LoggerFactory(), cache_logger_on_first_use=True, ) ``` -------------------------------- ### Configure RequestMiddleware Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/upgrade_guide.md For upgrades to 6.0+, replace any experimental or older request middleware with `django_structlog.middlewares.RequestMiddleware` in your Django `MIDDLEWARE` setting. ```python MIDDLEWARE += [ # "django_structlog.middlewares.request_middleware_router", # <- remove # "django_structlog.middlewares.requests.SyncRequestMiddleware", # <- remove # "django_structlog.middlewares.requests.AsyncRequestMiddleware", # <- remove "django_structlog.middlewares.RequestMiddleware", # <- make sure you use this one ] ``` -------------------------------- ### Add django-structlog to INSTALLED_APPS Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/getting_started.md Include 'django_structlog' in your Django project's INSTALLED_APPS setting. ```python INSTALLED_APP = [ # ... "django_structlog", # ... ] ``` -------------------------------- ### Replace logger.bind with bind_contextvars Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/upgrade_guide.md Update your code to use 'structlog.contextvars.bind_contextvars' instead of 'logger.bind' for context management. ```python @receiver(bind_extra_request_metadata) def bind_domain(request, logger, **kwargs): current_site = get_current_site(request) # logger.bind(domain=current_site.domain) structlog.contextvars.bind_contextvars(domain=current_site.domain) ``` -------------------------------- ### Adjust Structlog Processors for Exception Handling Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/upgrade_guide.md For upgrades to 2.0+, remove 'structlog.processors.ExceptionPrettyPrinter' and ensure 'structlog.processors.format_exc_info' is included for correct exception logging. ```python # remove structlog.processors.ExceptionPrettyPrinter(), of your processors. # make sure you have structlog.processors.format_exc_info, in your processors if you want appropriate exception logging. ``` -------------------------------- ### Bind extra request metadata using signals Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/getting_started.md Implement a signal receiver to bind additional metadata, such as the domain, to log entries based on the incoming request. ```python from django.contrib.sites.shortcuts import get_current_site from django.dispatch import receiver from django_structlog import signals import structlog @receiver(signals.bind_extra_request_metadata) def bind_domain(request, logger, **kwargs): current_site = get_current_site(request) structlog.contextvars.bind_contextvars(domain=current_site.domain) ``` -------------------------------- ### Search Flat Log Entries Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Use grep to search for specific log entries in a flat-line formatted log file using a request ID. ```bash $ cat logs/flat_line.log | grep request_id='3a8f801c-072b-4805-8f38-e1337f363ed4' ``` -------------------------------- ### Configure Standard Python Loggers with Structlog Processors Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Configure Django's LOGGING settings to use structlog processors for formatting logs. Ensure to add 'structlog.contextvars.merge_contextvars' to the 'foreign_pre_chain' in your formatter. ```python LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.JSONRenderer(), # Add this section: "foreign_pre_chain": [ structlog.contextvars.merge_contextvars, # <---- add this # customize the rest as you need structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), ], }, }, ... } ``` -------------------------------- ### Initialize Celery Worker with DjangoStructLogInitStep Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/celery.md In your Celery AppConfig, add DjangoStructLogInitStep to the worker's steps to initialize django-structlog for Celery. ```python import logging import structlog from celery import Celery from celery.signals import setup_logging from django_structlog.celery.steps import DjangoStructLogInitStep app = Celery("your_celery_project") # A step to initialize django-structlog app.steps['worker'].add(DjangoStructLogInitStep) ``` -------------------------------- ### Update Structlog Configuration for Contextvars Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Configure structlog to use contextvars by adding 'structlog.contextvars.merge_contextvars' as the first processor and removing 'context_class=structlog.threadlocal.wrap_dict(dict)'. ```python structlog.configure( processors=[ structlog.contextvars.merge_contextvars, # <---- add this structlog.stdlib.filter_by_level, structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), structlog.processors.StackInfoRenderer(), structlog.processors.format_exc_info, structlog.processors.UnicodeDecoder(), structlog.stdlib.ProcessorFormatter.wrap_for_formatter, ], # context_class=structlog.threadlocal.wrap_dict(dict), # <---- remove this logger_factory=structlog.stdlib.LoggerFactory(), cache_logger_on_first_use=True, ) # If you use standard logging LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.JSONRenderer(), "foreign_pre_chain": [ structlog.contextvars.merge_contextvars, # <---- add this # django_structlog.processors.inject_context_dict, # <---- remove this structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, ``` -------------------------------- ### Replace logger.bind with structlog.contextvars.bind_contextvars Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Use structlog.contextvars.bind_contextvars for binding context variables to the logger. This is a direct replacement for the older logger.bind method. ```python from django.dispatch import receiver from django.contrib.sites.shortcuts import get_current_site from structlog.stdlib import ProcessorFormatter from structlog.typing import Processor from .signals import bind_extra_request_metadata @receiver(bind_extra_request_metadata) def bind_domain(request, logger, **kwargs): current_site = get_current_site(request) # logger.bind(domain=current_site.domain) structlog.contextvars.bind_contextvars(domain=current_site.domain) ``` -------------------------------- ### Update Structlog Settings for Contextvars Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/upgrade_guide.md Modify structlog configuration to use context variables instead of thread-locals. Ensure 'structlog.contextvars.merge_contextvars' is the first processor and remove 'context_class'. ```python structlog.configure( processors=[ structlog.contextvars.merge_contextvars, # <---- add this structlog.stdlib.filter_by_level, structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), structlog.processors.StackInfoRenderer(), structlog.processors.format_exc_info, structlog.processors.UnicodeDecoder(), structlog.stdlib.ProcessorFormatter.wrap_for_formatter, ], # context_class=structlog.threadlocal.wrap_dict(dict), # <---- remove this logger_factory=structlog.stdlib.LoggerFactory(), cache_logger_on_first_use=True, ) ``` ```python # If you use standard logging LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.JSONRenderer(), "foreign_pre_chain": [ structlog.contextvars.merge_contextvars, # <---- add this # django_structlog.processors.inject_context_dict, # <---- remove this structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), ], }, }, ... } ``` -------------------------------- ### Enable Command Logging Configuration Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/commands.md Enable django-structlog's command logging feature by setting this boolean flag to True in your Django settings. ```python DJANGO_STRUCTLOG_COMMAND_LOGGING_ENABLED = True ``` -------------------------------- ### Enable Celery Integration in Django Settings Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/celery.md Add the RequestMiddleware to your Django settings and set DJANGO_STRUCTLOG_CELERY_ENABLED to True to enable Celery integration. ```python MIDDLEWARE = [ # ... 'django_structlog.middlewares.RequestMiddleware', ] DJANGO_STRUCTLOG_CELERY_ENABLED = True ``` -------------------------------- ### Search JSON Log Entries with jq Source: https://github.com/jrobichaud/django-structlog/blob/main/README.rst Use jq to filter and select specific log entries from a JSON formatted log file based on request ID. ```bash $ cat logs/json.log | jq '.[] | select(.request_id="3a8f801c-072b-4805-8f38-e1337f363ed4")' -s ``` -------------------------------- ### Configure foreign_pre_chain for standard loggers Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/getting_started.md Add the 'foreign_pre_chain' section to your LOGGING formatters to include structlog context variables when using standard Python loggers. ```python LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { "json_formatter": { "()": structlog.stdlib.ProcessorFormatter, "processor": structlog.processors.JSONRenderer(), # Add this section: "foreign_pre_chain": [ structlog.contextvars.merge_contextvars, # <---- add this # customize the rest as you need structlog.processors.TimeStamper(fmt="iso"), structlog.stdlib.add_logger_name, structlog.stdlib.add_log_level, structlog.stdlib.PositionalArgumentsFormatter(), ], }, }, ... } ``` -------------------------------- ### Enable Celery Integration via Settings Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/upgrade_guide.md If using Celery with django-structlog and upgrading to 6.0+, remove `CeleryMiddleware` from `MIDDLEWARE` and set `DJANGO_STRUCTLOG_CELERY_ENABLED = True`. ```python MIDDLEWARE += [ "django_structlog.middlewares.RequestMiddleware", # "django_structlog.middlewares.CeleryMiddleware", # <- remove this ] DJANGO_STRUCTLOG_CELERY_ENABLED = True # <-- add this ``` -------------------------------- ### Update requirements.txt for Celery Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/celery.md Replace the standard django-structlog package with the Celery-enabled version in your requirements.txt file. ```python django-structlog[celery]==X.Y.Z ``` -------------------------------- ### Bind AWS X-Amzn-Trace-Id Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/how_tos.md Bind the AWS 'X-Amzn-Trace-Id' header to the request context for tracing requests originating from an Application Load Balancer. It utilizes a helper function to retrieve request headers. ```python from django.dispatch import receiver from django_structlog import signals from django_structlog.middlewares.request import get_request_header import structlog @receiver(signals.bind_extra_request_metadata) def bind_trace_id(request, logger, **kwargs): trace_id = get_request_header( request, "x-amzn-trace-id", "HTTP_X_AMZN_TRACE_ID" ) if trace_id: structlog.contextvars.bind_contextvars(trace_id=trace_id) ``` -------------------------------- ### Integrate SignalCommand with Django Command Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/commands.md Add the @signalcommand decorator from django_extensions.management.utils to your Django management command class to enable signal handling for logging. ```python import structlog from django.core.management import BaseCommand from django_extensions.management.utils import signalcommand # <- add this logger = structlog.getLogger(__name__) class Command(BaseCommand): def add_arguments(self, parser): parser.add_argument("foo", type=str) @signalcommand # <- add this def handle(self, foo, *args, **options): logger.info("my log", foo=foo) return 0 ``` -------------------------------- ### Update Signal Receiver for log_kwargs Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/upgrade_guide.md When upgrading to 8.0+, update signal receivers to accept the new `log_kwargs` argument if you use optional signals like `bind_extra_request_metadata`. ```python from django.contrib.sites.shortcuts import get_current_site from django.dispatch import receiver from django_structlog import signals import structlog @receiver(signals.bind_extra_request_metadata) def my_receiver(request, logger, log_kwargs, **kwargs): # <- add `log_kwargs` if necessary ... @receiver(signals.bind_extra_request_finished_metadata) def my_receiver_finished(request, logger, response, log_kwargs, **kwargs): # <- add `log_kwargs` if necessary ... @receiver(signals.bind_extra_request_failed_metadata) def my_receiver_failed(request, logger, exception, log_kwargs, **kwargs): # <- add `log_kwargs` if necessary ... ``` -------------------------------- ### Add RequestMiddleware to MIDDLEWARE Source: https://github.com/jrobichaud/django-structlog/blob/main/docs/getting_started.md Incorporate the RequestMiddleware into your Django project's MIDDLEWARE setting to process incoming requests. ```python MIDDLEWARE = [ # ... "django_structlog.middlewares.RequestMiddleware", ] ```