-
Notifications
You must be signed in to change notification settings - Fork 433
fix(logger): correctly pick powertools or custom handler in custom environments #4295
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
leandrodamascena
merged 20 commits into
aws-powertools:develop
from
heitorlessa:fix/logger-handler
May 13, 2024
Merged
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
2f16889
fix(parameters): make cache aware of single vs multiple calls
heitorlessa 42ae968
chore: cleanup, add test for single and nested
heitorlessa cd2671c
chore(ci): add first centralized reusable workflow
heitorlessa 56b6c2b
fix: initial work on handler fix
heitorlessa 3602a80
fix: make it backwards compatible for children behaviour
heitorlessa e9d627b
chore: assert handlers; assert defaults create and use PT handler
heitorlessa d8ecb49
Delete bla.py
heitorlessa f6d252d
chore: ignore bla
heitorlessa f271b7a
Delete playground/app.mjs
hei
8000
torlessa f7c4387
Delete playground/.prettierrc
heitorlessa bb64924
chore: improve orphaned child side effect with an explicit error
heitorlessa 284c4e8
fix: orphan exception must not be shadowed by attr exc
heitorlessa f93fd77
chore: mypy
heitorlessa 40f136a
docs(logger): clarify child loggers side effects; cleanup over-used b…
heitorlessa afdfae3
Merge branch 'develop' into fix/logger-handler
heitorlessa ae1a112
chore: ignore type checking test coverage
heitorlessa 2dda52c
Merge branch 'develop' into fix/logger-handler
leandrodamascena e829300
Fixing small things
leandrodamascena 26ac439
Update constants.py
heitorlessa a91d8c3
Update constants.py
heitorlessa File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
# logger.powertools_handler is set with Powertools Logger handler; useful when there are many handlers | ||
LOGGER_ATTRIBUTE_POWERTOOLS_HANDLER = "powertools_handler" | ||
# logger.init attribute is set when Logger has been configured | ||
LOGGER_ATTRIBUTE_PRECONFIGURED = "init" | ||
heitorlessa marked this conversation as resolved.
Show resolved
Hide resolved
|
||
LOGGER_ATTRIBUTE_HANDLER = "logger_handler" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,6 @@ | ||
class InvalidLoggerSamplingRateError(Exception): | ||
pass | ||
|
||
|
||
class OrphanedChildLoggerError(Exception): | ||
pass |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -19,9 +19,15 @@ | |
Optional, | ||
TypeVar, | ||
Union, | ||
cast, | ||
overload, | ||
) | ||
|
||
from aws_lambda_powertools.logging.constants import ( | ||
LOGGER_ATTRIBUTE_HANDLER, | ||
LOGGER_ATTRIBUTE_POWERTOOLS_HANDLER, | ||
LOGGER_ATTRIBUTE_PRECONFIGURED, | ||
) | ||
from aws_lambda_powertools.shared import constants | ||
from aws_lambda_powertools.shared.functions import ( | ||
extract_event_from_common_models, | ||
|
@@ -31,7 +37,7 @@ | |
from aws_lambda_powertools.utilities import jmespath_utils | ||
|
||
from ..shared.types import AnyCallableT | ||
from .exceptions import InvalidLoggerSamplingRateError | ||
from .exceptions import InvalidLoggerSamplingRateError, OrphanedChildLoggerError | ||
from .filters import SuppressFilter | ||
from .formatter import ( | ||
RESERVED_FORMATTER_CUSTOM_KEYS, | ||
|
@@ -233,14 +239,14 @@ def __init__( | |
self.child = child | ||
self.logger_formatter = logger_formatter | ||
self._stream = stream or sys.stdout | ||
self.logger_handler = logger_handler or logging.StreamHandler(self._stream) | ||
self.log_uncaught_exceptions = log_uncaught_exceptions | ||
|
||
self._is_deduplication_disabled = resolve_truthy_env_var_choice( | ||
env=os.getenv(constants.LOGGER_LOG_DEDUPLICATION_ENV, "false"), | ||
) | ||
self._default_log_keys = {"service": self.service, "sampling_rate": self.sampling_rate} | ||
self._logger = self._get_logger() | ||
self.logger_handler = logger_handler or self._get_handler() | ||
|
||
# NOTE: This is primarily to improve UX, so IDEs can autocomplete LambdaPowertoolsFormatter options | ||
# previously, we masked all of them as kwargs thus limiting feature discovery | ||
|
@@ -264,7 +270,7 @@ def __init__( | |
|
||
# Prevent __getattr__ from shielding unknown attribute errors in type checkers | ||
# https://github.com/aws-powertools/powertools-lambda-python/issues/1660 | ||
if not TYPE_CHECKING: | ||
if not TYPE_CHECKING: # pragma: no cover | ||
|
||
def __getattr__(self, name): | ||
# Proxy attributes not found to actual logger to support backward compatibility | ||
|
@@ -279,6 +285,18 @@ def _get_logger(self) -> logging.Logger: | |
|
||
return logging.getLogger(logger_name) | ||
|
||
def _get_handler(self) -> logging.Handler: | ||
# is a logger handler already configured? | ||
if getattr(self, LOGGER_ATTRIBUTE_HANDLER, None): | ||
return self.logger_handler | ||
|
||
# for children, use parent's handler | ||
if self.child: | ||
return getattr(self._logger.parent, LOGGER_ATTRIBUTE_POWERTOOLS_HANDLER, None) # type: ignore[return-value] # always checked in formatting | ||
|
||
# otherwise, create a new stream handler (first time init) | ||
return logging.StreamHandler(self._stream) | ||
|
||
def _init_logger( | ||
self, | ||
formatter_options: Optional[Dict] = None, | ||
|
@@ -292,7 +310,7 @@ def _init_logger( | |
# a) multiple handlers being attached | ||
# b) different sampling mechanisms | ||
# c) multiple messages from being logged as handlers can be duplicated | ||
is_logger_preconfigured = getattr(self._logger, "init", False) | ||
is_logger_preconfigured = getattr(self._logger, LOGGER_ATTRIBUTE_PRECONFIGURED, False) | ||
if self.child or is_logger_preconfigured: | ||
return | ||
|
||
|
@@ -317,6 +335,7 @@ def _init_logger( | |
# std logging will return the same Logger with our attribute if name is reused | ||
logger.debug(f"Marking logger {self.service} as preconfigured") | ||
self._logger.init = True # type: ignore[attr-defined] | ||
self._logger.powertools_handler = self.logger_handler # type: ignore[attr-defined] | ||
|
||
def _configure_sampling(self) -> None: | ||
"""Dynamically set log level based on sampling rate | ||
|
@@ -613,7 +632,7 @@ def structure_logs(self, append: bool = False, formatter_options: Optional[Dict] | |
|
||
# Mode 1 | ||
log_keys = {**self._default_log_keys, **keys} | ||
is_logger_preconfigured = getattr(self._logger, "init", False) | ||
is_logger_preconfigured = getattr(self._logger, LOGGER_ATTRIBUTE_PRECONFIGURED, False) | ||
if not is_logger_preconfigured: | ||
formatter = self.logger_formatter or LambdaPowertoolsFormatter(**formatter_options, **log_keys) | ||
self.registered_handler.setFormatter(formatter) | ||
9E88
|
@@ -672,15 +691,20 @@ def removeFilter(self, filter: logging._FilterType) -> None: # noqa: A002 # fil | |
@property | ||
def registered_handler(self) -> logging.Handler: | ||
"""Convenience property to access the first logger handler""" | ||
# We ignore mypy here because self.child encodes whether or not self._logger.parent is | ||
# None, mypy can't see this from context but we can | ||
handlers = self._logger.parent.handlers if self.child else self._logger.handlers # type: ignore[union-attr] | ||
return handlers[0] | ||
return self._get_handler() | ||
|
||
@property | ||
def registered_formatter(self) -> BasePowertoolsFormatter: | ||
"""Convenience property to access the first logger formatter""" | ||
return self.registered_handler.formatter # type: ignore[return-value] | ||
handler = self.registered_handler | ||
if handler is None: | ||
raise OrphanedChildLoggerError( | ||
"Orphan child loggers cannot append nor remove keys until a parent is initialized first. " | ||
"To solve this issue, you can A) make sure a parent logger is initialized first, or B) move append/remove keys operations to a later stage." # noqa: E501 | ||
"Reference: https://docs.powertools.aws.dev/lambda/python/latest/core/logger/#reusing-logger-across-your-code", | ||
) | ||
|
||
return cast(BasePowertoolsFormatter, handler.formatter) | ||
|
||
@property | ||
def log_level(self) -> int: | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.