Feature request: Fix inconsistency between BatchProcessor success_handler and failure_handler call arguments #2816
Description
Use case
As mentioned in BatchProcessor's documentation (https://docs.powertools.aws.dev/lambda/python/latest/utilities/batch/#create-your-own-partial-processor), BatchProcessor can be extended by overriding success_handler and failure_handler methods:
class MyProcessor(BatchProcessor):
def failure_handler(self, record: SQSRecord, exception: ExceptionInfo) -> FailureResponse:
metrics.add_metric(name="BatchRecordFailures", unit=MetricUnit.Count, value=1)
return super().failure_handler(record, exception)
Reading this, I see and understand that the record is passed to the function as a Pydantic model. I would thus expect success_handler to take the same type of record as failure_handler. That is not currently the case, as record is passed as raw dict data in BatchProcessor's _process_record function:
powertools-lambda-python/aws_lambda_powertools/utilities/batch/base.py
Lines 485 to 502 in 021076e
In my opinion, record args should be homogeneous between the two functions, or otherwise devs wanting to override success_handler will encounter errors at runtime, have to dig in this tool's source code and understand the subtlety (like I did).
Solution/User Experience
First solution would be to add an example for success_handler in the documentation (https://docs.powertools.aws.dev/lambda/python/latest/utilities/batch/#extending-batchprocessor). This would notify developers of the differences between the functions' signature.
Second solution would be to pass the Pydantic model as record to success_handler. This should be a pretty straightforward change in the source code since this structure is already instantiated in _process_record. This would make the tool more homogeneous as well as improve experience in using the record's attributes in success_handler.
Alternative solutions
No response
Acknowledgment
- This feature request meets Powertools for AWS Lambda (Python) Tenets
- Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET