Homepage
A suite of utilities for AWS Lambda functions to ease adopting best practices such as tracing, structured logging, custom metrics, idempotency, batching, and more.
Tip
Powertools is also available for Java, TypeScript, and .NET
Support this project by becoming a reference customer, sharing your work, or using Layers/SAR
You can choose to support us in three ways:
1) Become a reference customers. This gives us permission to list your company in our documentation.
2) Share your work. Blog posts, video, sample projects you used Powertools!
3) Use Lambda Layers or SAR, if possible. This helps us understand who uses Powertools in a non-intrusive way, and helps us gain future investments for other Powertools languages.
When using Layers, you can add Powertools as a dev dependency (or as part of your virtual env) to not impact the development process.
Install¶
Powertools is available in the following formats:
- Lambda Layer (x86_64): arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPythonV2:16
- Lambda Layer (arm64): arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPythonV2-Arm64:16
- PyPi:
pip install "aws-lambda-powertools"
Some utilities require additional dependencies
You can stop reading if you're using Lambda Layer.
Tracer, Validation and Parser require additional dependencies. If you prefer to install all of them, use pip install "aws-lambda-powertools[all]"
.
For example:
- Tracer:
pip install "aws-lambda-powertools[tracer]"
- Validation:
pip install "aws-lambda-powertools[validation]"
- Parser:
pip install "aws-lambda-powertools[parser]"
- Tracer and Parser:
pip install "aws-lambda-powertools[tracer,parser]"
Local development¶
Powertools relies on the AWS SDK bundled in the Lambda runtime. This helps us achieve an optimal package size and initialization.
This means you need to add AWS SDK as a development dependency (not as a production dependency).
- Pip:
pip install "aws-lambda-powertools[aws-sdk]"
- Poetry:
poetry add "aws-lambda-powertools[aws-sdk]" --dev
- Pipenv:
pipenv install --dev "aws-lambda-powertools[aws-sdk]"
Local emulation
If you're running your code locally with AWS SAM CLI, and not with your Python/IDE interpreter directly, this is not necessary. SAM CLI already brings the AWS SDK in its emulation image.
Lambda Layer¶
Lambda Layer is a .zip file archive that can contain additional code, pre-packaged dependencies, data, or configuration files. Layers promote code sharing and separation of responsibilities so that you can iterate faster on writing business logic.
You can include Powertools Lambda Layer using AWS Lambda Console, or your preferred deployment framework.
Note: Click to expand and copy any regional Lambda Layer ARN
Note: Click to expand and copy code snippets for popular frameworks
1 2 3 4 5 |
|
1 2 3 4 5 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
Change {region} to your AWS region, e.g. eu-west-1
AWS CLI | |
---|---|
1 |
|
The pre-signed URL to download this Lambda Layer will be within Location
key.
1 2 3 4 5 6 |
|
1 2 3 4 5 6 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
Change {region} to your AWS region, e.g. eu-west-1
AWS CLI | |
---|---|
1 |
|
The pre-signed URL to download this Lambda Layer will be within Location
key.
Warning: Limitations
Container Image deployment (OCI) or inline Lambda functions do not support Lambda Layers.
SAR¶
Serverless Application Repository (SAR) App deploys a CloudFormation stack with a copy of our Lambda Layer in your AWS account and region.
Compared with the public Layer ARN option, SAR allows you to choose a semantic version and deploys a Layer in your target account.
App | ARN | Description |
---|---|---|
aws-lambda-powertools-python-layer | arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer | Contains all extra dependencies (e.g: pydantic). |
aws-lambda-powertools-python-layer-arm64 | arn:aws:serverlessrepo:eu-west-1:057560766410:applications/aws-lambda-powertools-python-layer-arm64 | Contains all extra dependencies (e.g: pydantic). For arm64 functions. |
Click to expand and copy SAR code snippets for popular frameworks
You can create a shared Lambda Layers stack and make this along with other account level layers stack.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
|
Credits to Dani Comnea for providing the Terraform equivalent.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
|
Example: Least-privileged IAM permissions to deploy Layer
Credits to mwarkentin for providing the scoped down IAM permissions.
The region and the account id for CloudFormationTransform
and GetCfnTemplate
are fixed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 |
|
Click to expand and copy an AWS CLI command to list all versions available in SAR
You can fetch available versions via SAR ListApplicationVersions API:
AWS CLI example | |
---|---|
1 2 |
|
Quick getting started¶
Hello world example using SAM CLI | |
---|---|
1 |
|
Features¶
Core utilities such as Tracing, Logging, Metrics, and Event Handler will be available across all Powertools languages. Additional utilities are subjective to each language ecosystem and customer demand.
Utility | Description |
---|---|
Tracing | Decorators and utilities to trace Lambda function handlers, and both synchronous and asynchronous functions |
Logger | Structured logging made easier, and decorator to enrich structured logging with key Lambda context details |
Metrics | Custom Metrics created asynchronously via CloudWatch Embedded Metric Format (EMF) |
Event handler: AppSync | AppSync event handler for Lambda Direct Resolver and Amplify GraphQL Transformer function |
Event handler: API Gateway, ALB and Lambda Function URL | Amazon API Gateway REST/HTTP API and ALB event handler for Lambda functions invoked using Proxy integration, and Lambda Function URL |
Middleware factory | Decorator factory to create your own middleware to run logic before, and after each Lambda invocation |
Parameters | Retrieve parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, or Amazon DynamoDB, and cache them for a specific amount of time |
Batch processing | Handle partial failures for AWS SQS batch processing |
Typing | Static typing classes to speedup development in your IDE |
Validation | JSON Schema validator for inbound events and responses |
Event source data classes | Data classes describing the schema of common Lambda event triggers |
Parser | Data parsing and deep validation using Pydantic |
Idempotency | Idempotent Lambda handler |
Feature Flags | A simple rule engine to evaluate when one or multiple features should be enabled depending on the input |
Streaming | Streams datasets larger than the available memory as streaming data. |
Environment variables¶
Info
Explicit parameters take precedence over environment variables
Environment variable | Description | Utility | Default |
---|---|---|---|
POWERTOOLS_SERVICE_NAME | Sets service name used for tracing namespace, metrics dimension and structured logging | All | "service_undefined" |
POWERTOOLS_METRICS_NAMESPACE | Sets namespace used for metrics | Metrics | None |
POWERTOOLS_TRACE_DISABLED | Explicitly disables tracing | Tracing | false |
POWERTOOLS_TRACER_CAPTURE_RESPONSE | Captures Lambda or method return as metadata. | Tracing | true |
POWERTOOLS_TRACER_CAPTURE_ERROR | Captures Lambda or method exception as metadata. | Tracing | true |
POWERTOOLS_TRACE_MIDDLEWARES | Creates sub-segment for each custom middleware | Middleware factory | false |
POWERTOOLS_LOGGER_LOG_EVENT | Logs incoming event | Logging | false |
POWERTOOLS_LOGGER_SAMPLE_RATE | Debug log sampling | Logging | 0 |
POWERTOOLS_LOG_DEDUPLICATION_DISABLED | Disables log deduplication filter protection to use Pytest Live Log feature | Logging | false |
POWERTOOLS_DEV | Increases verbosity across utilities | Multiple; see POWERTOOLS_DEV effect below | false |
LOG_LEVEL | Sets logging level | Logging | INFO |
Optimizing for non-production environments¶
Whether you're prototyping locally or against a non-production environment, you can use POWERTOOLS_DEV
to increase verbosity across multiple utilities.
Info
We will emit a warning when POWERTOOLS_DEV
is enabled to help you detect misuse in production environments.
When POWERTOOLS_DEV
is set to a truthy value (1
, true
), it'll have the following effects:
Utility | Effect |
---|---|
Logger | Increase JSON indentation to 4. This will ease local debugging when running functions locally under emulators or direct calls while not affecting unit tests |
Event Handler | Enable full traceback errors in the response, indent request/responses, and CORS in dev mode (* ). |
Tracer | Future-proof safety to disables tracing operations in non-Lambda environments. This already happens automatically in the Tracer utility. |
Debug mode¶
As a best practice for libraries, Powertools module logging statements are suppressed.
When necessary, you can use POWERTOOLS_DEBUG
environment variable to enable debugging. This will provide additional information on every internal operation.
Tenets¶
These are our core principles to guide our decision making.
- AWS Lambda only. We optimise for AWS Lambda function environments and supported runtimes only. Utilities might work with web frameworks and non-Lambda environments, though they are not officially supported.
- Eases the adoption of best practices. The main priority of the utilities is to facilitate best practices adoption, as defined in the AWS Well-Architected Serverless Lens; all other functionality is optional.
- Keep it lean. Additional dependencies are carefully considered for security and ease of maintenance, and prevent negatively impacting startup time.
- We strive for backwards compatibility. New features and changes should keep backwards compatibility. If a breaking change cannot be avoided, the deprecation and migration process should be clearly defined.
- We work backwards from the community. We aim to strike a balance of what would work best for 80% of customers. Emerging practices are considered and discussed via Requests for Comment (RFCs)
- Progressive. Utilities are designed to be incrementally adoptable for customers at any stage of their Serverless journey. They follow language idioms and their communityβs common practices.