Logging
The logging utility provides a Lambda optimized logger with output structured as JSON.
Key features¶
- Capture key fields from Lambda context, cold start and structures logging output as JSON
- Log Lambda event when instructed (disabled by default)
- Log sampling enables DEBUG log level for a percentage of requests (disabled by default)
- Append additional keys to structured log at any point in time
- Ahead-of-Time compilation to native code support AOT from version 1.6.0
Installation¶
Powertools for AWS Lambda (.NET) are available as NuGet packages. You can install the packages from NuGet Gallery or from Visual Studio editor by searching AWS.Lambda.Powertools*
to see various utilities available.
-
AWS.Lambda.Powertools.Logging:
dotnet add package AWS.Lambda.Powertools.Logging
Getting started¶
Info
AOT Support If loooking for AOT specific configurations navigate to the AOT section
Logging requires two settings:
Setting | Description | Environment variable | Attribute parameter |
---|---|---|---|
Service | Sets Service key that will be present across all log statements | POWERTOOLS_SERVICE_NAME |
Service |
Logging level | Sets how verbose Logger should be (Information, by default) | POWERTOOLS_LOG_LEVEL |
LogLevel |
Example using AWS Serverless Application Model (AWS SAM)¶
You can override log level by setting POWERTOOLS_LOG_LEVEL
environment variable in the AWS SAM template.
You can also explicitly set a service name via POWERTOOLS_SERVICE_NAME
environment variable. This sets Service key that will be present across all log statements.
Here is an example using the AWS SAM Globals section.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
Full list of environment variables¶
Environment variable | Description | Default |
---|---|---|
POWERTOOLS_SERVICE_NAME | Sets service name used for tracing namespace, metrics dimension and structured logging | "service_undefined" |
POWERTOOLS_LOG_LEVEL | Sets logging level | Information |
POWERTOOLS_LOGGER_CASE | Override the default casing for log keys | SnakeCase |
POWERTOOLS_LOGGER_LOG_EVENT | Logs incoming event | false |
POWERTOOLS_LOGGER_SAMPLE_RATE | Debug log sampling | 0 |
Using AWS Lambda Advanced Logging Controls (ALC)¶
When is it useful?
When you want to set a logging policy to drop informational or verbose logs for one or all AWS Lambda functions, regardless of runtime and logger used.
With AWS Lambda Advanced Logging Controls (ALC), you can enforce a minimum log level that Lambda will accept from your application code.
When enabled, you should keep Logger
and ALC log level in sync to avoid data loss.
When using AWS Lambda Advanced Logging Controls (ALC)
- When Powertools Logger output is set to
PascalCase
Level
property name will be replaced byLogLevel
as a property name. - ALC takes precedence over
POWERTOOLS_LOG_LEVEL
and when setting it in code using[Logging(LogLevel = )]
Here's a sequence diagram to demonstrate how ALC will drop both Information
and Debug
logs emitted from Logger
, when ALC log level is stricter than Logger
.
sequenceDiagram
title Lambda ALC allows WARN logs only
participant Lambda service
participant Lambda function
participant Application Logger
Note over Lambda service: AWS_LAMBDA_LOG_LEVEL="WARN"
Note over Application Logger: POWERTOOLS_LOG_LEVEL="DEBUG"
Lambda service->>Lambda function: Invoke (event)
Lambda function->>Lambda function: Calls handler
Lambda function->>Application Logger: Logger.Warning("Something happened")
Lambda function-->>Application Logger: Logger.Debug("Something happened")
Lambda function-->>Application Logger: Logger.Information("Something happened")
Lambda service->>Lambda service: DROP INFO and DEBUG logs
Lambda service->>CloudWatch Logs: Ingest error logs
Priority of log level settings in Powertools for AWS Lambda
We prioritise log level settings in this order:
- AWS_LAMBDA_LOG_LEVEL environment variable
- Setting the log level in code using
[Logging(LogLevel = )]
- POWERTOOLS_LOG_LEVEL environment variable
If you set Logger
level lower than ALC, we will emit a warning informing you that your messages will be discarded by Lambda.
NOTE With ALC enabled, we are unable to increase the minimum log level below the
AWS_LAMBDA_LOG_LEVEL
environment variable value, see AWS Lambda service documentation for more details.
Standard structured keys¶
Your logs will always include the following keys to your structured logging:
Key | Type | Example | Description |
---|---|---|---|
Timestamp | string | "2020-05-24 18:17:33,774" | Timestamp of actual log statement |
Level | string | "Information" | Logging level |
Name | string | "Powertools for AWS Lambda (.NET) Logger" | Logger name |
ColdStart | bool | true | ColdStart value. |
Service | string | "payment" | Service name defined. "service_undefined" will be used if unknown |
SamplingRate | int | 0.1 | Debug logging sampling rate in percentage e.g. 10% in this case |
Message | string | "Collecting payment" | Log statement value. Unserializable JSON values will be cast to string |
FunctionName | string | "example-powertools-HelloWorldFunction-1P1Z6B39FLU73" | |
FunctionVersion | string | "12" | |
FunctionMemorySize | string | "128" | |
FunctionArn | string | "arn:aws:lambda:eu-west-1:012345678910:function:example-powertools-HelloWorldFunction-1P1Z6B39FLU73" | |
XRayTraceId | string | "1-5759e988-bd862e3fe1be46a994272793" | X-Ray Trace ID when Lambda function has enabled Tracing |
FunctionRequestId | string | "899856cb-83d1-40d7-8611-9e78f15f32f4" | AWS Request ID from lambda context |
Logging incoming event¶
When debugging in non-production environments, you can instruct Logger to log the incoming event with LogEvent
parameter or via POWERTOOLS_LOGGER_LOG_EVENT
environment variable.
Warning
Log event is disabled by default to prevent sensitive info being logged.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Setting a Correlation ID¶
You can set a Correlation ID using CorrelationIdPath
parameter by passing a JSON Pointer expression.
Attention
The JSON Pointer expression is case sensitive
. In the bellow example /headers/my_request_id_header
would work but /Headers/my_request_id_header
would not find the element.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
1 2 3 4 5 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
We provide built-in JSON Pointer expression for known event sources, where either a request ID or X-Ray Trace ID are present.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
1 2 3 4 5 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
Appending additional keys¶
Custom keys are persisted across warm invocations
1 |
|
You can append your own keys to your existing logs via AppendKey
. Typically this value would be passed into the function via the event. Appended keys are added to all subsequent log entries in the current execution from the point the logger method is called. To ensure the key is added to all log entries, call this method as early as possible in the Lambda handler.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
Removing additional keys¶
You can remove any additional key from entry using Logger.RemoveKeys()
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
Extra Keys¶
Extra keys allow you to append additional keys to a log entry. Unlike AppendKey
, extra keys will only apply to the current log entry.
Extra keys argument is available for all log levels' methods, as implemented in the standard logging library - e.g. Logger.Information, Logger.Warning.
It accepts any dictionary, and all keyword arguments will be added as part of the root structure of the logs for that log statement.
Info
1 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
Clearing all state¶
Logger is commonly initialized in the global scope. Due to Lambda Execution Context reuse, this means that custom keys can be persisted across invocations. If you want all custom keys to be deleted, you can use ClearState=true
attribute on [Logging]
attribute.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
1 2 3 4 5 6 7 8 9 10 11 |
|
Sampling debug logs¶
You can dynamically set a percentage of your logs to DEBUG level via env var POWERTOOLS_LOGGER_SAMPLE_RATE
or
via SamplingRate
parameter on attribute.
Info
Configuration on environment variable is given precedence over sampling rate configuration on attribute, provided it's in valid value range.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
1 2 3 4 5 6 7 8 |
|
Configure Log Output Casing¶
By definition Powertools for AWS Lambda (.NET) outputs logging keys using snake case (e.g. "function_memory_size": 128). This allows developers using different Powertools for AWS Lambda (.NET) runtimes, to search logs across services written in languages such as Python or TypeScript.
If you want to override the default behavior you can either set the desired casing through attributes, as described in the example below, or by setting the POWERTOOLS_LOGGER_CASE
environment variable on your AWS Lambda function. Allowed values are: CamelCase
, PascalCase
and SnakeCase
.
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Below are some output examples for different casing.
1 2 3 4 5 6 7 8 9 10 11 |
|
1 2 3 4 5 6 7 8 9 10 11 |
|
1 2 3 4 5 6 7 8 9 10 11 |
|
Custom Log formatter (Bring Your Own Formatter)¶
You can customize the structure (keys and values) of your log entries by implementing a custom log formatter and override default log formatter using Logger.UseFormatter
method. You can implement a custom log formatter by inheriting the ILogFormatter
class and implementing the object FormatLogEntry(LogEntry logEntry)
method.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
|
AOT Support¶
Info
If you want to use the LogEvent
, Custom Log Formatter
features, or serialize your own types when Logging events, you need to make changes in your Lambda Main
method.
Info
Starting from version 1.6.0, it is required to update the Amazon.Lambda.Serialization.SystemTextJson NuGet package to version 2.4.3 in your csproj.
Configure¶
Replace SourceGeneratorLambdaJsonSerializer
with PowertoolsSourceGeneratorSerializer
.
This change enables Powertools to construct an instance of JsonSerializerOptions
used to customize the serialization and deserialization of Lambda JSON events and your own types.
1 2 3 4 |
|
1 2 3 4 |
|
For example when you have your own Demo type
1 2 3 4 5 |
|
To be able to serialize it in AOT you have to have your own JsonSerializerContext
1 2 3 4 5 6 |
|
When you update your code to use PowertoolsSourceGeneratorSerializer<MyCustomJsonSerializerContext>
, we combine your JsonSerializerContext
with Powertools' JsonSerializerContext
. This allows Powertools to serialize your types and Lambda events.
Custom Log Formatter¶
To use a custom log formatter with AOT, pass an instance of ILogFormatter
to PowertoolsSourceGeneratorSerializer
instead of using the static Logger.UseFormatter
in the Function constructor as you do in non-AOT Lambdas.
1 2 3 4 5 6 7 8 9 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 |
|
Anonymous types¶
Note
While we support anonymous type serialization by converting to a Dictionary<string, object>
, this is not a best practice and is not recommended when using native AOT.
We recommend using concrete classes and adding them to your JsonSerializerContext
.