Parameters
The Parameters utility provides high-level functions to retrieve one or multiple parameter values from AWS Systems Manager Parameter Store, AWS Secrets Manager, AWS AppConfig, Amazon DynamoDB, or your own parameter store.
Key features¶
- Retrieve one or multiple parameters from the underlying provider
- Cache parameter values for a given amount of time (defaults to 5 seconds)
- Transform parameter values from JSON or base64 encoded strings
- Bring Your Own Parameter Store Provider
Getting started¶
The Parameters Utility helps to retrieve parameters from the System Manager Parameter Store (SSM), secrets from the Secrets Manager, and application configuration from AppConfig. Additionally, the utility also offers support for a DynamoDB provider, enabling the retrieval of arbitrary parameters from specified tables.
Installation¶
Note
This utility supports AWS SDK for JavaScript v3 only. This allows the utility to be modular, and you to install only the SDK packages you need and keep your bundle size small.
Depending on the provider you want to use, install the library and the corresponding AWS SDK package:
1 |
|
1 |
|
1 |
|
1 |
|
Tip
If you are using the nodejs18.x
runtime or newer, the AWS SDK for JavaScript v3 is already installed and you can install the utility only.
IAM Permissions¶
This utility requires additional permissions to work as expected.
Note
Different parameter providers require different permissions.
Provider | Function/Method | IAM Permission |
---|---|---|
SSM | getParameter , SSMProvider.get |
ssm:GetParameter |
SSM | getParameters , SSMProvider.getMultiple |
ssm:GetParametersByPath |
SSM | getParametersByName , SSMProvider.getParametersByName |
ssm:GetParameter and ssm:GetParameters |
SSM | If using decrypt: true |
You must add an additional permission kms:Decrypt |
Secrets | getSecret , SecretsProvider.get |
secretsmanager:GetSecretValue |
DynamoDB | DynamoDBProvider.get |
dynamodb:GetItem |
DynamoDB | DynamoDBProvider.getMultiple |
dynamodb:Query |
AppConfig | getAppConfig , AppConfigProvider.getAppConfig |
appconfig:GetLatestConfiguration and appconfig:StartConfigurationSession |
Fetching parameters¶
You can retrieve a single parameter using the getParameter
high-level function.
Fetching a single parameter from SSM | |
---|---|
1 2 3 4 5 6 7 |
|
For multiple parameters, you can use either:
getParameters
to recursively fetch all parameters by path.getParametersByName
to fetch distinct parameters by their full name. It also accepts custom caching, transform, decrypt per parameter.
Fetching multiple parameters by path from SSM | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Fetching multiple parameters by names from SSM | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
|
getParametersByName
supports graceful error handling
By default, the provider will throw a GetParameterError
when any parameter fails to be fetched. You can override it by setting throwOnError: false
.
When disabled, instead the provider will take the following actions:
- Add failed parameter name in the
_errors
key, e.g.,{ _errors: [ '/param1', '/param2' ] }
- Keep only successful parameter names and their values in the response
- Throw
GetParameterError
if any of your parameters is named_errors
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
|
Fetching secrets¶
You can fetch secrets stored in Secrets Manager using getSecret
.
Fetching secrets | |
---|---|
1 2 3 4 5 6 7 |
|
Fetching app configurations¶
You can fetch application configurations in AWS AppConfig using getAppConfig
.
The following will retrieve the latest version and store it in the cache.
Fetching latest config from AppConfig | |
---|---|
1 2 3 4 5 6 7 8 9 10 |
|
Advanced¶
Adjusting cache TTL¶
By default, the provider will cache parameters retrieved in-memory for 5 seconds.
You can adjust how long values should be kept in cache by using the param maxAge
, when using get()
or getMultiple()
methods across all providers.
Tip
If you want to set the same TTL for all parameters, you can set the POWERTOOLS_PARAMETERS_MAX_AGE
environment variable. This will override the default TTL of 5 seconds but can be overridden by the maxAge
parameter.
Caching parameters values in memory for longer than 5 seconds | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
- Options passed to
get()
,getMultiple()
, andgetParametersByName()
will override the values set inPOWERTOOLS_PARAMETERS_MAX_AGE
environment variable.
Info
The maxAge
parameter is also available in high level functions like getParameter
, getSecret
, etc.
Always fetching the latest¶
If you'd like to always ensure you fetch the latest parameter from the store regardless if already available in cache, use the forceFetch
parameter.
Forcefully fetching the latest parameter whether TTL has expired or not | |
---|---|
1 2 3 4 5 6 7 |
|
Built-in provider class¶
For greater flexibility such as configuring the underlying SDK client used by built-in providers, you can use their respective Provider Classes directly.
Tip
This can be used to retrieve values from other regions, change the retry behavior, etc.
SSMProvider¶
Example with SSMProvider for further extensibility | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
The AWS Systems Manager Parameter Store provider supports two additional arguments for the get()
and getMultiple()
methods:
Parameter | Default | Description |
---|---|---|
decrypt | false |
Will automatically decrypt the parameter (see required IAM Permissions). |
recursive | true |
For getMultiple() only, will fetch all parameter values recursively based on a path prefix. |
Tip
If you want to always decrypt parameters, you can set the POWERTOOLS_PARAMETERS_SSM_DECRYPT=true
environment variable. This will override the default value of false
but can be overridden by the decrypt
parameter.
Example with get() and getMultiple() | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
- Options passed to
get()
,getMultiple()
, andgetParametersByName()
will override the values set inPOWERTOOLS_PARAMETERS_SSM_DECRYPT
environment variable.
SecretsProvider¶
Example with SecretsProvider for further extensibility | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 |
|
AppConfigProvider¶
The AWS AppConfig provider requires two arguments when initialized:
Parameter | Mandatory in constructor | Alternative | Description |
---|---|---|---|
application | No | POWERTOOLS_SERVICE_NAME env variable |
The application in which your config resides. |
environment | Yes | (N/A) | The environment that corresponds to your current config. |
Example with AppConfigProvider for further extensibility | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
DynamoDBProvider¶
The DynamoDB Provider does not have any high-level functions and needs to know the name of the DynamoDB table containing the parameters.
DynamoDB table structure for single parameters
For single parameters, you must use id
as the partition key for that table.
Example
DynamoDB table with id
partition key and value
as attribute
id | value |
---|---|
my-parameter | my-value |
With this table, await dynamoDBProvider.get('my-param')
will return my-value
.
1 2 3 4 5 6 7 8 9 |
|
You can initialize the DynamoDB provider pointing to DynamoDB Local using the endpoint
field in the clientConfig
parameter:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
DynamoDB table structure for multiple values parameters
You can retrieve multiple parameters sharing the same id
by having a sort key named sk
.
Example
DynamoDB table with id
primary key, sk
as sort key and value
as attribute
id | sk | value |
---|---|---|
my-hash-key | param-a | my-value-a |
my-hash-key | param-b | my-value-b |
my-hash-key | param-c | my-value-c |
With this table, await dynamoDBProvider.getMultiple('my-hash-key')
will return a dictionary response in the shape of sk:value
.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
1 2 3 4 5 |
|
Customizing DynamoDBProvider
DynamoDB provider can be customized at initialization to match your table structure:
Parameter | Mandatory | Default | Description |
---|---|---|---|
tableName | Yes | (N/A) | Name of the DynamoDB table containing the parameter values. |
keyAttr | No | id |
Hash key for the DynamoDB table. |
sortAttr | No | sk |
Range key for the DynamoDB table. You don't need to set this if you don't use the getMultiple() method. |
valueAttr | No | value |
Name of the attribute containing the parameter value. |
Customizing DynamoDBProvider to suit your table design | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
Create your own provider¶
You can create your own custom parameter store provider by extending the BaseProvider
class, and implementing the get()
and getMultiple()
methods, as well as its respective _get()
and _getMultiple()
private methods to retrieve a single, or multiple parameters from your custom store.
All caching logic is handled by the BaseProvider
, and provided that the return types of your store are compatible with the ones used in the BaseProvider
, all transformations will also work as expected.
Here's an example of implementing a custom parameter store using an external service like HashiCorp Vault, a widely popular key-value secret storage.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
Deserializing values with transform parameter¶
For parameters stored in JSON or Base64 format, you can use the transform
argument for deserialization.
Info
The transform
argument is available across all providers, including the high level functions.
1 2 3 4 5 6 7 8 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
Partial transform failures with getMultiple()
¶
If you use transform
with getMultiple()
, you can have a single malformed parameter value. To prevent failing the entire request, the method will return an undefined
value for the parameters that failed to transform.
You can override this by setting the throwOnTransformError
argument to true
. If you do so, a single transform error will throw a TransformParameterError
error.
For example, if you have three parameters, /param/a, /param/b and /param/c, but /param/c is malformed:
Throwing TransformParameterError at first malformed parameter | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 |
|
Auto-transform values on suffix¶
If you use transform
with getMultiple()
, you might want to retrieve and transform parameters encoded in different formats.
You can do this with a single request by using transform: 'auto'
. This will instruct any provider to to infer its type based on the suffix and transform it accordingly.
Info
transform: 'auto'
feature is available across all providers, including the high level functions.
Deserializing parameter values based on their suffix | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
For example, if you have three parameters: two with the following suffixes .json
and .binary
and one without any suffix:
Parameter name | Parameter value |
---|---|
/param/a | [some encoded value] |
/param/a.json | [some encoded value] |
/param/a.binary | [some encoded value] |
The return of await parametersProvider.getMultiple('/param', transform: 'auto');
call will be an object like:
1 2 3 4 5 |
|
The two parameters with a suffix will be decoded, while the one without a suffix will be returned as is.
Passing additional SDK arguments¶
You can use a special sdkOptions
object argument to pass any supported option directly to the underlying SDK method.
Specify a VersionId for a secret | |
---|---|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 |
|
Here is the mapping between this utility's functions and methods and the underlying SDK:
Provider | Function/Method | Client name | Function name |
---|---|---|---|
SSM Parameter Store | getParameter |
@aws-sdk/client-ssm |
GetParameterCommand |
SSM Parameter Store | getParameters |
@aws-sdk/client-ssm |
GetParametersByPathCommand |
SSM Parameter Store | SSMProvider.get |
@aws-sdk/client-ssm |
GetParameterCommand |
SSM Parameter Store | SSMProvider.getMultiple |
@aws-sdk/client-ssm |
GetParametersByPathCommand |
Secrets Manager | getSecret |
@aws-sdk/client-secrets-manager |
GetSecretValueCommand |
Secrets Manager | SecretsProvider.get |
@aws-sdk/client-secrets-manager |
GetSecretValueCommand |
AppConfig | AppConfigProvider.get |
@aws-sdk/client-appconfigdata |
StartConfigurationSessionCommand & GetLatestConfigurationCommand |
AppConfig | getAppConfig |
@aws-sdk/client-appconfigdata |
StartConfigurationSessionCommand & GetLatestConfigurationCommand |
DynamoDB | DynamoDBProvider.get |
@aws-sdk/client-dynamodb |
GetItemCommand |
DynamoDB | DynamoDBProvider.getMultiple |
@aws-sdk/client-dynamodb |
QueryCommand |
Bring your own AWS SDK v3 client¶
You can use the awsSdkV3Client
parameter via any of the available Provider Classes.
Provider | Client |
---|---|
SSMProvider | new SSMClient(); |
SecretsProvider | new SecretsManagerClient(); |
AppConfigProvider | new AppConfigDataClient(); |
DynamoDBProvider | new DynamoDBClient(); |
When is this useful?
Injecting a custom AWS SDK v3 client allows you to apply tracing or make unit/snapshot testing easier, including SDK customizations.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
Customizing AWS SDK v3 configuration¶
The clientConfig
parameter enables you to pass in a custom config object when constructing any of the built-in provider classes.
Tip
You can use a custom session for retrieving parameters cross-account/region and for snapshot testing.
When using VPC private endpoints, you can pass a custom client altogether. It's also useful for testing when injecting fake instances.
1 2 3 4 5 6 7 8 9 10 11 |
|
Testing your code¶
Mocking parameter values¶
For unit testing your applications, you can mock the calls to the parameters utility to avoid calling AWS APIs. This can be achieved in a number of ways - in this example, we use Jest mock functions to patch the getParameters
function.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
|
1 2 3 4 5 6 7 8 9 10 11 12 |
|
With this pattern in place, you can customize the return values of the mocked function to test different scenarios without calling AWS APIs.
A similar pattern can be applied also to any of the built-in provider classes - in this other example, we use Jest spyOn method to patch the get
function of the AppConfigProvider
class. This is useful also when you want to test that the correct arguments are being passed to the Parameters utility.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
|
In some other cases, you might want to mock the AWS SDK v3 client itself, in these cases we recommend using the aws-sdk-client-mock
and aws-sdk-client-mock-jest
libraries. This is useful when you want to test how your code behaves when the AWS SDK v3 client throws an error or a specific response.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
|
Clearing cache¶
Parameters utility caches all parameter values for performance and cost reasons. However, this can have unintended interference in tests using the same parameter name.
Within your tests, you can use clearCache
method available in every provider. When using multiple providers or higher level functions like getParameter
, use the clearCaches
standalone function to clear cache globally.
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|