Support both plain text and JSON format for Secrets Manager API key

[Gekko0114]

What does this PR do?

This PR enhances the get_api_key() function to support both plain text and JSON format values from AWS Secrets Manager. Previously, the function only supported plain text values. Now it can handle:

• Plain text API keys stored directly in Secrets Manager
• JSON format secrets with keys like DD_API_KEY or DATADOG_API_KEY

The function first treats the secret value as plain text. If the value looks like JSON (starts with { and ends with }), it attempts to parse it and extract the API key from common key names. If JSON parsing fails or no common key is found, it falls back to treating the value as plain text.

Motivation

Some users store their Datadog API keys in Secrets Manager as JSON objects (key-value pairs) rather than plain text. This change makes the library more flexible and compatible with different secret storage formats, improving the developer experience.

Testing Guidelines

Added comprehensive test cases covering:

• Plain text secrets (existing behavior)
• JSON format secrets with api_key key
• JSON format secrets with DD_API_KEY key
• JSON format secrets with custom keys (fallback to plain text)

All tests pass and maintain backward compatibility with existing plain text secrets.

Additional Notes

• The implementation prioritizes plain text handling first, ensuring backward compatibility
• JSON parsing only occurs when the value appears to be JSON (starts/ends with braces)
• Common key names checked: DD_API_KEY, DATADOG_API_KEY
• If JSON parsing fails or no common key is found, the original value is used as-is

Types of Changes

• New feature

Check all that apply

• This PR's description is comprehensive
• This PR's changes are covered by the automated tests

[astuyve]

Thanks for the PR, this is great! However this code path is only taken when the user is not using the datadog lambda extension (which is the default for the last ~3 years), and only when the user is sending custom metrics with the send_distribution_metric method.

Wanna write some rust? I think we should add this to the extension and release it at the same time as this change.

[Gekko0114]

@astuyve

Thanks!
I installed datadog-lambda via a container image by following the documentation below. Regarding the Secrets Manager, I have it registered as a Key-Value store rather than plain text. I encountered this error when I tried using lambda_metric.

https://docs.datadoghq.com/ja/serverless/aws_lambda/instrumentation/python/?tab=containerimage

This documentation appears to be up-to-date, but is this method no longer recommended?
Could you please let me know the best practice for monitoring Python running as a container image on AWS Lambda?