Skip to content

Commit 7a89c19

Browse files
sebstosebsto
committed
Add user-facing API for Streaming Lambda functions that receives JSON events
1 parent dede067 commit 7a89c19

File tree

10 files changed

+842
-2
lines changed

10 files changed

+842
-2
lines changed

‎.github/workflows/pull_request.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ jobs:
3636
# We pass the list of examples here, but we can't pass an array as argument
3737
# Instead, we pass a String with a valid JSON array.
3838
# The workaround is mentioned here https://github.com/orgs/community/discussions/11692
39-
examples: "[ 'APIGateway', 'APIGateway+LambdaAuthorizer', 'BackgroundTasks', 'HelloJSON', 'HelloWorld', 'ResourcesPackaging', 'S3EventNotifier', 'S3_AWSSDK', 'S3_Soto', 'Streaming', 'Testing', 'Tutorial' ]"
39+
examples: "[ 'APIGateway', 'APIGateway+LambdaAuthorizer', 'BackgroundTasks', 'HelloJSON', 'HelloWorld', 'ResourcesPackaging', 'S3EventNotifier', 'S3_AWSSDK', 'S3_Soto', 'Streaming', 'StreamingFromEvent', 'Testing', 'Tutorial' ]"
4040
archive_plugin_examples: "[ 'HelloWorld', 'ResourcesPackaging' ]"
4141
archive_plugin_enabled: true
4242

‎Examples/README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,9 @@ This directory contains example code for Lambda functions.
3434

3535
- **[S3_Soto](S3_Soto/README.md)**: a Lambda function that uses [Soto](https://github.com/soto-project/soto) to invoke an [Amazon S3](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html) API (requires [AWS SAM](https://aws.amazon.com/serverless/sam/)).
3636

37-
- **[Streaming]**: create a Lambda function exposed as an URL. The Lambda function streams its response over time. (requires [AWS SAM](https://aws.amazon.com/serverless/sam/)).
37+
- **[Streaming](Streaming/README.md)**: create a Lambda function exposed as an URL. The Lambda function streams its response over time. (requires [AWS SAM](https://aws.amazon.com/serverless/sam/)).
38+
39+
- **[StreamingFromEvent](StreamingFromEvent/README.md)**: a Lambda function that combines JSON input decoding with response streaming capabilities, demonstrating the new streaming codable interface (requires [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)).
3840

3941
- **[Testing](Testing/README.md)**: a test suite for Lambda functions.
4042

‎Examples/StreamingFromEvent/Package.swift

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
// swift-tools-version: 6.0
2+
3+
import PackageDescription
4+
5+
// needed for CI to test the local version of the library
6+
import struct Foundation.URL
7+
8+
let package = Package(
9+
name: "StreamingFromEvent",
10+
platforms: [.macOS(.v15)],
11+
dependencies: [
12+
// during CI, the dependency on local version of swift-aws-lambda-runtime is added dynamically below
13+
.package(url: "https://github.com/swift-server/swift-aws-lambda-runtime.git", branch: "main")
14+
],
15+
targets: [
16+
.executableTarget(
17+
name: "StreamingFromEvent",
18+
dependencies: [
19+
.product(name: "AWSLambdaRuntime", package: "swift-aws-lambda-runtime")
20+
]
21+
)
22+
]
23+
)
24+
25+
if let localDepsPath = Context.environment["LAMBDA_USE_LOCAL_DEPS"],
26+
localDepsPath != "",
27+
let v = try? URL(fileURLWithPath: localDepsPath).resourceValues(forKeys: [.isDirectoryKey]),
28+
v.isDirectory == true
29+
{
30+
// when we use the local runtime as deps, let's remove the dependency added above
31+
let indexToRemove = package.dependencies.firstIndex { dependency in
32+
if case .sourceControl(
33+
name: _,
34+
location: "https://github.com/swift-server/swift-aws-lambda-runtime.git",
35+
requirement: _
36+
) = dependency.kind {
37+
return true
38+
}
39+
return false
40+
}
41+
if let indexToRemove {
42+
package.dependencies.remove(at: indexToRemove)
43+
}
44+
45+
// then we add the dependency on LAMBDA_USE_LOCAL_DEPS' path (typically ../..)
46+
print("[INFO] Compiling against swift-aws-lambda-runtime located at \(localDepsPath)")
47+
package.dependencies += [
48+
.package(name: "swift-aws-lambda-runtime", path: localDepsPath)
49+
]
50+
}

‎Examples/StreamingFromEvent/README.md

Lines changed: 128 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,128 @@
1+
# Streaming Codable Lambda function
2+
3+
This example demonstrates how to use the new `StreamingLambdaHandlerWithEvent` protocol to create Lambda functions that:
4+
5+
1. **Receive JSON input**: Automatically decode JSON events into Swift structs
6+
2. **Stream responses**: Send data incrementally as it becomes available
7+
3. **Execute background work**: Perform additional processing after the response is sent
8+
9+
The example uses the new streaming codable interface that combines the benefits of:
10+
- Type-safe JSON input decoding (like regular `LambdaHandler`)
11+
- Response streaming capabilities (like `StreamingLambdaHandler`)
12+
- Background work execution after response completion
13+
14+
Streaming responses incurs a cost. For more information, see [AWS Lambda Pricing](https://aws.amazon.com/lambda/pricing/).
15+
16+
You can stream responses through [Lambda function URLs](https://docs.aws.amazon.com/lambda/latest/dg/urls-configuration.html), the AWS SDK, or using the Lambda [InvokeWithResponseStream](https://docs.aws.amazon.com/lambda/latest/dg/API_InvokeWithResponseStream.html) API.
17+
18+
## Code
19+
20+
The sample code creates a `StreamingFromEventHandler` struct that conforms to the `StreamingLambdaHandlerWithEvent` protocol provided by the Swift AWS Lambda Runtime.
21+
22+
The `handle(...)` method of this protocol receives incoming events as a decoded Swift struct (`StreamingRequest`) and returns the output through a `LambdaResponseStreamWriter`.
23+
24+
The Lambda function expects a JSON payload with the following structure:
25+
26+
```json
27+
{
28+
"count": 5,
29+
"message": "Hello from streaming Lambda!",
30+
"delayMs": 1000
31+
}
32+
```
33+
34+
Where:
35+
- `count`: Number of messages to stream (1-100)
36+
- `message`: The message content to repeat
37+
- `delayMs`: Optional delay between messages in milliseconds (defaults to 500ms)
38+
39+
The response is streamed through the `LambdaResponseStreamWriter`, which is passed as an argument in the `handle` function. The code calls the `write(_:)` function of the `LambdaResponseStreamWriter` with partial data repeatedly written before finally closing the response stream by calling `finish()`. Developers can also choose to return the entire output and not stream the response by calling `writeAndFinish(_:)`.
40+
41+
An error is thrown if `finish()` is called multiple times or if it is called after having called `writeAndFinish(_:)`.
42+
43+
The `handle(...)` method is marked as `mutating` to allow handlers to be implemented with a `struct`.
44+
45+
Once the struct is created and the `handle(...)` method is defined, the sample code creates a `LambdaRuntime` struct and initializes it with the handler just created. Then, the code calls `run()` to start the interaction with the AWS Lambda control plane.
46+
47+
Key features demonstrated:
48+
- **JSON Input Decoding**: The function automatically parses the JSON input into a `StreamingRequest` struct
49+
- **Input Validation**: Validates the count parameter and returns an error message if invalid
50+
- **Progressive Streaming**: Sends messages one by one with configurable delays
51+
- **Timestamped Output**: Each message includes an ISO8601 timestamp
52+
- **Background Processing**: Performs cleanup and logging after the response is complete
53+
- **Error Handling**: Gracefully handles invalid input with descriptive error messages
54+
55+
## Build & Package
56+
57+
To build & archive the package, type the following commands.
58+
59+
```bash
60+
swift package archive --allow-network-connections docker
61+
```
62+
63+
If there is no error, there is a ZIP file ready to deploy.
64+
The ZIP file is located at `.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingFromEvent/StreamingFromEvent.zip`
65+
66+
## Test locally
67+
68+
You can test the function locally before deploying:
69+
70+
```bash
71+
swift run &
72+
73+
# In another terminal, test with curl:
74+
curl -v \
75+
--header "Content-Type: application/json" \
76+
--data '{"count": 3, "message": "Hello World!", "delayMs": 1000}' \
77+
http://127.0.0.1:7000/invoke
78+
```
79+
80+
## Deploy with the AWS CLI
81+
82+
Here is how to deploy using the `aws` command line.
83+
84+
### Step 1: Create the function
85+
86+
```bash
87+
# Replace with your AWS Account ID
88+
AWS_ACCOUNT_ID=012345678901
89+
aws lambda create-function \
90+
--function-name StreamingFromEvent \
91+
--zip-file fileb://.build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingFromEvent/StreamingFromEvent.zip \
92+
--runtime provided.al2 \
93+
--handler provided \
94+
--architectures arm64 \
95+
--role arn:aws:iam::${AWS_ACCOUNT_ID}:role/lambda_basic_execution
96+
```
97+
98+
The `--architectures` flag is only required when you build the binary on an Apple Silicon machine (Apple M1 or more recent). It defaults to `x64`.
99+
100+
Be sure to set `AWS_ACCOUNT_ID` with your actual AWS account ID (for example: 012345678901).
101+
102+
### Invoke your Lambda function
103+
104+
To invoke the Lambda function, use the AWS CLI:
105+
106+
```bash
107+
aws lambda invoke \
108+
--function-name StreamingFromEvent \
109+
--payload $(echo '{"count": 5, "message": "Streaming from AWS!", "delayMs": 500}' | base64) \
110+
response.txt && cat response.txt
111+
```
112+
113+
This should output the following result, with configurable delays between each message:
114+
115+
```
116+
[2024-07-15T05:00:00Z] Message 1/3: Hello World!
117+
[2024-07-15T05:00:01Z] Message 2/3: Hello World!
118+
[2024-07-15T05:00:02Z] Message 3/3: Hello World!
119+
✅ Successfully sent 3 messages
120+
```
121+
122+
### Undeploy
123+
124+
When done testing, you can delete the Lambda function with this command.
125+
126+
```bash
127+
aws lambda delete-function --function-name StreamingFromEvent
128+
```

‎Examples/StreamingFromEvent/Sources/main.swift

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
//===----------------------------------------------------------------------===//
2+
//
3+
// This source file is part of the SwiftAWSLambdaRuntime open source project
4+
//
5+
// Copyright (c) 2024 Apple Inc. and the SwiftAWSLambdaRuntime project authors
6+
// Licensed under Apache License v2.0
7+
//
8+
// See LICENSE.txt for license information
9+
// See CONTRIBUTORS.txt for the list of SwiftAWSLambdaRuntime project authors
10+
//
11+
// SPDX-License-Identifier: Apache-2.0
12+
//
13+
//===----------------------------------------------------------------------===//
14+
15+
import AWSLambdaRuntime
16+
import NIOCore
17+
18+
#if canImport(FoundationEssentials)
19+
import FoundationEssentials
20+
#else
21+
import Foundation
22+
#endif
23+
24+
// Define your input event structure
25+
struct StreamingRequest: Decodable {
26+
let count: Int
27+
let message: String
28+
let delayMs: Int?
29+
30+
// Provide default values for optional fields
31+
var delay: Int {
32+
delayMs ?? 500
33+
}
34+
}
35+
36+
// Use the new streaming handler with JSON decoding
37+
let runtime = LambdaRuntime { (event: StreamingRequest, responseWriter, context: LambdaContext) in
38+
context.logger.info("Received request to send \(event.count) messages: '\(event.message)'")
39+
40+
// Validate input
41+
guard event.count > 0 && event.count <= 100 else {
42+
let errorMessage = "Count must be between 1 and 100, got: \(event.count)"
43+
context.logger.error("\(errorMessage)")
44+
try await responseWriter.writeAndFinish(ByteBuffer(string: "Error: \(errorMessage)\n"))
45+
return
46+
}
47+
48+
// Stream the messages
49+
for i in 1...event.count {
50+
let response = "[\(Date().ISO8601Format())] Message \(i)/\(event.count): \(event.message)\n"
51+
try await responseWriter.write(ByteBuffer(string: response))
52+
53+
// Optional delay between messages
54+
if event.delay > 0 {
55+
try await Task.sleep(for: .milliseconds(event.delay))
56+
}
57+
}
58+
59+
// Send completion message and finish the stream
60+
let completionMessage = "✅ Successfully sent \(event.count) messages\n"
61+
try await responseWriter.writeAndFinish(ByteBuffer(string: completionMessage))
62+
63+
// Optional: Do background work here after response is sent
64+
context.logger.info("Background work: cleaning up resources and logging metrics")
65+
66+
// Simulate some background processing
67+
try await Task.sleep(for: .milliseconds(100))
68+
context.logger.info("Background work completed")
69+
}
70+
71+
try await runtime.run()

‎Examples/StreamingFromEvent/events/sample-request.json

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
{
2+
"count": 5,
3+
"message": "Hello from streaming Lambda!",
4+
"delayMs": 1000
5+
}

‎Examples/StreamingFromEvent/template.yaml

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
AWSTemplateFormatVersion: '2010-09-09'
2+
Transform: AWS::Serverless-2016-10-31
3+
Description: SAM Template for Streaming Example
4+
5+
Resources:
6+
# Lambda function
7+
StreamingNumbers:
8+
Type: AWS::Serverless::Function
9+
Properties:
10+
CodeUri: .build/plugins/AWSLambdaPackager/outputs/AWSLambdaPackager/StreamingNumbers/StreamingNumbers.zip
11+
Timeout: 15
12+
Handler: swift.bootstrap # ignored by the Swift runtime
13+
Runtime: provided.al2
14+
MemorySize: 128
15+
Architectures:
16+
- arm64
17+
FunctionUrlConfig:
18+
AuthType: AWS_IAM
19+
InvokeMode: RESPONSE_STREAM
20+
21+
Outputs:
22+
# print Lambda function URL
23+
LambdaURL:
24+
Description: Lambda URL
25+
Value: !GetAtt StreamingNumbersUrl.FunctionUrl

0 commit comments

Comments
 (0)