-
Notifications
You must be signed in to change notification settings - Fork 2
Fix SQS Event Source LocalStack Integration #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
835431b
chore: update nuget packages including Aspire.Hosting to 9.5.0
Blind-Striker c0005ad
feat(sqs-lambda): implement SQS Event Source LocalStack URL fix
Blind-Striker d0dee6e
test(sqs-lambda): add analytics testing playground and simplify fix i…
Blind-Striker adb0b07
fix(tests): use explicit Assembly.Load for AWS assembly in ConstantsT…
Blind-Striker 4915669
fix(ci): resolve artifact upload error and enforce NuGet SemVer 2.0.0
Blind-Striker 3247838
fix(security): log injection vulnerability
Blind-Striker 19d701c
test(integration): add logging and analytics resource verification
Blind-Striker 3f574c1
refactor(tests): modernize integration tests with fixtures and focuse…
Blind-Striker c5754b7
test(integration): refine Phase 3 functional tests
Blind-Striker 07461e1
test(integration): increase wait time for Analyzer Lambda processing
Blind-Striker 5eedcfb
fix(hosting): configure AWS credentials for SQS Event Source on Linux
Blind-Striker b336419
test(integration): increase HTTP client timeout for Lambda tests
Blind-Striker 79edb2e
fix(hosting): correct AWS_SECRET_ACCESS_KEY assignment in LocalStack …
Blind-Striker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
110 changes: 110 additions & 0 deletions
110
playground/lambda/LocalStack.Lambda.Analyzer/Function.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| #pragma warning disable CA1822 // Member 'FunctionHandler' does not access instance data and can be marked as static | ||
| #pragma warning disable S2325 // Make 'FunctionHandler' a static method. | ||
| #pragma warning disable CA1812 // Error CA1812 : 'Function.ShortenRequest' is an internal class that is apparently never instantiated. | ||
| #pragma warning disable CA1031 // Modify 'FunctionHandler' to catch a more specific allowed exception type, or rethrow the exception | ||
|
|
||
| using System.Diagnostics; | ||
| using System.Globalization; | ||
| using System.Text.Json; | ||
| using Amazon.DynamoDBv2; | ||
| using Amazon.DynamoDBv2.Model; | ||
| using Amazon.Lambda.Core; | ||
| using Amazon.Lambda.SQSEvents; | ||
| using LocalStack.Client.Extensions; | ||
| using LocalStack.Playground.ServiceDefaults.ActivitySources; | ||
| using Microsoft.Extensions.DependencyInjection; | ||
| using Microsoft.Extensions.Hosting; | ||
| using OpenTelemetry.Instrumentation.AWSLambda; | ||
| using OpenTelemetry.Trace; | ||
|
|
||
| // Assembly attribute to enable the Lambda function's JSON input to be converted into a .NET class. | ||
| [assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))] | ||
|
|
||
| namespace LocalStack.Lambda.Analyzer; | ||
|
|
||
| public class Function | ||
| { | ||
| private readonly TracerProvider _traceProvider; | ||
|
|
||
| private readonly IAmazonDynamoDB _amazonDynamoDb; | ||
|
|
||
| private readonly string _analyticsTable; | ||
|
|
||
| public Function() | ||
| { | ||
| var builder = new HostApplicationBuilder(); | ||
|
|
||
| builder.AddServiceDefaults(); | ||
|
|
||
| builder.Services.AddLocalStack(builder.Configuration); | ||
| builder.Services.AddAwsService<IAmazonDynamoDB>(); | ||
|
|
||
| var host = builder.Build(); | ||
|
|
||
| _traceProvider = host.Services.GetRequiredService<TracerProvider>(); | ||
| _amazonDynamoDb = host.Services.GetRequiredService<IAmazonDynamoDB>(); | ||
|
|
||
| _analyticsTable = builder.Configuration["AWS:Resources:AnalyticsTableName"] ?? throw new InvalidOperationException("Missing AWS:Resources:AnalyticsTableName"); | ||
| } | ||
|
|
||
| public Task FunctionHandler(SQSEvent sqsEvent, ILambdaContext context) | ||
| { | ||
| return AWSLambdaWrapper.TraceAsync(_traceProvider, async (proxyRequest, lambdaContext) => | ||
| { | ||
| using var activity = RedirectorActivitySource.ActivitySource.StartActivity(nameof(FunctionHandler)); | ||
|
|
||
| lambdaContext.Logger.LogInformation($"Processing {proxyRequest.Records.Count} analytics events"); | ||
|
|
||
| foreach (var record in proxyRequest.Records) | ||
| { | ||
| try | ||
| { | ||
| var analyticsEvent = JsonSerializer.Deserialize<AnalyticsEvent>(record.Body); | ||
| if (analyticsEvent != null) | ||
| { | ||
| await ProcessAnalyticsEventAsync(analyticsEvent, lambdaContext).ConfigureAwait(false); | ||
| } | ||
| } | ||
| catch (Exception ex) | ||
| { | ||
| lambdaContext.Logger.LogError($"Error processing analytics event: {ex.Message}"); | ||
| activity?.SetStatus(ActivityStatusCode.Error, ex.Message); | ||
| } | ||
| } | ||
| }, sqsEvent, context); | ||
| } | ||
|
|
||
| private async Task ProcessAnalyticsEventAsync(AnalyticsEvent analyticsEvent, ILambdaContext context) | ||
| { | ||
| using var activity = RedirectorActivitySource.ActivitySource.StartActivity(nameof(ProcessAnalyticsEventAsync)); | ||
|
|
||
| activity?.AddTag("eventType", analyticsEvent.EventType); | ||
| activity?.AddTag("slug", analyticsEvent.Slug); | ||
|
|
||
| var item = new Dictionary<string, AttributeValue>(StringComparer.Ordinal) | ||
| { | ||
| ["EventId"] = new() { S = Guid.NewGuid().ToString() }, | ||
| ["Timestamp"] = new() { S = DateTime.UtcNow.ToString("O", CultureInfo.InvariantCulture) }, | ||
| ["EventType"] = new() { S = analyticsEvent.EventType }, | ||
| ["Slug"] = new() { S = analyticsEvent.Slug }, | ||
| ["OriginalUrl"] = new() { S = analyticsEvent.OriginalUrl }, | ||
| ["UserAgent"] = new() { S = analyticsEvent.UserAgent ?? "unknown" }, | ||
| ["IpAddress"] = new() { S = analyticsEvent.IpAddress ?? "unknown" }, | ||
| }; | ||
|
|
||
| await _amazonDynamoDb.PutItemAsync(new PutItemRequest | ||
| { | ||
| TableName = _analyticsTable, | ||
| Item = item, | ||
| }).ConfigureAwait(false); | ||
|
|
||
| context.Logger.LogInformation($"Processed {analyticsEvent.EventType} event for slug: {analyticsEvent.Slug}"); | ||
| } | ||
| } | ||
|
|
||
| public sealed record AnalyticsEvent( | ||
| string EventType, // "url_created" or "url_accessed" | ||
| string Slug, | ||
| string OriginalUrl, | ||
| string? UserAgent = null, | ||
| string? IpAddress = null); | ||
41 changes: 41 additions & 0 deletions
41
playground/lambda/LocalStack.Lambda.Analyzer/LocalStack.Lambda.Analyzer.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,41 @@ | ||
| <Project Sdk="Microsoft.NET.Sdk"> | ||
| <PropertyGroup> | ||
| <TargetFramework>$(DefaultTargetFramework)</TargetFramework> | ||
| <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles> | ||
| <AWSProjectType>Lambda</AWSProjectType> | ||
| <!-- This property makes the build directory similar to a publish directory and helps the AWS .NET Lambda Mock Test Tool find project dependencies. --> | ||
| <CopyLocalLockFileAssemblies>true</CopyLocalLockFileAssemblies> | ||
| <!-- Generate ready to run images during publishing to improve cold start time. --> | ||
| <PublishReadyToRun>true</PublishReadyToRun> | ||
| <IsPackable>false</IsPackable> | ||
| </PropertyGroup> | ||
|
|
||
| <ItemGroup> | ||
| <PackageReference Include="AWSSDK.Core"/> | ||
| <PackageReference Include="AWSSDK.DynamoDBv2"/> | ||
| <PackageReference Include="Amazon.Lambda.Core"/> | ||
| <PackageReference Include="Amazon.Lambda.SQSEvents"/> | ||
| <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson"/> | ||
|
|
||
| <PackageReference Include="LocalStack.Client"/> | ||
| <PackageReference Include="LocalStack.Client.Extensions"/> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <ProjectReference Include="..\..\LocalStack.Playground.ServiceDefaults\LocalStack.Playground.ServiceDefaults.csproj"/> | ||
| </ItemGroup> | ||
|
|
||
| <ItemGroup> | ||
| <Content Update="appsettings.json"> | ||
| <ExcludeFromSingleFile>true</ExcludeFromSingleFile> | ||
| <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> | ||
| <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory> | ||
| </Content> | ||
| <Content Update="appsettings.Development.json"> | ||
| <ExcludeFromSingleFile>true</ExcludeFromSingleFile> | ||
| <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory> | ||
| <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory> | ||
| <DependentUpon>appsettings.json</DependentUpon> | ||
| </Content> | ||
| </ItemGroup> | ||
| </Project> |
29 changes: 29 additions & 0 deletions
29
playground/lambda/LocalStack.Lambda.Analyzer/Properties/launchSettings.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| { | ||
| "$schema": "http://json.schemastore.org/launchsettings.json", | ||
| "profiles": { | ||
| "https": { | ||
| "commandName": "Project", | ||
| "dotnetRunMessages": true, | ||
| "launchBrowser": true, | ||
| "applicationUrl": "http://localhost:5366", | ||
| "environmentVariables": { | ||
| "ASPNETCORE_ENVIRONMENT": "Development" | ||
| } | ||
| }, | ||
| "http": { | ||
| "commandName": "Project", | ||
| "dotnetRunMessages": true, | ||
| "launchBrowser": true, | ||
| "applicationUrl": "https://localhost:7330;http://localhost:5366", | ||
| "environmentVariables": { | ||
| "ASPNETCORE_ENVIRONMENT": "Development" | ||
| } | ||
| }, | ||
| "Aspire_AnalyzerLambda": { | ||
| "commandName": "Executable", | ||
| "executablePath": "dotnet", | ||
| "commandLineArgs": "exec --depsfile ./LocalStack.Lambda.Analyzer.deps.json --runtimeconfig ./LocalStack.Lambda.Analyzer.runtimeconfig.json $(HOME)/.dotnet/tools/.store/amazon.lambda.testtool/0.11.0/amazon.lambda.testtool/0.11.0/content/Amazon.Lambda.RuntimeSupport/net8.0/Amazon.Lambda.RuntimeSupport.dll LocalStack.Lambda.Analyzer::LocalStack.Lambda.Analyzer.Function::FunctionHandler", | ||
| "workingDirectory": "./bin/$(Configuration)/net8.0" | ||
| } | ||
| } | ||
| } |
11 changes: 11 additions & 0 deletions
11
playground/lambda/LocalStack.Lambda.Analyzer/appsettings.Development.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| { | ||
| "Logging": { | ||
| "LogLevel": { | ||
| "Default": "Information", | ||
| "Microsoft.AspNetCore": "Warning" | ||
| } | ||
| }, | ||
| "LocalStack": { | ||
| "UseLocalStack": true | ||
| } | ||
| } |
12 changes: 12 additions & 0 deletions
12
playground/lambda/LocalStack.Lambda.Analyzer/appsettings.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "Logging": { | ||
| "LogLevel": { | ||
| "Default": "Information", | ||
| "Microsoft.AspNetCore": "Warning" | ||
| } | ||
| }, | ||
| "AllowedHosts": "*", | ||
| "LocalStack": { | ||
| "UseLocalStack": false | ||
| } | ||
| } |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Telemetry Misattribution in Analyzer Lambda
The Analyzer Lambda uses
RedirectorActivitySource.ActivitySourcein bothFunctionHandlerandProcessAnalyticsEventAsync. This misattributes telemetry, making it appear the Redirector service is performing Analyzer operations.Additional Locations (1)
playground/lambda/LocalStack.Lambda.Analyzer/Function.cs#L78-L79