Project Overview
The Silent Scalper project addresses the need for a scalable, serverless pipeline for processing file uploads. It validates files, ensures secure handling, and classifies them into appropriate buckets while automating monitoring and notifications.
Technologies Used
- AWS Lambda: Serverless file processing and validation.
- Amazon S3: Input, output, and quarantine bucket storage.
- Amazon DynamoDB: Metadata storage for processed files.
- Amazon SNS: Real-time notifications for events and errors.
- Amazon CloudWatch: Logging, monitoring, and alarms.
- Terraform: Infrastructure as Code for consistent deployments.
- Python: File processing logic within Lambda functions.
Challenges and Solutions
- Terraform configuration issues: Solved through detailed validation and incremental testing.
- SNS notification setup: Used environment variables to securely store sensitive information.
- Debugging file routing: Validated each component step by step for accurate processing.
Impact and Results
- Reduced processing time with scalable serverless architecture.
- Minimized operational costs using pay-as-you-go AWS services.
- Demonstrated expertise in building secure, efficient cloud solutions.
Architecture Diagram
