Nginx Log Parser - Terminal Bench Challenge

A comprehensive nginx access log analysis tool that generates detailed summary reports using standard Unix command-line tools. This project demonstrates advanced bash scripting and system administration skills through practical log processing.

🚀 Overview

This task parses nginx access logs and generates a structured summary report containing:

• Request Statistics: Total requests and unique IP addresses
• Status Code Analysis: Breakdown of HTTP response codes
• Traffic Patterns: Top requested paths and most active IP addresses
• Bandwidth Usage: Total bytes transferred

📊 Sample Output

NGINX ACCESS LOG SUMMARY REPORT
================================

Total Requests: 12
Unique IP Addresses: 4

Status Code Breakdown:
- 200: 8 requests
- 500: 1 requests
- 404: 1 requests
- 403: 1 requests
- 302: 1 requests

Top 3 Most Requested Paths:
1. /styles.css - 1 requests
2. /nonexistent.html - 1 requests
3. /index.html - 1 requests

Top 3 IP Addresses by Request Volume:
1. 192.168.1.100 - 4 requests
2. 203.0.113.25 - 3 requests
3. 10.0.0.50 - 3 requests

Total Bytes Transferred: 19136 bytes

🛠️ Technical Implementation

Core Technologies

• Bash scripting with advanced text processing
• AWK for field extraction and data manipulation
• Unix tools: grep, sort, uniq, wc for data analysis
• Docker for containerized execution environment

Key Features

• Robust log parsing with proper field extraction from quoted HTTP requests
• Efficient data processing using Unix pipelines
• Comprehensive error handling with input validation
• Deterministic output suitable for automated testing

📁 Project Structure

parse-nginx-logs/
├── access.log              # Sample nginx access log data
├── solution.sh             # Main bash script for log analysis
├── task.yaml               # Task configuration and metadata
├── Dockerfile              # Container environment setup
├── docker-compose.yaml     # Multi-container orchestration
├── run-tests.sh            # Test execution script
└── tests/
    └── test_outputs.py     # Comprehensive pytest test suite (9 tests)

🧪 Testing & Validation

The project includes a comprehensive test suite that validates:

• ✅ Report file generation
• ✅ Correct data structure and formatting
• ✅ Accurate request counting and IP analysis
• ✅ Proper status code breakdown
• ✅ Top paths and IPs identification
• ✅ Byte transfer calculations

Test Results: 9/9 tests passing (100% success rate)

🚀 Quick Start

Prerequisites

• Docker and Docker Compose
• Unix-like environment (Linux/macOS)

Running the Analysis

1. Execute the log parser:

   bash solution.sh

2. Run comprehensive tests:

   bash run-tests.sh

3. View the generated report:

   cat summary_report.txt

Using Terminal-Bench (if available)

# Run with oracle agent
uv run tb run --agent oracle --task-id parse-nginx-logs

# Interactive debugging
uv run tb tasks interact -t parse-nginx-logs

📋 Log Format Support

The parser handles standard nginx access log format:

IP - - [timestamp] "METHOD path HTTP/version" status size "referer" "user-agent"

Example:

192.168.1.100 - - [03/Aug/2025:10:15:32 +0000] "GET /index.html HTTP/1.1" 200 1024 "-" "Mozilla/5.0..."

💡 Key Learning Outcomes

This project demonstrates:

• Advanced bash scripting with complex text processing
• Unix tool mastery for efficient data pipeline construction
• System administration skills for log analysis workflows
• Container deployment with proper environment configuration
• Test-driven development with comprehensive validation

🎯 Use Cases

• Web server monitoring and traffic analysis
• Performance optimization through request pattern identification
• Security analysis via IP and path monitoring
• Capacity planning using bandwidth and request metrics
• Automated reporting for system administration tasks

📈 Performance Characteristics

• Efficient processing of large log files using Unix pipelines
• Memory optimized through streaming data processing
• Fast execution leveraging optimized system tools
• Scalable architecture suitable for production environments

📝 License

This project is part of an assessment submission and demonstrates practical bash/Linux expertise for educational and evaluation purposes.

---

Created by: David Arnal Date: August 3, 2025