Implement Testcontainers for development and testing

README.md

@@ -1,161 +1,101 @@
-# GenAI Application Demo using Model Runner
+# GenAI Application Testing 
 
-A modern chat application demonstrating integration of frontend technologies with local Large Language Models (LLMs).
+This GitHub branch specifically checks the integration between your application and the model runner. It performs several key testing operations:
 
-## Overview
+- Environment Setup Testing: It verifies that Testcontainers could successfully set up the required environment, including creating and configuring a socat container for networking.
+- API Endpoint Testing: It tests various endpoints of the model runner service:
+   - Checks the /models endpoint (to list available models)
+   - Tests the /engines endpoint (to verify the service was running)
+- Model Creation Testing: It attempts to create a model (as an optional test) and verified this operation was successful.
+- Connectivity Testing: The test verified that the model runner was accessible at a specific URL and provided the correct configuration URL to use in the application.
 
-This project is a full-stack GenAI chat application that showcases how to build a Generative AI interface with a React frontend and Go backend, connected to Llama 3.2 running on Ollama.
+This type of testing is crucial for GenAI applications because it ensures that:
 
-## Two Methods
+- The infrastructure components (containers, networking) work correctly
+- The model service is properly configured and responding
+- Basic model operations (like model loading/creation) function as expected
 
-There are two ways you can use Model Runner:
+Why testcontainers? The approach implemented with Testcontainers is particularly valuable for GenAI testing because it provides an isolated, reproducible environment that can be used across development, testing, and CI/CD workflows, ensuring consistent behavior of your GenAI application regardless of where it runs.
 
-- Using Internal DNS
-- Using TCP
 
-
-### Using Internal DNS
-
-This menthods points to the same Model Runner (llama.cpp engine) but through different connection method. 
-It uses the internal Docker DNS resolution (model-runner.docker.internal)
-
-
-
-#### Architecture
-
-The application consists of three main components:
-
-1. **Frontend**: React TypeScript application providing a responsive chat interface
-2. **Backend**: Go server that handles API requests and connects to the LLM
-3. **Model Runner**: Ollama service running the Llama 3.2 (1B parameter) model
+## Getting Started
 
 ```
-┌─────────────┐     ┌─────────────┐     ┌─────────────┐
-│   Frontend  │ >>> │   Backend   │ >>> │    Ollama   │
-│  (React/TS) │     │    (Go)     │     │  (Llama 3.2)│
-└─────────────┘     └─────────────┘     └─────────────┘
-      :3000              :8080              :11434
+chmod +x setup-tc.sh
 ```
 
-##### Features
-
-- Real-time chat interface with message history
-- Streaming AI responses (tokens appear as they're generated)
-- Dockerized deployment for easy setup
-- Local LLM integration (no cloud API dependencies)
-- Cross-origin resource sharing (CORS) enabled
-- Comprehensive integration tests using Testcontainers
-
-##### Prerequisites
 
-- Docker and Docker Compose
-- Git
-- Go 1.19 or higher
+## Running the script
 
-##### Quick Start
-
-1. Clone this repository:
-   ```bash
-   git clone https://github.com/ajeetraina/genai-app-demo.git
-   cd genai-app-demo
-   ```
-
-2. Start the application using Docker Compose:
-   ```bash
-   docker compose up -d -build
-   ```
-
-3. Access the frontend at [http://localhost:3000](http://localhost:3000)
-
-##### Development Setup
-
-### Frontend
-
-The frontend is a React TypeScript application using Vite:
-
-```bash
-cd frontend
-npm install
-npm run dev
 ```
-
-### Backend
-
-The Go backend can be run directly:
-
-```bash
-go mod download
-go run main.go
+./setup-tc.sh
 ```
 
-Make sure to set the environment variables in `backend.env` or provide them directly.
-
-
-## Using TCP 
-
-This menthods points to the same Model Runner (`llama.cpp engine`) but through different connection method. 
-It uses the host-side TCP support via `host.docker.internal:12434`
-
-
-
-
-## Testing
-
-The application includes comprehensive integration tests using Testcontainers in Go.
+You will see the following result:
 
-### Running Tests
-
-```bash
-# Run all tests
-cd tests
-go test -v ./integration
-
-# Run specific test categories
-go test -v ./integration -run TestGenAIAppIntegration    # API tests
-go test -v ./integration -run TestFrontendIntegration    # UI tests
-go test -v ./integration -run TestGenAIQuality           # Quality tests
-go test -v ./integration -run TestGenAIPerformance       # Performance tests
-
-# Run tests in short mode (faster)
-go test -v ./integration -short
-
-# Run tests with Docker Compose instead of Testcontainers
-export USE_DOCKER_COMPOSE=true
-go test -v ./integration -run TestWithDockerCompose
 ```
-
-Alternatively, you can use the provided Makefile:
-
-```bash
-# Run all tests
-make -C tests test
-
-# Run specific test categories
-make -C tests test-api
-make -C tests test-frontend
-make -C tests test-quality
-make -C tests test-performance
-
-# Clean up test resources
-make -C tests clean
+./setup-tc.sh
+=== RUN   TestModelRunnerIntegration
+2025/03/22 11:30:01 github.com/testcontainers/testcontainers-go - Connected to docker:
+  Server Version: 28.0.2 (via Testcontainers Desktop 1.18.1)
+  API Version: 1.43
+  Operating System: Docker Desktop
+  Total Memory: 9937 MB
+  Resolved Docker Host: tcp://127.0.0.1:49295
+  Resolved Docker Socket Path: /var/run/docker.sock
+  Test SessionID: 2cfa4d78c19d284c11acd58ccc7793aefda24b4dccf93701d9895a3d6e080814
+  Test ProcessID: 6b7d8317-5c2a-42e3-a1a2-5a490caafa13
+2025/03/22 11:30:01 🐳 Creating container for image testcontainers/ryuk:0.6.0
+2025/03/22 11:30:01 ✅ Container created: c4b4712ee09f
+2025/03/22 11:30:01 🐳 Starting container: c4b4712ee09f
+2025/03/22 11:30:01 ✅ Container started: c4b4712ee09f
+2025/03/22 11:30:01 🚧 Waiting for container id c4b4712ee09f image: testcontainers/ryuk:0.6.0. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
+2025/03/22 11:30:01 🐳 Creating container for image alpine/socat
+2025/03/22 11:30:01 ✅ Container created: 89fae4dea4e4
+2025/03/22 11:30:01 🐳 Starting container: 89fae4dea4e4
+2025/03/22 11:30:01 ✅ Container started: 89fae4dea4e4
+2025/03/22 11:30:01 🚧 Waiting for container id 89fae4dea4e4 image: alpine/socat. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
+    model_runner_test.go:64: Testing GET /models endpoint...
+    model_runner_test.go:85: Available models: 0 found
+    model_runner_test.go:96: Testing /engines endpoint...
+    model_runner_test.go:108: Engines endpoint response: Docker Model Runner
+
+        The service is running.
+    model_runner_test.go:130: Attempting to create model (optional test)...
+    model_runner_test.go:148: Successfully created model
+    model_runner_test.go:159: Model Runner test completed successfully!
+    model_runner_test.go:160: Model Runner is accessible via: http://127.0.0.1:55536
+    model_runner_test.go:161: Use this URL in your application config: http://127.0.0.1:55536/engines/llama.cpp/v1
+2025/03/22 11:30:08 🐳 Terminating container: 89fae4dea4e4
+2025/03/22 11:30:09 🚫 Container terminated: 89fae4dea4e4
+--- PASS: TestModelRunnerIntegration (7.80s)
+PASS
+ok  	github.com/ajeetraina/genai-app-demo/tests/integration	(cached)
 ```
 
-## Configuration
 
-The backend connects to the LLM service using environment variables defined in `backend.env`:
 
-- `BASE_URL`: URL for the model runner
-- `MODEL`: Model identifier to use
-- `API_KEY`: API key for authentication (defaults to "ollama")
+Here's a breakdown of what's happening:
 
-## Deployment
+- Test Initialization:
+  - The test connects to your Docker environment (Docker Desktop version 28.0.2)
+  - It reports memory (9937 MB) and connection details
+  - A unique test session ID is generated to track this test run
 
-The application is configured for easy deployment using Docker Compose. See the `compose.yaml` and `ollama-ci.yaml` files for details.
+- Container Setup:
+   - Ryuk Container: First, a container using the testcontainers/ryuk:0.6.0 image is created and started. Ryuk is a cleanup service that ensures containers are removed when tests complete, even if they terminate unexpectedly.
+   - Socat Container: Then, an alpine/socat container is created and started. This container acts as a network proxy, forwarding traffic between your tests and the model runner service.
 
-## License
+- Testing Process:
+    - Models Endpoint: The test checks the /models endpoint and reports that 0 models were found.
+    - Engines Endpoint: The test verifies the /engines endpoint is responding with "Docker Model Runner The service is running."
+    - Model Creation: The test attempts to create a model and reports success.
 
-MIT
 
-## Contributing
+- Test Completion:
+   - The test passes successfully (in 7.80 seconds)
+   - The socat container is terminated
+   - The test provides important URL information:
+   - Model Runner is accessible via: http://127.0.0.1:55536
+   - Use this URL in your application config: http://127.0.0.1:55536/engines/llama.cpp/v1
 
-Contributions are welcome! Please feel free to submit a Pull Request.

TESTCONTAINERS_USAGE.md

@@ -0,0 +1,84 @@
+# Using Testcontainers for Development and Testing
+
+This branch introduces a Testcontainers-based approach for both development and testing of the GenAI App. It eliminates the need for multiple Docker Compose files by leveraging Testcontainers to manage dependencies programmatically.
+
+## Features
+
+- **Development Mode**: Automatically starts and configures dependencies when running the app in dev mode
+- **TCP Connection Support**: Uses a socat container to forward traffic to the Docker Model Runner
+- **Graceful Shutdown**: Properly cleans up resources when the application exits
+- **Integrated Testing**: Tests Model Runner API directly with Testcontainers
+
+## Development Usage
+
+To run the application in development mode with Testcontainers managing the Model Runner connection:
+
+```bash
+# Make sure Docker Model Runner is enabled in Docker Desktop
+# with "Enable host-side TCP support" checked
+
+# Run the application in development mode
+go run -tags dev .
+```
+
+This will:
+1. Start a socat container that forwards traffic to model-runner.docker.internal
+2. Configure the application to use this TCP connection
+3. Automatically clean up containers when the application exits
+
+## Testing
+
+The Model Runner integration test is located at `tests/integration/model_runner_test.go`. It tests:
+
+1. Connectivity to the Model Runner API
+2. Listing available models
+3. Creating a new model
+4. Verifying the model was added
+5. Deleting the model
+6. Verifying the model was removed
+
+To run the tests:
+
+```bash
+cd tests
+go test -v ./integration -run TestModelRunnerIntegration
+```
+
+## Benefits Over Docker Compose
+
+1. **Simplified Configuration**: No need for multiple compose files
+2. **Dynamic Port Allocation**: Avoids port conflicts
+3. **Programmatic Control**: Start/stop containers directly from Go code
+4. **Unified Approach**: Same approach for both development and testing
+5. **Automatic Cleanup**: Graceful shutdown of containers when the app exits
+
+## Implementation Details
+
+### Development Mode
+
+The file `dev_dependencies.go` is built only when the `dev` build tag is used. It uses Go's `init()` function to:
+
+- Start a socat container to forward traffic to the Docker Model Runner
+- Configure environment variables for the application
+- Set up signal handlers for graceful shutdown
+
+### Integration Testing
+
+The Model Runner test uses a similar approach to start a socat container and test the Model Runner API endpoints.
+
+## Upgrading Testcontainers
+
+This implementation currently uses Testcontainers v0.27.0. It's recommended to upgrade to v0.35.0 for the latest features and improvements. To upgrade:
+
+```bash
+cd tests
+go get github.com/testcontainers/[email protected]
+go mod tidy
+```
+
+Then in the root module:
+
+```bash
+go get github.com/testcontainers/[email protected]
+go mod tidy
+```

backend.env

@@ -1,3 +1,3 @@
-BASE_URL: http://model-runner.docker.internal/engines/llama.cpp/v1/
-MODEL: ignaciolopezluna020/llama3.2:1B
-API_KEY: ${API_KEY:-ollama}
+BASE_URL=http://localhost:12434/engines/llama.cpp/v1/
+MODEL=ignaciolopezluna020/llama3.2:1B
+API_KEY=${API_KEY:-ollama}