Skip to content
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -713,3 +713,6 @@ CONTRIBUTION_GUIDE.md
.qodo
.windsurfrules
.windsurf/rules

# MCP servers
.serena
4 changes: 2 additions & 2 deletions backend/sample.env
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ DEFAULT_LOG_LEVEL="INFO"
PATH_PREFIX="api/v1"

# Django settings
DJANGO_APP_BACKEND_URL=http://frontend.unstract.localhost
DJANGO_APP_BACKEND_URL=http://frontend.unstract.localhost:8081
DJANGO_SECRET_KEY="1(xf&nc6!y7!l&!5xe&i_rx7e^m@fcut9fduv86ft=-b@2g6"

# Postgres DB envs
Expand Down Expand Up @@ -51,7 +51,7 @@ GOOGLE_OAUTH2_SECRET=
SESSION_EXPIRATION_TIME_IN_SECOND=7200

# FE Web Application Dependencies
WEB_APP_ORIGIN_URL="http://frontend.unstract.localhost"
WEB_APP_ORIGIN_URL="http://frontend.unstract.localhost:8081"

# API keys for trusted services
INTERNAL_SERVICE_API_KEY=
Expand Down
146 changes: 146 additions & 0 deletions docker/CONTAINER_RUNTIME.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Container Runtime Support (Docker & Podman)

The Unstract docker-compose configuration supports both **Docker** (default) and **Podman**.

## Socket Detection

The configuration defaults to Docker and supports Podman via environment variable:
- **Docker**: Default (`/var/run/docker.sock`) - no configuration needed
- **Podman**: Set `DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock`

## Using Docker

Docker works out of the box with no additional configuration:

```bash
VERSION=main docker-compose -f docker-compose.yaml up -d
```

The Docker socket at `/var/run/docker.sock` is used automatically.

## Using Podman

### Prerequisites

1. **Enable Podman socket** (required for Traefik to discover containers):
```bash
systemctl --user enable podman.socket
systemctl --user start podman.socket
```

2. **Verify socket is running**:
```bash
systemctl --user status podman.socket
# Should show: active (listening)
```

### Run with Podman

Set the `DOCKER_SOCKET` environment variable to point to Podman socket, then run podman-compose:

```bash
export DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
VERSION=main podman-compose -f docker-compose.yaml up -d
```

**Note**: The `DOCKER_SOCKET` environment variable must be set to use Podman instead of the default Docker socket.

## Custom Socket Path

If you need to specify a custom socket path, set the `DOCKER_SOCKET` environment variable:

```bash
# Example: Custom Docker socket location
export DOCKER_SOCKET=/custom/path/docker.sock
VERSION=main docker-compose -f docker-compose.yaml up -d

# Example: Alternative Podman socket location
export DOCKER_SOCKET=/run/user/$(id -u)/podman/podman.sock
VERSION=main podman-compose -f docker-compose.yaml up -d
```

## Troubleshooting

### Traefik shows "Cannot connect to Docker daemon"

**For Podman users**:
1. Check if Podman socket is running:
```bash
systemctl --user status podman.socket
```

2. If inactive, start it:
```bash
systemctl --user start podman.socket
```

3. Verify socket file exists:
```bash
ls -la $XDG_RUNTIME_DIR/podman/podman.sock
# Should show: srw-rw---- (socket file, not directory)
```

4. If it's a directory (wrong), remove and restart:
```bash
rmdir $XDG_RUNTIME_DIR/podman/podman.sock
systemctl --user restart podman.socket
```

**For Docker users**:
1. Check if Docker daemon is running:
```bash
systemctl status docker
```

2. Verify socket permissions:
```bash
ls -la /var/run/docker.sock
```

### Port 8081 not accessible

This is the Traefik HTTP port for Podman rootless compatibility.

1. Check if Traefik container is running:
```bash
podman ps | grep unstract-proxy
# or
docker ps | grep unstract-proxy
```

2. Check Traefik logs:
```bash
podman logs unstract-proxy
# or
docker logs unstract-proxy
```

## Socket Path Priority

The configuration uses this simple priority:

1. `$DOCKER_SOCKET` - if explicitly set (use this for Podman or custom paths)
2. `/var/run/docker.sock` - default (Docker standard socket)

**Docker**: No configuration needed - uses default socket
**Podman**: Set `export DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock`

## Technical Details

The docker-compose files use this volume mount configuration:

```yaml
volumes:
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
```

This means:
- If `DOCKER_SOCKET` is set → use that path (for Podman or custom Docker socket)
- Else → use `/var/run/docker.sock` (Docker default)

**For Podman users:**
```bash
export DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
```

This overrides the default Docker socket with the Podman socket path.
8 changes: 8 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Docker Commands

## Container Runtime Support

This project supports both **Docker** and **Podman**. See [CONTAINER_RUNTIME.md](CONTAINER_RUNTIME.md) for detailed information.

**Quick Start**:
- **Docker**: Use `docker compose` commands as shown below
- **Podman**: Use `podman-compose` commands AND enable the socket: `systemctl --user enable --now podman.socket`

## Docker Build

```bash
Expand Down
14 changes: 8 additions & 6 deletions docker/docker-compose-dev-essentials.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,13 +77,15 @@ services:
--providers.docker=true --providers.docker.network=unstract-network
--providers.file.filename=/proxy_overrides.yaml --providers.file.watch=true
ports:
# The HTTP port
- "80:80"
# The HTTP port (changed to 8081 for rootless Podman compatibility)
- "8081:80"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hari-kuriakose why do we need to make this change exactly?

# The Web UI (enabled by --api.insecure=true)
- "8080:8080"
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
# Socket mount for container runtime (defaults to Docker)
# Docker: /var/run/docker.sock (default)
# Podman: Set DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
# Proxy overrides for components run directly in Docker host
- ./proxy_overrides.yaml:/proxy_overrides.yaml
# Since any proxy overrides need to point to Docker host for relevant routes.
Expand Down Expand Up @@ -142,8 +144,8 @@ services:
env_file:
- ./essentials.env
ports:
- "5672:5672" # AMQP port
- "15672:15672" # Management UI port
- "5672:5672" # AMQP port
- "15672:15672" # Management UI port
volumes:
- rabbitmq_data:/var/lib/rabbitmq

Expand Down
18 changes: 10 additions & 8 deletions docker/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ services:
- ./workflow_data:/data
- ${TOOL_REGISTRY_CONFIG_SRC_PATH}:/data/tool_registry_config


# Celery worker for managing logs and periodic tasks
worker-logging:
image: unstract/backend:${VERSION}
Expand Down Expand Up @@ -153,8 +152,8 @@ services:
- ../backend/.env
- ./essentials.env
depends_on:
- db
- rabbitmq
- db
- rabbitmq
environment:
- ENVIRONMENT=development
- APPLICATION_NAME=unstract-celery-beat
Expand All @@ -165,7 +164,7 @@ services:
container_name: unstract-frontend
restart: unless-stopped
ports:
- "3000:80"
- "3000:8080"
depends_on:
- backend
- reverse-proxy
Expand All @@ -174,6 +173,7 @@ services:
labels:
- traefik.enable=true
- traefik.http.routers.frontend.rule=Host(`frontend.unstract.localhost`) && !PathPrefix(`/api/v1`, `/deployment`)
- traefik.http.services.frontend.loadbalancer.server.port=8080
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hari-kuriakose is this config necessary? looking at our existing backend configuration I can't see such a configuration


platform-service:
image: unstract/platform-service:${VERSION}
Expand Down Expand Up @@ -231,8 +231,10 @@ services:
- ../runner/.env
volumes:
- ./workflow_data:/data
# Docker socket bind mount to spawn tool containers
- /var/run/docker.sock:/var/run/docker.sock
# Socket mount for container runtime (defaults to Docker)
# Docker: /var/run/docker.sock (default)
# Podman: Set DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
depends_on:
- redis
- rabbitmq
Expand Down Expand Up @@ -309,8 +311,8 @@ services:
image: unstract/worker-unified:${VERSION}
container_name: unstract-worker-file-processing-v2
restart: unless-stopped
# command: ["file-processing"]
command: [".venv/bin/celery", "-A", "worker", "worker", "--queues=file_processing,api_file_processing,file_processing_priority", "--loglevel=INFO", "--pool=prefork", "--concurrency=4", "--prefetch-multiplier=1", "--without-gossip", "--without-mingle", "--without-heartbeat"]
entrypoint: .venv/bin/celery
command: "-A worker worker --queues=file_processing,api_file_processing,file_processing_priority --loglevel=INFO --pool=prefork --concurrency=4 --prefetch-multiplier=1 --without-gossip --without-mingle --without-heartbeat"
ports:
- "8087:8082"
env_file:
Expand Down
2 changes: 1 addition & 1 deletion docker/dockerfiles/frontend.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ RUN mkdir -p /usr/share/nginx/html/config && \
COPY ../frontend/generate-runtime-config.sh /docker-entrypoint.d/40-env.sh
RUN chmod +x /docker-entrypoint.d/40-env.sh

EXPOSE 80
EXPOSE 8080

USER nginx

Expand Down
2 changes: 1 addition & 1 deletion frontend/nginx.conf
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ http {
scgi_temp_path /tmp/scgi_temp 1 2;

server {
listen 80;
listen 8080;
root /usr/share/nginx/html;
include /etc/nginx/mime.types;

Expand Down
2 changes: 1 addition & 1 deletion frontend/sample.env
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
REACT_APP_BACKEND_URL=http://localhost:8000
REACT_APP_BACKEND_URL=http://frontend.unstract.localhost:8081

# For development
NODE_ENV=development
Expand Down