Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -713,3 +713,6 @@ CONTRIBUTION_GUIDE.md
.qodo
.windsurfrules
.windsurf/rules

# MCP servers
.serena
4 changes: 2 additions & 2 deletions backend/sample.env
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ DEFAULT_LOG_LEVEL="INFO"
PATH_PREFIX="api/v1"

# Django settings
DJANGO_APP_BACKEND_URL=http://frontend.unstract.localhost
DJANGO_APP_BACKEND_URL=http://frontend.unstract.localhost:8081
DJANGO_SECRET_KEY="1(xf&nc6!y7!l&!5xe&i_rx7e^m@fcut9fduv86ft=-b@2g6"

# Postgres DB envs
Expand Down Expand Up @@ -51,7 +51,7 @@ GOOGLE_OAUTH2_SECRET=
SESSION_EXPIRATION_TIME_IN_SECOND=7200

# FE Web Application Dependencies
WEB_APP_ORIGIN_URL="http://frontend.unstract.localhost"
WEB_APP_ORIGIN_URL="http://frontend.unstract.localhost:8081"

# API keys for trusted services
INTERNAL_SERVICE_API_KEY=
Expand Down
146 changes: 146 additions & 0 deletions docker/CONTAINER_RUNTIME.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
# Container Runtime Support (Docker & Podman)

The Unstract docker-compose configuration supports both **Docker** (default) and **Podman**.

## Socket Detection

The configuration defaults to Docker and supports Podman via environment variable:
- **Docker**: Default (`/var/run/docker.sock`) - no configuration needed
- **Podman**: Set `DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock`

## Using Docker

Docker works out of the box with no additional configuration:

```bash
VERSION=main docker-compose -f docker-compose.yaml up -d
```

The Docker socket at `/var/run/docker.sock` is used automatically.

## Using Podman

### Prerequisites

1. **Enable Podman socket** (required for Traefik to discover containers):
```bash
systemctl --user enable podman.socket
systemctl --user start podman.socket
```

2. **Verify socket is running**:
```bash
systemctl --user status podman.socket
# Should show: active (listening)
```

### Run with Podman

Set the `DOCKER_SOCKET` environment variable to point to Podman socket, then run podman-compose:

```bash
export DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
VERSION=main podman-compose -f docker-compose.yaml up -d
```

**Note**: The `DOCKER_SOCKET` environment variable must be set to use Podman instead of the default Docker socket.

## Custom Socket Path

If you need to specify a custom socket path, set the `DOCKER_SOCKET` environment variable:

```bash
# Example: Custom Docker socket location
export DOCKER_SOCKET=/custom/path/docker.sock
VERSION=main docker-compose -f docker-compose.yaml up -d

# Example: Alternative Podman socket location
export DOCKER_SOCKET=/run/user/$(id -u)/podman/podman.sock
VERSION=main podman-compose -f docker-compose.yaml up -d
```

## Troubleshooting

### Traefik shows "Cannot connect to Docker daemon"

**For Podman users**:
1. Check if Podman socket is running:
```bash
systemctl --user status podman.socket
```

2. If inactive, start it:
```bash
systemctl --user start podman.socket
```

3. Verify socket file exists:
```bash
ls -la $XDG_RUNTIME_DIR/podman/podman.sock
# Should show: srw-rw---- (socket file, not directory)
```

4. If it's a directory (wrong), remove and restart:
```bash
rmdir $XDG_RUNTIME_DIR/podman/podman.sock
systemctl --user restart podman.socket
```

**For Docker users**:
1. Check if Docker daemon is running:
```bash
systemctl status docker
```

2. Verify socket permissions:
```bash
ls -la /var/run/docker.sock
```

### Port 8081 not accessible

This is the Traefik HTTP port for Podman rootless compatibility.

1. Check if Traefik container is running:
```bash
podman ps | grep unstract-proxy
# or
docker ps | grep unstract-proxy
```

2. Check Traefik logs:
```bash
podman logs unstract-proxy
# or
docker logs unstract-proxy
```

## Socket Path Priority

The configuration uses this simple priority:

1. `$DOCKER_SOCKET` - if explicitly set (use this for Podman or custom paths)
2. `/var/run/docker.sock` - default (Docker standard socket)

**Docker**: No configuration needed - uses default socket
**Podman**: Set `export DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock`

## Technical Details

The docker-compose files use this volume mount configuration:

```yaml
volumes:
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
```

This means:
- If `DOCKER_SOCKET` is set → use that path (for Podman or custom Docker socket)
- Else → use `/var/run/docker.sock` (Docker default)

**For Podman users:**
```bash
export DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
```

This overrides the default Docker socket with the Podman socket path.
8 changes: 8 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# Docker Commands

## Container Runtime Support

This project supports both **Docker** and **Podman**. See [CONTAINER_RUNTIME.md](CONTAINER_RUNTIME.md) for detailed information.

**Quick Start**:
- **Docker**: Use `docker compose` commands as shown below
- **Podman**: Use `podman-compose` commands AND enable the socket: `systemctl --user enable --now podman.socket`

## Docker Build

```bash
Expand Down
14 changes: 8 additions & 6 deletions docker/docker-compose-dev-essentials.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -77,13 +77,15 @@ services:
--providers.docker=true --providers.docker.network=unstract-network
--providers.file.filename=/proxy_overrides.yaml --providers.file.watch=true
ports:
# The HTTP port
- "80:80"
# The HTTP port (changed to 8081 for rootless Podman compatibility)
- "8081:80"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hari-kuriakose why do we need to make this change exactly?

# The Web UI (enabled by --api.insecure=true)
- "8080:8080"
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
# Socket mount for container runtime (defaults to Docker)
# Docker: /var/run/docker.sock (default)
# Podman: Set DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
# Proxy overrides for components run directly in Docker host
- ./proxy_overrides.yaml:/proxy_overrides.yaml
# Since any proxy overrides need to point to Docker host for relevant routes.
Expand Down Expand Up @@ -142,8 +144,8 @@ services:
env_file:
- ./essentials.env
ports:
- "5672:5672" # AMQP port
- "15672:15672" # Management UI port
- "5672:5672" # AMQP port
- "15672:15672" # Management UI port
volumes:
- rabbitmq_data:/var/lib/rabbitmq

Expand Down
18 changes: 10 additions & 8 deletions docker/docker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ services:
- ./workflow_data:/data
- ${TOOL_REGISTRY_CONFIG_SRC_PATH}:/data/tool_registry_config


# Celery worker for managing logs and periodic tasks
worker-logging:
image: unstract/backend:${VERSION}
Expand Down Expand Up @@ -154,8 +153,8 @@ services:
- ../backend/.env
- ./essentials.env
depends_on:
- db
- rabbitmq
- db
- rabbitmq
environment:
- ENVIRONMENT=development
- APPLICATION_NAME=unstract-celery-beat
Expand All @@ -166,7 +165,7 @@ services:
container_name: unstract-frontend
restart: unless-stopped
ports:
- "3000:80"
- "3000:8080"
depends_on:
- backend
- reverse-proxy
Expand All @@ -175,6 +174,7 @@ services:
labels:
- traefik.enable=true
- traefik.http.routers.frontend.rule=Host(`frontend.unstract.localhost`) && !PathPrefix(`/api/v1`, `/deployment`)
- traefik.http.services.frontend.loadbalancer.server.port=8080
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hari-kuriakose is this config necessary? looking at our existing backend configuration I can't see such a configuration


platform-service:
image: unstract/platform-service:${VERSION}
Expand Down Expand Up @@ -232,8 +232,10 @@ services:
- ../runner/.env
volumes:
- ./workflow_data:/data
# Docker socket bind mount to spawn tool containers
- /var/run/docker.sock:/var/run/docker.sock
# Socket mount for container runtime (defaults to Docker)
# Docker: /var/run/docker.sock (default)
# Podman: Set DOCKER_SOCKET=${XDG_RUNTIME_DIR}/podman/podman.sock
- ${DOCKER_SOCKET:-/var/run/docker.sock}:/var/run/docker.sock
depends_on:
- redis
- rabbitmq
Expand Down Expand Up @@ -310,8 +312,8 @@ services:
image: unstract/worker-unified:${VERSION}
container_name: unstract-worker-file-processing-v2
restart: unless-stopped
# command: ["file-processing"]
command: [".venv/bin/celery", "-A", "worker", "worker", "--queues=file_processing,api_file_processing,file_processing_priority", "--loglevel=INFO", "--pool=prefork", "--concurrency=4", "--prefetch-multiplier=1", "--without-gossip", "--without-mingle", "--without-heartbeat"]
entrypoint: .venv/bin/celery
command: "-A worker worker --queues=file_processing,api_file_processing,file_processing_priority --loglevel=INFO --pool=prefork --concurrency=4 --prefetch-multiplier=1 --without-gossip --without-mingle --without-heartbeat"
ports:
- "8087:8082"
env_file:
Expand Down
2 changes: 1 addition & 1 deletion docker/dockerfiles/frontend.Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ RUN mkdir -p /usr/share/nginx/html/config && \
COPY ../frontend/generate-runtime-config.sh /docker-entrypoint.d/40-env.sh
RUN chmod +x /docker-entrypoint.d/40-env.sh

EXPOSE 80
EXPOSE 8080

USER nginx

Expand Down
2 changes: 1 addition & 1 deletion frontend/nginx.conf
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ http {
scgi_temp_path /tmp/scgi_temp 1 2;

server {
listen 80;
listen 8080;
root /usr/share/nginx/html;
include /etc/nginx/mime.types;

Expand Down
2 changes: 1 addition & 1 deletion frontend/sample.env
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
REACT_APP_BACKEND_URL=http://localhost:8000
REACT_APP_BACKEND_URL=http://frontend.unstract.localhost:8081

# Analytics - PostHog (set to false to disable tracking)
# Default: false for local development
Expand Down