Skip to content

loonylabs-dev/searxng-proxy

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

6 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

SearXNG Proxy

Docker API SearXNG Tests Health

A simple proxy server for SearXNG meta-search engine with authentication, designed to provide secure access to privacy-respecting search.

πŸ“‹ Table of Contents

✨ Features

  • API Key Authentication: Secure access control via Bearer tokens
  • Cloudflare Tunnel Integration: Built-in support for secure external access
  • Docker Support: Complete containerized setup with Docker Compose
  • Health Check Endpoints: Monitor proxy status (authenticated + unauthenticated)
  • Multi-Engine Search: Aggregates results from Google, Bing, DuckDuckGo, Qwant, Yahoo, Brave
  • Flexible Configuration: Environment-based configuration
  • Comprehensive Tests: Unit and integration tests with Jest
  • Unlimited Searches: No rate limits (self-hosted)
  • JSON & HTML Output: Support for both JSON API and HTML responses

πŸš€ Quick Start

πŸ“‹ Prerequisites
  • Docker and Docker Compose
  • Node.js 18+ (for local development)
  • Cloudflare account (optional, for tunnel)

🐳 Docker Deployment (Recommended)

  1. Clone the repository:

    git clone https://github.com/loonylabs-dev/searxng-proxy.git
    cd searxng-proxy
  2. Set up environment:

    cp .env.example .env
    # Edit .env with your API key
  3. Generate API Key:

    # Generate a secure API key
    openssl rand -base64 32
    
    # Add to .env:
    # API_KEY=your_generated_key_here
  4. Start services:

    docker-compose up -d
  5. Verify:

    # Check services
    docker-compose ps
    
    # Test local endpoint (requires port mapping, see Configuration)
    curl -H "Authorization: Bearer YOUR_API_KEY" \
      "http://localhost:3000/search?q=test&format=json"
πŸ’» Local Development
  1. Install dependencies:

    npm install
  2. Set up environment:

    cp .env.example .env
    # Edit .env with your configuration (use SEARXNG_URL=http://localhost:8080)
  3. Start SearXNG (Docker):

    docker run -d -p 8080:8080 -v ./searxng:/etc/searxng searxng/searxng:latest
  4. Start the proxy:

    npm run dev          # Development mode
    # or
    npm run build && npm start  # Production mode

πŸ”Œ API Usage

The proxy provides secure access to SearXNG with API key authentication.

πŸ” Authentication

All requests (except /healthz) require a Bearer token:

Authorization: Bearer YOUR_API_KEY

πŸ“ Endpoints

GET /search

Search with SearXNG.

Parameters:

  • q (required) - Search query
  • format (optional) - Response format: json (default) or html
  • engines (optional) - Specific engines to use
  • Additional SearXNG parameters (language, time_range, etc.)

Example:

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://search.loonylabs.dev/search?q=typescript+best+practices&format=json"

Response (JSON):

{
  "query": "typescript best practices",
  "number_of_results": 42,
  "results": [
    {
      "title": "TypeScript Best Practices",
      "url": "https://example.com/typescript-best-practices",
      "content": "Learn the best practices for TypeScript...",
      "engine": "google"
    }
  ]
}

GET /health

Health check (requires authentication).

curl -H "Authorization: Bearer YOUR_API_KEY" \
  "https://search.loonylabs.dev/health"

Response:

{
  "status": "ok",
  "service": "searxng-proxy"
}

GET /healthz

Health check (no authentication required - for Cloudflare monitoring).

curl "https://search.loonylabs.dev/healthz"

Response:

{
  "status": "ok",
  "service": "searxng-proxy"
}

🎯 Integration Example

// searxng-client.ts
import fetch from 'node-fetch';

const SEARXNG_URL = 'https://search.loonylabs.dev';
const SEARXNG_API_KEY = process.env.SEARXNG_API_KEY;

export async function search(query: string, maxResults = 10) {
  const response = await fetch(
    `${SEARXNG_URL}/search?q=${encodeURIComponent(query)}&format=json`,
    {
      headers: {
        'Authorization': `Bearer ${SEARXNG_API_KEY}`,
      },
    }
  );

  if (!response.ok) {
    throw new Error(`Search failed: ${response.statusText}`);
  }

  const data = await response.json();
  return data.results.slice(0, maxResults);
}

Usage:

const results = await search('machine learning tutorials', 10);
console.log(results);

☁️ Cloudflare Tunnel Setup

For secure external access:

  1. Set up Cloudflare Tunnel:

    # Install cloudflared and create a tunnel
    cloudflared tunnel create searxng-proxy
  2. Configure tunnel:

    cp cloudflare/config.example.yml cloudflare/config.yml
    # Edit config.yml with your tunnel ID and domain
  3. Add tunnel credentials: Place your tunnel credentials JSON file in cloudflare/

  4. Configure DNS:

    cloudflared tunnel route dns searxng-proxy search.loonylabs.dev
  5. The tunnel will automatically start with Docker Compose

See cloudflare/README.md for detailed setup instructions.

βš™οΈ Configuration

πŸ”§ Environment Variables

Create .env from .env.example:

Variable Default Description
API_KEY Required Authentication key for API access
SEARXNG_URL http://searxng:8080 (Docker)
http://localhost:8080 (local)
SearXNG server URL
PORT 3000 Proxy server port (local dev only)
PROXY_URL - Proxy URL for integration tests (optional)

Example .env:

API_KEY=your_secret_api_key_here
SEARXNG_URL=http://searxng:8080

# For integration tests (optional)
PROXY_URL=https://search.loonylabs.dev
πŸ“‹ SearXNG Settings

Edit searxng/settings.yml to customize:

  • Search engines
  • UI theme
  • Rate limiting
  • Result formatting
  • Language preferences

See SearXNG Documentation for all options.

🐳 Docker Configuration

The Docker setup includes:

  • SearXNG container: Meta-search engine (internal only)
  • Proxy container: Express server with API key authentication
  • Cloudflared container: Cloudflare Tunnel client (optional)

Architecture:

Internet (HTTPS)
    ↓
Cloudflare Tunnel
    ↓
Proxy (Express + API Key Auth)
    ↓
SearXNG (Meta-Search Engine)

Port Exposure: By default, no ports are exposed to the host for security. Access is via Cloudflare Tunnel.

To expose port 3000 for local testing, add to docker-compose.yml:

proxy:
  ports:
    - "3000:3000"  # Add this line

πŸ§ͺ Running Tests

The project includes comprehensive unit and integration tests.

Quick Test Commands

# Run all tests
npm test

# Run only unit tests (no Docker required)
npm run test:unit

# Run only integration tests (Docker required)
npm run test:integration

# Run tests in watch mode
npm run test:unit:watch

# Run tests with coverage
npm run test:unit:coverage

Test Structure

/tests
β”œβ”€β”€ /unit              # Unit tests (no Docker required)
β”‚   └── /api           # API endpoint tests
β”‚       β”œβ”€β”€ health.test.ts      # Health endpoint tests (9 tests)
β”‚       └── search.test.ts      # Search logic tests (21 tests)
β”œβ”€β”€ /integration       # Integration tests (Docker required)
β”‚   └── searxng-integration.test.ts  # End-to-end search tests (17 tests)
└── README.md          # Detailed testing documentation

Running Integration Tests

Integration tests require running Docker containers:

# 1. Start services
docker-compose up -d

# 2. Configure PROXY_URL in .env
# Option A: Use Cloudflare Tunnel URL (recommended)
PROXY_URL=https://search.loonylabs.dev

# Option B: Use localhost (requires port mapping)
# Add "ports: - 3000:3000" to proxy service in docker-compose.yml
PROXY_URL=http://localhost:3000

# 3. Run integration tests
npm run test:integration

Test Coverage:

  • βœ… 47+ tests across unit and integration suites
  • βœ… Health check endpoints (/health, /healthz)
  • βœ… Search functionality (JSON/HTML formats)
  • βœ… Authentication & authorization
  • βœ… Error handling (missing params, invalid API keys)
  • βœ… Response validation
  • βœ… Special characters & unicode support

For detailed testing documentation, see tests/README.md.

πŸ“‚ Project Structure

searxng-proxy/
β”œβ”€β”€ src/
β”‚   └── index.ts              # Express proxy server with API key auth
β”œβ”€β”€ tests/                    # Test suite
β”‚   β”œβ”€β”€ unit/                 # Unit tests
β”‚   β”‚   └── api/              # API endpoint unit tests
β”‚   β”‚       β”œβ”€β”€ health.test.ts
β”‚   β”‚       └── search.test.ts
β”‚   β”œβ”€β”€ integration/          # Integration tests
β”‚   β”‚   └── searxng-integration.test.ts
β”‚   └── README.md             # Testing documentation
β”œβ”€β”€ searxng/
β”‚   └── settings.yml          # SearXNG configuration
β”œβ”€β”€ cloudflare/
β”‚   β”œβ”€β”€ config.example.yml    # Cloudflare tunnel template
β”‚   └── README.md             # Cloudflare setup guide
β”œβ”€β”€ docker-compose.yml        # Complete stack (searxng + proxy + cloudflared)
β”œβ”€β”€ Dockerfile                # Proxy service image
β”œβ”€β”€ .env.example              # Environment variables template
β”œβ”€β”€ jest.config.js            # Jest test configuration
β”œβ”€β”€ jest.setup.js             # Jest setup file
β”œβ”€β”€ package.json              # Node.js dependencies
β”œβ”€β”€ tsconfig.json             # TypeScript configuration
└── README.md                 # This file

πŸ”’ Security Notes

  • API Key: Use a strong, randomly generated key (32+ characters)
  • HTTPS Only: Cloudflare Tunnel provides automatic HTTPS
  • Credentials: Never commit .env, config.yml, or *.json files
  • Rate Limiting: Consider adding rate limiting in production
  • Monitoring: Monitor usage via Cloudflare Analytics
  • Port Exposure: By default, no ports are exposed to host (secure)

πŸ”§ Troubleshooting

Services won't start

# Check Docker
docker --version
docker-compose --version

# Check logs
docker-compose logs

# Check specific service
docker-compose logs proxy
docker-compose logs searxng

401 Unauthorized

  • Verify API key in .env matches your request header
  • Check: Authorization: Bearer YOUR_API_KEY (space after "Bearer")
  • API key is case-sensitive
  • Ensure no extra spaces or newlines in API key

Cloudflare Tunnel not connecting

# Check cloudflared logs
docker-compose logs cloudflared

# Verify tunnel configuration
cat cloudflare/config.yml

# Test tunnel status (on Cloudflare dashboard)
# https://one.dash.cloudflare.com/

SearXNG not returning results

# Check SearXNG logs
docker-compose logs searxng

# Test SearXNG directly (inside Docker network)
docker-compose exec proxy wget -O- "http://searxng:8080/search?q=test&format=json"

# Check if SearXNG is running
docker-compose ps searxng

Integration tests fail with ECONNREFUSED

Cause: Proxy port not exposed to host (intentional for security)

Solutions:

  1. Use Cloudflare Tunnel (recommended):

    # In .env
    PROXY_URL=https://search.yourdomain.com
  2. Temporarily expose port for testing:

    # In docker-compose.yml
    proxy:
      ports:
        - "3000:3000"

    Then restart: docker-compose down && docker-compose up -d

NPM package installation fails

# Clear npm cache
npm cache clean --force

# Remove node_modules and package-lock.json
rm -rf node_modules package-lock.json

# Reinstall
npm install

πŸ“„ License

MIT License - see LICENSE file for details.

🀝 Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Add tests if applicable
  5. Run tests: npm test
  6. Submit a pull request

For questions or support, please open an issue on GitHub.


πŸ“š Similar Projects

This project follows the same pattern as:

  • ollama-proxy - Ollama API with authentication and Cloudflare Tunnel

πŸ”— Links


Made with ❀️ by loonylabs-dev

Privacy-Respecting Meta-Search API βœ…

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published