Transform any query into comprehensive, AI-analyzed insights from across the web
xdSearch is an intelligent search engine that combines web scraping, content analysis, and AI synthesis to deliver detailed, contextual answers to complex questions. Unlike traditional search engines that return links, xdSearch processes and analyzes content to provide comprehensive responses with cost tracking and source transparency.
Instead of getting a list of links, xdSearch transforms this:
xds "How will climate change affect global agriculture in the next 20 years?"Into a comprehensive analysis covering crop yield impacts, regional variations, adaptation strategies, and economic implications - all synthesized from multiple authoritative sources with complete cost and token tracking.
xdSearch employs a sophisticated three-phase approach that mimics how a research analyst would tackle a complex question:
Phase 1: Intelligent Query Generation The system analyzes your question and generates multiple optimized search queries using AI. This ensures comprehensive coverage of the topic from different angles, rather than relying on a single search approach.
Phase 2: Smart Content Gathering & Analysis xdSearch performs concurrent web scraping with built-in resilience patterns (circuit breakers, retry logic) and intelligently chunks content while preserving context. Each piece of content is then summarized by AI, maintaining relevance to your original question.
Phase 3: Synthesis & Final Analysis All gathered information is synthesized by advanced AI models into a comprehensive, coherent response. The system automatically selects the most appropriate model based on content complexity and length.
Cost-Effective Intelligence: Typical queries cost between $0.005-$0.02, making it significantly more affordable than comparable commercial solutions while providing enterprise-grade analysis.
Cross-Platform Reliability: Native installers for Windows, macOS, and Linux with automatic dependency detection and configuration. The system handles everything from Node.js installation to API key management.
Professional Error Handling: Implements circuit breaker patterns, exponential backoff, and smart retry logic to ensure reliable operation even when individual web sources are unavailable.
Transparent Operations: Complete visibility into token usage, cost breakdown by phase, execution time, and source attribution. Every search provides detailed metrics for optimization and budgeting.
Trial Access: Get started immediately with 10 free queries. Perfect for evaluating the system's capabilities before committing to API costs.
Research & Analysis: Academic research, market analysis, competitive intelligence, and policy research where you need comprehensive understanding rather than just links.
Professional Decision Making: Strategic planning, technology evaluation, investment research, and due diligence where thorough analysis is critical.
Content Creation: Blog research, journalism, educational content, and documentation where you need authoritative, well-sourced information.
Learning & Education: Complex topic exploration, exam preparation, and knowledge building where traditional search results are too fragmented.
-
Clone the repository:
git clone https://github.com/jsle97/xdSearch.git cd xdSearch -
Install and run the application:
Linux/macOS:
./install-xds.sh
Windows:
.\install-xds.ps1
During installation, you will be prompted to configure your API keys. You can also start with a free trial (10 free queries).
xdSearch requires two types of API access:
OpenAI API (for AI analysis): Follow our detailed setup guide Google Custom Search API (for web search): Follow our detailed setup guide
The configuration process is interactive and includes validation to ensure your keys are working correctly before your first search.
xds "What are the latest developments in quantum computing applications?"xds "Compare the economic impacts of renewable energy adoption across different European countries in the past 5 years"xds "How do recent advances in CRISPR gene editing technology address previous safety concerns?"Each query provides a detailed execution summary including cost breakdown, token usage, processing time, and source count.
Launch the interactive interface for comprehensive system management:
./xds-ui # Cross-platform interactive interfaceThe interface provides navigation for configuration management, system diagnostics, cache operations, and advanced settings with both keyboard navigation and fallback numeric menus.
For high-volume usage, consider adjusting concurrent processing limits, implementing request batching, and monitoring token usage patterns to optimize costs and performance.
Every search provides transparent cost tracking:
[$] Total Estimated Cost: $0.005350
[↑] Total Input Tokens: 37971
[↓] Total Output Tokens: 3445
[∞] Total Execution Time: 42.05s
Phase Details:
[★] Phase 1 (Query Generation): $0.000234
[★] Phase 2 (Content Analysis): $0.004626
[★] Phase 3 (Final Synthesis): $0.000491
This granular tracking helps you understand cost patterns and optimize usage for your specific needs.
Local Processing: All data processing occurs on your local machine. Content is fetched, analyzed, and discarded - nothing is stored permanently unless you choose to save results.
API Key Security: Keys are stored locally in encrypted environment files. The system includes validation and secure handling practices.
No Data Collection: Unlike commercial alternatives, xdSearch doesn't track your queries, build profiles, or collect usage analytics.
The open-source version manages heavy workloads, while the enterprise edition processes thousands of URLs, analyzes PDFs, DOCs, XLS files, and images, supports extended time limits, advanced scraping techniques, and custom AI models, automatically re-evaluates results, and generates new queries if initial results are insufficient for a complete answer.
For enterprise inquiries and custom implementations, contact the development team (just me :)) through the repository issues or email.
Documentation: Comprehensive guides available in the DOC/ directory
Issues: Report bugs and request features through GitHub Issues
Discussions: Community support and feature discussions through GitHub Discussions
Email: Direct developer contact for enterprise and complex technical questions
CC BY-NC License - see LICENSE file for details.
Built with modern JavaScript, leveraging OpenAI's language models and Google's search infrastructure. Special thanks to the open-source community for tools and libraries that make this project possible.
Ready to transform how you research and analyze information? Start with the free trial and experience the power of AI-driven search synthesis.