Ananta KV Store

Ananta is a next-generation, distributed Key-Value store built on Erlang actors for massively scalable and fault-tolerant applications. Designed to leverage the full power of Erlang/OTP, Ananta provides enterprise-grade features with exceptional performance characteristics.

📋 Table of Contents

A fundamental question arises: why develop another Key-Value Store when numerous options already exist? The primary impetus behind its creation was to transparently communicate the underlying costs and benefits of the software to end consumers. This empowers users to leverage the Key-Value Store with maximum efficiency and effectiveness, drawing significant advantages from Erlang/OTP.

• Features
• Performance
• Quick Start
• API Documentation
• Installation
• Configuration
• Usage Examples
• Cluster Management
• Monitoring
• Development
• Testing
• Contributing
• License

✨ Features

🚀 High Performance

• 146,410 requests/sec throughput
• 6.27ms average latency
• Support for 100,000+ concurrent connections
• Non-blocking I/O with Erlang actors

The high-level plan encompasses several key objectives:
Horizontal Scalability: Implement an easy and efficient method for horizontal scaling.
Cost-Effective Distributed KV Store: Utilize a cheap yet powerful distributed Key-Value store.
Large Volume Data Storage: Store a significant volume of data at a reduced cost.
Enhanced Real-time Monitoring: Provide more comprehensive real-time monitoring capabilities.

🌐 Distributed Architecture

• Horizontal scaling through clustering
• Automatic load balancing across nodes
• Node discovery and management
• Fault tolerance with supervisor trees

🔒 Data Reliability

• Write-Ahead Logging (WAL) for durability
• Data replication with quorum logic
• Consistent snapshots for backup/recovery
• LRU cache with size-based eviction

📊 Monitoring & Observability

• Real-time metrics with histogram statistics
• Performance counters for all operations
• Cluster health monitoring
• Erlang observer integration

🔧 Developer Experience

• RESTful HTTP API with comprehensive documentation
• Interactive Swagger UI for API exploration
• Client SDK generation support
• Comprehensive logging with Lager

🚀 Performance

Ananta delivers exceptional performance characteristics:

Metric	Value
Throughput	146,410 requests/sec
Average Latency	6.27ms
Max Connections	100,000+
99th Percentile	14.14ms

Detailed Latency Distribution

Running 30s test @ http://localhost:8001/kv/mykey
4 threads and 100,000 connections

Thread Stats   Avg      Stdev     Max   +/- Stdev
  Latency     6.27ms    8.43ms   1.62s    99.23%
  Req/Sec    36.95k    11.64k   68.75k    65.02%

Latency Distribution
  50%    5.81ms
  75%    7.64ms  
  90%    9.57ms
  99%   14.14ms

4,407,404 requests in 30.10s, 575.91MB read
Requests/sec: 146,410.59
Transfer/sec: 19.13MB

🏃 Quick Start

1. Start the Server

# Clone the repository
git clone https://github.com/your-org/ananta.git
cd ananta

# Build and run in shell mode
./rebar3.1 shell --apps ananta

2. Basic Operations

Store a value:

curl -X PUT http://localhost:8001/kv/user:123 \
  -H "Content-Type: application/json" \
  -d '{"name": "John Doe", "age": 30, "active": true}'

Retrieve a value:

curl -X GET http://localhost:8001/kv/user:123

Delete a value:

curl -X DELETE http://localhost:8001/kv/user:123

3. Verify Installation

# Check server health
curl -X GET http://localhost:8001/metrics

# View cluster status
curl -X GET http://localhost:8001/cluster/members

📚 API Documentation

Ananta provides comprehensive API documentation with interactive testing capabilities:

Interactive Documentation

Access the built-in Swagger UI through your running server:

# Start the server
./rebar3.1 shell --apps ananta

# Open documentation in browser
open http://localhost:8001/docs

OpenAPI Specification

• File: swagger.yaml
• Format: OpenAPI 3.0
• Coverage: All endpoints with examples and schemas

API Overview

Endpoint	Method	Description
/kv/{key}	GET, PUT, DELETE	Core key-value operations
/lb/{key}	GET	Load-balanced reads
/cluster/join	POST	Add node to cluster
/cluster/leave	POST	Remove node from cluster
/cluster/members	GET	List cluster members
/metrics	GET	Performance metrics

📖 See API_DOCS.md for detailed usage instructions

🛠 Installation

Prerequisites

Ubuntu/Debian:

# Install Erlang/OTP
wget https://packages.erlang-solutions.com/erlang-solutions_2.0_all.deb
sudo dpkg -i erlang-solutions_2.0_all.deb
sudo apt-get update
sudo apt-get install erlang

# Install build dependencies
sudo apt-get install build-essential libssl-dev

macOS:

# Using Homebrew
brew install erlang

Windows:

• Download Erlang from erlang-solutions.com
• Install Visual Studio Build Tools

Build from Source

# Clone the repository
git clone https://github.com/your-org/ananta.git
cd ananta

# Clean build (optional)
./rebar3.1 clean -a

# Development build
./rebar3.1 compile

# Release build
./rebar3.1 release

# Production build
./rebar3.1 as prod tar

⚙️ Configuration

System Configuration

Edit config/sys.config for application settings:

[
    {ananta, [
        {port, 8001},
        {max_connections, 10000},
        {cache_size, 1000000}
    ]},
    {lager, [
        {log_root, "logs"},
        {crash_log, "crash.log"}
    ]}
].

VM Arguments

Edit config/vm.args for Erlang VM settings:

-name [email protected]
-setcookie 'SomeCookie'
-heart
+K true
+A 64

High-Performance Tuning

For production environments, consider:

# System limits (in /etc/security/limits.conf)
* soft nofile 65536
* hard nofile 65536

# Kernel parameters (in /etc/sysctl.conf)
net.core.somaxconn = 32768
net.ipv4.tcp_max_syn_backlog = 16384

💡 Usage Examples

JSON Data Storage

# Store user profile
curl -X PUT http://localhost:8001/kv/user:alice \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Alice Johnson",
    "email": "[email protected]",
    "preferences": {
      "theme": "dark",
      "notifications": true
    },
    "tags": ["premium", "early-adopter"]
  }'

# Retrieve user profile
curl -X GET http://localhost:8001/kv/user:alice

Session Management

# Store session data
curl -X PUT http://localhost:8001/kv/session:abc123 \
  -H "Content-Type: application/json" \
  -d '{
    "user_id": "user:alice",
    "expires_at": "2024-01-15T10:30:00Z",
    "permissions": ["read", "write"]
  }'

Caching API Responses

# Cache API response
curl -X PUT http://localhost:8001/kv/api:weather:nyc \
  -H "Content-Type: application/json" \
  -d '{
    "temperature": 22,
    "condition": "sunny",
    "cached_at": "2024-01-10T15:45:00Z"
  }'

🌐 Cluster Management

Setting Up a Cluster

Start multiple nodes:

# Node 1
./rebar3.1 shell --apps ananta --name [email protected]

# Node 2 (in another terminal)
./rebar3.1 shell --apps ananta --name [email protected] --setcookie SameCookie

Join nodes to cluster:

# Add node2 to the cluster
curl -X POST http://localhost:8001/cluster/join \
  -H "Content-Type: text/plain" \
  -d "[email protected]"

# Verify cluster membership
curl -X GET http://localhost:8001/cluster/members

Load Balanced Operations

# Read through load balancer (automatically picks a node)
curl -X GET http://localhost:8001/lb/user:alice

📊 Monitoring

Real-time Metrics

# Get performance metrics
curl -X GET http://localhost:8001/metrics | jq .

Sample metrics output:

[
  {
    "name": "ananta_kv_get",
    "value": 15420
  },
  {
    "name": "ananta_kv_get_histogram", 
    "stats": {
      "min": 0.5,
      "max": 25.3,
      "mean": 6.27,
      "median": 5.81,
      "95": 12.5,
      "99": 14.14
    }
  }
]

Erlang Observer

# Connect to running node for detailed monitoring
erl -name [email protected] -setcookie 'SomeCookie' -run observer

🛠 Development

Development Workflow

# Start development shell
./rebar3.1 shell --apps ananta

# Run with auto-recompilation
./rebar3.1 auto --apps ananta

# Run specific module tests
./rebar3.1 eunit --module=ananta_kv_server

# Generate documentation
./rebar3.1 edoc

Code Style

We use Elvis for code style enforcement:

# Check entire project
./elvis rock

# Check specific file
./elvis rock src/ananta_kv_server.erl

Hot Code Reloading

In development shell:

% Recompile and reload module
c(ananta_kv_server).

% Reload all changed modules
l(ananta_kv_server).

🧪 Testing

Unit Testing

# Run all tests
./rebar3.1 eunit

# Run with coverage
./rebar3.1 cover

# Run specific test suite
./rebar3.1 eunit --module=ananta_kv_server_tests

Integration Testing

# Start test cluster
./rebar3.1 shell --apps ananta --name [email protected]

# Run API tests
curl -X GET http://localhost:8001/metrics

Load Testing

# Using wrk
wrk -t4 -c100 -d30s http://localhost:8001/kv/test_key

# Using Apache Bench
ab -n 10000 -c 100 http://localhost:8001/kv/test_key

🤝 Contributing

We welcome contributions! Please follow these steps:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Make your changes and ensure code style compliance: ./elvis rock
4. Add tests for new functionality
5. Update documentation including API specs if needed
6. Commit your changes: git commit -m 'Add amazing feature'
7. Push to the branch: git push origin feature/amazing-feature
8. Open a Pull Request

Development Setup

git clone https://github.com/your-org/ananta.git
cd ananta
./rebar3.1 deps
./rebar3.1 compile
./rebar3.1 shell --apps ananta

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Erlang/OTP Team - For the incredible actor-based platform
• Cowboy - High-performance HTTP server
• Folsom - Metrics collection library
• Community - For feedback and contributions

---

📧 Support

• Documentation: API_DOCS.md
• Issues: GitHub Issues
• Discussions: GitHub Discussions

Built with ❤️ using Erlang/OTP