diff --git a/CHANGELOG.md b/CHANGELOG.md index 62ed13e..ac7a4f1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,23 @@ The format is based on [Keep a Changelog][keepachangelog] and adheres to [Semant ### Deprecated ### Fixed +## [0.54.0] - 2025-11-03 + +### Added +- Integrated Olric distributed cache for high-speed key-value storage and caching. +- Added new HTTP Gateway endpoints for cache operations (GET, PUT, DELETE, SCAN) via `/v1/cache/`. +- Added `olric_servers` and `olric_timeout` configuration options to the Gateway. +- Updated the automated installation script (`install-debros-network.sh`) to include Olric installation, configuration, and firewall rules (ports 3320, 3322). + +### Changed +- Refactored README for better clarity and organization, focusing on quick start and core features. + +### Deprecated + +### Removed + +### Fixed +\n ## [0.53.18] - 2025-11-03 ### Added diff --git a/Makefile b/Makefile index 22f1d5c..03a1f0b 100644 --- a/Makefile +++ b/Makefile @@ -21,7 +21,7 @@ test-e2e: .PHONY: build clean test run-node run-node2 run-node3 run-example deps tidy fmt vet lint clear-ports install-hooks -VERSION := 0.53.18 +VERSION := 0.54.0 COMMIT ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo unknown) DATE ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ) LDFLAGS := -X 'main.version=$(VERSION)' -X 'main.commit=$(COMMIT)' -X 'main.date=$(DATE)' diff --git a/README.md b/README.md index f0826fd..54fe138 100644 --- a/README.md +++ b/README.md @@ -1,966 +1,155 @@ # DeBros Network - Distributed P2P Database System -A robust, decentralized peer-to-peer network built in Go, providing distributed SQL database, key-value storage, pub/sub messaging, and resilient peer management. Designed for applications needing reliable, scalable, and secure data sharing without centralized infrastructure. - ---- +DeBros Network is a decentralized peer-to-peer data platform built in Go. It combines distributed SQL (RQLite), pub/sub messaging, and resilient peer discovery so applications can share state without central infrastructure. ## Table of Contents -- [Features](#features) -- [Architecture Overview](#architecture-overview) -- [System Requirements](#system-requirements) +- [At a Glance](#at-a-glance) - [Quick Start](#quick-start) -- [Deployment & Installation](#deployment--installation) -- [Configuration](#configuration) -- [CLI Usage](#cli-usage) +- [Components & Ports](#components--ports) +- [Configuration Cheatsheet](#configuration-cheatsheet) +- [CLI Highlights](#cli-highlights) - [HTTP Gateway](#http-gateway) -- [Development](#development) -- [Database Client (Go ORM-like)](#database-client-go-orm-like) - [Troubleshooting](#troubleshooting) -- [License](#license) +- [Resources](#resources) ---- +## At a Glance -## Features - -- **Distributed SQL Database:** RQLite-backed, Raft-consensus, ACID transactions, automatic failover. -- **Pub/Sub Messaging:** Topic-based, real-time, namespaced, automatic cleanup. -- **Peer Discovery & Management:** Nodes discover peers, bootstrap support, health monitoring. -- **Application Isolation:** Namespace-based multi-tenancy, per-app config. -- **Secure by Default:** Noise/TLS transport, peer identity, systemd hardening. -- **Simple Client API:** Lightweight Go client for apps and CLI tools. - ---- - -## Architecture Overview - -``` -┌─────────────────────────────────────────────────────────────┐ -│ DeBros Network Cluster │ -├─────────────────────────────────────────────────────────────┤ -│ Application Layer │ -│ ┌─────────────┐ ┌─────────────┐ ┌────────────────────────┐ │ -│ │ Anchat │ │ Custom App │ │ CLI Tools │ │ -│ └─────────────┘ └─────────────┘ └────────────────────────┘ │ -├─────────────────────────────────────────────────────────────┤ -│ Client API │ -│ ┌─────────────┐ ┌────────────────────────┐ │ -│ │ Database │ │ PubSub │ │ -│ │ Client │ │ Client │ │ -│ └─────────────┘ └────────────────────────┘ │ -├─────────────────────────────────────────────────────────────┤ -│ Network Node Layer │ -│ ┌─────────────┐ ┌─────────────┐ ┌────────────────────────┐ │ -│ │ Discovery │ │ PubSub │ │ Database │ │ -│ │ Manager │ │ Manager │ │ (RQLite) │ │ -│ └─────────────┘ └─────────────┘ └────────────────────────┘ │ -├─────────────────────────────────────────────────────────────┤ -│ Transport Layer │ -│ ┌─────────────┐ ┌─────────────┐ ┌────────────────────────┐ │ -│ │ LibP2P │ │ Noise/TLS │ │ RQLite │ │ -│ │ Host │ │ Encryption │ │ Database │ │ -│ └─────────────┘ └─────────────┘ └────────────────────────┘ │ -└─────────────────────────────────────────────────────────────┘ -``` - -- **Node:** Full P2P participant, runs services, handles peer discovery, database, pubsub. -- **Client:** Lightweight, connects only to bootstrap peers, consumes services, no peer discovery. - ---- - -## System Requirements - -### Software - -- **Go:** 1.21+ (recommended) -- **RQLite:** 8.x (distributed SQLite) -- **Git:** For source management -- **Make:** For build automation (recommended) - -### Hardware - -- **Minimum:** 2 CPU cores, 4GB RAM, 10GB disk, stable internet -- **Recommended:** 4+ cores, 8GB+ RAM, 50GB+ SSD, low-latency network - -### Network Ports - -- **4001:** LibP2P P2P communication -- **5001:** RQLite HTTP API -- **7001:** RQLite Raft consensus - -### Filesystem Permissions - -DeBros Network stores all configuration and data in `~/.debros/` directory. Ensure you have: - -- **Read/Write access** to your home directory (`~`) -- **Available disk space**: At least 10GB for database and logs -- **No restrictive mount options**: The home directory must not be mounted read-only -- **Unix permissions**: Standard user permissions are sufficient (no root/sudo required) - -#### Directory Structure - -DeBros automatically creates the following directory structure: - -``` -~/.debros/ -├── bootstrap.yaml # Bootstrap node config -├── node.yaml # Node config -├── gateway.yaml # Gateway config -├── bootstrap/ # Bootstrap node data (auto-created) -│ ├── rqlite/ # RQLite database files -│ │ ├── db.sqlite # Main database -│ │ ├── raft/ # Raft consensus data -│ │ └── rsnapshots/ # Raft snapshots -│ ├── peer.info # Node multiaddr (created at startup) -│ └── identity.key # Node private key (created at startup) -├── node/ # Node data (auto-created) -│ ├── rqlite/ # RQLite database files -│ ├── raft/ # Raft data -│ ├── peer.info # Node multiaddr (created at startup) -│ └── identity.key # Node private key (created at startup) -└── node2/ # Additional node configs (if running multiple) - └── rqlite/ # RQLite database files -``` - -**Files Created at Startup:** -- `identity.key` - LibP2P private key for the node (generated once, reused) -- `peer.info` - The node's multiaddr (e.g., `/ip4/0.0.0.0/tcp/4001/p2p/12D3KooW...`) - -**Automatic Creation**: The node automatically creates all necessary data directories when started. You only need to ensure: -1. `~/.debros/` is writable -2. Sufficient disk space available -3. Correct config files exist - -**Permission Check:** - -```bash -# Verify home directory is writable -touch ~/test-write && rm ~/test-write && echo "✓ Home directory is writable" - -# Check available disk space -df -h ~ -``` - -**If you get permission errors:** - -``` -Error: Failed to create/access config directory -Please ensure: - 1. Home directory is accessible - 2. You have write permissions to home directory - 3. Disk space is available -``` - -**Solution:** - -- Ensure you're not running with overly restrictive umask: `umask` should show `0022` or similar -- Check home directory permissions: `ls -ld ~` should show your user as owner -- For sandboxed/containerized environments: Ensure `/home/` is writable - ---- +- Distributed SQL backed by RQLite and Raft consensus +- Topic-based pub/sub with automatic cleanup +- Namespace isolation for multi-tenant apps +- Secure transport using libp2p plus Noise/TLS +- Lightweight Go client and CLI tooling ## Quick Start -### 1. Clone and Setup +1. Clone and build the project: -```bash -git clone https://github.com/DeBrosOfficial/network.git -cd network -``` + ```bash + git clone https://github.com/DeBrosOfficial/network.git + cd network + make build + ``` -### 2. Build All Executables +2. Generate local configuration (bootstrap, node2, node3, gateway): -```bash -make build -``` + ```bash + ./bin/network-cli config init + ``` -### 3. Generate Configuration Files +3. Launch the full development stack: -```bash -# Generate all configs (bootstrap, node2, node3, gateway) with one command -./bin/network-cli config init -``` + ```bash + make dev + ``` -This creates: -- `~/.debros/bootstrap.yaml` - Bootstrap node -- `~/.debros/node2.yaml` - Regular node 2 -- `~/.debros/node3.yaml` - Regular node 3 -- `~/.debros/gateway.yaml` - HTTP Gateway + This starts three nodes and the HTTP gateway. Stop with `Ctrl+C`. -Plus auto-generated identities for each node. +4. Validate the network from another terminal: -### 4. Start the Complete Network Stack + ```bash + ./bin/network-cli health + ./bin/network-cli peers + ./bin/network-cli pubsub publish notifications "Hello World" + ./bin/network-cli pubsub subscribe notifications 10s + ``` -```bash -make dev -``` +## Components & Ports -This starts: -- Bootstrap node (P2P: 4001, RQLite HTTP: 5001, Raft: 7001) -- Node 2 (P2P: 4002, RQLite HTTP: 5002, Raft: 7002) -- Node 3 (P2P: 4003, RQLite HTTP: 5003, Raft: 7003) -- Gateway (HTTP: 6001) +- **Bootstrap node**: P2P `4001`, RQLite HTTP `5001`, Raft `7001` +- **Additional nodes** (`node2`, `node3`): Incrementing ports (`400{2,3}`, `500{2,3}`, `700{2,3}`) +- **Gateway**: HTTP `6001` exposes REST/WebSocket APIs +- **Data directory**: `~/.debros/` stores configs, identities, and RQLite data -Logs stream to terminal. Press **Ctrl+C** to stop all processes. +Use `make dev` for the complete stack or run binaries individually with `go run ./cmd/node --config ` and `go run ./cmd/gateway --config gateway.yaml`. -### 5. Test with CLI (in another terminal) +## Configuration Cheatsheet -```bash -./bin/network-cli health -./bin/network-cli peers -./bin/network-cli pubsub publish notifications "Hello World" -./bin/network-cli pubsub subscribe notifications 10s -``` +All runtime configuration lives in `~/.debros/`. ---- +- `bootstrap.yaml`: `type: bootstrap`, blank `database.rqlite_join_address` +- `node*.yaml`: `type: node`, set `database.rqlite_join_address` (e.g. `127.0.0.1:7001`) and include the bootstrap `discovery.bootstrap_peers` +- `gateway.yaml`: configure `gateway.bootstrap_peers`, `gateway.namespace`, and optional auth flags -## Deployment & Installation +Validation reminders: -### Automated Production Install +- HTTP and Raft ports must differ +- Non-bootstrap nodes require a join address and bootstrap peers +- Bootstrap nodes cannot define a join address +- Multiaddrs must end with `/p2p/` -Run the install script for a secure, production-ready setup: +Regenerate configs any time with `./bin/network-cli config init --force`. -```bash -curl -sSL https://github.com/DeBrosOfficial/network/raw/main/scripts/install-debros-network.sh | sudo bash -``` +## CLI Highlights -**What the Script Does:** +All commands accept `--format json`, `--timeout `, and `--bootstrap `. -- Detects OS, installs Go, RQLite, dependencies -- Creates `debros` system user, secure directory structure -- Generates LibP2P identity keys -- Clones source, builds binaries -- Sets up systemd service (`debros-node`) -- Configures firewall (UFW) for required ports -- Generates YAML config in `/opt/debros/configs/node.yaml` +- **Auth** -**Directory Structure:** + ```bash + ./bin/network-cli auth login + ./bin/network-cli auth status + ./bin/network-cli auth logout + ``` -``` -/opt/debros/ -├── bin/ # Binaries -├── configs/ # YAML configs -├── keys/ # Identity keys -├── data/ # RQLite DB, storage -├── logs/ # Node logs -├── src/ # Source code -``` +- **Network** -**Service Management:** + ```bash + ./bin/network-cli health + ./bin/network-cli status + ./bin/network-cli peers + ``` -```bash -sudo systemctl status debros-node -sudo systemctl start debros-node -sudo systemctl stop debros-node -sudo systemctl restart debros-node -sudo journalctl -u debros-node.service -f -``` +- **Database** ---- + ```bash + ./bin/network-cli query "SELECT * FROM users" + ./bin/network-cli query "CREATE TABLE users (id INTEGER PRIMARY KEY)" + ./bin/network-cli transaction --file ops.json + ``` -## Configuration +- **Pub/Sub** -### Configuration Files Location + ```bash + ./bin/network-cli pubsub publish + ./bin/network-cli pubsub subscribe 30s + ./bin/network-cli pubsub topics + ``` -All configuration files are stored in `~/.debros/` for both local development and production deployments: - -- `~/.debros/node.yaml` - Node configuration -- `~/.debros/node.yaml` - Bootstrap node configuration -- `~/.debros/gateway.yaml` - Gateway configuration - -The system will **only** load config from `~/.debros/` and will error if required config files are missing. - -### Generating Configuration Files - -Use the `network-cli config init` command to generate configuration files: - -### Generate Complete Stack (Recommended) - -```bash -# Generate bootstrap, node2, node3, and gateway configs in one command -./bin/network-cli config init - -# Force regenerate (overwrites existing configs) -./bin/network-cli config init --force -``` - -This is the **recommended way** to get started with a local development network. - -### Generate Individual Configs (Advanced) - -For custom setups or production deployments, you can generate individual configs: - -#### Generate a Single Node Config - -```bash -# Generate basic node config with bootstrap peers -./bin/network-cli config init --type node --bootstrap-peers "/ip4/127.0.0.1/tcp/4001/p2p/QmXxx" - -# With custom ports -./bin/network-cli config init --type node --name node2.yaml \ - --listen-port 4002 --rqlite-http-port 5002 --rqlite-raft-port 7002 \ - --join localhost:5001 --bootstrap-peers "/ip4/127.0.0.1/tcp/4001/p2p/QmXxx" - -# Force overwrite existing config -./bin/network-cli config init --type node --force -``` - -#### Generate a Bootstrap Node Config - -```bash -# Generate bootstrap node (no join address required) -./bin/network-cli config init --type bootstrap - -# With custom ports -./bin/network-cli config init --type bootstrap --listen-port 4001 --rqlite-http-port 5001 --rqlite-raft-port 7001 -``` - -#### Generate a Gateway Config - -```bash -# Generate gateway config -./bin/network-cli config init --type gateway - -# With bootstrap peers -./bin/network-cli config init --type gateway --bootstrap-peers "/ip4/127.0.0.1/tcp/4001/p2p/QmXxx" -``` - -### Running the Network - -Once configs are generated, start the complete stack with: - -```bash -make dev -``` - -Or start individual components (in separate terminals): - -```bash -# Terminal 1 - Bootstrap node -go run ./cmd/node --config bootstrap.yaml - -# Terminal 2 - Node 2 -go run ./cmd/node --config node2.yaml - -# Terminal 3 - Node 3 -go run ./cmd/node --config node3.yaml - -# Terminal 4 - Gateway -go run ./cmd/gateway --config gateway.yaml -``` - -### Running Multiple Nodes on the Same Machine - -The default `make dev` creates a 3-node setup. For additional nodes, generate individual configs: - -```bash -# Generate additional node configs with unique ports -./bin/network-cli config init --type node --name node4.yaml \ - --listen-port 4004 --rqlite-http-port 5004 --rqlite-raft-port 7004 \ - --join localhost:5001 \ - --bootstrap-peers "/ip4/127.0.0.1/tcp/4001/p2p/" - -# Start the additional node -go run ./cmd/node --config node4.yaml -``` - -#### Key Points for Multiple Nodes - -- **Each node needs unique ports**: P2P port, RQLite HTTP port, and RQLite Raft port must all be different -- **Join address**: Non-bootstrap nodes need `rqlite_join_address` pointing to the bootstrap or an existing node (use Raft port) -- **Bootstrap peers**: All nodes need the bootstrap node's multiaddr in `discovery.bootstrap_peers` -- **Config files**: Store all configs in `~/.debros/` with different filenames -- **--config flag**: Specify which config file to load - -⚠️ **Common Mistake - Same Ports:** -If all nodes use the same ports (e.g., 5001, 7001), they will try to bind to the same addresses and fail to communicate. Verify each node has unique ports: - -```bash -# Bootstrap -grep "rqlite_port\|rqlite_raft_port" ~/.debros/bootstrap.yaml -# Should show: rqlite_port: 5001, rqlite_raft_port: 7001 - -# Node 2 -grep "rqlite_port\|rqlite_raft_port" ~/.debros/node2.yaml -# Should show: rqlite_port: 5002, rqlite_raft_port: 7002 - -# Node 3 -grep "rqlite_port\|rqlite_raft_port" ~/.debros/node3.yaml -# Should show: rqlite_port: 5003, rqlite_raft_port: 7003 -``` - -If ports are wrong, regenerate the config with `--force`: - -```bash -./bin/network-cli config init --type node --name node.yaml \ - --listen-port 4002 --rqlite-http-port 5002 --rqlite-raft-port 7002 \ - --join localhost:5001 --bootstrap-peers '' --force -``` - -### Validating Configuration - -DeBros Network performs strict validation of all configuration files at startup. This ensures invalid configurations are caught immediately rather than causing silent failures later. - -#### Validation Features - -- **Strict YAML Parsing:** Unknown configuration keys are rejected with helpful error messages -- **Format Validation:** Multiaddrs, ports, durations, and other formats are validated for correctness -- **Cross-Field Validation:** Configuration constraints (e.g., bootstrap nodes don't join clusters) are enforced -- **Aggregated Error Reporting:** All validation errors are reported together, not one-by-one - -#### Common Validation Errors - -**Missing or Invalid `node.type`** -``` -node.type: must be one of [bootstrap node]; got "invalid" -``` -Solution: Set `type: "bootstrap"` or `type: "node"` - -**Invalid Bootstrap Peer Format** -``` -discovery.bootstrap_peers[0]: invalid multiaddr; expected /ip{4,6}/.../tcp//p2p/ -discovery.bootstrap_peers[0]: missing /p2p/ component -``` -Solution: Use full multiaddr format: `/ip4/127.0.0.1/tcp/4001/p2p/12D3KooW...` - -**Port Conflicts** -``` -database.rqlite_raft_port: must differ from database.rqlite_port (5001) -``` -Solution: Use different ports for HTTP and Raft (e.g., 5001 and 7001) - -**RQLite Join Address Issues (Nodes)** -``` -database.rqlite_join_address: required for node type (non-bootstrap) -database.rqlite_join_address: invalid format; expected host:port -``` -Solution: Non-bootstrap nodes must specify where to join the cluster. Use Raft port: `127.0.0.1:7001` - -**Bootstrap Nodes Cannot Join** -``` -database.rqlite_join_address: must be empty for bootstrap type -``` -Solution: Bootstrap nodes should have `rqlite_join_address: ""` - -**Invalid Listen Addresses** -``` -node.listen_addresses[0]: invalid TCP port 99999; port must be between 1 and 65535 -``` -Solution: Use valid ports [1-65535], e.g., `/ip4/0.0.0.0/tcp/4001` - -**Unknown Configuration Keys** -``` -invalid config: yaml: unmarshal errors: - line 42: field migrations_path not found in type config.DatabaseConfig -``` -Solution: Remove unsupported keys. Supported keys are documented in the YAML Reference section above. - ---- - -## CLI Usage - -### Authentication Commands - -```bash -./bin/network-cli auth login # Authenticate with wallet -./bin/network-cli auth whoami # Show current authentication status -./bin/network-cli auth status # Show detailed authentication info -./bin/network-cli auth logout # Clear stored credentials -``` - -### Network Operations - -```bash -./bin/network-cli health # Check network health -./bin/network-cli status # Get network status -./bin/network-cli peers # List connected peers -``` - -### Database Operations - -```bash -./bin/network-cli query "SELECT * FROM table" # Execute SQL -./bin/network-cli query "CREATE TABLE users (id INTEGER)" # DDL operations -``` - -### Pub/Sub Messaging - -```bash -./bin/network-cli pubsub publish # Send message -./bin/network-cli pubsub subscribe [duration] # Listen for messages -./bin/network-cli pubsub topics # List active topics -``` - -### CLI Options - -```bash ---format json # Output in JSON format ---timeout 30s # Set operation timeout ---bootstrap # Override bootstrap peer ---production # Use production bootstrap peers -``` - -### Database Operations (Gateway REST) - -```http -POST /v1/rqlite/exec # Body: {"sql": "INSERT/UPDATE/DELETE/DDL ...", "args": [...]} -POST /v1/rqlite/find # Body: {"table":"...", "criteria":{"col":val,...}, "options":{...}} -POST /v1/rqlite/find-one # Body: same as /find, returns a single row (404 if not found) -POST /v1/rqlite/select # Body: {"table":"...", "select":[...], "where":[...], "joins":[...], "order_by":[...], "limit":N, "offset":N, "one":false} -POST /v1/rqlite/transaction # Body: {"ops":[{"kind":"exec|query","sql":"...","args":[...]}], "return_results": true} -POST /v1/rqlite/query # Body: {"sql": "SELECT ...", "args": [..]} (legacy-friendly SELECT) -GET /v1/rqlite/schema # Returns tables/views + create SQL -POST /v1/rqlite/create-table # Body: {"schema": "CREATE TABLE ..."} -POST /v1/rqlite/drop-table # Body: {"table": "table_name"} -``` - -Common workflows: - -```bash -# Exec (INSERT/UPDATE/DELETE/DDL) -curl -X POST "$GW/v1/rqlite/exec" \ - -H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \ - -d '{"sql":"INSERT INTO users(name,email) VALUES(?,?)","args":["Alice","alice@example.com"]}' - -# Find (criteria + options) -curl -X POST "$GW/v1/rqlite/find" \ - -H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \ - -d '{ - "table":"users", - "criteria":{"active":true}, - "options":{"select":["id","email"],"order_by":["created_at DESC"],"limit":25} - }' - -# Select (fluent builder via JSON) -curl -X POST "$GW/v1/rqlite/select" \ - -H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \ - -d '{ - "table":"orders o", - "select":["o.id","o.total","u.email AS user_email"], - "joins":[{"kind":"INNER","table":"users u","on":"u.id = o.user_id"}], - "where":[{"conj":"AND","expr":"o.total > ?","args":[100]}], - "order_by":["o.created_at DESC"], - "limit":10 - }' - -# Transaction (atomic batch) -curl -X POST "$GW/v1/rqlite/transaction" \ - -H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \ - -d '{ - "return_results": true, - "ops": [ - {"kind":"exec","sql":"INSERT INTO users(email) VALUES(?)","args":["bob@example.com"]}, - {"kind":"query","sql":"SELECT last_insert_rowid() AS id","args":[]} - ] - }' - -# Schema -curl "$GW/v1/rqlite/schema" -H "Authorization: Bearer $API_KEY" - -# DDL helpers -curl -X POST "$GW/v1/rqlite/create-table" -H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \ - -d '{"schema":"CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT, email TEXT)"}' -curl -X POST "$GW/v1/rqlite/drop-table" -H "Authorization: Bearer $API_KEY" -H 'Content-Type: application/json' \ - -d '{"table":"users"}' -``` - -### Authentication - -The CLI features an enhanced authentication system with explicit command support and automatic wallet detection: - -#### Explicit Authentication Commands - -Use the `auth` command to manage your credentials: - -```bash -# Authenticate with your wallet (opens browser for signature) -./bin/network-cli auth login - -# Check if you're authenticated -./bin/network-cli auth whoami - -# View detailed authentication info -./bin/network-cli auth status - -# Clear all stored credentials -./bin/network-cli auth logout -``` - -Credentials are stored securely in `~/.debros/credentials.json` with restricted file permissions (readable only by owner). - -#### Key Features - -- **Explicit Authentication:** Use `auth login` command to authenticate with your wallet -- **Automatic Authentication:** Commands that require auth (query, pubsub, etc.) automatically prompt if needed -- **Multi-Wallet Management:** Seamlessly switch between multiple wallet credentials -- **Persistent Sessions:** Wallet credentials are automatically saved and restored between sessions -- **Enhanced User Experience:** Streamlined authentication flow with better error handling and user feedback - -#### Automatic Authentication Flow - -When using operations that require authentication (query, pubsub publish/subscribe), the CLI will automatically: - -1. Check for existing valid credentials -2. Prompt for wallet authentication if needed -3. Handle signature verification -4. Persist credentials for future use - -**Example with automatic authentication:** - -```bash -# First time - will prompt for wallet authentication when needed -./bin/network-cli pubsub publish notifications "Hello World" -``` - -#### Environment Variables - -You can override the gateway URL used for authentication: - -```bash -export DEBROS_GATEWAY_URL="http://localhost:6001" -./bin/network-cli auth login -``` - ---- +Credentials live at `~/.debros/credentials.json` with user-only permissions. ## HTTP Gateway -The DeBros Network includes a powerful HTTP/WebSocket gateway that provides a modern REST API and WebSocket interface over the P2P network, featuring an enhanced authentication system with multi-wallet support. +Start locally with `make run-gateway` or `go run ./cmd/gateway --config gateway.yaml`. -### Quick Start +Environment overrides: ```bash -make run-gateway -# Or manually: -go run ./cmd/gateway -``` - -### Configuration - -The gateway can be configured via configs/gateway.yaml and environment variables (env override YAML): - -```bash -# Basic Configuration export GATEWAY_ADDR="0.0.0.0:6001" export GATEWAY_NAMESPACE="my-app" -export GATEWAY_BOOTSTRAP_PEERS="/ip4/127.0.0.1/tcp/4001/p2p/YOUR_PEER_ID" - -# Authentication Configuration +export GATEWAY_BOOTSTRAP_PEERS="/ip4/127.0.0.1/tcp/4001/p2p/" export GATEWAY_REQUIRE_AUTH=true export GATEWAY_API_KEYS="key1:namespace1,key2:namespace2" ``` -### Enhanced Authentication System +Common endpoints (see `openapi/gateway.yaml` for the full spec): -The gateway features a significantly improved authentication system with the following capabilities: - -#### Key Features - -- **Automatic Authentication:** No manual auth commands required - authentication happens automatically when needed -- **Multi-Wallet Support:** Seamlessly manage multiple wallet credentials with automatic switching -- **Persistent Sessions:** Wallet credentials are automatically saved and restored -- **Enhanced User Experience:** Streamlined authentication flow with better error handling - -#### Authentication Methods - -**Wallet-Based Authentication (Ethereum EIP-191)** - -- Uses `personal_sign` for secure wallet verification -- Supports multiple wallets with automatic detection -- Addresses are case-insensitive with normalized signature handling - -**JWT Tokens** - -- Issued by the gateway with configurable expiration -- JWKS endpoints available at `/v1/auth/jwks` and `/.well-known/jwks.json` -- Automatic refresh capability - -**API Keys** - -- Support for pre-configured API keys via `Authorization: Bearer ` or `X-API-Key` headers -- Optional namespace mapping for multi-tenant applications - -### API Endpoints - -#### Health & Status - -```http -GET /health # Basic health check -GET /v1/health # Detailed health status -GET /v1/status # Network status -GET /v1/version # Version information -``` - -#### Authentication (Public Endpoints) - -```http -POST /v1/auth/challenge # Generate wallet challenge -POST /v1/auth/verify # Verify wallet signature -POST /v1/auth/register # Register new wallet -POST /v1/auth/refresh # Refresh JWT token -POST /v1/auth/logout # Clear authentication -GET /v1/auth/whoami # Current auth status -POST /v1/auth/api-key # Generate API key (authenticated) -``` - -#### RQLite HTTP ORM Gateway (/v1/db) - -The gateway now exposes a full HTTP interface over the Go ORM-like client (see `pkg/rqlite/gateway.go`) so you can build SDKs in any language. - -- Base path: `/v1/db` -- Endpoints: - - `POST /v1/rqlite/exec` — Execute write/DDL SQL; returns `{ rows_affected, last_insert_id }` - - `POST /v1/rqlite/find` — Map-based criteria; returns `{ items: [...], count: N }` - - `POST /v1/rqlite/find-one` — Single row; 404 if not found - - `POST /v1/rqlite/select` — Fluent SELECT via JSON (joins, where, order, group, limit, offset) - - `POST /v1/rqlite/transaction` — Atomic batch of exec/query ops, optional per-op results - - `POST /v1/rqlite/query` — Arbitrary SELECT (legacy-friendly), returns `items` - - `GET /v1/rqlite/schema` — List user tables/views + create SQL - - `POST /v1/rqlite/create-table` — Convenience for DDL - - `POST /v1/rqlite/drop-table` — Safe drop (identifier validated) - -Payload examples are shown in the [Database Operations (Gateway REST)](#database-operations-gateway-rest) section. - -#### Network Operations - -```http -GET /v1/network/status # Network status -GET /v1/network/peers # Connected peers -POST /v1/network/connect # Connect to peer -POST /v1/network/disconnect # Disconnect from peer -``` - -#### Pub/Sub Messaging - -**WebSocket Interface** - -```http -GET /v1/pubsub/ws?topic= # WebSocket connection for real-time messaging -``` - -**REST Interface** - -```http -POST /v1/pubsub/publish # Publish message to topic -GET /v1/pubsub/topics # List active topics -``` - ---- - -## SDK Authoring Guide - -### Base concepts - -- OpenAPI: a machine-readable spec is available at `openapi/gateway.yaml` for SDK code generation. -- **Auth**: send `X-API-Key: ` or `Authorization: Bearer ` with every request. -- **Versioning**: all endpoints are under `/v1/`. -- **Responses**: mutations return `{status:"ok"}`; queries/lists return JSON; errors return `{ "error": "message" }` with proper HTTP status. - -### Key HTTP endpoints for SDKs - -- **Database** - - Exec: `POST /v1/rqlite/exec` `{sql, args?}` → `{rows_affected,last_insert_id}` - - Find: `POST /v1/rqlite/find` `{table, criteria, options?}` → `{items,count}` - - FindOne: `POST /v1/rqlite/find-one` `{table, criteria, options?}` → single object or 404 - - Select: `POST /v1/rqlite/select` `{table, select?, joins?, where?, order_by?, group_by?, limit?, offset?, one?}` - - Transaction: `POST /v1/rqlite/transaction` `{ops:[{kind,sql,args?}], return_results?}` - - Query: `POST /v1/rqlite/query` `{sql, args?}` → `{items,count}` - - Schema: `GET /v1/rqlite/schema` - - Create Table: `POST /v1/rqlite/create-table` `{schema}` - - Drop Table: `POST /v1/rqlite/drop-table` `{table}` -- **PubSub** - - WS Subscribe: `GET /v1/pubsub/ws?topic=` - - Publish: `POST /v1/pubsub/publish` `{topic, data_base64}` → `{status:"ok"}` - - Topics: `GET /v1/pubsub/topics` → `{topics:[...]}` - ---- +- `GET /health`, `GET /v1/status`, `GET /v1/version` +- `POST /v1/auth/challenge`, `POST /v1/auth/verify`, `POST /v1/auth/refresh` +- `POST /v1/rqlite/exec`, `POST /v1/rqlite/find`, `POST /v1/rqlite/select`, `POST /v1/rqlite/transaction` +- `GET /v1/rqlite/schema` +- `POST /v1/pubsub/publish`, `GET /v1/pubsub/topics`, `GET /v1/pubsub/ws?topic=` ## Troubleshooting -### Configuration & Permissions +- **Config directory errors**: Ensure `~/.debros/` exists, is writable, and has free disk space (`touch ~/.debros/test && rm ~/.debros/test`). +- **Port conflicts**: Inspect with `lsof -i :4001` (or other ports) and stop conflicting processes or regenerate configs with new ports. +- **Missing configs**: Run `./bin/network-cli config init` before starting nodes. +- **Cluster join issues**: Confirm the bootstrap node is running, `peer.info` multiaddr matches `bootstrap_peers`, and firewall rules allow the P2P ports. -**Error: "Failed to create/access config directory"** +## Resources -This happens when DeBros cannot access or create `~/.debros/` directory. - -**Causes:** -1. Home directory is not writable -2. Home directory doesn't exist -3. Filesystem is read-only (sandboxed/containerized environment) -4. Permission denied (running with wrong user/umask) - -**Solutions:** - -```bash -# Check home directory exists and is writable -ls -ld ~ -touch ~/test-write && rm ~/test-write - -# Check umask (should be 0022 or 0002) -umask - -# If umask is too restrictive, change it -umask 0022 - -# Check disk space -df -h ~ - -# For containerized environments, ensure /home/ is mounted with write permissions -docker run -v /home:/home --user $(id -u):$(id -g) debros-network -``` - -**Error: "Config file not found at ~/.debros/node.yaml"** - -The node requires a config file to exist before starting. - -**Solution:** - -Generate config files first: - -```bash -# Build CLI -make build - -# Generate configs -./bin/network-cli config init --type bootstrap -./bin/network-cli config init --type node --bootstrap-peers '' -./bin/network-cli config init --type gateway -``` - -### Node Startup Issues - -**Error: "node.data_dir: parent directory not writable"** - -The data directory parent is not accessible. - -**Solution:** - -Ensure `~/.debros` is writable and has at least 10GB free space: - -```bash -# Check permissions -ls -ld ~/.debros - -# Check available space -df -h ~/.debros - -# Recreate if corrupted -rm -rf ~/.debros -./bin/network-cli config init --type bootstrap -``` - -**Error: "failed to create data directory"** - -The node cannot create its data directory in `~/.debros`. - -**Causes:** -1. `~/.debros` is not writable -2. Parent directory path in config uses `~` which isn't expanded properly -3. Disk is full - -**Solutions:** - -```bash -# Check ~/.debros exists and is writable -mkdir -p ~/.debros -ls -ld ~/.debros - -# Verify data_dir in config uses ~ (e.g., ~/.debros/node) -cat ~/.debros/node.yaml | grep data_dir - -# Check disk space -df -h ~ - -# Ensure user owns ~/.debros -chown -R $(whoami) ~/.debros - -# Retry node startup -make run-node -``` - -**Error: "stat ~/.debros: no such file or directory"** - -**Port Already in Use** - -If you get "address already in use" errors: - -```bash -# Find processes using ports -lsof -i :4001 # P2P port -lsof -i :5001 # RQLite HTTP -lsof -i :7001 # RQLite Raft - -# Kill if needed -kill -9 - -# Or use different ports in config -./bin/network-cli config init --type node --listen-port 4002 --rqlite-http-port 5002 --rqlite-raft-port 7002 -``` - -### Common Configuration Errors - -**Error: "discovery.bootstrap_peers: required for node type"** - -Nodes (non-bootstrap) must specify bootstrap peers to discover the network. - -**Solution:** - -Generate node config with bootstrap peers: - -```bash -./bin/network-cli config init --type node --bootstrap-peers '/ip4/127.0.0.1/tcp/4001/p2p/12D3KooW...' -``` - -**Error: "database.rqlite_join_address: required for node type"** - -Non-bootstrap nodes must specify which node to join in the Raft cluster. - -**Solution:** - -Generate config with join address: - -```bash -./bin/network-cli config init --type node --join localhost:5001 -``` - -**Error: "database.rqlite_raft_port: must differ from database.rqlite_port"** - -HTTP and Raft ports cannot be the same. - -**Solution:** - -Use different ports (RQLite HTTP and Raft must be on different ports): - -```bash -./bin/network-cli config init --type node \ - --rqlite-http-port 5001 \ - --rqlite-raft-port 7001 -``` - -### Peer Discovery Issues - -If nodes can't find each other: - -1. **Verify bootstrap node is running:** - ```bash - ./bin/network-cli health - ./bin/network-cli peers - ``` - -2. **Check bootstrap peer multiaddr is correct:** - ```bash - cat ~/.debros/bootstrap/peer.info # On bootstrap node - # Should match value in other nodes' discovery.bootstrap_peers - ``` - -3. **Ensure all nodes have same bootstrap peers in config** - -4. **Check firewall/network:** - ```bash - # Verify P2P port is open - nc -zv 127.0.0.1 4001 - ``` - ---- - -## License \ No newline at end of file +- Go modules: `go mod tidy`, `go test ./...` +- Automation: `make build`, `make dev`, `make run-gateway`, `make lint` +- API reference: `openapi/gateway.yaml` +- Code of Conduct: [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) diff --git a/cmd/gateway/config.go b/cmd/gateway/config.go index e10763c..d8d1864 100644 --- a/cmd/gateway/config.go +++ b/cmd/gateway/config.go @@ -5,6 +5,7 @@ import ( "os" "path/filepath" "strings" + "time" "github.com/DeBrosOfficial/network/pkg/config" "github.com/DeBrosOfficial/network/pkg/gateway" @@ -57,6 +58,8 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config { EnableHTTPS bool `yaml:"enable_https"` DomainName string `yaml:"domain_name"` TLSCacheDir string `yaml:"tls_cache_dir"` + OlricServers []string `yaml:"olric_servers"` + OlricTimeout string `yaml:"olric_timeout"` } data, err := os.ReadFile(configPath) @@ -86,6 +89,8 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config { EnableHTTPS: false, DomainName: "", TLSCacheDir: "", + OlricServers: nil, + OlricTimeout: 0, } if v := strings.TrimSpace(y.ListenAddr); v != "" { @@ -125,6 +130,18 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config { } } + // Olric configuration + if len(y.OlricServers) > 0 { + cfg.OlricServers = y.OlricServers + } + if v := strings.TrimSpace(y.OlricTimeout); v != "" { + if parsed, err := time.ParseDuration(v); err == nil { + cfg.OlricTimeout = parsed + } else { + logger.ComponentWarn(logging.ComponentGeneral, "invalid olric_timeout, using default", zap.String("value", v), zap.Error(err)) + } + } + // Validate configuration if errs := cfg.ValidateConfig(); len(errs) > 0 { fmt.Fprintf(os.Stderr, "\nGateway configuration errors (%d):\n", len(errs)) diff --git a/go.mod b/go.mod index 0b1a4b6..d2cec41 100644 --- a/go.mod +++ b/go.mod @@ -11,21 +11,28 @@ require ( github.com/libp2p/go-libp2p-pubsub v0.14.2 github.com/mackerelio/go-osstat v0.2.6 github.com/multiformats/go-multiaddr v0.15.0 + github.com/olric-data/olric v0.7.0 github.com/rqlite/gorqlite v0.0.0-20250609141355-ac86a4a1c9a8 go.uber.org/zap v1.27.0 + golang.org/x/crypto v0.40.0 golang.org/x/net v0.42.0 gopkg.in/yaml.v3 v3.0.1 ) require ( + github.com/RoaringBitmap/roaring v1.9.4 // indirect + github.com/armon/go-metrics v0.4.1 // indirect github.com/benbjohnson/clock v1.3.5 // indirect github.com/beorn7/perks v1.0.1 // indirect + github.com/bits-and-blooms/bitset v1.22.0 // indirect github.com/btcsuite/btcd/btcec/v2 v2.2.0 // indirect + github.com/buraksezer/consistent v0.10.0 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/containerd/cgroups v1.1.0 // indirect github.com/coreos/go-systemd/v22 v22.5.0 // indirect github.com/davidlazar/go-crypto v0.0.0-20200604182044-b73af7476f6c // indirect github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 // indirect + github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect github.com/docker/go-units v0.5.0 // indirect github.com/elastic/gosigar v0.14.3 // indirect github.com/flynn/noise v1.1.0 // indirect @@ -33,10 +40,20 @@ require ( github.com/go-task/slim-sprig/v3 v3.0.0 // indirect github.com/godbus/dbus/v5 v5.1.0 // indirect github.com/gogo/protobuf v1.3.2 // indirect + github.com/google/btree v1.1.3 // indirect github.com/google/gopacket v1.1.19 // indirect github.com/google/pprof v0.0.0-20250208200701-d0013a598941 // indirect github.com/google/uuid v1.6.0 // indirect + github.com/hashicorp/errwrap v1.1.0 // indirect + github.com/hashicorp/go-immutable-radix v1.3.1 // indirect + github.com/hashicorp/go-metrics v0.5.4 // indirect + github.com/hashicorp/go-msgpack/v2 v2.1.3 // indirect + github.com/hashicorp/go-multierror v1.1.1 // indirect + github.com/hashicorp/go-sockaddr v1.0.7 // indirect + github.com/hashicorp/golang-lru v1.0.2 // indirect github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect + github.com/hashicorp/logutils v1.0.0 // indirect + github.com/hashicorp/memberlist v0.5.3 // indirect github.com/holiman/uint256 v1.2.4 // indirect github.com/huin/goupnp v1.3.0 // indirect github.com/ipfs/go-cid v0.5.0 // indirect @@ -60,6 +77,7 @@ require ( github.com/mikioh/tcpopt v0.0.0-20190314235656-172688c1accc // indirect github.com/minio/sha256-simd v1.0.1 // indirect github.com/mr-tron/base58 v1.2.0 // indirect + github.com/mschoch/smat v0.2.0 // indirect github.com/multiformats/go-base32 v0.1.0 // indirect github.com/multiformats/go-base36 v0.2.0 // indirect github.com/multiformats/go-multiaddr-dns v0.4.1 // indirect @@ -101,14 +119,20 @@ require ( github.com/quic-go/quic-go v0.50.1 // indirect github.com/quic-go/webtransport-go v0.8.1-0.20241018022711-4ac2c9250e66 // indirect github.com/raulk/go-watchdog v1.3.0 // indirect + github.com/redis/go-redis/v9 v9.8.0 // indirect github.com/rogpeppe/go-internal v1.13.1 // indirect + github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 // indirect github.com/spaolacci/murmur3 v1.1.0 // indirect + github.com/tidwall/btree v1.7.0 // indirect + github.com/tidwall/match v1.1.1 // indirect + github.com/tidwall/redcon v1.6.2 // indirect + github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect + github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect github.com/wlynxg/anet v0.0.5 // indirect go.uber.org/dig v1.18.0 // indirect go.uber.org/fx v1.23.0 // indirect go.uber.org/mock v0.5.0 // indirect go.uber.org/multierr v1.11.0 // indirect - golang.org/x/crypto v0.40.0 // indirect golang.org/x/exp v0.0.0-20250718183923-645b1fa84792 // indirect golang.org/x/mod v0.26.0 // indirect golang.org/x/sync v0.16.0 // indirect @@ -116,5 +140,6 @@ require ( golang.org/x/text v0.27.0 // indirect golang.org/x/tools v0.35.0 // indirect google.golang.org/protobuf v1.36.6 // indirect + gopkg.in/yaml.v2 v2.4.0 // indirect lukechampine.com/blake3 v1.4.1 // indirect ) diff --git a/go.sum b/go.sum index 33dd50c..69f9844 100644 --- a/go.sum +++ b/go.sum @@ -8,22 +8,45 @@ dmitri.shuralyov.com/service/change v0.0.0-20181023043359-a85b471d5412/go.mod h1 dmitri.shuralyov.com/state v0.0.0-20180228185332-28bcc343414c/go.mod h1:0PRwlb0D6DFvNNtx+9ybjezNCa8XF0xaYcETyp6rHWU= git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999/go.mod h1:fPE2ZNJGynbRyZ4dJvy6G277gSllfV2HJqblrnkyeyg= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= +github.com/DataDog/datadog-go v3.2.0+incompatible/go.mod h1:LButxg5PwREeZtORoXG3tL4fMGNddJ+vMq1mwgfaqoQ= +github.com/RoaringBitmap/roaring v1.9.4 h1:yhEIoH4YezLYT04s1nHehNO64EKFTop/wBhxv2QzDdQ= +github.com/RoaringBitmap/roaring v1.9.4/go.mod h1:6AXUsoIEzDTFFQCe1RbGA6uFONMhvejWj5rqITANK90= +github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/template v0.0.0-20190718012654-fb15b899a751/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc= +github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alecthomas/units v0.0.0-20190717042225-c3de453c63f4/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0= +github.com/alecthomas/units v0.0.0-20190924025748-f65c72e2690d/go.mod h1:rBZYJk541a8SKzHPHnH3zbiI+7dagKZ0cgpgrD7Fyho= github.com/anmitsu/go-shlex v0.0.0-20161002113705-648efa622239/go.mod h1:2FmKhYUyUczH0OGQWaF5ceTx0UBShxjsH6f8oGKYe2c= +github.com/armon/go-metrics v0.4.1 h1:hR91U9KYmb6bLBYLQjyM+3j+rcd/UhE+G78SFnF8gJA= +github.com/armon/go-metrics v0.4.1/go.mod h1:E6amYzXo6aW1tqzoZGT755KkbgrJsSdpwZ+3JqfkOG4= github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/benbjohnson/clock v1.3.5 h1:VvXlSJBzZpA/zum6Sj74hxwYI2DIxRWuNIoXAzHZz5o= github.com/benbjohnson/clock v1.3.5/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= +github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= +github.com/bits-and-blooms/bitset v1.12.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8= +github.com/bits-and-blooms/bitset v1.22.0 h1:Tquv9S8+SGaS3EhyA+up3FXzmkhxPGjQQCkcs2uw7w4= +github.com/bits-and-blooms/bitset v1.22.0/go.mod h1:7hO7Gc7Pp1vODcmWvKMRA9BNmbv6a/7QIWpPxHddWR8= github.com/bradfitz/go-smtpd v0.0.0-20170404230938-deb6d6237625/go.mod h1:HYsPBTaaSFSlLx/70C2HPIMNZpVV8+vt/A+FMnYP11g= +github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs= +github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c= +github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA= +github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0= github.com/btcsuite/btcd/btcec/v2 v2.2.0 h1:fzn1qaOt32TuLjFlkzYSsBC35Q3KUjT1SwPxiMSCF5k= github.com/btcsuite/btcd/btcec/v2 v2.2.0/go.mod h1:U7MHm051Al6XmscBQ0BoNydpOTsFAn707034b5nY8zU= github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1 h1:q0rUy8C/TYNBQS1+CGKw68tLOFYSNEs0TFnxxnS9+4U= github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1/go.mod h1:7SFka0XMvUgj3hfZtydOrQY2mwhPclbT2snogU7SQQc= github.com/buger/jsonparser v0.0.0-20181115193947-bf1c66bbce23/go.mod h1:bbYlZJ7hK1yFx9hf58LP0zeX7UjIGs20ufpu3evjr+s= +github.com/buraksezer/consistent v0.10.0 h1:hqBgz1PvNLC5rkWcEBVAL9dFMBWz6I0VgUCW25rrZlU= +github.com/buraksezer/consistent v0.10.0/go.mod h1:6BrVajWq7wbKZlTOUPs/XVfR8c0maujuPowduSpZqmw= +github.com/cespare/xxhash/v2 v2.1.1/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs= github.com/cilium/ebpf v0.2.0/go.mod h1:To2CFviqOWL/M0gIMsvSMlqe7em/l1ALkX1PyjrX2Qs= +github.com/circonus-labs/circonus-gometrics v2.3.1+incompatible/go.mod h1:nmEj6Dob7S7YxXgwXpfOuvO54S+tGdZdw9fuRZt25Ag= +github.com/circonus-labs/circonusllhist v0.1.3/go.mod h1:kMXHVDlOchFAehlya5ePtbp5jckzBHf4XRpQvBOLI+I= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/containerd/cgroups v0.0.0-20201119153540-4cbc285b3327/go.mod h1:ZJeTFisyysqgcCdecO57Dj79RfL0LNeGiFUqLYQRYLE= github.com/containerd/cgroups v1.1.0 h1:v8rEWFl6EoqHB+swVNjVoCJE8o3jX7e8nqBGPLaDFBM= @@ -43,6 +66,8 @@ github.com/decred/dcrd/crypto/blake256 v1.1.0 h1:zPMNGQCm0g4QTY27fOCorQW7EryeQ/U github.com/decred/dcrd/crypto/blake256 v1.1.0/go.mod h1:2OfgNZ5wDpcsFmHmCK5gZTPcCXqlm2ArzUIkw9czNJo= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0 h1:NMZiJj8QnKe1LgsbDayM4UoHwbvwDRwnI3hwNaAHRnc= github.com/decred/dcrd/dcrec/secp256k1/v4 v4.4.0/go.mod h1:ZXNYxsqcloTdSy/rNShjYzMhyjf0LaoftYK0p+A3h40= +github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78= +github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc= github.com/docker/go-units v0.4.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4= github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk= @@ -61,8 +86,15 @@ github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMo github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= github.com/gliderlabs/ssh v0.1.1/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0= github.com/go-errors/errors v1.0.1/go.mod h1:f4zRHt4oKfwPJE5k8C9vpYG+aDHdBFUsgrm6/TyX73Q= +github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-kit/kit v0.9.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as= +github.com/go-kit/log v0.1.0/go.mod h1:zbhenjAZHb184qTLMA9ZjW7ThYL0H2mk7Q6pNt4vbaY= +github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE= +github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk= +github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A= github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY= github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY= +github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY= github.com/go-task/slim-sprig/v3 v3.0.0 h1:sUs3vkvUymDpBKi3qH1YSqBQk9+9D/8M2mN1vB6EwHI= github.com/go-task/slim-sprig/v3 v3.0.0/go.mod h1:W848ghGpv3Qj3dhTPRyJypKRiqCdHZiAzKg9hl15HA8= github.com/godbus/dbus/v5 v5.0.3/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA= @@ -79,13 +111,29 @@ github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfb github.com/golang/mock v1.2.0/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= +github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= +github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= +github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= +github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= +github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= +github.com/golang/protobuf v1.4.2/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= +github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= github.com/google/btree v0.0.0-20180813153112-4030bb1f1f0c/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ= +github.com/google/btree v1.1.3 h1:CVpQJjYgC4VbzxeGVHfvZrv1ctoYCAI8vbl07Fcxlyg= +github.com/google/btree v1.1.3/go.mod h1:qOPhT0dTNdNzV6Z/lhRX0YXUafgPLFUh+gZMl761Gm4= github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= +github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= +github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.2/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.4/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= +github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU= github.com/google/go-github v17.0.0+incompatible/go.mod h1:zLgOLi98H3fifZn+44m+umXrS52loVEgC2AApnigrVQ= github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck= +github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gopacket v1.1.19 h1:ves8RnFZPGiFnTS0uPQStjwru6uO6h+nlr9j6fL7kF8= github.com/google/gopacket v1.1.19/go.mod h1:iJ8V8n6KS+z2U1A8pUwu8bW5SyEMkXJB8Yo/Vo+TKTo= github.com/google/martian v2.1.0+incompatible/go.mod h1:9I4somxYTbIHy5NJKHRl3wXiIaQGbYVAs8BPL6v8lEs= @@ -101,8 +149,33 @@ github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aN github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE= github.com/gregjones/httpcache v0.0.0-20180305231024-9cad4c3443a7/go.mod h1:FecbI9+v66THATjSRHfNgh1IVFe/9kFxbXtjV0ctIMA= github.com/grpc-ecosystem/grpc-gateway v1.5.0/go.mod h1:RSKVYQBd5MCa4OVpNdGskqpgL2+G+NZTnrVHpWWfpdw= +github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= +github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I= +github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4= +github.com/hashicorp/go-cleanhttp v0.5.0/go.mod h1:JpRdi6/HCYpAwUzNwuwqhbovhLtngrth3wmdIIUrZ80= +github.com/hashicorp/go-immutable-radix v1.0.0/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= +github.com/hashicorp/go-immutable-radix v1.3.1 h1:DKHmCUm2hRBK510BaiZlwvpD40f8bJFeZnpfm2KLowc= +github.com/hashicorp/go-immutable-radix v1.3.1/go.mod h1:0y9vanUI8NX6FsYoO3zeMjhV/C5i9g4Q3DwcSNZ4P60= +github.com/hashicorp/go-metrics v0.5.4 h1:8mmPiIJkTPPEbAiV97IxdAGNdRdaWwVap1BU6elejKY= +github.com/hashicorp/go-metrics v0.5.4/go.mod h1:CG5yz4NZ/AI/aQt9Ucm/vdBnbh7fvmv4lxZ350i+QQI= +github.com/hashicorp/go-msgpack/v2 v2.1.3 h1:cB1w4Zrk0O3jQBTcFMKqYQWRFfsSQ/TYKNyUUVyCP2c= +github.com/hashicorp/go-msgpack/v2 v2.1.3/go.mod h1:SjlwKKFnwBXvxD/I1bEcfJIBbEJ+MCUn39TxymNR5ZU= +github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo= +github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM= +github.com/hashicorp/go-retryablehttp v0.5.3/go.mod h1:9B5zBasrRhHXnJnui7y6sL7es7NDiJgTc6Er0maI1Xs= +github.com/hashicorp/go-sockaddr v1.0.7 h1:G+pTkSO01HpR5qCxg7lxfsFEZaG+C0VssTy/9dbT+Fw= +github.com/hashicorp/go-sockaddr v1.0.7/go.mod h1:FZQbEYa1pxkQ7WLpyXJ6cbjpT8q0YgQaK/JakXqGyWw= +github.com/hashicorp/go-uuid v1.0.0 h1:RS8zrF7PhGwyNPOtxSClXXj9HA8feRnJzgnI1RJCSnM= +github.com/hashicorp/go-uuid v1.0.0/go.mod h1:6SBZvOh/SIDV7/2o3Jml5SYk/TvGqwFJ/bN7x4byOro= +github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= +github.com/hashicorp/golang-lru v1.0.2 h1:dV3g9Z/unq5DpblPpw+Oqcv4dU/1omnb4Ok8iPY6p1c= +github.com/hashicorp/golang-lru v1.0.2/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k= github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM= +github.com/hashicorp/logutils v1.0.0 h1:dLEQVugN8vlakKOUE3ihGLTZJRB4j+M2cdTm/ORI65Y= +github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= +github.com/hashicorp/memberlist v0.5.3 h1:tQ1jOCypD0WvMemw/ZhhtH+PWpzcftQvgCorLu0hndk= +github.com/hashicorp/memberlist v0.5.3/go.mod h1:h60o12SZn/ua/j0B6iKAZezA4eDaGsIuPO70eOaJ6WE= github.com/holiman/uint256 v1.2.4 h1:jUc4Nk8fm9jZabQuqr2JzednajVmBpC+oiTiXZJEApU= github.com/holiman/uint256 v1.2.4/go.mod h1:EOMSn4q6Nyt9P6efbI3bueV4e1b3dGlUCXeiRV4ng7E= github.com/huin/goupnp v1.3.0 h1:UvLUlWDNpoUdYzb2TCn+MuTWtcjXKSza2n6CBdQ0xXc= @@ -116,8 +189,14 @@ github.com/jackpal/go-nat-pmp v1.0.2/go.mod h1:QPH045xvCAeXUZOxsnwmrtiCoxIr9eob+ github.com/jbenet/go-temp-err-catcher v0.1.0 h1:zpb3ZH6wIE8Shj2sKS+khgRvf7T7RABoLk/+KKHggpk= github.com/jbenet/go-temp-err-catcher v0.1.0/go.mod h1:0kJRvmDZXNMIiJirNPEYfhpPwbGVtZVWC34vc5WLsDk= github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1/go.mod h1:E0B/fFc00Y+Rasa88328GlI/XbtyysCtTHZS8h7IrBU= +github.com/jpillora/backoff v1.0.0/go.mod h1:J/6gKK9jxlEcS3zixgDgUAsiuZ7yrSoa/FX5e0EB2j4= github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU= +github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= +github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4= github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU= +github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w= +github.com/julienschmidt/httprouter v1.3.0/go.mod h1:JR6WtHb+2LUe8TCKY3cZOxFyyO8IZAc4RVcycCCAKdM= github.com/kisielk/errcheck v1.2.0/go.mod h1:/BMXB+zMLi60iA8Vv6Ksmxu/1UDYcXs4uQLJ+jE2L00= github.com/kisielk/errcheck v1.5.0/go.mod h1:pFxgyoBC7bSaBwPgfKdkLd5X25qrDl4LWUI2bnpBCr8= github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck= @@ -125,8 +204,11 @@ github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zt github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ= github.com/klauspost/cpuid/v2 v2.2.10 h1:tBs3QSyvjDyFTq3uoc/9xFpCuOsJQFNPiAhYdw2skhE= github.com/klauspost/cpuid/v2 v2.2.10/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0= +github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= +github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/koron/go-ssdp v0.0.5 h1:E1iSMxIs4WqxTbIBLtmNBeOOC+1sCIXQeqTWVnpmwhk= github.com/koron/go-ssdp v0.0.5/go.mod h1:Qm59B7hpKpDqfyRNWRNr00jGwLdXjDyZh6y7rH6VS0w= +github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc= github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo= github.com/kr/pretty v0.2.1/go.mod h1:ipq/a2n7PKx3OHsz4KJII5eveXtPO4qwEXGdVfWzfnI= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE= @@ -178,11 +260,15 @@ github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1/go.mod h1:pD8Rv github.com/minio/sha256-simd v0.1.1-0.20190913151208-6de447530771/go.mod h1:B5e1o+1/KgNmWrSQK08Y6Z1Vb5pwIktudl0J58iy0KM= github.com/minio/sha256-simd v1.0.1 h1:6kaan5IFmwTNynnKKpDHe6FWHohJOHhCPchzK49dzMM= github.com/minio/sha256-simd v1.0.1/go.mod h1:Pz6AKMiUdngCLpeTL/RJY1M9rUuPMYujV5xJjtbRSN8= +github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= +github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0= github.com/mr-tron/base58 v1.1.2/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc= github.com/mr-tron/base58 v1.2.0 h1:T/HDJBh4ZCPbU39/+c3rRvE0uKBQlU27+QI8LJ4t64o= github.com/mr-tron/base58 v1.2.0/go.mod h1:BinMc/sQntlIE1frQmRFPUoPA1Zkr8VRgBdjWI2mNwc= +github.com/mschoch/smat v0.2.0 h1:8imxQsjDm8yFEAVBe7azKmKSgzSkZXDuKkSq9374khM= +github.com/mschoch/smat v0.2.0/go.mod h1:kc9mz7DoBKqDyiRL7VZN8KvXQMWeTaVnttLRXOlotKw= github.com/multiformats/go-base32 v0.1.0 h1:pVx9xoSPqEIQG8o+UbAe7DNi51oej1NtK+aGkbLYxPE= github.com/multiformats/go-base32 v0.1.0/go.mod h1:Kj3tFY6zNr+ABYMqeUNeGvkIC/UYgtWibDcT0rExnbI= github.com/multiformats/go-base36 v0.2.0 h1:lFsAbNOGeKtuKozrtBsAkSVhv1p9D0/qedU9rQyccr0= @@ -207,8 +293,12 @@ github.com/multiformats/go-varint v0.0.7 h1:sWSGR+f/eu5ABZA2ZpYKBILXTTs9JWpdEM/n github.com/multiformats/go-varint v0.0.7/go.mod h1:r8PUYw/fD/SjBCiKOoDlGF6QawOELpZAu9eioSos/OU= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA= github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= +github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= +github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo= github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM= +github.com/olric-data/olric v0.7.0 h1:EKN2T6ZTtdu8Un0jV0KOWVxWm9odptJpefmDivfZdjE= +github.com/olric-data/olric v0.7.0/go.mod h1:+ZnPpgc8JkNkza8rETCKGn0P/QPF6HhZY0EbCKAOslo= github.com/onsi/ginkgo/v2 v2.22.2 h1:/3X8Panh8/WwhU/3Ssa6rCKqPLuAkVY2I0RoyDLySlU= github.com/onsi/ginkgo/v2 v2.22.2/go.mod h1:oeMosUL+8LtarXBHu/c0bx2D/K9zyQ6uX3cTyztHwsk= github.com/onsi/gomega v1.36.2 h1:koNYke6TVk6ZmnyHrCXba/T/MoLBXFjeC1PtvYgw0A8= @@ -217,6 +307,8 @@ github.com/opencontainers/runtime-spec v1.0.2/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/ github.com/opencontainers/runtime-spec v1.2.0 h1:z97+pHb3uELt/yiAWD691HNHQIF07bE7dzrbT927iTk= github.com/opencontainers/runtime-spec v1.2.0/go.mod h1:jwyrGlmzljRJv/Fgzds9SsS/C5hL+LL3ko9hs6T5lQ0= github.com/openzipkin/zipkin-go v0.1.1/go.mod h1:NtoC/o8u3JlF1lSlyPNswIbeQH9bJTmOf0Erfk+hxe8= +github.com/pascaldekloe/goe v0.1.0 h1:cBOtyMzM9HTpWjXfbbunk26uA6nG3a8n06Wieeh0MwY= +github.com/pascaldekloe/goe v0.1.0/go.mod h1:lzWF7FIEvWOWxwDKqyGYQf6ZUaNfKdP144TG7ZOy1lc= github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58 h1:onHthvaw9LFnH4t2DcNVpwGmV9E1BkGknEliJkfwQj0= github.com/pbnjay/memory v0.0.0-20210728143218-7b4eea64cf58/go.mod h1:DXv8WO4yhMYhSNPKjeNKa5WY9YCIEBRbNzFFPJbWO6Y= github.com/pion/datachannel v1.5.10 h1:ly0Q26K1i6ZkGf42W7D4hQYR90pZwzFOjTq5AuCKk4o= @@ -261,21 +353,38 @@ github.com/pion/turn/v4 v4.0.0 h1:qxplo3Rxa9Yg1xXDxxH8xaqcyGUtbHYw4QSCvmFWvhM= github.com/pion/turn/v4 v4.0.0/go.mod h1:MuPDkm15nYSklKpN8vWJ9W2M0PlyQZqYt1McGuxG7mA= github.com/pion/webrtc/v4 v4.0.10 h1:Hq/JLjhqLxi+NmCtE8lnRPDr8H4LcNvwg8OxVcdv56Q= github.com/pion/webrtc/v4 v4.0.10/go.mod h1:ViHLVaNpiuvaH8pdiuQxuA9awuE6KVzAXx3vVWilOck= +github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4= github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_golang v0.8.0/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= +github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw= +github.com/prometheus/client_golang v1.0.0/go.mod h1:db9x61etRT2tGnBNRi70OPL5FsnadC4Ky3P0J6CfImo= +github.com/prometheus/client_golang v1.4.0/go.mod h1:e9GMxYsXl05ICDXkRhurwBS4Q3OK1iX/F2sw+iXX5zU= +github.com/prometheus/client_golang v1.7.1/go.mod h1:PY5Wy2awLA44sXw4AOSfFBetzPP4j5+D6mVACh+pe2M= +github.com/prometheus/client_golang v1.11.1/go.mod h1:Z6t4BnS23TR94PD6BsDNk8yVqroYurpAkEiz0P2BEV0= github.com/prometheus/client_golang v1.22.0 h1:rb93p9lokFEsctTys46VnV1kLCDpVZ0a/Y92Vm0Zc6Q= github.com/prometheus/client_golang v1.22.0/go.mod h1:R7ljNsLXhuQXYZYtw6GAE9AZg8Y7vEW5scdCXrWRXC0= github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo= +github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/prometheus/client_model v0.2.0/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk= github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE= github.com/prometheus/common v0.0.0-20180801064454-c7de2306084e/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro= +github.com/prometheus/common v0.4.1/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4= +github.com/prometheus/common v0.9.1/go.mod h1:yhUN8i9wzaXS3w1O07YhxHEBxD+W35wd8bs7vj7HSQ4= +github.com/prometheus/common v0.10.0/go.mod h1:Tlit/dnDKsSWFlCLTWaA1cyBgKHSMdTB80sz/V91rCo= +github.com/prometheus/common v0.26.0/go.mod h1:M7rCNAaPfAosfx8veZJCuw84e35h3Cfd9VFqTh1DIvc= github.com/prometheus/common v0.63.0 h1:YR/EIY1o3mEFP/kZCD7iDMnLPlGyuU2Gb3HIcXnA98k= github.com/prometheus/common v0.63.0/go.mod h1:VVFF/fBIoToEnWRVkYoXEkq3R3paCoxG9PXP74SnV18= github.com/prometheus/procfs v0.0.0-20180725123919-05ee40e3a273/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk= +github.com/prometheus/procfs v0.0.2/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA= +github.com/prometheus/procfs v0.0.8/go.mod h1:7Qr8sr6344vo1JqZ6HhLceV9o3AJ1Ff+GxbHq6oeK9A= +github.com/prometheus/procfs v0.1.3/go.mod h1:lV6e/gmhEcM9IjHGsFOCxxuZ+z1YqCvr4OA4YeYWdaU= +github.com/prometheus/procfs v0.6.0/go.mod h1:cz+aTbrPOrUb4q7XlbU9ygM+/jj0fzG6c1xBZuNvfVA= github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg= github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is= github.com/quic-go/qpack v0.5.1 h1:giqksBPnT/HDtZ6VhtFKgoLOWmlyo9Ei6u9PqzIMbhI= @@ -286,12 +395,16 @@ github.com/quic-go/webtransport-go v0.8.1-0.20241018022711-4ac2c9250e66 h1:4WFk6 github.com/quic-go/webtransport-go v0.8.1-0.20241018022711-4ac2c9250e66/go.mod h1:Vp72IJajgeOL6ddqrAhmp7IM9zbTcgkQxD/YdxrVwMw= github.com/raulk/go-watchdog v1.3.0 h1:oUmdlHxdkXRJlwfG0O9omj8ukerm8MEQavSiDTEtBsk= github.com/raulk/go-watchdog v1.3.0/go.mod h1:fIvOnLbF0b0ZwkB9YU4mOW9Did//4vPZtDqv66NfsMU= +github.com/redis/go-redis/v9 v9.8.0 h1:q3nRvjrlge/6UD7eTu/DSg2uYiU2mCL0G/uzBWqhicI= +github.com/redis/go-redis/v9 v9.8.0/go.mod h1:huWgSWd8mW6+m0VPhJjSSQ+d6Nh1VICQ6Q5lHuCH/Iw= github.com/rogpeppe/go-internal v1.13.1 h1:KvO1DLK/DRN07sQ1LQKScxyZJuNnedQ5/wKSR38lUII= github.com/rogpeppe/go-internal v1.13.1/go.mod h1:uMEvuHeurkdAXX61udpOXGD/AzZDWNMNyH2VO9fmH0o= github.com/rqlite/gorqlite v0.0.0-20250609141355-ac86a4a1c9a8 h1:BoxiqWvhprOB2isgM59s8wkgKwAoyQH66Twfmof41oE= github.com/rqlite/gorqlite v0.0.0-20250609141355-ac86a4a1c9a8/go.mod h1:xF/KoXmrRyahPfo5L7Szb5cAAUl53dMWBh9cMruGEZg= github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g= github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= +github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529 h1:nn5Wsu0esKSJiIVhscUtVbo7ada43DJhG55ua/hjS5I= +github.com/sean-/seed v0.0.0-20170313163322-e2103e2c3529/go.mod h1:DxrIzT+xaE7yg65j358z/aeFdxmN0P9QXhEzd20vsDc= github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo= github.com/shurcooL/component v0.0.0-20170202220835-f88ec8f54cc4/go.mod h1:XhFIlyj5a1fBNx5aJTbKoIq0mNaPvOagO+HjB3EtxrY= github.com/shurcooL/events v0.0.0-20181021180414-410e4ca65f48/go.mod h1:5u70Mqkb5O5cxEA8nxTsgrgLehJeAw6Oc4Ab1c/P1HM= @@ -316,16 +429,22 @@ github.com/shurcooL/sanitized_anchor_name v0.0.0-20170918181015-86672fcb3f95/go. github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc= github.com/shurcooL/users v0.0.0-20180125191416-49c67e49c537/go.mod h1:QJTqeLYEDaXHZDBsXlPCDqdhQuJkuw4NOtaxYe3xii4= github.com/shurcooL/webdavfs v0.0.0-20170829043945-18c3829fa133/go.mod h1:hKmq5kWdCj2z2KEozexVbfEZIWiTjhE0+UjmZgPqehw= +github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= +github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= +github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0= github.com/sourcegraph/annotate v0.0.0-20160123013949-f4cad6c6324d/go.mod h1:UdhH50NIW0fCiwBSr0co2m7BnFLdv4fQTgdqdJTHFeE= github.com/sourcegraph/syntaxhighlight v0.0.0-20170531221838-bd320f5d308e/go.mod h1:HuIsMU8RRBOtsCgI77wP899iHVBQpCmg4ErYMZB+2IA= github.com/spaolacci/murmur3 v1.1.0 h1:7c1g84S4BPRrfL5Xrdp6fOJ206sU9y293DDHaoy0bLI= github.com/spaolacci/murmur3 v1.1.0/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo= github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs= +github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI= github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4= +github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.3/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo= @@ -333,9 +452,21 @@ github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXl github.com/stretchr/testify v1.10.0 h1:Xv5erBjTwe/5IxqUQTdXv5kgmIvbHo3QQyRwhJsOfJA= github.com/stretchr/testify v1.10.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07/go.mod h1:kDXzergiv9cbyO7IOYJZWg1U88JhDg3PB6klq9Hg2pA= +github.com/tidwall/btree v1.1.0/go.mod h1:TzIRzen6yHbibdSfK6t8QimqbUnoxUSrZfeW7Uob0q4= +github.com/tidwall/btree v1.7.0 h1:L1fkJH/AuEh5zBnnBbmTwQ5Lt+bRJ5A8EWecslvo9iI= +github.com/tidwall/btree v1.7.0/go.mod h1:twD9XRA5jj9VUQGELzDO4HPQTNJsoWWfYEL+EUQ2cKY= +github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA= +github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM= +github.com/tidwall/redcon v1.6.2 h1:5qfvrrybgtO85jnhSravmkZyC0D+7WstbfCs3MmPhow= +github.com/tidwall/redcon v1.6.2/go.mod h1:p5Wbsgeyi2VSTBWOcA5vRXrOb9arFTcU2+ZzFjqV75Y= +github.com/tv42/httpunix v0.0.0-20150427012821-b75d8614f926/go.mod h1:9ESjWnEqriFuLhtthL60Sar/7RFoluCcXsuvEwTV5KM= github.com/urfave/cli v1.22.2/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0= github.com/viant/assertly v0.4.8/go.mod h1:aGifi++jvCrUaklKEKT0BU95igDNaqkvz+49uaYMPRU= github.com/viant/toolbox v0.24.0/go.mod h1:OxMCG57V0PXuIP2HNQrtJf2CjqdmbrOx5EkMILuUhzM= +github.com/vmihailenco/msgpack/v5 v5.4.1 h1:cQriyiUvjTwOHg8QZaPihLWeRAAVoCpE00IUPn0Bjt8= +github.com/vmihailenco/msgpack/v5 v5.4.1/go.mod h1:GaZTsDaehaPpQVyxrf5mtQlH+pc21PIudVV/E3rRQok= +github.com/vmihailenco/tagparser/v2 v2.0.0 h1:y09buUbR+b5aycVFQs/g70pqKVZNBmxwAhO7/IwNM9g= +github.com/vmihailenco/tagparser/v2 v2.0.0/go.mod h1:Wri+At7QHww0WTrCBeu4J6bNtoV6mEfg5OIWRZA9qds= github.com/wlynxg/anet v0.0.3/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA= github.com/wlynxg/anet v0.0.5 h1:J3VJGi1gvo0JwZ/P1/Yc/8p63SoW98B5dHkYDmpgvvU= github.com/wlynxg/anet v0.0.5/go.mod h1:eay5PRQr7fIVAMbTbchTnO9gG65Hg/uYGdc7mguHxoA= @@ -357,6 +488,7 @@ go.uber.org/zap v1.27.0 h1:aJMhYGrd5QSmlpLMr2MftRKl7t8J8PTZPA732ud/XR8= go.uber.org/zap v1.27.0/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E= go4.org v0.0.0-20180809161055-417644f6feb5/go.mod h1:MkTOUMDaeVYJUOUsaDXIhWPZYa1yOyC1qaOBpL57BhE= golang.org/x/build v0.0.0-20190111050920-041ab4dc3f9d/go.mod h1:OWs+y06UdEOHN4y+MfF/py+xQ/tYqIWW03b70/CG9Rw= +golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20181030102418-4d3f4d9ffa16/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= golang.org/x/crypto v0.0.0-20190313024323-a1f597ede03a/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= @@ -390,12 +522,15 @@ golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73r golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181029044818-c44066c5c816/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20181106065722-10aee1819953/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= +golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190313220215-9f648a60d977/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= +golang.org/x/net v0.0.0-20190613194153-d28f0bde5980/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= golang.org/x/net v0.0.0-20200226121028-0de0cce0169b/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s= +golang.org/x/net v0.0.0-20200625001655-4c5254603344/go.mod h1:/O7V0waA8r7cgGh81Ro3o1hOxt32SMVPicZroKQ2sZA= golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20210119194325-5f4716e94777/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= @@ -419,6 +554,7 @@ golang.org/x/sync v0.0.0-20190227155943-e225da77a7e6/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190911185100-cd5d95a43a6e/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= @@ -426,16 +562,26 @@ golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw= golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA= golang.org/x/sys v0.0.0-20180810173357-98c5dad5d1a0/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181029174526-d69651ed3497/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= +golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190316082340-a2f829d7f35f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200106162015-b016eb3dc98e/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200122134326-e047566fdf82/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200124204421-9fbb57f87de9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200602225109-6fdc65e7d980/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200615200032-f1bc736245b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20200625212154-ddb9806d33ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210124154548-22da62e12c0c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210603081109-ebe580a85c40/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= @@ -456,6 +602,7 @@ golang.org/x/term v0.11.0/go.mod h1:zC9APTIj3jG3FdV/Ons+XE1riIZXG4aZ4GTHiPZJPIU= golang.org/x/term v0.16.0/go.mod h1:yn7UURbUtPyrVJPGPq404EukNFxcm/foM+bV/bfcDsY= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/text v0.3.1-0.20180807135948-17ff2d5776d2/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= +golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk= golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ= golang.org/x/text v0.3.7/go.mod h1:u+2+/6zg+i71rQMx5EYifcz6MCKuco9NR6JIITiCfzQ= golang.org/x/text v0.7.0/go.mod h1:mrYo+phRRbMaCq/xk9113O4dZlRixOauAjOtrjsXDZ8= @@ -502,15 +649,29 @@ google.golang.org/grpc v1.14.0/go.mod h1:yo6s7OP7yaDglbqo1J04qKzAhqBH6lvTonzMVmE google.golang.org/grpc v1.16.0/go.mod h1:0JHn/cJsOMiMfNA9+DeHDlAU7KAAB5GDlYFpa9MZMio= google.golang.org/grpc v1.17.0/go.mod h1:6QZJwpn2B+Zp71q/5VxRsJ6NXXVCE5NRUHRo+f3cWCs= google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= +google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= +google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= +google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= +google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= +google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= +google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= +google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.36.6 h1:z1NpPI8ku2WgiWnf+t9wTPsn6eP1L7ksHUlkfLvd9xY= google.golang.org/protobuf v1.36.6/go.mod h1:jduwjTPXsFjZGTmRluh+L6NjiWu7pchiJ2/5YcXBHnY= +gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/check.v1 v1.0.0-20190902080502-41f04d3bba15/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/inf.v0 v0.9.1/go.mod h1:cWUDdTG/fYaXco+Dcufb5Vnc6Gp2YChqWtbxRZE0mXw= gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.4/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.2.5/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI= +gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= +gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/pkg/cli/setup.go b/pkg/cli/setup.go index c1a7ed7..bc245d1 100644 --- a/pkg/cli/setup.go +++ b/pkg/cli/setup.go @@ -62,11 +62,12 @@ func HandleSetupCommand(args []string) { fmt.Printf(" 3. Install Go 1.21+ (if needed)\n") fmt.Printf(" 4. Install RQLite database\n") fmt.Printf(" 5. Install Anyone Relay (Anon) for anonymous networking\n") - fmt.Printf(" 6. Create directories (/home/debros/bin, /home/debros/src)\n") - fmt.Printf(" 7. Clone and build DeBros Network\n") - fmt.Printf(" 8. Generate configuration files\n") - fmt.Printf(" 9. Create systemd services (debros-node, debros-gateway)\n") - fmt.Printf(" 10. Start and enable services\n") + fmt.Printf(" 6. Install Olric cache server\n") + fmt.Printf(" 7. Create directories (/home/debros/bin, /home/debros/src)\n") + fmt.Printf(" 8. Clone and build DeBros Network\n") + fmt.Printf(" 9. Generate configuration files\n") + fmt.Printf(" 10. Create systemd services (debros-node, debros-gateway, debros-olric)\n") + fmt.Printf(" 11. Start and enable services\n") fmt.Printf(strings.Repeat("=", 70) + "\n\n") fmt.Printf("Ready to begin setup? (yes/no): ") @@ -92,6 +93,9 @@ func HandleSetupCommand(args []string) { // Step 4.5: Install Anon (Anyone relay) installAnon() + // Step 4.6: Install Olric cache server + installOlric() + // Step 5: Setup directories setupDirectories() @@ -1037,6 +1041,132 @@ func configureFirewallForAnon() { fmt.Printf(" No active firewall detected\n") } +func installOlric() { + fmt.Printf("💾 Installing Olric cache server...\n") + + // Check if already installed + if _, err := exec.LookPath("olric-server"); err == nil { + fmt.Printf(" ✓ Olric already installed\n") + configureFirewallForOlric() + return + } + + // Ensure Go is available (required for go install) + if _, err := exec.LookPath("go"); err != nil { + fmt.Fprintf(os.Stderr, "⚠️ Go not found - cannot install Olric. Please install Go first.\n") + return + } + + fmt.Printf(" Installing Olric server via go install...\n") + cmd := exec.Command("go", "install", "github.com/olric-data/olric/cmd/olric-server@v0.7.0") + cmd.Env = append(os.Environ(), "GOBIN=/usr/local/bin") + if output, err := cmd.CombinedOutput(); err != nil { + fmt.Fprintf(os.Stderr, "⚠️ Failed to install Olric: %v\n", err) + if len(output) > 0 { + fmt.Fprintf(os.Stderr, " Output: %s\n", string(output)) + } + fmt.Fprintf(os.Stderr, " You can manually install with: go install github.com/olric-data/olric/cmd/olric-server@v0.7.0\n") + return + } + + // Verify installation + if _, err := exec.LookPath("olric-server"); err != nil { + fmt.Fprintf(os.Stderr, "⚠️ Olric installation verification failed: binary not found in PATH\n") + fmt.Fprintf(os.Stderr, " Make sure /usr/local/bin is in PATH\n") + return + } + + fmt.Printf(" ✓ Olric installed\n") + + // Configure firewall + configureFirewallForOlric() + + // Create Olric config directory + olricConfigDir := "/home/debros/.debros/olric" + if err := os.MkdirAll(olricConfigDir, 0755); err == nil { + configPath := olricConfigDir + "/config.yaml" + if _, err := os.Stat(configPath); os.IsNotExist(err) { + configContent := `memberlist: + bind-addr: "0.0.0.0" + bind-port: 3322 +client: + bind-addr: "0.0.0.0" + bind-port: 3320 + +# Durability and replication configuration +# Replicates data across entire network for fault tolerance +dmaps: + default: + replication: + mode: sync # Synchronous replication for durability + replica_count: 2 # Replicate to 2 backup nodes (3 total copies: 1 primary + 2 backups) + write_quorum: 2 # Require 2 nodes to acknowledge writes + read_quorum: 1 # Read from 1 node (faster reads) + read_repair: true # Enable read-repair for consistency + +# Split-brain protection +member_count_quorum: 2 # Require at least 2 nodes to operate (prevents split-brain) +` + if err := os.WriteFile(configPath, []byte(configContent), 0644); err == nil { + exec.Command("chown", "debros:debros", configPath).Run() + fmt.Printf(" ✓ Olric config created at %s\n", configPath) + } + } + exec.Command("chown", "-R", "debros:debros", olricConfigDir).Run() + } +} + +func configureFirewallForOlric() { + fmt.Printf(" Checking firewall configuration for Olric...\n") + + // Check for UFW + if _, err := exec.LookPath("ufw"); err == nil { + output, _ := exec.Command("ufw", "status").CombinedOutput() + if strings.Contains(string(output), "Status: active") { + fmt.Printf(" Adding UFW rules for Olric...\n") + exec.Command("ufw", "allow", "3320/tcp", "comment", "Olric HTTP API").Run() + exec.Command("ufw", "allow", "3322/tcp", "comment", "Olric Memberlist").Run() + fmt.Printf(" ✓ UFW rules added for Olric\n") + return + } + } + + // Check for firewalld + if _, err := exec.LookPath("firewall-cmd"); err == nil { + output, _ := exec.Command("firewall-cmd", "--state").CombinedOutput() + if strings.Contains(string(output), "running") { + fmt.Printf(" Adding firewalld rules for Olric...\n") + exec.Command("firewall-cmd", "--permanent", "--add-port=3320/tcp").Run() + exec.Command("firewall-cmd", "--permanent", "--add-port=3322/tcp").Run() + exec.Command("firewall-cmd", "--reload").Run() + fmt.Printf(" ✓ firewalld rules added for Olric\n") + return + } + } + + // Check for iptables + if _, err := exec.LookPath("iptables"); err == nil { + output, _ := exec.Command("iptables", "-L", "-n").CombinedOutput() + if strings.Contains(string(output), "Chain INPUT") { + fmt.Printf(" Adding iptables rules for Olric...\n") + exec.Command("iptables", "-A", "INPUT", "-p", "tcp", "--dport", "3320", "-j", "ACCEPT", "-m", "comment", "--comment", "Olric HTTP API").Run() + exec.Command("iptables", "-A", "INPUT", "-p", "tcp", "--dport", "3322", "-j", "ACCEPT", "-m", "comment", "--comment", "Olric Memberlist").Run() + + // Try to save rules + if _, err := exec.LookPath("netfilter-persistent"); err == nil { + exec.Command("netfilter-persistent", "save").Run() + } else if _, err := exec.LookPath("iptables-save"); err == nil { + cmd := exec.Command("sh", "-c", "iptables-save > /etc/iptables/rules.v4") + cmd.Run() + } + fmt.Printf(" ✓ iptables rules added for Olric\n") + return + } + } + + fmt.Printf(" No active firewall detected for Olric\n") +} + func setupDirectories() { fmt.Printf("📁 Creating directories...\n") @@ -1285,6 +1415,19 @@ func generateConfigsInteractive(force bool) { // Fix ownership exec.Command("chown", "debros:debros", nodeConfigPath).Run() fmt.Printf(" ✓ Node config created: %s\n", nodeConfigPath) + + // Generate Olric config file for this node (uses multicast discovery) + var olricConfigPath string + if isBootstrap { + olricConfigPath = "/home/debros/.debros/bootstrap/olric-config.yaml" + } else { + olricConfigPath = "/home/debros/.debros/node/olric-config.yaml" + } + if err := generateOlricConfig(olricConfigPath, vpsIP, 3320, 3322); err != nil { + fmt.Fprintf(os.Stderr, "⚠️ Failed to generate Olric config: %v\n", err) + } else { + fmt.Printf(" ✓ Olric config created: %s\n", olricConfigPath) + } } // Generate gateway config @@ -1334,9 +1477,20 @@ func generateConfigsInteractive(force bool) { } } + // For Olric servers, use localhost for local dev, or current node IP + // In production, gateway will discover Olric nodes via LibP2P network + var olricServers []string + if bootstrapPeers == "" { + // Local development - use localhost + olricServers = []string{"localhost:3320"} + } else { + // Production - start with current node, will discover others via LibP2P + olricServers = []string{fmt.Sprintf("%s:3320", vpsIP)} + } + // Gateway config should include bootstrap peers if this is a regular node // (bootstrap nodes don't need bootstrap peers since they are the bootstrap) - gatewayConfig := generateGatewayConfigDirect(bootstrapPeers, enableHTTPS, domain, tlsCacheDir) + gatewayConfig := generateGatewayConfigDirect(bootstrapPeers, enableHTTPS, domain, tlsCacheDir, olricServers) if err := os.WriteFile(gatewayPath, []byte(gatewayConfig), 0644); err != nil { fmt.Fprintf(os.Stderr, "❌ Failed to write gateway config: %v\n", err) os.Exit(1) @@ -1429,6 +1583,10 @@ func generateNodeConfigWithIP(name, id string, listenPort, rqliteHTTPPort, rqlit joinAddr = fmt.Sprintf("localhost:%d", rqliteHTTPPort) } + // Generate Olric config file for regular node (uses multicast discovery) + olricConfigPath := "/home/debros/.debros/node/olric-config.yaml" + generateOlricConfig(olricConfigPath, ipAddr, 3320, 3322) + return fmt.Sprintf(`node: id: "%s" type: "node" @@ -1468,7 +1626,7 @@ logging: } // generateGatewayConfigDirect generates gateway config directly -func generateGatewayConfigDirect(bootstrapPeers string, enableHTTPS bool, domain, tlsCacheDir string) string { +func generateGatewayConfigDirect(bootstrapPeers string, enableHTTPS bool, domain, tlsCacheDir string, olricServers []string) string { var peers []string if bootstrapPeers != "" { for _, p := range strings.Split(bootstrapPeers, ",") { @@ -1499,12 +1657,71 @@ func generateGatewayConfigDirect(bootstrapPeers string, enableHTTPS bool, domain fmt.Fprintf(&httpsYAML, "enable_https: false\n") } + // Olric servers configuration + var olricYAML strings.Builder + if len(olricServers) > 0 { + olricYAML.WriteString("olric_servers:\n") + for _, server := range olricServers { + fmt.Fprintf(&olricYAML, " - \"%s\"\n", server) + } + } else { + // Default to localhost for local development + olricYAML.WriteString("olric_servers:\n") + olricYAML.WriteString(" - \"localhost:3320\"\n") + } + return fmt.Sprintf(`listen_addr: ":6001" client_namespace: "default" rqlite_dsn: "" %s %s -`, peersYAML.String(), httpsYAML.String()) +%s +`, peersYAML.String(), httpsYAML.String(), olricYAML.String()) +} + +// generateOlricConfig generates an Olric configuration file +// Uses multicast discovery - peers will be discovered dynamically via LibP2P network +func generateOlricConfig(configPath, bindIP string, httpPort, memberlistPort int) error { + // Ensure directory exists + dir := filepath.Dir(configPath) + if err := os.MkdirAll(dir, 0755); err != nil { + return fmt.Errorf("failed to create Olric config directory: %w", err) + } + + var config strings.Builder + config.WriteString("memberlist:\n") + config.WriteString(fmt.Sprintf(" bind-addr: \"%s\"\n", bindIP)) + config.WriteString(fmt.Sprintf(" bind-port: %d\n", memberlistPort)) + config.WriteString(" # Multicast discovery enabled - peers discovered dynamically via LibP2P network\n") + + config.WriteString("client:\n") + config.WriteString(fmt.Sprintf(" bind-addr: \"%s\"\n", bindIP)) + config.WriteString(fmt.Sprintf(" bind-port: %d\n", httpPort)) + + // Durability and replication settings + config.WriteString("\n# Durability and replication configuration\n") + config.WriteString("# Replicates data across entire network for fault tolerance\n") + config.WriteString("dmaps:\n") + config.WriteString(" default:\n") + config.WriteString(" replication:\n") + config.WriteString(" mode: sync # Synchronous replication for durability\n") + config.WriteString(" replica_count: 2 # Replicate to 2 backup nodes (3 total copies: 1 primary + 2 backups)\n") + config.WriteString(" write_quorum: 2 # Require 2 nodes to acknowledge writes\n") + config.WriteString(" read_quorum: 1 # Read from 1 node (faster reads)\n") + config.WriteString(" read_repair: true # Enable read-repair for consistency\n") + + // Split-brain protection + config.WriteString("\n# Split-brain protection\n") + config.WriteString("member_count_quorum: 2 # Require at least 2 nodes to operate (prevents split-brain)\n") + + // Write config file + if err := os.WriteFile(configPath, []byte(config.String()), 0644); err != nil { + return fmt.Errorf("failed to write Olric config: %w", err) + } + + // Fix ownership + exec.Command("chown", "debros:debros", configPath).Run() + return nil } func createSystemdServices() { diff --git a/pkg/config/config.go b/pkg/config/config.go index 5784597..4314198 100644 --- a/pkg/config/config.go +++ b/pkg/config/config.go @@ -41,6 +41,10 @@ type DatabaseConfig struct { ClusterSyncInterval time.Duration `yaml:"cluster_sync_interval"` // default: 30s PeerInactivityLimit time.Duration `yaml:"peer_inactivity_limit"` // default: 24h MinClusterSize int `yaml:"min_cluster_size"` // default: 1 + + // Olric cache configuration + OlricHTTPPort int `yaml:"olric_http_port"` // Olric HTTP API port (default: 3320) + OlricMemberlistPort int `yaml:"olric_memberlist_port"` // Olric memberlist port (default: 3322) } // DiscoveryConfig contains peer discovery configuration @@ -116,6 +120,10 @@ func DefaultConfig() *Config { ClusterSyncInterval: 30 * time.Second, PeerInactivityLimit: 24 * time.Hour, MinClusterSize: 1, + + // Olric cache configuration + OlricHTTPPort: 3320, + OlricMemberlistPort: 3322, }, Discovery: DiscoveryConfig{ BootstrapPeers: []string{}, diff --git a/pkg/gateway/cache_handlers.go b/pkg/gateway/cache_handlers.go new file mode 100644 index 0000000..1796b7e --- /dev/null +++ b/pkg/gateway/cache_handlers.go @@ -0,0 +1,356 @@ +package gateway + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "strings" + "time" + + olriclib "github.com/olric-data/olric" +) + +// Cache HTTP handlers for Olric distributed cache + +func (g *Gateway) cacheHealthHandler(w http.ResponseWriter, r *http.Request) { + if g.olricClient == nil { + writeError(w, http.StatusServiceUnavailable, "Olric cache client not initialized") + return + } + + ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second) + defer cancel() + + err := g.olricClient.Health(ctx) + if err != nil { + writeError(w, http.StatusServiceUnavailable, fmt.Sprintf("cache health check failed: %v", err)) + return + } + + writeJSON(w, http.StatusOK, map[string]any{ + "status": "ok", + "service": "olric", + }) +} + +func (g *Gateway) cacheGetHandler(w http.ResponseWriter, r *http.Request) { + if g.olricClient == nil { + writeError(w, http.StatusServiceUnavailable, "Olric cache client not initialized") + return + } + + if r.Method != http.MethodPost { + writeError(w, http.StatusMethodNotAllowed, "method not allowed") + return + } + + var req struct { + DMap string `json:"dmap"` // Distributed map name + Key string `json:"key"` // Key to retrieve + } + + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid json body") + return + } + + if strings.TrimSpace(req.DMap) == "" || strings.TrimSpace(req.Key) == "" { + writeError(w, http.StatusBadRequest, "dmap and key are required") + return + } + + ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second) + defer cancel() + + client := g.olricClient.GetClient() + dm, err := client.NewDMap(req.DMap) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to create DMap: %v", err)) + return + } + + gr, err := dm.Get(ctx, req.Key) + if err != nil { + if err == olriclib.ErrKeyNotFound { + writeError(w, http.StatusNotFound, "key not found") + return + } + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to get key: %v", err)) + return + } + + // Try to decode the value from Olric + // Values stored as JSON bytes need to be deserialized, while basic types + // (strings, numbers, bools) can be retrieved directly + var value any + + // First, try to get as bytes (for JSON-serialized complex types) + var bytesVal []byte + if err := gr.Scan(&bytesVal); err == nil && len(bytesVal) > 0 { + // Try to deserialize as JSON + var jsonVal any + if err := json.Unmarshal(bytesVal, &jsonVal); err == nil { + value = jsonVal + } else { + // If JSON unmarshal fails, treat as string + value = string(bytesVal) + } + } else { + // Try as string (for simple string values) + if strVal, err := gr.String(); err == nil { + value = strVal + } else { + // Fallback: try to scan as any type + var anyVal any + if err := gr.Scan(&anyVal); err == nil { + value = anyVal + } else { + // Last resort: try String() again, ignoring error + strVal, _ := gr.String() + value = strVal + } + } + } + + writeJSON(w, http.StatusOK, map[string]any{ + "key": req.Key, + "value": value, + "dmap": req.DMap, + }) +} + +func (g *Gateway) cachePutHandler(w http.ResponseWriter, r *http.Request) { + if g.olricClient == nil { + writeError(w, http.StatusServiceUnavailable, "Olric cache client not initialized") + return + } + + if r.Method != http.MethodPost { + writeError(w, http.StatusMethodNotAllowed, "method not allowed") + return + } + + var req struct { + DMap string `json:"dmap"` // Distributed map name + Key string `json:"key"` // Key to store + Value any `json:"value"` // Value to store + TTL string `json:"ttl"` // Optional TTL (duration string like "1h", "30m") + } + + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid json body") + return + } + + if strings.TrimSpace(req.DMap) == "" || strings.TrimSpace(req.Key) == "" { + writeError(w, http.StatusBadRequest, "dmap and key are required") + return + } + + if req.Value == nil { + writeError(w, http.StatusBadRequest, "value is required") + return + } + + ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second) + defer cancel() + + client := g.olricClient.GetClient() + dm, err := client.NewDMap(req.DMap) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to create DMap: %v", err)) + return + } + + // TODO: TTL support - need to check Olric v0.7 API for TTL/expiry options + // For now, ignore TTL if provided + if req.TTL != "" { + _, err := time.ParseDuration(req.TTL) + if err != nil { + writeError(w, http.StatusBadRequest, fmt.Sprintf("invalid ttl format: %v", err)) + return + } + // TTL parsing succeeded but not yet implemented in API + // Will be added once we confirm the correct Olric API method + } + + // Serialize complex types (maps, slices) to JSON bytes for Olric storage + // Olric can handle basic types (string, number, bool) directly, but complex + // types need to be serialized to bytes + var valueToStore any + switch req.Value.(type) { + case map[string]any: + // Serialize maps to JSON bytes + jsonBytes, err := json.Marshal(req.Value) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to marshal value: %v", err)) + return + } + valueToStore = jsonBytes + case []any: + // Serialize slices to JSON bytes + jsonBytes, err := json.Marshal(req.Value) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to marshal value: %v", err)) + return + } + valueToStore = jsonBytes + case string: + // Basic string type can be stored directly + valueToStore = req.Value + case float64: + // Basic number type can be stored directly + valueToStore = req.Value + case int: + // Basic int type can be stored directly + valueToStore = req.Value + case int64: + // Basic int64 type can be stored directly + valueToStore = req.Value + case bool: + // Basic bool type can be stored directly + valueToStore = req.Value + case nil: + // Nil can be stored directly + valueToStore = req.Value + default: + // For any other type, serialize to JSON to be safe + jsonBytes, err := json.Marshal(req.Value) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to marshal value: %v", err)) + return + } + valueToStore = jsonBytes + } + + err = dm.Put(ctx, req.Key, valueToStore) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to put key: %v", err)) + return + } + + writeJSON(w, http.StatusOK, map[string]any{ + "status": "ok", + "key": req.Key, + "dmap": req.DMap, + }) +} + +func (g *Gateway) cacheDeleteHandler(w http.ResponseWriter, r *http.Request) { + if g.olricClient == nil { + writeError(w, http.StatusServiceUnavailable, "Olric cache client not initialized") + return + } + + if r.Method != http.MethodPost { + writeError(w, http.StatusMethodNotAllowed, "method not allowed") + return + } + + var req struct { + DMap string `json:"dmap"` // Distributed map name + Key string `json:"key"` // Key to delete + } + + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid json body") + return + } + + if strings.TrimSpace(req.DMap) == "" || strings.TrimSpace(req.Key) == "" { + writeError(w, http.StatusBadRequest, "dmap and key are required") + return + } + + ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second) + defer cancel() + + client := g.olricClient.GetClient() + dm, err := client.NewDMap(req.DMap) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to create DMap: %v", err)) + return + } + + deletedCount, err := dm.Delete(ctx, req.Key) + if err != nil { + if err == olriclib.ErrKeyNotFound { + writeError(w, http.StatusNotFound, "key not found") + return + } + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to delete key: %v", err)) + return + } + if deletedCount == 0 { + writeError(w, http.StatusNotFound, "key not found") + return + } + + writeJSON(w, http.StatusOK, map[string]any{ + "status": "ok", + "key": req.Key, + "dmap": req.DMap, + }) +} + +func (g *Gateway) cacheScanHandler(w http.ResponseWriter, r *http.Request) { + if g.olricClient == nil { + writeError(w, http.StatusServiceUnavailable, "Olric cache client not initialized") + return + } + + if r.Method != http.MethodPost { + writeError(w, http.StatusMethodNotAllowed, "method not allowed") + return + } + + var req struct { + DMap string `json:"dmap"` // Distributed map name + Match string `json:"match"` // Optional regex pattern to match keys + } + + if err := json.NewDecoder(r.Body).Decode(&req); err != nil { + writeError(w, http.StatusBadRequest, "invalid json body") + return + } + + if strings.TrimSpace(req.DMap) == "" { + writeError(w, http.StatusBadRequest, "dmap is required") + return + } + + ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second) + defer cancel() + + client := g.olricClient.GetClient() + dm, err := client.NewDMap(req.DMap) + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to create DMap: %v", err)) + return + } + + var iterator olriclib.Iterator + if req.Match != "" { + iterator, err = dm.Scan(ctx, olriclib.Match(req.Match)) + } else { + iterator, err = dm.Scan(ctx) + } + + if err != nil { + writeError(w, http.StatusInternalServerError, fmt.Sprintf("failed to scan: %v", err)) + return + } + defer iterator.Close() + + var keys []string + for iterator.Next() { + keys = append(keys, iterator.Key()) + } + + writeJSON(w, http.StatusOK, map[string]any{ + "keys": keys, + "count": len(keys), + "dmap": req.DMap, + }) +} diff --git a/pkg/gateway/cache_handlers_test.go b/pkg/gateway/cache_handlers_test.go new file mode 100644 index 0000000..6f2a5f8 --- /dev/null +++ b/pkg/gateway/cache_handlers_test.go @@ -0,0 +1,202 @@ +package gateway + +import ( + "bytes" + "context" + "encoding/json" + "net/http" + "net/http/httptest" + "testing" + "time" + + "github.com/DeBrosOfficial/network/pkg/logging" + "github.com/DeBrosOfficial/network/pkg/olric" + "go.uber.org/zap" +) + +func TestCacheHealthHandler(t *testing.T) { + // Create a test logger + logger, _ := logging.NewDefaultLogger(logging.ComponentGeneral) + + // Create gateway without Olric client (should return service unavailable) + cfg := &Config{ + ListenAddr: ":6001", + ClientNamespace: "test", + } + gw := &Gateway{ + logger: logger, + cfg: cfg, + } + + req := httptest.NewRequest("GET", "/v1/cache/health", nil) + w := httptest.NewRecorder() + + gw.cacheHealthHandler(w, req) + + if w.Code != http.StatusServiceUnavailable { + t.Errorf("expected status %d, got %d", http.StatusServiceUnavailable, w.Code) + } + + var resp map[string]any + if err := json.NewDecoder(w.Body).Decode(&resp); err != nil { + t.Fatalf("failed to decode response: %v", err) + } + + if resp["error"] == nil { + t.Error("expected error in response") + } +} + +func TestCacheGetHandler_MissingClient(t *testing.T) { + logger, _ := logging.NewDefaultLogger(logging.ComponentGeneral) + + cfg := &Config{ + ListenAddr: ":6001", + ClientNamespace: "test", + } + gw := &Gateway{ + logger: logger, + cfg: cfg, + } + + reqBody := map[string]string{ + "dmap": "test-dmap", + "key": "test-key", + } + bodyBytes, _ := json.Marshal(reqBody) + req := httptest.NewRequest("POST", "/v1/cache/get", bytes.NewReader(bodyBytes)) + w := httptest.NewRecorder() + + gw.cacheGetHandler(w, req) + + if w.Code != http.StatusServiceUnavailable { + t.Errorf("expected status %d, got %d", http.StatusServiceUnavailable, w.Code) + } +} + +func TestCacheGetHandler_InvalidBody(t *testing.T) { + logger, _ := logging.NewDefaultLogger(logging.ComponentGeneral) + + cfg := &Config{ + ListenAddr: ":6001", + ClientNamespace: "test", + } + gw := &Gateway{ + logger: logger, + cfg: cfg, + olricClient: &olric.Client{}, // Mock client + } + + req := httptest.NewRequest("POST", "/v1/cache/get", bytes.NewReader([]byte("invalid json"))) + w := httptest.NewRecorder() + + gw.cacheGetHandler(w, req) + + if w.Code != http.StatusBadRequest { + t.Errorf("expected status %d, got %d", http.StatusBadRequest, w.Code) + } +} + +func TestCachePutHandler_MissingFields(t *testing.T) { + logger, _ := logging.NewDefaultLogger(logging.ComponentGeneral) + + cfg := &Config{ + ListenAddr: ":6001", + ClientNamespace: "test", + } + gw := &Gateway{ + logger: logger, + cfg: cfg, + olricClient: &olric.Client{}, + } + + // Test missing dmap + reqBody := map[string]string{ + "key": "test-key", + } + bodyBytes, _ := json.Marshal(reqBody) + req := httptest.NewRequest("POST", "/v1/cache/put", bytes.NewReader(bodyBytes)) + w := httptest.NewRecorder() + + gw.cachePutHandler(w, req) + + if w.Code != http.StatusBadRequest { + t.Errorf("expected status %d, got %d", http.StatusBadRequest, w.Code) + } + + // Test missing key + reqBody = map[string]string{ + "dmap": "test-dmap", + } + bodyBytes, _ = json.Marshal(reqBody) + req = httptest.NewRequest("POST", "/v1/cache/put", bytes.NewReader(bodyBytes)) + w = httptest.NewRecorder() + + gw.cachePutHandler(w, req) + + if w.Code != http.StatusBadRequest { + t.Errorf("expected status %d, got %d", http.StatusBadRequest, w.Code) + } +} + +func TestCacheDeleteHandler_WrongMethod(t *testing.T) { + logger, _ := logging.NewDefaultLogger(logging.ComponentGeneral) + + cfg := &Config{ + ListenAddr: ":6001", + ClientNamespace: "test", + } + gw := &Gateway{ + logger: logger, + cfg: cfg, + olricClient: &olric.Client{}, + } + + req := httptest.NewRequest("GET", "/v1/cache/delete", nil) + w := httptest.NewRecorder() + + gw.cacheDeleteHandler(w, req) + + if w.Code != http.StatusMethodNotAllowed { + t.Errorf("expected status %d, got %d", http.StatusMethodNotAllowed, w.Code) + } +} + +func TestCacheScanHandler_InvalidBody(t *testing.T) { + logger, _ := logging.NewDefaultLogger(logging.ComponentGeneral) + + cfg := &Config{ + ListenAddr: ":6001", + ClientNamespace: "test", + } + gw := &Gateway{ + logger: logger, + cfg: cfg, + olricClient: &olric.Client{}, + } + + req := httptest.NewRequest("POST", "/v1/cache/scan", bytes.NewReader([]byte("invalid"))) + w := httptest.NewRecorder() + + gw.cacheScanHandler(w, req) + + if w.Code != http.StatusBadRequest { + t.Errorf("expected status %d, got %d", http.StatusBadRequest, w.Code) + } +} + +// Test Olric client wrapper +func TestOlricClientConfig(t *testing.T) { + logger := zap.NewNop() + + // Test default servers + cfg := olric.Config{} + client, err := olric.NewClient(cfg, logger) + if err == nil { + // If client creation succeeds, test that it has default servers + // This will fail if Olric server is not running, which is expected in tests + ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second) + defer cancel() + _ = client.Close(ctx) + } +} diff --git a/pkg/gateway/gateway.go b/pkg/gateway/gateway.go index 62354cb..f941c42 100644 --- a/pkg/gateway/gateway.go +++ b/pkg/gateway/gateway.go @@ -5,13 +5,16 @@ import ( "crypto/rand" "crypto/rsa" "database/sql" + "net" "strconv" "sync" "time" "github.com/DeBrosOfficial/network/pkg/client" "github.com/DeBrosOfficial/network/pkg/logging" + "github.com/DeBrosOfficial/network/pkg/olric" "github.com/DeBrosOfficial/network/pkg/rqlite" + "github.com/multiformats/go-multiaddr" "go.uber.org/zap" _ "github.com/rqlite/gorqlite/stdlib" @@ -31,6 +34,10 @@ type Config struct { EnableHTTPS bool // Enable HTTPS with ACME (Let's Encrypt) DomainName string // Domain name for HTTPS certificate TLSCacheDir string // Directory to cache TLS certificates (default: ~/.debros/tls-cache) + + // Olric cache configuration + OlricServers []string // List of Olric server addresses (e.g., ["localhost:3320"]). If empty, defaults to ["localhost:3320"] + OlricTimeout time.Duration // Timeout for Olric operations (default: 10s) } type Gateway struct { @@ -46,6 +53,9 @@ type Gateway struct { ormClient rqlite.Client ormHTTP *rqlite.HTTPGateway + // Olric cache client + olricClient *olric.Client + // Local pub/sub bypass for same-gateway subscribers localSubscribers map[string][]*localSubscriber // topic+namespace -> subscribers mu sync.RWMutex @@ -132,6 +142,42 @@ func New(logger *logging.ColoredLogger, cfg *Config) (*Gateway, error) { ) } + logger.ComponentInfo(logging.ComponentGeneral, "Initializing Olric cache client...") + + // Discover Olric servers dynamically from LibP2P peers if not explicitly configured + olricServers := cfg.OlricServers + if len(olricServers) == 0 { + logger.ComponentInfo(logging.ComponentGeneral, "Olric servers not configured, discovering from LibP2P peers...") + discovered := discoverOlricServers(c, logger.Logger) + if len(discovered) > 0 { + olricServers = discovered + logger.ComponentInfo(logging.ComponentGeneral, "Discovered Olric servers from LibP2P peers", + zap.Strings("servers", olricServers)) + } else { + // Fallback to localhost for local development + olricServers = []string{"localhost:3320"} + logger.ComponentInfo(logging.ComponentGeneral, "No Olric servers discovered, using localhost fallback") + } + } else { + logger.ComponentInfo(logging.ComponentGeneral, "Using explicitly configured Olric servers", + zap.Strings("servers", olricServers)) + } + + olricCfg := olric.Config{ + Servers: olricServers, + Timeout: cfg.OlricTimeout, + } + olricClient, olricErr := olric.NewClient(olricCfg, logger.Logger) + if olricErr != nil { + logger.ComponentWarn(logging.ComponentGeneral, "failed to initialize Olric cache client; cache endpoints disabled", zap.Error(olricErr)) + } else { + gw.olricClient = olricClient + logger.ComponentInfo(logging.ComponentGeneral, "Olric cache client ready", + zap.Strings("servers", olricCfg.Servers), + zap.Duration("timeout", olricCfg.Timeout), + ) + } + logger.ComponentInfo(logging.ComponentGeneral, "Gateway creation completed, returning...") return gw, nil } @@ -151,6 +197,13 @@ func (g *Gateway) Close() { if g.sqlDB != nil { _ = g.sqlDB.Close() } + if g.olricClient != nil { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + if err := g.olricClient.Close(ctx); err != nil { + g.logger.ComponentWarn(logging.ComponentGeneral, "error during Olric client close", zap.Error(err)) + } + } } // getLocalSubscribers returns all local subscribers for a given topic and namespace @@ -161,3 +214,96 @@ func (g *Gateway) getLocalSubscribers(topic, namespace string) []*localSubscribe } return nil } + +// discoverOlricServers discovers Olric server addresses from LibP2P peers +// Returns a list of IP:port addresses where Olric servers are expected to run (port 3320) +func discoverOlricServers(networkClient client.NetworkClient, logger *zap.Logger) []string { + // Get network info to access peer information + networkInfo := networkClient.Network() + if networkInfo == nil { + logger.Debug("Network info not available for Olric discovery") + return nil + } + + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + + peers, err := networkInfo.GetPeers(ctx) + if err != nil { + logger.Debug("Failed to get peers for Olric discovery", zap.Error(err)) + return nil + } + + olricServers := make([]string, 0) + seen := make(map[string]bool) + + for _, peer := range peers { + for _, addrStr := range peer.Addresses { + // Parse multiaddr + ma, err := multiaddr.NewMultiaddr(addrStr) + if err != nil { + continue + } + + // Extract IP address + var ip string + if ipv4, err := ma.ValueForProtocol(multiaddr.P_IP4); err == nil && ipv4 != "" { + ip = ipv4 + } else if ipv6, err := ma.ValueForProtocol(multiaddr.P_IP6); err == nil && ipv6 != "" { + ip = ipv6 + } else { + continue + } + + // Skip localhost loopback addresses (we'll use localhost:3320 as fallback) + if ip == "127.0.0.1" || ip == "::1" || ip == "localhost" { + continue + } + + // Build Olric server address (standard port 3320) + olricAddr := net.JoinHostPort(ip, "3320") + if !seen[olricAddr] { + olricServers = append(olricServers, olricAddr) + seen[olricAddr] = true + } + } + } + + // Also check bootstrap peers from config + if cfg := networkClient.Config(); cfg != nil { + for _, bootstrapAddr := range cfg.BootstrapPeers { + ma, err := multiaddr.NewMultiaddr(bootstrapAddr) + if err != nil { + continue + } + + var ip string + if ipv4, err := ma.ValueForProtocol(multiaddr.P_IP4); err == nil && ipv4 != "" { + ip = ipv4 + } else if ipv6, err := ma.ValueForProtocol(multiaddr.P_IP6); err == nil && ipv6 != "" { + ip = ipv6 + } else { + continue + } + + // Skip localhost + if ip == "127.0.0.1" || ip == "::1" || ip == "localhost" { + continue + } + + olricAddr := net.JoinHostPort(ip, "3320") + if !seen[olricAddr] { + olricServers = append(olricServers, olricAddr) + seen[olricAddr] = true + } + } + } + + // If we found servers, log them + if len(olricServers) > 0 { + logger.Info("Discovered Olric servers from LibP2P network", + zap.Strings("servers", olricServers)) + } + + return olricServers +} diff --git a/pkg/gateway/routes.go b/pkg/gateway/routes.go index cce24e8..25d09a0 100644 --- a/pkg/gateway/routes.go +++ b/pkg/gateway/routes.go @@ -47,5 +47,12 @@ func (g *Gateway) Routes() http.Handler { // anon proxy (authenticated users only) mux.HandleFunc("/v1/proxy/anon", g.anonProxyHandler) + // cache endpoints (Olric) + mux.HandleFunc("/v1/cache/health", g.cacheHealthHandler) + mux.HandleFunc("/v1/cache/get", g.cacheGetHandler) + mux.HandleFunc("/v1/cache/put", g.cachePutHandler) + mux.HandleFunc("/v1/cache/delete", g.cacheDeleteHandler) + mux.HandleFunc("/v1/cache/scan", g.cacheScanHandler) + return g.withMiddleware(mux) } diff --git a/pkg/olric/client.go b/pkg/olric/client.go new file mode 100644 index 0000000..d2b78bd --- /dev/null +++ b/pkg/olric/client.go @@ -0,0 +1,103 @@ +package olric + +import ( + "context" + "fmt" + "time" + + olriclib "github.com/olric-data/olric" + "go.uber.org/zap" +) + +// Client wraps an Olric cluster client for distributed cache operations +type Client struct { + client olriclib.Client + logger *zap.Logger +} + +// Config holds configuration for the Olric client +type Config struct { + // Servers is a list of Olric server addresses (e.g., ["localhost:3320"]) + // If empty, defaults to ["localhost:3320"] + Servers []string + + // Timeout is the timeout for client operations + // If zero, defaults to 10 seconds + Timeout time.Duration +} + +// NewClient creates a new Olric client wrapper +func NewClient(cfg Config, logger *zap.Logger) (*Client, error) { + servers := cfg.Servers + if len(servers) == 0 { + servers = []string{"localhost:3320"} + } + + client, err := olriclib.NewClusterClient(servers) + if err != nil { + return nil, fmt.Errorf("failed to create Olric cluster client: %w", err) + } + + timeout := cfg.Timeout + if timeout == 0 { + timeout = 10 * time.Second + } + + return &Client{ + client: client, + logger: logger, + }, nil +} + +// Health checks if the Olric client is healthy +func (c *Client) Health(ctx context.Context) error { + // Create a DMap to test connectivity + dm, err := c.client.NewDMap("_health_check") + if err != nil { + return fmt.Errorf("failed to create DMap for health check: %w", err) + } + + // Try a simple put/get operation + testKey := fmt.Sprintf("_health_%d", time.Now().UnixNano()) + testValue := "ok" + + ctx, cancel := context.WithTimeout(ctx, 5*time.Second) + defer cancel() + + err = dm.Put(ctx, testKey, testValue) + if err != nil { + return fmt.Errorf("health check put failed: %w", err) + } + + gr, err := dm.Get(ctx, testKey) + if err != nil { + return fmt.Errorf("health check get failed: %w", err) + } + + val, err := gr.String() + if err != nil { + return fmt.Errorf("health check value decode failed: %w", err) + } + + if val != testValue { + return fmt.Errorf("health check value mismatch: expected %q, got %q", testValue, val) + } + + // Clean up test key + _, _ = dm.Delete(ctx, testKey) + + return nil +} + +// Close closes the Olric client connection +func (c *Client) Close(ctx context.Context) error { + if c.client == nil { + return nil + } + return c.client.Close(ctx) +} + +// GetClient returns the underlying Olric client +func (c *Client) GetClient() olriclib.Client { + return c.client +} diff --git a/scripts/install-debros-network.sh b/scripts/install-debros-network.sh index efbc1ba..a1bd4e5 100755 --- a/scripts/install-debros-network.sh +++ b/scripts/install-debros-network.sh @@ -396,6 +396,50 @@ configure_firewall_for_anon() { log "No active firewall detected, skipping firewall configuration" } +configure_firewall_for_olric() { + log "Checking firewall configuration for Olric..." + + # Check for UFW + if command -v ufw &>/dev/null && sudo ufw status | grep -q "Status: active"; then + log "UFW detected and active, adding Olric ports..." + sudo ufw allow 3320/tcp comment 'Olric HTTP API' 2>/dev/null || true + sudo ufw allow 3322/tcp comment 'Olric Memberlist' 2>/dev/null || true + success "UFW rules added for Olric" + return 0 + fi + + # Check for firewalld + if command -v firewall-cmd &>/dev/null && sudo firewall-cmd --state 2>/dev/null | grep -q "running"; then + log "firewalld detected and active, adding Olric ports..." + sudo firewall-cmd --permanent --add-port=3320/tcp 2>/dev/null || true + sudo firewall-cmd --permanent --add-port=3322/tcp 2>/dev/null || true + sudo firewall-cmd --reload 2>/dev/null || true + success "firewalld rules added for Olric" + return 0 + fi + + # Check for iptables + if command -v iptables &>/dev/null; then + # Check if iptables has any rules (indicating it's in use) + if sudo iptables -L -n | grep -q "Chain INPUT"; then + log "iptables detected, adding Olric ports..." + sudo iptables -A INPUT -p tcp --dport 3320 -j ACCEPT -m comment --comment "Olric HTTP API" 2>/dev/null || true + sudo iptables -A INPUT -p tcp --dport 3322 -j ACCEPT -m comment --comment "Olric Memberlist" 2>/dev/null || true + + # Try to save rules if iptables-persistent is available + if command -v netfilter-persistent &>/dev/null; then + sudo netfilter-persistent save 2>/dev/null || true + elif command -v iptables-save &>/dev/null; then + sudo iptables-save | sudo tee /etc/iptables/rules.v4 >/dev/null 2>&1 || true + fi + success "iptables rules added for Olric" + return 0 + fi + fi + + log "No active firewall detected for Olric, skipping firewall configuration" +} + run_setup() { echo -e "" echo -e "${BLUE}========================================${NOCOLOR}"