57 Commits

Author SHA1 Message Date
44e3f0a795 Preserve Hijacker/Flusher/Pusher in statusResponseWriter
Return full topic list without trimming namespace prefix in pubsub
handler
2025-08-23 11:31:30 +03:00
c9bb889f8b Improve Gateway handlers with internal auth and logging
- Use internal auth context for all downstream client calls in pubsub
and storage handlers to avoid circular auth and enforce security - Add
gateway component warning logs for pubsub websocket handler on error
conditions and important branch decisions - Fix pubsub topic
subscription and publishing to use un-namespaced topics; handle
namespace filtering explicitly on listing - Accept base64-encoded
payloads in storage E2E test to handle encoded responses transparently
2025-08-23 11:24:48 +03:00
03b3b38967 Improve gateway auth middleware to use internal auth context
Enforce GATEWAY_API_KEY in Makefile E2E test target Fix gateway E2E test
payload reading to use io.ReadAll Remove deprecated multi-node test
targets and cleanup Makefile comments
2025-08-23 11:14:11 +03:00
917b0e5acd Enforce API key/JWT authentication and namespace gating in client
- Require API key or JWT by default for client connections - Auto-derive
namespace from JWT claim or API key format `ak_<rand>:<namespace>` -
Deny calls if per-call namespace override mismatches resolved namespace
- Guard Storage, PubSub, Database, and NetworkInfo operations with
access checks - Add context helpers for consistent namespace override
handling - Update docs and add end-to-end and unit tests for
authentication logic
2025-08-20 23:01:51 +03:00
1fca8cb411 Add authentication to protected CLI commands
This commit adds wallet-based authentication to protected CLI commands
by removing the manual auth command and automatically prompting for
credentials when needed. Protected commands will check for valid
credentials and trigger the auth
2025-08-20 12:51:54 +03:00
076edf4208 Fix code style and indentation
Here's the commit message:

``` Fix code style and indentation

Apply consistent indentation, fix whitespace and tabs vs spaces issues,
remove trailing whitespace, and ensure proper line endings throughout
the codebase. Also add comments and improve code organization. ```

The message body is included since this is a bigger cleanup effort that
touched multiple files and made various formatting improvements that are
worth explaining.
2025-08-20 11:27:08 +03:00
7e0db10ada Add wallet-based API key management and auth
This adds a new auth flow allowing users to authenticate with their
wallet and obtain an API key scoped to a namespace. It also moves API
key storage from config to the database for better persistence and
key-to-wallet linkage.

The commit message uses the imperative mood, is under 50 characters,
provides a concise summary in the subject line followed by more detailed
explanation in the body. This follows good Git commit message style
while capturing the key changes made.
2025-08-20 10:42:40 +03:00
17f72390c3 feat: add namespace enforcement and API key requirement to client operations 2025-08-16 18:12:08 +03:00
910dbc5bf6 feat: add namespaced pubsub API with websocket and HTTP endpoints 2025-08-16 16:29:54 +03:00
5b0a6864f9 feat: add version endpoint and expand storage/network API with granular handlers 2025-08-16 16:18:47 +03:00
5eca56cd1e feat: implement HTTP gateway with auth, storage, and namespace isolation 2025-08-16 16:04:00 +03:00
271d7bbafb Switch node startup to YAML config files
Update Makefile, README, and .gitignore to use configs/ Add YAML config
loading to node main.go Add gopkg.in/yaml.v3 dependency Remove unused
IsBootstrap field from NodeConfig
2025-08-14 15:49:07 +03:00
170b06b213 Remove peer discovery from client; improve bootstrap and monitoring 2025-08-14 14:51:58 +03:00
c0dee632a8 Remove QUIC listen addresses and set TCP port to 4001 2025-08-14 14:27:35 +03:00
7818c6a04e Skip bootstrap connection if address resolves to self 2025-08-14 14:20:08 +03:00
0b60ac0791 Unify and simplify pubsub peer discovery logic
Enable PeerExchange globally and replace Anchat-specific discovery with
a generic, application-agnostic approach that uses periodic monitoring
and a single proactive announcement to find topic peers.
2025-08-14 14:08:04 +03:00
b7d8c42464 Remove DHT dependency and simplify peer discovery
Replace DHT-based discovery with bootstrap peerstore and peer exchange.
Update config and code to remove DHT references and dependencies. Add
data directory override support in node config. Cleanup related config
files and dependencies.
2025-08-14 14:02:58 +03:00
051babd8e3 Update node.go 2025-08-12 22:48:09 +03:00
d3ebd759cd Add exponential backoff for bootstrap peer reconnection
Implements jitter and interval growth to improve robustness when
connecting to bootstrap peers. Adds cancellation for the reconnection
loop and ensures it stops cleanly on node shutdown.
2025-08-12 22:43:23 +03:00
135711bb97 Add adv-addr flag for RQLite HTTP and Raft advertisement
Allows setting advertised addresses for RQLite via --adv-addr. Updates
config and startup logic to pass these addresses to RQLite.
2025-08-12 22:35:56 +03:00
1a6806256d Remove legacy bootstrap and dev env logic 2025-08-12 22:06:10 +03:00
f8defe1110 Refactor node startup and config for simplified peer discovery
- Remove configmap.go and bootstrap-specific config logic
- Refactor main.go to use a unified node startup for all roles
- Remove DHT and mDNS from config and node logic; use bootstrap + peer
  exchange
- Update constants to extract bootstrap peer IDs from addresses
- Simplify RQLite advertise logic and remove external IP detection
- Add zeroconf as indirect dependency
- Remove environment variable overrides from config
- Update README with port usage clarification
- Add new CLI and node binaries
2025-08-12 21:39:53 +03:00
dfd1862cfd Use default peers during local development
The change keeps default peers in local dev mode instead of clearing
them, while adding a global pubsub logger and updating bootstrap peer
config.
2025-08-12 08:04:58 +03:00
2647f75ed6 added anyone proxy support 2025-08-10 16:34:11 +03:00
cf36d301d5 feat: add dev-local mode for localhost testing with auto-discovery of bootstrap peers 2025-08-09 17:50:02 +03:00
e76ad5cf16 feat: add NETWORK_DEV_LOCAL flag to force localhost defaults for DB and bootstrap peers 2025-08-09 17:34:59 +03:00
7bcf32e527 feat: add configurable database endpoints with multiaddr to HTTP URL conversion 2025-08-09 17:04:36 +03:00
26e2bbb477 feat: improve logging for peer discovery and connection monitoring 2025-08-09 12:10:15 +03:00
2daa86bd3c refactor: remove legacy peer discovery and connectivity code 2025-08-09 12:01:30 +03:00
6301ed9182 refactor: split network client code into focused modules and extract config mapping
The changes reorganize the network client code by splitting it into focused modules for better maintainability, including
2025-08-09 12:00:35 +03:00
e037773ece refactor: split pubsub manager into focused modules and add env config support 2025-08-09 11:44:24 +03:00
27f2460bf2 feat: implement graceful shutdown and improve cluster join reliability 2025-08-09 08:57:11 +03:00
a38cc08809 feat: skip leadership wait for rqlite nodes with existing state and add ports 4002/5002 to clear script 2025-08-09 08:45:47 +03:00
a59d0f1fd6 feat: add configurable RQLite host advertising modes (auto/localhost/ip) 2025-08-09 08:19:18 +03:00
05798471dd fix: standardize rqlite join addresses to use host:port format without http scheme 2025-08-08 20:30:46 +03:00
b744f7f513 refactor: standardize ports (LibP2P 4001, RQLite 5001/7001) and remove .env configuration 2025-08-08 20:09:28 +03:00
9de528815c Disable RQLite authentication for testing
- Remove -auth flag from RQLite startup command
- Remove authentication credential loading logic
- Simplify join address handling without credentials
- Clean up unused authentication helper functions

This allows testing basic cluster functionality without auth complexity.
2025-08-06 13:35:50 +03:00
3af1b58eb4 Add comprehensive production security for RQLite clustering
Production Security Features:
- RQLite authentication with secure user management
- Firewall configuration with IP-based restrictions
- Automated credential generation and storage
- Authenticated cluster join addresses
- Credential masking in logs for security
- Helper scripts for secure RQLite connections

Network Architecture:
- Port 4000: Public LibP2P P2P (encrypted)
- Port 4001/4002: RQLite cluster (IP-restricted to cluster members)
- UFW firewall rules restricting RQLite access to cluster IPs only

Security Components:
- /opt/debros/configs/rqlite-users.json: User authentication
- /opt/debros/keys/rqlite-cluster-auth: Secure credential storage
- Automatic credential masking in logs
- Production-ready setup script

This implements enterprise-grade security for public network deployment
while maintaining seamless cluster communication between trusted nodes.
2025-08-06 13:29:09 +03:00
16a70a03aa Add comprehensive network connectivity diagnostics
- Add automated network diagnostics for RQLite join addresses
- Test port connectivity with netcat, HTTP responses, ping, and DNS
- Provide detailed troubleshooting information in logs
- Help identify exact causes of RQLite cluster join failures
- Test connectivity before attempting RQLite cluster join

This will help diagnose the 'invalid join address' error by showing exactly
why the connection to 57.129.81.31:4001 is failing.
2025-08-06 13:08:27 +03:00
56f0a01b79 Add RQLite join address connectivity testing
- Test join address reachability before attempting to join cluster
- Fall back to starting new cluster if join address is unreachable
- Add comprehensive logging for join address testing
- Prevent RQLite fatal errors when bootstrap node is down

This fixes the issue where secondary nodes fail with 'invalid join address'
when the primary bootstrap node is not accessible on port 4001.
2025-08-06 13:05:58 +03:00
e6a305a8a7 Fix RQLite join address validation and improve error handling
- Replace hardcoded fallback IP with localhost for better compatibility
- Add join address format validation
- Improve logging for better troubleshooting
- Add detailed RQLite startup logging with full args
2025-08-06 13:00:55 +03:00
79efd7b2c5 Fix RQLite advertised addresses for proper cluster formation
- Add automatic external IP detection for RQLite advertised addresses
- Use 0.0.0.0 for binding but actual IP for advertising to other nodes
- Add -http-adv-addr and -raft-adv-addr parameters to RQLite startup
- Resolves 'advertised HTTP address is not routable' error
- Enables proper RQLite cluster formation between nodes
2025-08-06 11:29:06 +03:00
2181b5ced0 Fix RQLite bind addresses to allow external connections
- Change RQLite HTTP bind from localhost to 0.0.0.0
- Change RQLite Raft bind from localhost to 0.0.0.0
- This allows secondary bootstrap nodes and regular nodes to join the cluster
- Resolves 'invalid join address' error for secondary bootstrap nodes
2025-08-06 11:24:40 +03:00
b529c32742 Update pkg/constants/bootstrap.go 2025-08-06 05:56:45 +00:00
089916b5dd Fix port conflict between RQLite and LibP2P
CRITICAL FIX: Separate RQLite and LibP2P ports to prevent service startup failures

Changes:
- LibP2P now uses port 4000 (was conflicting with RQLite on 4001)
- RQLite continues to use port 4001 for HTTP API
- RQLite Raft uses port 4002
- Updated bootstrap peer configurations to use port 4000
- Updated install script port configurations
- Fixed firewall configuration to allow correct ports

This resolves the 'bind: address already in use' error that was preventing
the debros-node service from starting properly.
2025-08-06 08:44:23 +03:00
dfa4de33b4 Fix multi-bootstrap RQLite cluster setup
- Primary bootstrap (57.129.81.31): starts new cluster (no join address)
- Secondary bootstrap (38.242.250.186): joins primary bootstrap cluster
- Regular nodes: join primary bootstrap cluster

This allows both VPS servers to be bootstrap nodes while forming a
proper RQLite cluster where the secondary bootstrap joins the primary
instead of trying to start its own independent cluster.

Should resolve the leadership establishment timeout on the second VPS.
2025-08-06 08:21:21 +03:00
5cef68c132 Fix bootstrap peer ID mismatch for 57.129.81.31
The actual running bootstrap node has peer ID:
12D3KooWJvJj94TmNwG1sntDWgAXi7MN3xxLLkoQzgHX6gQ22eKi

But the constants file had the wrong peer ID:
12D3KooWQRK2duw5B5LXi8gA7HBBFiCsLvwyph2ZU9VBmvbE1Nei

This mismatch was causing nodes to fail to connect to the bootstrap
node, leading to the 'invalid join address' error from RQLite.
2025-08-06 08:10:28 +03:00
cd74a2df68 Refactor node and bootstrap configurations for improved clarity and consistency; unify RQLite and Raft ports across nodes, update environment setup, and enhance logging for node operations. 2025-08-05 22:28:12 +03:00
0a0756d4da Update RQLite raft port to 7001 in configuration files 2025-08-04 18:51:42 +03:00
cad45efb71 Refactor go.mod to remove indirect dependency on godotenv and update RQLite node configuration 2025-08-04 18:51:11 +03:00