mirror of
https://github.com/DeBrosOfficial/network.git
synced 2025-12-12 22:58:49 +00:00
69d7ccf4c7
- Added automatic setup for IPFS and IPFS Cluster during the network setup process. - Implemented initialization of IPFS repositories and Cluster configurations for each node. - Enhanced Makefile to support starting IPFS and Cluster daemons with improved logging. - Introduced a new documentation guide for IPFS Cluster setup, detailing configuration and verification steps. - Updated changelog to reflect the new features and improvements.
172 lines
4.6 KiB
Markdown
172 lines
4.6 KiB
Markdown
# IPFS Cluster Setup Guide
|
|
|
|
This guide explains how IPFS Cluster is configured to run on every DeBros Network node.
|
|
|
|
## Overview
|
|
|
|
Each DeBros Network node runs its own IPFS Cluster peer, enabling distributed pinning and replication across the network. The cluster uses CRDT consensus for automatic peer discovery.
|
|
|
|
## Architecture
|
|
|
|
- **IPFS (Kubo)**: Runs on each node, handles content storage and retrieval
|
|
- **IPFS Cluster**: Runs on each node, manages pinning and replication
|
|
- **Cluster Consensus**: Uses CRDT (instead of Raft) for simpler multi-node setup
|
|
|
|
## Automatic Setup
|
|
|
|
When you run `network-cli setup`, the following happens automatically:
|
|
|
|
1. IPFS (Kubo) and IPFS Cluster are installed
|
|
2. IPFS repository is initialized for each node
|
|
3. IPFS Cluster service.json config is generated
|
|
4. Systemd services are created and started:
|
|
- `debros-ipfs` - IPFS daemon
|
|
- `debros-ipfs-cluster` - IPFS Cluster service
|
|
- `debros-node` - DeBros Network node (depends on cluster)
|
|
- `debros-gateway` - HTTP Gateway (depends on node)
|
|
|
|
## Configuration
|
|
|
|
### Node Configs
|
|
|
|
Each node config (`~/.debros/bootstrap.yaml`, `~/.debros/node.yaml`, etc.) includes:
|
|
|
|
```yaml
|
|
database:
|
|
ipfs:
|
|
cluster_api_url: "http://localhost:9094" # Local cluster API
|
|
api_url: "http://localhost:5001" # Local IPFS API
|
|
replication_factor: 3 # Desired replication
|
|
```
|
|
|
|
### Cluster Service Config
|
|
|
|
Cluster service configs are stored at:
|
|
|
|
- Bootstrap: `~/.debros/bootstrap/ipfs-cluster/service.json`
|
|
- Nodes: `~/.debros/node/ipfs-cluster/service.json`
|
|
|
|
Key settings:
|
|
|
|
- **Consensus**: CRDT (automatic peer discovery)
|
|
- **API Listen**: `0.0.0.0:9094` (REST API)
|
|
- **Cluster Listen**: `0.0.0.0:9096` (peer-to-peer)
|
|
- **Secret**: Shared cluster secret stored at `~/.debros/cluster-secret`
|
|
|
|
## Verification
|
|
|
|
### Check Cluster Peers
|
|
|
|
From any node, verify all cluster peers are connected:
|
|
|
|
```bash
|
|
sudo -u debros ipfs-cluster-ctl --host http://localhost:9094 peers ls
|
|
```
|
|
|
|
You should see all cluster peers listed (bootstrap, node1, node2, etc.).
|
|
|
|
### Check IPFS Daemon
|
|
|
|
Verify IPFS is running:
|
|
|
|
```bash
|
|
sudo -u debros ipfs daemon --repo-dir=~/.debros/bootstrap/ipfs/repo
|
|
# Or for regular nodes:
|
|
sudo -u debros ipfs daemon --repo-dir=~/.debros/node/ipfs/repo
|
|
```
|
|
|
|
### Check Service Status
|
|
|
|
```bash
|
|
network-cli service status all
|
|
```
|
|
|
|
Should show:
|
|
|
|
- `debros-ipfs` - running
|
|
- `debros-ipfs-cluster` - running
|
|
- `debros-node` - running
|
|
- `debros-gateway` - running
|
|
|
|
## Troubleshooting
|
|
|
|
### Cluster Peers Not Connecting
|
|
|
|
If peers aren't discovering each other:
|
|
|
|
1. **Check firewall**: Ensure ports 9096 (cluster swarm) and 9094 (cluster API) are open
|
|
2. **Verify secret**: All nodes must use the same cluster secret from `~/.debros/cluster-secret`
|
|
3. **Check logs**: `journalctl -u debros-ipfs-cluster -f`
|
|
|
|
### Not Enough Peers Error
|
|
|
|
If you see "not enough peers to allocate CID" errors:
|
|
|
|
- The cluster needs at least `replication_factor` peers running
|
|
- Check that all nodes have `debros-ipfs-cluster` service running
|
|
- Verify with `ipfs-cluster-ctl peers ls`
|
|
|
|
### IPFS Not Starting
|
|
|
|
If IPFS daemon fails to start:
|
|
|
|
1. Check IPFS repo exists: `ls -la ~/.debros/bootstrap/ipfs/repo/`
|
|
2. Check permissions: `chown -R debros:debros ~/.debros/bootstrap/ipfs/`
|
|
3. Check logs: `journalctl -u debros-ipfs -f`
|
|
|
|
## Manual Setup (If Needed)
|
|
|
|
If automatic setup didn't work, you can manually initialize:
|
|
|
|
### 1. Initialize IPFS
|
|
|
|
```bash
|
|
sudo -u debros ipfs init --profile=server --repo-dir=~/.debros/bootstrap/ipfs/repo
|
|
sudo -u debros ipfs config --json Addresses.API '["/ip4/localhost/tcp/5001"]' --repo-dir=~/.debros/bootstrap/ipfs/repo
|
|
```
|
|
|
|
### 2. Initialize Cluster
|
|
|
|
```bash
|
|
# Generate or get cluster secret
|
|
CLUSTER_SECRET=$(cat ~/.debros/cluster-secret)
|
|
|
|
# Initialize cluster (will create service.json)
|
|
sudo -u debros ipfs-cluster-service init --consensus crdt
|
|
```
|
|
|
|
### 3. Start Services
|
|
|
|
```bash
|
|
systemctl start debros-ipfs
|
|
systemctl start debros-ipfs-cluster
|
|
systemctl start debros-node
|
|
systemctl start debros-gateway
|
|
```
|
|
|
|
## Ports
|
|
|
|
- **4001**: IPFS swarm (LibP2P)
|
|
- **5001**: IPFS HTTP API
|
|
- **8080**: IPFS Gateway (optional)
|
|
- **9094**: IPFS Cluster REST API
|
|
- **9096**: IPFS Cluster swarm (LibP2P)
|
|
|
|
## Replication Factor
|
|
|
|
The default replication factor is 3, meaning content is pinned to 3 cluster peers. This requires at least 3 nodes running cluster peers.
|
|
|
|
To change replication factor, edit node configs:
|
|
|
|
```yaml
|
|
database:
|
|
ipfs:
|
|
replication_factor: 1 # For single-node development
|
|
```
|
|
|
|
## Security Notes
|
|
|
|
- Cluster secret is stored at `~/.debros/cluster-secret` (mode 0600)
|
|
- Cluster API (port 9094) should be firewalled in production
|
|
- IPFS API (port 5001) should only be accessible locally
|