feat: enhance binary installation and ownership management

- Improved the binary installation process by adding checks for the existence and contents of the source binary directory, ensuring a successful build. - Implemented individual file copying to avoid issues with wildcard expansion, enhancing reliability during the installation. - Added ownership correction for directories and files created during service initialization, ensuring proper permissions for the debros user. - Enhanced error handling for ownership changes, providing detailed output in case of failures.
2025-12-11 08:18:49 +00:00 · 2025-11-11 09:33:01 +02:00 · 2025-11-11 09:33:01 +02:00 · 7c3378a8ec
commit 7c3378a8ec
parent bd4542ef56
6 changed files with 228 additions and 21 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -13,6 +13,23 @@ The format is based on [Keep a Changelog][keepachangelog] and adheres to [Semant
 ### Deprecated

 ### Fixed
+## [0.69.0] - 2025-11-11
+
+### Added
+- Added comprehensive documentation for setting up HTTPS using a domain name, including configuration steps for both installation and existing setups.
+- Added the `--force` flag to the `install` command for reconfiguring all settings.
+- Added new log targets (`ipfs-cluster`, `rqlite`, `olric`) and improved the `dbn prod logs` command documentation.
+
+### Changed
+- Improved the IPFS Cluster configuration logic to ensure the cluster secret and IPFS API port are correctly synchronized during updates.
+- Refined the directory structure creation process to ensure node-specific data directories are created only when initializing services.
+
+### Deprecated
+
+### Removed
+
+### Fixed
+\n
 ## [0.68.1] - 2025-11-11

 ### Added
--- a/2
+++ b/2
@ -19,7 +19,7 @@ test-e2e:

 .PHONY: build clean test run-node run-node2 run-node3 run-example deps tidy fmt vet lint clear-ports install-hooks kill

-VERSION := 0.68.1
+VERSION := 0.69.0
 COMMIT  ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo unknown)
 DATE    ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
 LDFLAGS := -X 'main.version=$(VERSION)' -X 'main.commit=$(COMMIT)' -X 'main.date=$(DATE)'
--- a/README.md
+++ b/README.md
@ -111,8 +111,9 @@ sudo dbn prod install \
 **Optional flags:**

 - `--branch`: Git branch to use (`main` or `nightly`, default: `main`)
- `--domain`: Domain name for HTTPS (enables ACME/Let's Encrypt)
+- `--domain`: Domain name for HTTPS (enables ACME/Let's Encrypt) - see [HTTPS Setup](#https-setup-with-domain) below
 - `--bootstrap-join`: Raft join address for secondary bootstrap nodes
+- `--force`: Reconfigure all settings (use with caution)

 #### Secondary Bootstrap Node

@ -174,6 +175,103 @@ The upgrade process:

 **Note**: The upgrade automatically detects your node type (bootstrap vs. regular node) and preserves all secrets, data, and configurations.

+**Note**: Currently, the `upgrade` command does not support adding a domain via `--domain` flag. To enable HTTPS after installation, see [Adding Domain After Installation](#adding-domain-after-installation) below.
+
+### HTTPS Setup with Domain
+
+DeBros Gateway supports automatic HTTPS with Let's Encrypt certificates via ACME. This enables secure connections on ports 80 (HTTP redirect) and 443 (HTTPS).
+
+#### Prerequisites
+
+- Domain name pointing to your server's public IP address
+- Ports 80 and 443 open and accessible from the internet
+- Gateway service running
+
+#### Adding Domain During Installation
+
+Specify your domain during installation:
+
+```bash
+# Bootstrap node with HTTPS
+sudo dbn prod install --bootstrap --domain node-kv4la8.debros.network --branch nightly
+
+# Secondary node with HTTPS
+sudo dbn prod install \
+  --vps-ip <your_public_ip> \
+  --peers /ip4/<bootstrap_ip>/tcp/4001/p2p/<peer_id> \
+  --domain example.com \
+  --branch nightly
+```
+
+The gateway will automatically:
+
+- Obtain Let's Encrypt certificates via ACME
+- Serve HTTP on port 80 (redirects to HTTPS)
+- Serve HTTPS on port 443
+- Renew certificates automatically
+
+#### Adding Domain After Installation
+
+Currently, the `upgrade` command doesn't support `--domain` flag. To enable HTTPS on an existing installation:
+
+1. **Edit the gateway configuration:**
+
+```bash
+sudo nano /home/debros/.debros/configs/gateway.yaml
+```
+
+2. **Update the configuration:**
+
+```yaml
+listen_addr: ":6001"
+client_namespace: "default"
+rqlite_dsn: ""
+bootstrap_peers: []
+enable_https: true
+domain_name: "your-domain.com"
+tls_cache_dir: "/home/debros/.debros/tls-cache"
+olric_servers:
+  - "127.0.0.1:3320"
+olric_timeout: "10s"
+ipfs_cluster_api_url: "http://localhost:9094"
+ipfs_api_url: "http://localhost:4501"
+ipfs_timeout: "60s"
+ipfs_replication_factor: 3
+```
+
+3. **Ensure ports 80 and 443 are available:**
+
+```bash
+# Check if ports are in use
+sudo lsof -i :80
+sudo lsof -i :443
+
+# If needed, stop conflicting services
+```
+
+4. **Restart the gateway:**
+
+```bash
+sudo systemctl restart debros-gateway.service
+```
+
+5. **Verify HTTPS is working:**
+
+```bash
+# Check gateway logs
+sudo journalctl -u debros-gateway.service -f
+
+# Test HTTPS endpoint
+curl https://your-domain.com/health
+```
+
+**Important Notes:**
+
+- The gateway will automatically obtain Let's Encrypt certificates on first start
+- Certificates are cached in `/home/debros/.debros/tls-cache`
+- Certificate renewal happens automatically
+- Ensure your domain's DNS A record points to the server's public IP before enabling HTTPS
+
 ### Service Management

 All services run as systemd units under the `debros` user.
@ -193,7 +291,7 @@ systemctl status debros-gateway
 #### View Logs

 ```bash
-# View recent logs
+# View recent logs (last 50 lines)
 dbn prod logs node

 # Follow logs in real-time
@ -201,10 +299,22 @@ dbn prod logs node --follow

 # View specific service logs
 dbn prod logs ipfs --follow
+dbn prod logs ipfs-cluster --follow
+dbn prod logs rqlite --follow
+dbn prod logs olric --follow
 dbn prod logs gateway --follow
 ```

-Available log targets: `node`, `ipfs`, `ipfs-cluster`, `rqlite`, `olric`, `gateway`
+**Available log service names:**
+
+- `node` - DeBros Network Node (bootstrap or regular)
+- `ipfs` - IPFS Daemon
+- `ipfs-cluster` - IPFS Cluster Service
+- `rqlite` - RQLite Database
+- `olric` - Olric Cache Server
+- `gateway` - DeBros Gateway
+
+**Note:** The `logs` command uses journalctl and accepts the full systemd service name. Use the short names above for convenience.

 #### Service Control Commands

@ -240,6 +350,59 @@ sudo systemctl start debros-*
 sudo systemctl enable debros-*
 ```

+### Complete Production Commands Reference
+
+#### Installation & Upgrade
+
+```bash
+# Install bootstrap node
+sudo dbn prod install --bootstrap [--domain DOMAIN] [--branch BRANCH]
+
+# Install secondary node
+sudo dbn prod install --vps-ip IP --peers ADDRS [--domain DOMAIN] [--branch BRANCH]
+
+# Install secondary bootstrap
+sudo dbn prod install --bootstrap --vps-ip IP --bootstrap-join ADDR [--domain DOMAIN] [--branch BRANCH]
+
+# Upgrade installation
+sudo dbn prod upgrade [--restart] [--branch BRANCH]
+```
+
+#### Service Management
+
+```bash
+# Check service status (no sudo required)
+dbn prod status
+
+# Start all services
+sudo dbn prod start
+
+# Stop all services
+sudo dbn prod stop
+
+# Restart all services
+sudo dbn prod restart
+```
+
+#### Logs
+
+```bash
+# View recent logs
+dbn prod logs <service>
+
+# Follow logs in real-time
+dbn prod logs <service> --follow
+
+# Available services: node, ipfs, ipfs-cluster, rqlite, olric, gateway
+```
+
+#### Uninstall
+
+```bash
+# Remove all services (preserves data and configs)
+sudo dbn prod uninstall
+```
+
 ### Directory Structure

 Production installations use `/home/debros/.debros/`:
@ -275,7 +438,9 @@ Remove all production services (preserves data and configs):
 sudo dbn prod uninstall
 ```

-This stops and removes all systemd services but keeps `/home/debros/.debros/` intact. To completely remove:
+This stops and removes all systemd services but keeps `/home/debros/.debros/` intact. You'll be prompted to confirm before uninstalling.
+
+**To completely remove everything:**

 ```bash
 sudo dbn prod uninstall
--- a/pkg/environments/production/installers.go
+++ b/pkg/environments/production/installers.go
@ -551,12 +551,12 @@ func (bi *BinaryInstaller) InitializeIPFSClusterConfig(nodeType, clusterPath, cl
 		}
 	}

-	// Always update the cluster secret (for both new and existing configs)
-	// This ensures existing installations get the secret synchronized
+	// Always update the cluster secret and IPFS port (for both new and existing configs)
+	// This ensures existing installations get the secret and port synchronized
 	if clusterSecret != "" {
-		fmt.Fprintf(bi.logWriter.(interface{ Write([]byte) (int, error) }), "    Updating cluster secret...\n")
-		if err := bi.updateClusterSecret(clusterPath, clusterSecret); err != nil {
-			return fmt.Errorf("failed to update cluster secret: %w", err)
+		fmt.Fprintf(bi.logWriter.(interface{ Write([]byte) (int, error) }), "    Updating cluster secret and IPFS port...\n")
+		if err := bi.updateClusterConfig(clusterPath, clusterSecret, ipfsAPIPort); err != nil {
+			return fmt.Errorf("failed to update cluster config: %w", err)
 		}
 	}

@ -566,8 +566,8 @@ func (bi *BinaryInstaller) InitializeIPFSClusterConfig(nodeType, clusterPath, cl
 	return nil
 }

-// updateClusterSecret updates the secret field in IPFS Cluster service.json
-func (bi *BinaryInstaller) updateClusterSecret(clusterPath, secret string) error {
+// updateClusterConfig updates the secret and IPFS port in IPFS Cluster service.json
+func (bi *BinaryInstaller) updateClusterConfig(clusterPath, secret string, ipfsAPIPort int) error {
 	serviceJSONPath := filepath.Join(clusterPath, "service.json")

 	// Read existing config
@ -591,6 +591,21 @@ func (bi *BinaryInstaller) updateClusterSecret(clusterPath, secret string) error
 		}
 	}

+	// Update IPFS port in IPFS Proxy configuration
+	ipfsNodeMultiaddr := fmt.Sprintf("/ip4/127.0.0.1/tcp/%d", ipfsAPIPort)
+	if api, ok := config["api"].(map[string]interface{}); ok {
+		if ipfsproxy, ok := api["ipfsproxy"].(map[string]interface{}); ok {
+			ipfsproxy["node_multiaddress"] = ipfsNodeMultiaddr
+		}
+	}
+
+	// Update IPFS port in IPFS Connector configuration
+	if ipfsConnector, ok := config["ipfs_connector"].(map[string]interface{}); ok {
+		if ipfshttp, ok := ipfsConnector["ipfshttp"].(map[string]interface{}); ok {
+			ipfshttp["node_multiaddress"] = ipfsNodeMultiaddr
+		}
+	}
+
 	// Write back
 	updatedData, err := json.MarshalIndent(config, "", "  ")
 	if err != nil {
--- a/pkg/environments/production/orchestrator.go
+++ b/pkg/environments/production/orchestrator.go
@ -200,8 +200,8 @@ func (ps *ProductionSetup) Phase2ProvisionEnvironment() error {
 		}
 	}

-	// Create directory structure
-	if err := ps.fsProvisioner.EnsureDirectoryStructure(); err != nil {
+	// Create directory structure (base directories only - node-specific dirs created in Phase2c)
+	if err := ps.fsProvisioner.EnsureDirectoryStructure(""); err != nil {
 		return fmt.Errorf("failed to create directory structure: %w", err)
 	}
 	ps.logf("  ✓ Directory structure created")
@ -259,6 +259,11 @@ func (ps *ProductionSetup) Phase2bInstallBinaries() error {
 func (ps *ProductionSetup) Phase2cInitializeServices(nodeType string) error {
 	ps.logf("Phase 2c: Initializing services...")

+	// Ensure node-specific directories exist
+	if err := ps.fsProvisioner.EnsureDirectoryStructure(nodeType); err != nil {
+		return fmt.Errorf("failed to create node-specific directories: %w", err)
+	}
+
 	// Build paths with nodeType awareness to match systemd unit definitions
 	dataDir := filepath.Join(ps.debrosDir, "data", nodeType)

--- a/pkg/environments/production/provisioner.go
+++ b/pkg/environments/production/provisioner.go
@ -24,18 +24,14 @@ func NewFilesystemProvisioner(debrosHome string) *FilesystemProvisioner {
 }

 // EnsureDirectoryStructure creates all required directories
-func (fp *FilesystemProvisioner) EnsureDirectoryStructure() error {
+// nodeType can be "bootstrap", "node", or "" (empty string means create base directories only)
+func (fp *FilesystemProvisioner) EnsureDirectoryStructure(nodeType string) error {
+	// Base directories that are always needed
 	dirs := []string{
 		fp.debrosDir,
 		filepath.Join(fp.debrosDir, "configs"),
 		filepath.Join(fp.debrosDir, "secrets"),
 		filepath.Join(fp.debrosDir, "data"),
-		filepath.Join(fp.debrosDir, "data", "bootstrap", "ipfs", "repo"),
-		filepath.Join(fp.debrosDir, "data", "bootstrap", "ipfs-cluster"),
-		filepath.Join(fp.debrosDir, "data", "bootstrap", "rqlite"),
-		filepath.Join(fp.debrosDir, "data", "node", "ipfs", "repo"),
-		filepath.Join(fp.debrosDir, "data", "node", "ipfs-cluster"),
-		filepath.Join(fp.debrosDir, "data", "node", "rqlite"),
 		filepath.Join(fp.debrosDir, "logs"),
 		filepath.Join(fp.debrosDir, "tls-cache"),
 		filepath.Join(fp.debrosDir, "backups"),
@ -43,6 +39,15 @@ func (fp *FilesystemProvisioner) EnsureDirectoryStructure() error {
 		filepath.Join(fp.debrosHome, "src"),
 	}

+	// Only create directories for the requested node type
+	if nodeType == "bootstrap" || nodeType == "node" {
+		dirs = append(dirs,
+			filepath.Join(fp.debrosDir, "data", nodeType, "ipfs", "repo"),
+			filepath.Join(fp.debrosDir, "data", nodeType, "ipfs-cluster"),
+			filepath.Join(fp.debrosDir, "data", nodeType, "rqlite"),
+		)
+	}
+
 	for _, dir := range dirs {
 		if err := os.MkdirAll(dir, 0755); err != nil {
 			return fmt.Errorf("failed to create directory %s: %w", dir, err)