mirror of
https://github.com/DeBrosOfficial/orama.git
synced 2026-03-17 06:23:00 +00:00
Added hatzhner support for clustering cli orama to spin up clusters
This commit is contained in:
parent
6898f47e2e
commit
fade8f89ed
208
docs/SANDBOX.md
Normal file
208
docs/SANDBOX.md
Normal file
@ -0,0 +1,208 @@
|
|||||||
|
# Sandbox: Ephemeral Hetzner Cloud Clusters
|
||||||
|
|
||||||
|
Spin up temporary 5-node Orama clusters on Hetzner Cloud for development and testing. Total cost: ~€0.04/hour.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# One-time setup (API key, domain, floating IPs, SSH key)
|
||||||
|
orama sandbox setup
|
||||||
|
|
||||||
|
# Create a cluster (~5 minutes)
|
||||||
|
orama sandbox create --name my-feature
|
||||||
|
|
||||||
|
# Check health
|
||||||
|
orama sandbox status
|
||||||
|
|
||||||
|
# SSH into a node
|
||||||
|
orama sandbox ssh 1
|
||||||
|
|
||||||
|
# Deploy code changes
|
||||||
|
orama sandbox rollout
|
||||||
|
|
||||||
|
# Tear it down
|
||||||
|
orama sandbox destroy
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
### 1. Hetzner Cloud Account
|
||||||
|
|
||||||
|
Create a project at [console.hetzner.cloud](https://console.hetzner.cloud) and generate an API token with read/write permissions under **Security > API Tokens**.
|
||||||
|
|
||||||
|
### 2. Domain with Glue Records
|
||||||
|
|
||||||
|
You need a domain (or subdomain) that points to Hetzner Floating IPs. The `orama sandbox setup` wizard will guide you through this.
|
||||||
|
|
||||||
|
**Example:** Using `sbx.dbrs.space`
|
||||||
|
|
||||||
|
At your domain registrar:
|
||||||
|
1. Create glue records (Personal DNS Servers):
|
||||||
|
- `ns1.sbx.dbrs.space` → `<floating-ip-1>`
|
||||||
|
- `ns2.sbx.dbrs.space` → `<floating-ip-2>`
|
||||||
|
2. Set custom nameservers for `sbx.dbrs.space`:
|
||||||
|
- `ns1.sbx.dbrs.space`
|
||||||
|
- `ns2.sbx.dbrs.space`
|
||||||
|
|
||||||
|
DNS propagation can take up to 48 hours.
|
||||||
|
|
||||||
|
### 3. Binary Archive
|
||||||
|
|
||||||
|
Build the binary archive before creating a cluster:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
orama build
|
||||||
|
```
|
||||||
|
|
||||||
|
This creates `/tmp/orama-<version>-linux-amd64.tar.gz` with all pre-compiled binaries.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
Run the interactive setup wizard:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
orama sandbox setup
|
||||||
|
```
|
||||||
|
|
||||||
|
This will:
|
||||||
|
1. Prompt for your Hetzner API token and validate it
|
||||||
|
2. Ask for your sandbox domain
|
||||||
|
3. Create or reuse 2 Hetzner Floating IPs (~$0.005/hr each)
|
||||||
|
4. Create a firewall with sandbox rules
|
||||||
|
5. Generate an SSH keypair at `~/.orama/sandbox_key`
|
||||||
|
6. Upload the public key to Hetzner
|
||||||
|
7. Display DNS configuration instructions
|
||||||
|
|
||||||
|
Config is saved to `~/.orama/sandbox.yaml`.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
### `orama sandbox create [--name <name>]`
|
||||||
|
|
||||||
|
Creates a new 5-node cluster. If `--name` is omitted, a random name is generated (e.g., "swift-falcon").
|
||||||
|
|
||||||
|
**Cluster layout:**
|
||||||
|
- Nodes 1-2: Nameservers (CoreDNS + Caddy + all services)
|
||||||
|
- Nodes 3-5: Regular nodes (all services except CoreDNS)
|
||||||
|
|
||||||
|
**Phases:**
|
||||||
|
1. Provision 5 CX22 servers on Hetzner (parallel, ~90s)
|
||||||
|
2. Assign floating IPs to nameserver nodes (~10s)
|
||||||
|
3. Upload binary archive to all nodes (parallel, ~60s)
|
||||||
|
4. Install genesis node + generate invite tokens (~120s)
|
||||||
|
5. Join remaining 4 nodes (serial with health checks, ~180s)
|
||||||
|
6. Verify cluster health (~15s)
|
||||||
|
|
||||||
|
**One sandbox at a time.** Since the floating IPs are shared, only one sandbox can own the nameservers. Destroy the active sandbox before creating a new one.
|
||||||
|
|
||||||
|
### `orama sandbox destroy [--name <name>] [--force]`
|
||||||
|
|
||||||
|
Tears down a cluster:
|
||||||
|
1. Unassigns floating IPs
|
||||||
|
2. Deletes all 5 servers (parallel)
|
||||||
|
3. Removes state file
|
||||||
|
|
||||||
|
Use `--force` to skip confirmation.
|
||||||
|
|
||||||
|
### `orama sandbox list`
|
||||||
|
|
||||||
|
Lists all sandboxes with their status. Also checks Hetzner for orphaned servers that don't have a corresponding state file.
|
||||||
|
|
||||||
|
### `orama sandbox status [--name <name>]`
|
||||||
|
|
||||||
|
Shows per-node health including:
|
||||||
|
- Service status (active/inactive)
|
||||||
|
- RQLite role (Leader/Follower)
|
||||||
|
- Cluster summary (commit index, voter count)
|
||||||
|
|
||||||
|
### `orama sandbox rollout [--name <name>]`
|
||||||
|
|
||||||
|
Deploys code changes:
|
||||||
|
1. Uses the latest binary archive from `/tmp/` (run `orama build` first)
|
||||||
|
2. Pushes to all nodes
|
||||||
|
3. Rolling upgrade: followers first, leader last, 15s between nodes
|
||||||
|
|
||||||
|
### `orama sandbox ssh <node-number>`
|
||||||
|
|
||||||
|
Opens an interactive SSH session to a sandbox node (1-5).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
orama sandbox ssh 1 # SSH into node 1 (genesis/ns1)
|
||||||
|
orama sandbox ssh 3 # SSH into node 3 (regular node)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Floating IPs
|
||||||
|
|
||||||
|
Hetzner Floating IPs are persistent IPv4 addresses that can be reassigned between servers. They solve the DNS chicken-and-egg problem:
|
||||||
|
|
||||||
|
- Glue records at the registrar point to 2 Floating IPs (configured once)
|
||||||
|
- Each new sandbox assigns the Floating IPs to its nameserver nodes
|
||||||
|
- DNS works instantly — no propagation delay between clusters
|
||||||
|
|
||||||
|
### SSH Authentication
|
||||||
|
|
||||||
|
Sandbox uses a standalone ed25519 keypair at `~/.orama/sandbox_key`, separate from the production wallet-derived keys. The public key is uploaded to Hetzner during setup and injected into every server at creation time.
|
||||||
|
|
||||||
|
### Server Naming
|
||||||
|
|
||||||
|
Servers: `sbx-<name>-<N>` (e.g., `sbx-swift-falcon-1` through `sbx-swift-falcon-5`)
|
||||||
|
|
||||||
|
### State Files
|
||||||
|
|
||||||
|
Sandbox state is stored at `~/.orama/sandboxes/<name>.yaml`. This tracks server IDs, IPs, roles, and cluster status.
|
||||||
|
|
||||||
|
## Cost
|
||||||
|
|
||||||
|
| Resource | Cost | Qty | Total |
|
||||||
|
|----------|------|-----|-------|
|
||||||
|
| CX22 (2 vCPU, 4GB) | €0.006/hr | 5 | €0.03/hr |
|
||||||
|
| Floating IPv4 | €0.005/hr | 2 | €0.01/hr |
|
||||||
|
| **Total** | | | **~€0.04/hr** |
|
||||||
|
|
||||||
|
Servers are billed per hour. Floating IPs are billed as long as they exist (even unassigned). Destroy the sandbox when not in use to save on server costs.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### "sandbox not configured"
|
||||||
|
|
||||||
|
Run `orama sandbox setup` first.
|
||||||
|
|
||||||
|
### "no binary archive found"
|
||||||
|
|
||||||
|
Run `orama build` to create the binary archive.
|
||||||
|
|
||||||
|
### "sandbox X is already active"
|
||||||
|
|
||||||
|
Only one sandbox can be active at a time. Destroy it first:
|
||||||
|
```bash
|
||||||
|
orama sandbox destroy --name <name>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Server creation fails
|
||||||
|
|
||||||
|
Check:
|
||||||
|
- Hetzner API token is valid and has read/write permissions
|
||||||
|
- You haven't hit Hetzner's server limit (default: 10 per project)
|
||||||
|
- The selected location has CX22 capacity
|
||||||
|
|
||||||
|
### Genesis install fails
|
||||||
|
|
||||||
|
SSH into the node to debug:
|
||||||
|
```bash
|
||||||
|
orama sandbox ssh 1
|
||||||
|
journalctl -u orama-node -f
|
||||||
|
```
|
||||||
|
|
||||||
|
The sandbox will be left in "error" state. You can destroy and recreate it.
|
||||||
|
|
||||||
|
### DNS not resolving
|
||||||
|
|
||||||
|
1. Verify glue records are configured at your registrar
|
||||||
|
2. Check propagation: `dig NS sbx.dbrs.space @8.8.8.8`
|
||||||
|
3. Propagation can take 24-48 hours for new domains
|
||||||
|
|
||||||
|
### Orphaned servers
|
||||||
|
|
||||||
|
If `orama sandbox list` shows orphaned servers, delete them manually at [console.hetzner.cloud](https://console.hetzner.cloud). Sandbox servers are labeled `orama-sandbox=<name>` for easy identification.
|
||||||
121
pkg/cli/cmd/sandboxcmd/sandbox.go
Normal file
121
pkg/cli/cmd/sandboxcmd/sandbox.go
Normal file
@ -0,0 +1,121 @@
|
|||||||
|
package sandboxcmd
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/cli/sandbox"
|
||||||
|
"github.com/spf13/cobra"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Cmd is the root command for sandbox operations.
|
||||||
|
var Cmd = &cobra.Command{
|
||||||
|
Use: "sandbox",
|
||||||
|
Short: "Manage ephemeral Hetzner Cloud clusters for testing",
|
||||||
|
Long: `Spin up temporary 5-node Orama clusters on Hetzner Cloud for development and testing.
|
||||||
|
|
||||||
|
Setup (one-time):
|
||||||
|
orama sandbox setup
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
orama sandbox create [--name <name>] Create a new 5-node cluster
|
||||||
|
orama sandbox destroy [--name <name>] Tear down a cluster
|
||||||
|
orama sandbox list List active sandboxes
|
||||||
|
orama sandbox status [--name <name>] Show cluster health
|
||||||
|
orama sandbox rollout [--name <name>] Build + push + rolling upgrade
|
||||||
|
orama sandbox ssh <node-number> SSH into a sandbox node (1-5)`,
|
||||||
|
}
|
||||||
|
|
||||||
|
var setupCmd = &cobra.Command{
|
||||||
|
Use: "setup",
|
||||||
|
Short: "Interactive setup: Hetzner API key, domain, floating IPs, SSH key",
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
return sandbox.Setup()
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
var createCmd = &cobra.Command{
|
||||||
|
Use: "create",
|
||||||
|
Short: "Create a new 5-node sandbox cluster (~5 min)",
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
name, _ := cmd.Flags().GetString("name")
|
||||||
|
return sandbox.Create(name)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
var destroyCmd = &cobra.Command{
|
||||||
|
Use: "destroy",
|
||||||
|
Short: "Destroy a sandbox cluster and release resources",
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
name, _ := cmd.Flags().GetString("name")
|
||||||
|
force, _ := cmd.Flags().GetBool("force")
|
||||||
|
return sandbox.Destroy(name, force)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
var listCmd = &cobra.Command{
|
||||||
|
Use: "list",
|
||||||
|
Short: "List active sandbox clusters",
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
return sandbox.List()
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
var statusCmd = &cobra.Command{
|
||||||
|
Use: "status",
|
||||||
|
Short: "Show cluster health report",
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
name, _ := cmd.Flags().GetString("name")
|
||||||
|
return sandbox.Status(name)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
var rolloutCmd = &cobra.Command{
|
||||||
|
Use: "rollout",
|
||||||
|
Short: "Build + push + rolling upgrade to sandbox cluster",
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
name, _ := cmd.Flags().GetString("name")
|
||||||
|
return sandbox.Rollout(name)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
var sshCmd = &cobra.Command{
|
||||||
|
Use: "ssh <node-number>",
|
||||||
|
Short: "SSH into a sandbox node (1-5)",
|
||||||
|
Args: cobra.ExactArgs(1),
|
||||||
|
RunE: func(cmd *cobra.Command, args []string) error {
|
||||||
|
name, _ := cmd.Flags().GetString("name")
|
||||||
|
var nodeNum int
|
||||||
|
if _, err := fmt.Sscanf(args[0], "%d", &nodeNum); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "Invalid node number: %s (expected 1-5)\n", args[0])
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
return sandbox.SSHInto(name, nodeNum)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
func init() {
|
||||||
|
// create flags
|
||||||
|
createCmd.Flags().String("name", "", "Sandbox name (random if not specified)")
|
||||||
|
|
||||||
|
// destroy flags
|
||||||
|
destroyCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||||
|
destroyCmd.Flags().Bool("force", false, "Skip confirmation")
|
||||||
|
|
||||||
|
// status flags
|
||||||
|
statusCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||||
|
|
||||||
|
// rollout flags
|
||||||
|
rolloutCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||||
|
|
||||||
|
// ssh flags
|
||||||
|
sshCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||||
|
|
||||||
|
Cmd.AddCommand(setupCmd)
|
||||||
|
Cmd.AddCommand(createCmd)
|
||||||
|
Cmd.AddCommand(destroyCmd)
|
||||||
|
Cmd.AddCommand(listCmd)
|
||||||
|
Cmd.AddCommand(statusCmd)
|
||||||
|
Cmd.AddCommand(rolloutCmd)
|
||||||
|
Cmd.AddCommand(sshCmd)
|
||||||
|
}
|
||||||
153
pkg/cli/sandbox/config.go
Normal file
153
pkg/cli/sandbox/config.go
Normal file
@ -0,0 +1,153 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config holds sandbox configuration, stored at ~/.orama/sandbox.yaml.
|
||||||
|
type Config struct {
|
||||||
|
HetznerAPIToken string `yaml:"hetzner_api_token"`
|
||||||
|
Domain string `yaml:"domain"`
|
||||||
|
Location string `yaml:"location"` // Hetzner datacenter (default: fsn1)
|
||||||
|
ServerType string `yaml:"server_type"` // Hetzner server type (default: cx22)
|
||||||
|
FloatingIPs []FloatIP `yaml:"floating_ips"`
|
||||||
|
SSHKey SSHKeyConfig `yaml:"ssh_key"`
|
||||||
|
FirewallID int64 `yaml:"firewall_id,omitempty"` // Hetzner firewall resource ID
|
||||||
|
}
|
||||||
|
|
||||||
|
// FloatIP holds a Hetzner floating IP reference.
|
||||||
|
type FloatIP struct {
|
||||||
|
ID int64 `yaml:"id"`
|
||||||
|
IP string `yaml:"ip"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// SSHKeyConfig holds SSH key paths and the Hetzner resource ID.
|
||||||
|
type SSHKeyConfig struct {
|
||||||
|
HetznerID int64 `yaml:"hetzner_id"`
|
||||||
|
PrivateKeyPath string `yaml:"private_key_path"`
|
||||||
|
PublicKeyPath string `yaml:"public_key_path"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// configDir returns ~/.orama/, creating it if needed.
|
||||||
|
func configDir() (string, error) {
|
||||||
|
home, err := os.UserHomeDir()
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("get home directory: %w", err)
|
||||||
|
}
|
||||||
|
dir := filepath.Join(home, ".orama")
|
||||||
|
if err := os.MkdirAll(dir, 0700); err != nil {
|
||||||
|
return "", fmt.Errorf("create config directory: %w", err)
|
||||||
|
}
|
||||||
|
return dir, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// configPath returns the full path to ~/.orama/sandbox.yaml.
|
||||||
|
func configPath() (string, error) {
|
||||||
|
dir, err := configDir()
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return filepath.Join(dir, "sandbox.yaml"), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadConfig reads the sandbox config from ~/.orama/sandbox.yaml.
|
||||||
|
// Returns an error if the file doesn't exist (user must run setup first).
|
||||||
|
func LoadConfig() (*Config, error) {
|
||||||
|
path, err := configPath()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil, fmt.Errorf("sandbox not configured, run: orama sandbox setup")
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("read config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var cfg Config
|
||||||
|
if err := yaml.Unmarshal(data, &cfg); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse config %s: %w", path, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := cfg.validate(); err != nil {
|
||||||
|
return nil, fmt.Errorf("invalid config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg.Defaults()
|
||||||
|
|
||||||
|
return &cfg, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SaveConfig writes the sandbox config to ~/.orama/sandbox.yaml.
|
||||||
|
func SaveConfig(cfg *Config) error {
|
||||||
|
path, err := configPath()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
data, err := yaml.Marshal(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("marshal config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||||
|
return fmt.Errorf("write config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// validate checks that required fields are present.
|
||||||
|
func (c *Config) validate() error {
|
||||||
|
if c.HetznerAPIToken == "" {
|
||||||
|
return fmt.Errorf("hetzner_api_token is required")
|
||||||
|
}
|
||||||
|
if c.Domain == "" {
|
||||||
|
return fmt.Errorf("domain is required")
|
||||||
|
}
|
||||||
|
if len(c.FloatingIPs) < 2 {
|
||||||
|
return fmt.Errorf("2 floating IPs required, got %d", len(c.FloatingIPs))
|
||||||
|
}
|
||||||
|
if c.SSHKey.PrivateKeyPath == "" {
|
||||||
|
return fmt.Errorf("ssh_key.private_key_path is required")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Defaults fills in default values for optional fields.
|
||||||
|
func (c *Config) Defaults() {
|
||||||
|
if c.Location == "" {
|
||||||
|
c.Location = "fsn1"
|
||||||
|
}
|
||||||
|
if c.ServerType == "" {
|
||||||
|
c.ServerType = "cx22"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExpandedPrivateKeyPath returns the absolute path to the SSH private key.
|
||||||
|
func (c *Config) ExpandedPrivateKeyPath() string {
|
||||||
|
return expandHome(c.SSHKey.PrivateKeyPath)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ExpandedPublicKeyPath returns the absolute path to the SSH public key.
|
||||||
|
func (c *Config) ExpandedPublicKeyPath() string {
|
||||||
|
return expandHome(c.SSHKey.PublicKeyPath)
|
||||||
|
}
|
||||||
|
|
||||||
|
// expandHome replaces a leading ~ with the user's home directory.
|
||||||
|
func expandHome(path string) string {
|
||||||
|
if len(path) < 2 || path[:2] != "~/" {
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
home, err := os.UserHomeDir()
|
||||||
|
if err != nil {
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
return filepath.Join(home, path[2:])
|
||||||
|
}
|
||||||
554
pkg/cli/sandbox/create.go
Normal file
554
pkg/cli/sandbox/create.go
Normal file
@ -0,0 +1,554 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/cli/remotessh"
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Create orchestrates the creation of a new sandbox cluster.
|
||||||
|
func Create(name string) error {
|
||||||
|
cfg, err := LoadConfig()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check for existing active sandbox
|
||||||
|
active, err := FindActiveSandbox()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if active != nil {
|
||||||
|
return fmt.Errorf("sandbox %q is already active (status: %s)\nDestroy it first: orama sandbox destroy --name %s",
|
||||||
|
active.Name, active.Status, active.Name)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Generate name if not provided
|
||||||
|
if name == "" {
|
||||||
|
name = GenerateName()
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Printf("Creating sandbox %q (%s, %d nodes)\n\n", name, cfg.Domain, 5)
|
||||||
|
|
||||||
|
client := NewHetznerClient(cfg.HetznerAPIToken)
|
||||||
|
|
||||||
|
state := &SandboxState{
|
||||||
|
Name: name,
|
||||||
|
CreatedAt: time.Now().UTC(),
|
||||||
|
Domain: cfg.Domain,
|
||||||
|
Status: StatusCreating,
|
||||||
|
}
|
||||||
|
|
||||||
|
// Phase 1: Provision servers
|
||||||
|
fmt.Println("Phase 1: Provisioning servers...")
|
||||||
|
if err := phase1ProvisionServers(client, cfg, state); err != nil {
|
||||||
|
cleanupFailedCreate(client, state)
|
||||||
|
return fmt.Errorf("provision servers: %w", err)
|
||||||
|
}
|
||||||
|
SaveState(state)
|
||||||
|
|
||||||
|
// Phase 2: Assign floating IPs
|
||||||
|
fmt.Println("\nPhase 2: Assigning floating IPs...")
|
||||||
|
if err := phase2AssignFloatingIPs(client, cfg, state); err != nil {
|
||||||
|
return fmt.Errorf("assign floating IPs: %w", err)
|
||||||
|
}
|
||||||
|
SaveState(state)
|
||||||
|
|
||||||
|
// Phase 3: Upload binary archive
|
||||||
|
fmt.Println("\nPhase 3: Uploading binary archive...")
|
||||||
|
if err := phase3UploadArchive(cfg, state); err != nil {
|
||||||
|
return fmt.Errorf("upload archive: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Phase 4: Install genesis node
|
||||||
|
fmt.Println("\nPhase 4: Installing genesis node...")
|
||||||
|
tokens, err := phase4InstallGenesis(cfg, state)
|
||||||
|
if err != nil {
|
||||||
|
state.Status = StatusError
|
||||||
|
SaveState(state)
|
||||||
|
return fmt.Errorf("install genesis: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Phase 5: Join remaining nodes
|
||||||
|
fmt.Println("\nPhase 5: Joining remaining nodes...")
|
||||||
|
if err := phase5JoinNodes(cfg, state, tokens); err != nil {
|
||||||
|
state.Status = StatusError
|
||||||
|
SaveState(state)
|
||||||
|
return fmt.Errorf("join nodes: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Phase 6: Verify cluster
|
||||||
|
fmt.Println("\nPhase 6: Verifying cluster...")
|
||||||
|
phase6Verify(cfg, state)
|
||||||
|
|
||||||
|
state.Status = StatusRunning
|
||||||
|
SaveState(state)
|
||||||
|
|
||||||
|
printCreateSummary(cfg, state)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// phase1ProvisionServers creates 5 Hetzner servers in parallel.
|
||||||
|
func phase1ProvisionServers(client *HetznerClient, cfg *Config, state *SandboxState) error {
|
||||||
|
type serverResult struct {
|
||||||
|
index int
|
||||||
|
server *HetznerServer
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
results := make(chan serverResult, 5)
|
||||||
|
|
||||||
|
for i := 0; i < 5; i++ {
|
||||||
|
go func(idx int) {
|
||||||
|
role := "node"
|
||||||
|
if idx < 2 {
|
||||||
|
role = "nameserver"
|
||||||
|
}
|
||||||
|
|
||||||
|
serverName := fmt.Sprintf("sbx-%s-%d", state.Name, idx+1)
|
||||||
|
labels := map[string]string{
|
||||||
|
"orama-sandbox": state.Name,
|
||||||
|
"orama-sandbox-role": role,
|
||||||
|
}
|
||||||
|
|
||||||
|
req := CreateServerRequest{
|
||||||
|
Name: serverName,
|
||||||
|
ServerType: cfg.ServerType,
|
||||||
|
Image: "ubuntu-24.04",
|
||||||
|
Location: cfg.Location,
|
||||||
|
SSHKeys: []int64{cfg.SSHKey.HetznerID},
|
||||||
|
Labels: labels,
|
||||||
|
}
|
||||||
|
if cfg.FirewallID > 0 {
|
||||||
|
req.Firewalls = []struct {
|
||||||
|
Firewall int64 `json:"firewall"`
|
||||||
|
}{{Firewall: cfg.FirewallID}}
|
||||||
|
}
|
||||||
|
|
||||||
|
srv, err := client.CreateServer(req)
|
||||||
|
results <- serverResult{index: idx, server: srv, err: err}
|
||||||
|
}(i)
|
||||||
|
}
|
||||||
|
|
||||||
|
servers := make([]ServerState, 5)
|
||||||
|
for i := 0; i < 5; i++ {
|
||||||
|
r := <-results
|
||||||
|
if r.err != nil {
|
||||||
|
return fmt.Errorf("server %d: %w", r.index+1, r.err)
|
||||||
|
}
|
||||||
|
fmt.Printf(" Created %s (ID: %d, initializing...)\n", r.server.Name, r.server.ID)
|
||||||
|
role := "node"
|
||||||
|
if r.index < 2 {
|
||||||
|
role = "nameserver"
|
||||||
|
}
|
||||||
|
servers[r.index] = ServerState{
|
||||||
|
ID: r.server.ID,
|
||||||
|
Name: r.server.Name,
|
||||||
|
Role: role,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for all servers to reach "running"
|
||||||
|
fmt.Print(" Waiting for servers to boot...")
|
||||||
|
for i := range servers {
|
||||||
|
srv, err := client.WaitForServer(servers[i].ID, 3*time.Minute)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("wait for %s: %w", servers[i].Name, err)
|
||||||
|
}
|
||||||
|
servers[i].IP = srv.PublicNet.IPv4.IP
|
||||||
|
fmt.Print(".")
|
||||||
|
}
|
||||||
|
fmt.Println(" OK")
|
||||||
|
|
||||||
|
// Assign floating IPs to nameserver entries
|
||||||
|
if len(cfg.FloatingIPs) >= 2 {
|
||||||
|
servers[0].FloatingIP = cfg.FloatingIPs[0].IP
|
||||||
|
servers[1].FloatingIP = cfg.FloatingIPs[1].IP
|
||||||
|
}
|
||||||
|
|
||||||
|
state.Servers = servers
|
||||||
|
|
||||||
|
for _, srv := range servers {
|
||||||
|
fmt.Printf(" %s: %s (%s)\n", srv.Name, srv.IP, srv.Role)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// phase2AssignFloatingIPs assigns floating IPs and configures loopback.
|
||||||
|
func phase2AssignFloatingIPs(client *HetznerClient, cfg *Config, state *SandboxState) error {
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
|
||||||
|
for i := 0; i < 2 && i < len(cfg.FloatingIPs) && i < len(state.Servers); i++ {
|
||||||
|
fip := cfg.FloatingIPs[i]
|
||||||
|
srv := state.Servers[i]
|
||||||
|
|
||||||
|
// Unassign if currently assigned elsewhere (ignore "not assigned" errors)
|
||||||
|
fmt.Printf(" Assigning %s to %s...\n", fip.IP, srv.Name)
|
||||||
|
if err := client.UnassignFloatingIP(fip.ID); err != nil {
|
||||||
|
// Log but continue — may fail if not currently assigned, which is fine
|
||||||
|
fmt.Printf(" Note: unassign %s: %v (continuing)\n", fip.IP, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := client.AssignFloatingIP(fip.ID, srv.ID); err != nil {
|
||||||
|
return fmt.Errorf("assign %s to %s: %w", fip.IP, srv.Name, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Configure floating IP on the server's loopback interface
|
||||||
|
// Hetzner floating IPs require this: ip addr add <floating_ip>/32 dev lo
|
||||||
|
node := inspector.Node{
|
||||||
|
User: "root",
|
||||||
|
Host: srv.IP,
|
||||||
|
SSHKey: sshKeyPath,
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for SSH to be ready on freshly booted servers
|
||||||
|
if err := waitForSSH(node, 2*time.Minute); err != nil {
|
||||||
|
return fmt.Errorf("SSH not ready on %s: %w", srv.Name, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
cmd := fmt.Sprintf("ip addr add %s/32 dev lo 2>/dev/null || true", fip.IP)
|
||||||
|
if err := remotessh.RunSSHStreaming(node, cmd); err != nil {
|
||||||
|
return fmt.Errorf("configure loopback on %s: %w", srv.Name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// waitForSSH polls until SSH is responsive on the node.
|
||||||
|
func waitForSSH(node inspector.Node, timeout time.Duration) error {
|
||||||
|
deadline := time.Now().Add(timeout)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
_, err := runSSHOutput(node, "echo ok")
|
||||||
|
if err == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
time.Sleep(3 * time.Second)
|
||||||
|
}
|
||||||
|
return fmt.Errorf("timeout after %s", timeout)
|
||||||
|
}
|
||||||
|
|
||||||
|
// phase3UploadArchive builds (if needed) and uploads the binary archive to all nodes.
|
||||||
|
func phase3UploadArchive(cfg *Config, state *SandboxState) error {
|
||||||
|
// Find existing archive
|
||||||
|
archivePath := findNewestArchive()
|
||||||
|
if archivePath == "" {
|
||||||
|
fmt.Println(" No binary archive found, run `orama build` first")
|
||||||
|
return fmt.Errorf("no binary archive found in /tmp/ (run `orama build` first)")
|
||||||
|
}
|
||||||
|
|
||||||
|
info, _ := os.Stat(archivePath)
|
||||||
|
fmt.Printf(" Archive: %s (%s)\n", filepath.Base(archivePath), formatBytes(info.Size()))
|
||||||
|
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
remotePath := "/tmp/" + filepath.Base(archivePath)
|
||||||
|
|
||||||
|
// Upload to all 5 nodes in parallel
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
errs := make([]error, len(state.Servers))
|
||||||
|
|
||||||
|
for i, srv := range state.Servers {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(idx int, srv ServerState) {
|
||||||
|
defer wg.Done()
|
||||||
|
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||||
|
|
||||||
|
if err := remotessh.UploadFile(node, archivePath, remotePath); err != nil {
|
||||||
|
errs[idx] = fmt.Errorf("upload to %s: %w", srv.Name, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract + install CLI
|
||||||
|
extractCmd := fmt.Sprintf("mkdir -p /opt/orama && tar xzf %s -C /opt/orama && rm -f %s && cp /opt/orama/bin/orama /usr/local/bin/orama && chmod +x /usr/local/bin/orama",
|
||||||
|
remotePath, remotePath)
|
||||||
|
if err := remotessh.RunSSHStreaming(node, extractCmd); err != nil {
|
||||||
|
errs[idx] = fmt.Errorf("extract on %s: %w", srv.Name, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Printf(" Uploaded to %s\n", srv.Name)
|
||||||
|
}(i, srv)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
|
||||||
|
for _, err := range errs {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// phase4InstallGenesis installs the genesis node and generates invite tokens.
|
||||||
|
func phase4InstallGenesis(cfg *Config, state *SandboxState) ([]string, error) {
|
||||||
|
genesis := state.GenesisServer()
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
node := inspector.Node{User: "root", Host: genesis.IP, SSHKey: sshKeyPath}
|
||||||
|
|
||||||
|
// Install genesis
|
||||||
|
installCmd := fmt.Sprintf("orama node install --vps-ip %s --domain %s --base-domain %s --nameserver --skip-checks",
|
||||||
|
genesis.IP, cfg.Domain, cfg.Domain)
|
||||||
|
fmt.Printf(" Installing on %s (%s)...\n", genesis.Name, genesis.IP)
|
||||||
|
if err := remotessh.RunSSHStreaming(node, installCmd); err != nil {
|
||||||
|
return nil, fmt.Errorf("install genesis: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for RQLite leader
|
||||||
|
fmt.Print(" Waiting for RQLite leader...")
|
||||||
|
if err := waitForRQLiteHealth(node, 3*time.Minute); err != nil {
|
||||||
|
return nil, fmt.Errorf("genesis health: %w", err)
|
||||||
|
}
|
||||||
|
fmt.Println(" OK")
|
||||||
|
|
||||||
|
// Generate invite tokens (one per remaining node)
|
||||||
|
fmt.Print(" Generating invite tokens...")
|
||||||
|
remaining := len(state.Servers) - 1
|
||||||
|
tokens := make([]string, remaining)
|
||||||
|
|
||||||
|
for i := 0; i < remaining; i++ {
|
||||||
|
token, err := generateInviteToken(node)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("generate invite token %d: %w", i+1, err)
|
||||||
|
}
|
||||||
|
tokens[i] = token
|
||||||
|
fmt.Print(".")
|
||||||
|
}
|
||||||
|
fmt.Println(" OK")
|
||||||
|
|
||||||
|
return tokens, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// phase5JoinNodes joins the remaining 4 nodes to the cluster (serial).
|
||||||
|
func phase5JoinNodes(cfg *Config, state *SandboxState, tokens []string) error {
|
||||||
|
genesisIP := state.GenesisServer().IP
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
|
||||||
|
for i := 1; i < len(state.Servers); i++ {
|
||||||
|
srv := state.Servers[i]
|
||||||
|
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||||
|
token := tokens[i-1]
|
||||||
|
|
||||||
|
var installCmd string
|
||||||
|
if srv.Role == "nameserver" {
|
||||||
|
installCmd = fmt.Sprintf("orama node install --join http://%s --token %s --vps-ip %s --domain %s --base-domain %s --nameserver --skip-checks",
|
||||||
|
genesisIP, token, srv.IP, cfg.Domain, cfg.Domain)
|
||||||
|
} else {
|
||||||
|
installCmd = fmt.Sprintf("orama node install --join http://%s --token %s --vps-ip %s --base-domain %s --skip-checks",
|
||||||
|
genesisIP, token, srv.IP, cfg.Domain)
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Printf(" [%d/%d] Joining %s (%s, %s)...\n", i, len(state.Servers)-1, srv.Name, srv.IP, srv.Role)
|
||||||
|
if err := remotessh.RunSSHStreaming(node, installCmd); err != nil {
|
||||||
|
return fmt.Errorf("join %s: %w", srv.Name, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for node health before proceeding
|
||||||
|
fmt.Printf(" Waiting for %s health...", srv.Name)
|
||||||
|
if err := waitForRQLiteHealth(node, 3*time.Minute); err != nil {
|
||||||
|
fmt.Printf(" WARN: %v\n", err)
|
||||||
|
} else {
|
||||||
|
fmt.Println(" OK")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// phase6Verify runs a basic cluster health check.
|
||||||
|
func phase6Verify(cfg *Config, state *SandboxState) {
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
genesis := state.GenesisServer()
|
||||||
|
node := inspector.Node{User: "root", Host: genesis.IP, SSHKey: sshKeyPath}
|
||||||
|
|
||||||
|
// Check RQLite cluster
|
||||||
|
out, err := runSSHOutput(node, "curl -s http://localhost:5001/status | grep -o '\"state\":\"[^\"]*\"' | head -1")
|
||||||
|
if err == nil {
|
||||||
|
fmt.Printf(" RQLite: %s\n", strings.TrimSpace(out))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check DNS (if floating IPs configured, only with safe domain names)
|
||||||
|
if len(cfg.FloatingIPs) > 0 && isSafeDNSName(cfg.Domain) {
|
||||||
|
out, err = runSSHOutput(node, fmt.Sprintf("dig +short @%s test.%s 2>/dev/null || echo 'DNS not responding'",
|
||||||
|
cfg.FloatingIPs[0].IP, cfg.Domain))
|
||||||
|
if err == nil {
|
||||||
|
fmt.Printf(" DNS: %s\n", strings.TrimSpace(out))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// waitForRQLiteHealth polls RQLite until it reports Leader or Follower state.
|
||||||
|
func waitForRQLiteHealth(node inspector.Node, timeout time.Duration) error {
|
||||||
|
deadline := time.Now().Add(timeout)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
out, err := runSSHOutput(node, "curl -sf http://localhost:5001/status 2>/dev/null | grep -o '\"state\":\"[^\"]*\"'")
|
||||||
|
if err == nil {
|
||||||
|
result := strings.TrimSpace(out)
|
||||||
|
if strings.Contains(result, "Leader") || strings.Contains(result, "Follower") {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Second)
|
||||||
|
}
|
||||||
|
return fmt.Errorf("timeout waiting for RQLite health after %s", timeout)
|
||||||
|
}
|
||||||
|
|
||||||
|
// generateInviteToken runs `orama node invite` on the node and parses the token.
|
||||||
|
func generateInviteToken(node inspector.Node) (string, error) {
|
||||||
|
out, err := runSSHOutput(node, "orama node invite --expiry 1h 2>&1")
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("invite command failed: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse token from output — the invite command outputs:
|
||||||
|
// "sudo orama install --join https://... --token <64-char-hex> --vps-ip ..."
|
||||||
|
// Look for the --token flag value first
|
||||||
|
fields := strings.Fields(out)
|
||||||
|
for i, field := range fields {
|
||||||
|
if field == "--token" && i+1 < len(fields) {
|
||||||
|
candidate := fields[i+1]
|
||||||
|
if len(candidate) == 64 && isHex(candidate) {
|
||||||
|
return candidate, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Fallback: look for any standalone 64-char hex string
|
||||||
|
for _, word := range fields {
|
||||||
|
if len(word) == 64 && isHex(word) {
|
||||||
|
return word, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", fmt.Errorf("could not parse token from invite output:\n%s", out)
|
||||||
|
}
|
||||||
|
|
||||||
|
// isSafeDNSName returns true if the string is safe to use in shell commands.
|
||||||
|
func isSafeDNSName(s string) bool {
|
||||||
|
for _, c := range s {
|
||||||
|
if !((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '.' || c == '-') {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return len(s) > 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// isHex returns true if s contains only hex characters.
|
||||||
|
func isHex(s string) bool {
|
||||||
|
for _, c := range s {
|
||||||
|
if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// runSSHOutput runs a command via SSH and returns stdout as a string.
|
||||||
|
func runSSHOutput(node inspector.Node, command string) (string, error) {
|
||||||
|
args := []string{
|
||||||
|
"ssh", "-n",
|
||||||
|
"-o", "StrictHostKeyChecking=accept-new",
|
||||||
|
"-o", "ConnectTimeout=10",
|
||||||
|
"-o", "BatchMode=yes",
|
||||||
|
"-i", node.SSHKey,
|
||||||
|
fmt.Sprintf("%s@%s", node.User, node.Host),
|
||||||
|
command,
|
||||||
|
}
|
||||||
|
|
||||||
|
out, err := execCommand(args[0], args[1:]...)
|
||||||
|
return string(out), err
|
||||||
|
}
|
||||||
|
|
||||||
|
// execCommand runs a command and returns its output.
|
||||||
|
func execCommand(name string, args ...string) ([]byte, error) {
|
||||||
|
return exec.Command(name, args...).Output()
|
||||||
|
}
|
||||||
|
|
||||||
|
// findNewestArchive finds the newest binary archive in /tmp/.
|
||||||
|
func findNewestArchive() string {
|
||||||
|
entries, err := os.ReadDir("/tmp")
|
||||||
|
if err != nil {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
var best string
|
||||||
|
var bestMod int64
|
||||||
|
for _, entry := range entries {
|
||||||
|
name := entry.Name()
|
||||||
|
if strings.HasPrefix(name, "orama-") && strings.Contains(name, "-linux-") && strings.HasSuffix(name, ".tar.gz") {
|
||||||
|
info, err := entry.Info()
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if info.ModTime().Unix() > bestMod {
|
||||||
|
best = filepath.Join("/tmp", name)
|
||||||
|
bestMod = info.ModTime().Unix()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return best
|
||||||
|
}
|
||||||
|
|
||||||
|
// formatBytes formats a byte count as human-readable.
|
||||||
|
func formatBytes(b int64) string {
|
||||||
|
const unit = 1024
|
||||||
|
if b < unit {
|
||||||
|
return fmt.Sprintf("%d B", b)
|
||||||
|
}
|
||||||
|
div, exp := int64(unit), 0
|
||||||
|
for n := b / unit; n >= unit; n /= unit {
|
||||||
|
div *= unit
|
||||||
|
exp++
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%.1f %cB", float64(b)/float64(div), "KMGTPE"[exp])
|
||||||
|
}
|
||||||
|
|
||||||
|
// printCreateSummary prints the cluster summary after creation.
|
||||||
|
func printCreateSummary(cfg *Config, state *SandboxState) {
|
||||||
|
fmt.Printf("\nSandbox %q ready (%d nodes)\n", state.Name, len(state.Servers))
|
||||||
|
fmt.Println()
|
||||||
|
|
||||||
|
fmt.Println("Nameservers:")
|
||||||
|
for _, srv := range state.NameserverNodes() {
|
||||||
|
floating := ""
|
||||||
|
if srv.FloatingIP != "" {
|
||||||
|
floating = fmt.Sprintf(" (floating: %s)", srv.FloatingIP)
|
||||||
|
}
|
||||||
|
fmt.Printf(" %s: %s%s\n", srv.Name, srv.IP, floating)
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Println("Nodes:")
|
||||||
|
for _, srv := range state.RegularNodes() {
|
||||||
|
fmt.Printf(" %s: %s\n", srv.Name, srv.IP)
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Printf("Domain: %s\n", cfg.Domain)
|
||||||
|
fmt.Printf("Gateway: https://%s\n", cfg.Domain)
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("SSH: orama sandbox ssh 1")
|
||||||
|
fmt.Println("Destroy: orama sandbox destroy")
|
||||||
|
}
|
||||||
|
|
||||||
|
// cleanupFailedCreate deletes any servers that were created during a failed provision.
|
||||||
|
func cleanupFailedCreate(client *HetznerClient, state *SandboxState) {
|
||||||
|
if len(state.Servers) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
fmt.Println("\nCleaning up failed creation...")
|
||||||
|
for _, srv := range state.Servers {
|
||||||
|
if srv.ID > 0 {
|
||||||
|
client.DeleteServer(srv.ID)
|
||||||
|
fmt.Printf(" Deleted %s\n", srv.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
DeleteState(state.Name)
|
||||||
|
}
|
||||||
122
pkg/cli/sandbox/destroy.go
Normal file
122
pkg/cli/sandbox/destroy.go
Normal file
@ -0,0 +1,122 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Destroy tears down a sandbox cluster.
|
||||||
|
func Destroy(name string, force bool) error {
|
||||||
|
cfg, err := LoadConfig()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resolve sandbox name
|
||||||
|
state, err := resolveSandbox(name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Confirm destruction
|
||||||
|
if !force {
|
||||||
|
reader := bufio.NewReader(os.Stdin)
|
||||||
|
fmt.Printf("Destroy sandbox %q? This deletes %d servers. [y/N]: ", state.Name, len(state.Servers))
|
||||||
|
choice, _ := reader.ReadString('\n')
|
||||||
|
choice = strings.TrimSpace(strings.ToLower(choice))
|
||||||
|
if choice != "y" && choice != "yes" {
|
||||||
|
fmt.Println("Aborted.")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
state.Status = StatusDestroying
|
||||||
|
SaveState(state) // best-effort status update
|
||||||
|
|
||||||
|
client := NewHetznerClient(cfg.HetznerAPIToken)
|
||||||
|
|
||||||
|
// Step 1: Unassign floating IPs from nameserver nodes
|
||||||
|
fmt.Println("Unassigning floating IPs...")
|
||||||
|
for _, srv := range state.NameserverNodes() {
|
||||||
|
if srv.FloatingIP == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// Find the floating IP ID from config
|
||||||
|
for _, fip := range cfg.FloatingIPs {
|
||||||
|
if fip.IP == srv.FloatingIP {
|
||||||
|
if err := client.UnassignFloatingIP(fip.ID); err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, " Warning: could not unassign floating IP %s: %v\n", fip.IP, err)
|
||||||
|
} else {
|
||||||
|
fmt.Printf(" Unassigned %s from %s\n", fip.IP, srv.Name)
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 2: Delete all servers in parallel
|
||||||
|
fmt.Printf("Deleting %d servers...\n", len(state.Servers))
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
var mu sync.Mutex
|
||||||
|
var failed []string
|
||||||
|
|
||||||
|
for _, srv := range state.Servers {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(srv ServerState) {
|
||||||
|
defer wg.Done()
|
||||||
|
if err := client.DeleteServer(srv.ID); err != nil {
|
||||||
|
// Treat 404 as already deleted (idempotent)
|
||||||
|
if strings.Contains(err.Error(), "404") || strings.Contains(err.Error(), "not found") {
|
||||||
|
fmt.Printf(" %s (ID %d): already deleted\n", srv.Name, srv.ID)
|
||||||
|
} else {
|
||||||
|
mu.Lock()
|
||||||
|
failed = append(failed, fmt.Sprintf("%s (ID %d): %v", srv.Name, srv.ID, err))
|
||||||
|
mu.Unlock()
|
||||||
|
fmt.Fprintf(os.Stderr, " Warning: failed to delete %s: %v\n", srv.Name, err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
fmt.Printf(" Deleted %s (ID %d)\n", srv.Name, srv.ID)
|
||||||
|
}
|
||||||
|
}(srv)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
|
||||||
|
if len(failed) > 0 {
|
||||||
|
fmt.Fprintf(os.Stderr, "\nFailed to delete %d server(s):\n", len(failed))
|
||||||
|
for _, f := range failed {
|
||||||
|
fmt.Fprintf(os.Stderr, " %s\n", f)
|
||||||
|
}
|
||||||
|
fmt.Fprintf(os.Stderr, "\nManual cleanup: delete servers at https://console.hetzner.cloud\n")
|
||||||
|
state.Status = StatusError
|
||||||
|
SaveState(state)
|
||||||
|
return fmt.Errorf("failed to delete %d server(s)", len(failed))
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 3: Remove state file
|
||||||
|
if err := DeleteState(state.Name); err != nil {
|
||||||
|
return fmt.Errorf("delete state: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Printf("\nSandbox %q destroyed (%d servers deleted)\n", state.Name, len(state.Servers))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// resolveSandbox finds a sandbox by name or returns the active one.
|
||||||
|
func resolveSandbox(name string) (*SandboxState, error) {
|
||||||
|
if name != "" {
|
||||||
|
return LoadState(name)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Find the active sandbox
|
||||||
|
active, err := FindActiveSandbox()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if active == nil {
|
||||||
|
return nil, fmt.Errorf("no active sandbox found, specify --name")
|
||||||
|
}
|
||||||
|
return active, nil
|
||||||
|
}
|
||||||
438
pkg/cli/sandbox/hetzner.go
Normal file
438
pkg/cli/sandbox/hetzner.go
Normal file
@ -0,0 +1,438 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const hetznerBaseURL = "https://api.hetzner.cloud/v1"
|
||||||
|
|
||||||
|
// HetznerClient is a minimal Hetzner Cloud API client.
|
||||||
|
type HetznerClient struct {
|
||||||
|
token string
|
||||||
|
httpClient *http.Client
|
||||||
|
}
|
||||||
|
|
||||||
|
// NewHetznerClient creates a new Hetzner API client.
|
||||||
|
func NewHetznerClient(token string) *HetznerClient {
|
||||||
|
return &HetznerClient{
|
||||||
|
token: token,
|
||||||
|
httpClient: &http.Client{
|
||||||
|
Timeout: 30 * time.Second,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Request helpers ---
|
||||||
|
|
||||||
|
func (c *HetznerClient) doRequest(method, path string, body interface{}) ([]byte, int, error) {
|
||||||
|
var bodyReader io.Reader
|
||||||
|
if body != nil {
|
||||||
|
data, err := json.Marshal(body)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, fmt.Errorf("marshal request body: %w", err)
|
||||||
|
}
|
||||||
|
bodyReader = bytes.NewReader(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
req, err := http.NewRequest(method, hetznerBaseURL+path, bodyReader)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, fmt.Errorf("create request: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||||
|
if body != nil {
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
}
|
||||||
|
|
||||||
|
resp, err := c.httpClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, 0, fmt.Errorf("request %s %s: %w", method, path, err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
respBody, err := io.ReadAll(resp.Body)
|
||||||
|
if err != nil {
|
||||||
|
return nil, resp.StatusCode, fmt.Errorf("read response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return respBody, resp.StatusCode, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *HetznerClient) get(path string) ([]byte, error) {
|
||||||
|
body, status, err := c.doRequest("GET", path, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if status < 200 || status >= 300 {
|
||||||
|
return nil, parseHetznerError(body, status)
|
||||||
|
}
|
||||||
|
return body, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *HetznerClient) post(path string, payload interface{}) ([]byte, error) {
|
||||||
|
body, status, err := c.doRequest("POST", path, payload)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if status < 200 || status >= 300 {
|
||||||
|
return nil, parseHetznerError(body, status)
|
||||||
|
}
|
||||||
|
return body, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c *HetznerClient) delete(path string) error {
|
||||||
|
_, status, err := c.doRequest("DELETE", path, nil)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if status < 200 || status >= 300 {
|
||||||
|
return fmt.Errorf("delete %s: HTTP %d", path, status)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- API types ---
|
||||||
|
|
||||||
|
// HetznerServer represents a Hetzner Cloud server.
|
||||||
|
type HetznerServer struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Status string `json:"status"` // initializing, running, off, ...
|
||||||
|
PublicNet HetznerPublicNet `json:"public_net"`
|
||||||
|
Labels map[string]string `json:"labels"`
|
||||||
|
ServerType struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
} `json:"server_type"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// HetznerPublicNet holds public networking info for a server.
|
||||||
|
type HetznerPublicNet struct {
|
||||||
|
IPv4 struct {
|
||||||
|
IP string `json:"ip"`
|
||||||
|
} `json:"ipv4"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// HetznerFloatingIP represents a Hetzner floating IP.
|
||||||
|
type HetznerFloatingIP struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
IP string `json:"ip"`
|
||||||
|
Server *int64 `json:"server"` // nil if unassigned
|
||||||
|
Labels map[string]string `json:"labels"`
|
||||||
|
Description string `json:"description"`
|
||||||
|
HomeLocation struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
} `json:"home_location"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// HetznerSSHKey represents a Hetzner SSH key.
|
||||||
|
type HetznerSSHKey struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Fingerprint string `json:"fingerprint"`
|
||||||
|
PublicKey string `json:"public_key"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// HetznerFirewall represents a Hetzner firewall.
|
||||||
|
type HetznerFirewall struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Rules []HetznerFWRule `json:"rules"`
|
||||||
|
Labels map[string]string `json:"labels"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// HetznerFWRule represents a firewall rule.
|
||||||
|
type HetznerFWRule struct {
|
||||||
|
Direction string `json:"direction"`
|
||||||
|
Protocol string `json:"protocol"`
|
||||||
|
Port string `json:"port"`
|
||||||
|
SourceIPs []string `json:"source_ips"`
|
||||||
|
Description string `json:"description,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// HetznerError represents an API error response.
|
||||||
|
type HetznerError struct {
|
||||||
|
Error struct {
|
||||||
|
Code string `json:"code"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
} `json:"error"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseHetznerError(body []byte, status int) error {
|
||||||
|
var he HetznerError
|
||||||
|
if err := json.Unmarshal(body, &he); err == nil && he.Error.Message != "" {
|
||||||
|
return fmt.Errorf("hetzner API error (HTTP %d): %s — %s", status, he.Error.Code, he.Error.Message)
|
||||||
|
}
|
||||||
|
return fmt.Errorf("hetzner API error: HTTP %d", status)
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Server operations ---
|
||||||
|
|
||||||
|
// CreateServerRequest holds parameters for server creation.
|
||||||
|
type CreateServerRequest struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
ServerType string `json:"server_type"`
|
||||||
|
Image string `json:"image"`
|
||||||
|
Location string `json:"location"`
|
||||||
|
SSHKeys []int64 `json:"ssh_keys"`
|
||||||
|
Labels map[string]string `json:"labels"`
|
||||||
|
Firewalls []struct {
|
||||||
|
Firewall int64 `json:"firewall"`
|
||||||
|
} `json:"firewalls,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// CreateServer creates a new server and returns it.
|
||||||
|
func (c *HetznerClient) CreateServer(req CreateServerRequest) (*HetznerServer, error) {
|
||||||
|
body, err := c.post("/servers", req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("create server %q: %w", req.Name, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
Server HetznerServer `json:"server"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse create server response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &resp.Server, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetServer retrieves a server by ID.
|
||||||
|
func (c *HetznerClient) GetServer(id int64) (*HetznerServer, error) {
|
||||||
|
body, err := c.get("/servers/" + strconv.FormatInt(id, 10))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("get server %d: %w", id, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
Server HetznerServer `json:"server"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse server response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &resp.Server, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteServer deletes a server by ID.
|
||||||
|
func (c *HetznerClient) DeleteServer(id int64) error {
|
||||||
|
return c.delete("/servers/" + strconv.FormatInt(id, 10))
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListServersByLabel lists servers filtered by a label selector.
|
||||||
|
func (c *HetznerClient) ListServersByLabel(selector string) ([]HetznerServer, error) {
|
||||||
|
body, err := c.get("/servers?label_selector=" + selector)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("list servers: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
Servers []HetznerServer `json:"servers"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse servers response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return resp.Servers, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// WaitForServer polls until the server reaches "running" status.
|
||||||
|
func (c *HetznerClient) WaitForServer(id int64, timeout time.Duration) (*HetznerServer, error) {
|
||||||
|
deadline := time.Now().Add(timeout)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
srv, err := c.GetServer(id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if srv.Status == "running" {
|
||||||
|
return srv, nil
|
||||||
|
}
|
||||||
|
time.Sleep(3 * time.Second)
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("server %d did not reach running state within %s", id, timeout)
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Floating IP operations ---
|
||||||
|
|
||||||
|
// CreateFloatingIP creates a new floating IP.
|
||||||
|
func (c *HetznerClient) CreateFloatingIP(location, description string, labels map[string]string) (*HetznerFloatingIP, error) {
|
||||||
|
payload := map[string]interface{}{
|
||||||
|
"type": "ipv4",
|
||||||
|
"home_location": location,
|
||||||
|
"description": description,
|
||||||
|
"labels": labels,
|
||||||
|
}
|
||||||
|
|
||||||
|
body, err := c.post("/floating_ips", payload)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("create floating IP: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
FloatingIP HetznerFloatingIP `json:"floating_ip"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse floating IP response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &resp.FloatingIP, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListFloatingIPsByLabel lists floating IPs filtered by label.
|
||||||
|
func (c *HetznerClient) ListFloatingIPsByLabel(selector string) ([]HetznerFloatingIP, error) {
|
||||||
|
body, err := c.get("/floating_ips?label_selector=" + selector)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("list floating IPs: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
FloatingIPs []HetznerFloatingIP `json:"floating_ips"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse floating IPs response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return resp.FloatingIPs, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// AssignFloatingIP assigns a floating IP to a server.
|
||||||
|
func (c *HetznerClient) AssignFloatingIP(floatingIPID, serverID int64) error {
|
||||||
|
payload := map[string]int64{"server": serverID}
|
||||||
|
_, err := c.post("/floating_ips/"+strconv.FormatInt(floatingIPID, 10)+"/actions/assign", payload)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("assign floating IP %d to server %d: %w", floatingIPID, serverID, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnassignFloatingIP removes a floating IP assignment.
|
||||||
|
func (c *HetznerClient) UnassignFloatingIP(floatingIPID int64) error {
|
||||||
|
_, err := c.post("/floating_ips/"+strconv.FormatInt(floatingIPID, 10)+"/actions/unassign", struct{}{})
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("unassign floating IP %d: %w", floatingIPID, err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- SSH Key operations ---
|
||||||
|
|
||||||
|
// UploadSSHKey uploads a public key to Hetzner.
|
||||||
|
func (c *HetznerClient) UploadSSHKey(name, publicKey string) (*HetznerSSHKey, error) {
|
||||||
|
payload := map[string]string{
|
||||||
|
"name": name,
|
||||||
|
"public_key": publicKey,
|
||||||
|
}
|
||||||
|
|
||||||
|
body, err := c.post("/ssh_keys", payload)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("upload SSH key: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
SSHKey HetznerSSHKey `json:"ssh_key"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse SSH key response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &resp.SSHKey, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// GetSSHKey retrieves an SSH key by ID.
|
||||||
|
func (c *HetznerClient) GetSSHKey(id int64) (*HetznerSSHKey, error) {
|
||||||
|
body, err := c.get("/ssh_keys/" + strconv.FormatInt(id, 10))
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("get SSH key %d: %w", id, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
SSHKey HetznerSSHKey `json:"ssh_key"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse SSH key response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &resp.SSHKey, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Firewall operations ---
|
||||||
|
|
||||||
|
// CreateFirewall creates a firewall with the given rules.
|
||||||
|
func (c *HetznerClient) CreateFirewall(name string, rules []HetznerFWRule, labels map[string]string) (*HetznerFirewall, error) {
|
||||||
|
payload := map[string]interface{}{
|
||||||
|
"name": name,
|
||||||
|
"rules": rules,
|
||||||
|
"labels": labels,
|
||||||
|
}
|
||||||
|
|
||||||
|
body, err := c.post("/firewalls", payload)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("create firewall: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
Firewall HetznerFirewall `json:"firewall"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse firewall response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &resp.Firewall, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListFirewallsByLabel lists firewalls filtered by label.
|
||||||
|
func (c *HetznerClient) ListFirewallsByLabel(selector string) ([]HetznerFirewall, error) {
|
||||||
|
body, err := c.get("/firewalls?label_selector=" + selector)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("list firewalls: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resp struct {
|
||||||
|
Firewalls []HetznerFirewall `json:"firewalls"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(body, &resp); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse firewalls response: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return resp.Firewalls, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteFirewall deletes a firewall by ID.
|
||||||
|
func (c *HetznerClient) DeleteFirewall(id int64) error {
|
||||||
|
return c.delete("/firewalls/" + strconv.FormatInt(id, 10))
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Validation ---
|
||||||
|
|
||||||
|
// ValidateToken checks if the API token is valid by making a simple request.
|
||||||
|
func (c *HetznerClient) ValidateToken() error {
|
||||||
|
_, err := c.get("/servers?per_page=1")
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("invalid Hetzner API token: %w", err)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- Sandbox firewall rules ---
|
||||||
|
|
||||||
|
// SandboxFirewallRules returns the standard firewall rules for sandbox nodes.
|
||||||
|
func SandboxFirewallRules() []HetznerFWRule {
|
||||||
|
allIPv4 := []string{"0.0.0.0/0"}
|
||||||
|
allIPv6 := []string{"::/0"}
|
||||||
|
allIPs := append(allIPv4, allIPv6...)
|
||||||
|
|
||||||
|
return []HetznerFWRule{
|
||||||
|
{Direction: "in", Protocol: "tcp", Port: "22", SourceIPs: allIPs, Description: "SSH"},
|
||||||
|
{Direction: "in", Protocol: "tcp", Port: "53", SourceIPs: allIPs, Description: "DNS TCP"},
|
||||||
|
{Direction: "in", Protocol: "udp", Port: "53", SourceIPs: allIPs, Description: "DNS UDP"},
|
||||||
|
{Direction: "in", Protocol: "tcp", Port: "80", SourceIPs: allIPs, Description: "HTTP"},
|
||||||
|
{Direction: "in", Protocol: "tcp", Port: "443", SourceIPs: allIPs, Description: "HTTPS"},
|
||||||
|
{Direction: "in", Protocol: "udp", Port: "51820", SourceIPs: allIPs, Description: "WireGuard"},
|
||||||
|
}
|
||||||
|
}
|
||||||
303
pkg/cli/sandbox/hetzner_test.go
Normal file
303
pkg/cli/sandbox/hetzner_test.go
Normal file
@ -0,0 +1,303 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestValidateToken_Success(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Header.Get("Authorization") != "Bearer test-token" {
|
||||||
|
t.Errorf("unexpected auth header: %s", r.Header.Get("Authorization"))
|
||||||
|
}
|
||||||
|
w.WriteHeader(200)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{"servers": []interface{}{}})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
if err := client.ValidateToken(); err != nil {
|
||||||
|
t.Errorf("ValidateToken() error = %v, want nil", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestValidateToken_InvalidToken(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(401)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||||
|
"error": map[string]string{
|
||||||
|
"code": "unauthorized",
|
||||||
|
"message": "unable to authenticate",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "bad-token")
|
||||||
|
if err := client.ValidateToken(); err == nil {
|
||||||
|
t.Error("ValidateToken() expected error for invalid token")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCreateServer(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != "POST" || r.URL.Path != "/v1/servers" {
|
||||||
|
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||||
|
}
|
||||||
|
|
||||||
|
var req CreateServerRequest
|
||||||
|
json.NewDecoder(r.Body).Decode(&req)
|
||||||
|
|
||||||
|
if req.Name != "sbx-test-1" {
|
||||||
|
t.Errorf("unexpected server name: %s", req.Name)
|
||||||
|
}
|
||||||
|
if req.ServerType != "cx22" {
|
||||||
|
t.Errorf("unexpected server type: %s", req.ServerType)
|
||||||
|
}
|
||||||
|
|
||||||
|
w.WriteHeader(201)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||||
|
"server": map[string]interface{}{
|
||||||
|
"id": 12345,
|
||||||
|
"name": req.Name,
|
||||||
|
"status": "initializing",
|
||||||
|
"public_net": map[string]interface{}{
|
||||||
|
"ipv4": map[string]string{"ip": "1.2.3.4"},
|
||||||
|
},
|
||||||
|
"labels": req.Labels,
|
||||||
|
"server_type": map[string]string{"name": "cx22"},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
server, err := client.CreateServer(CreateServerRequest{
|
||||||
|
Name: "sbx-test-1",
|
||||||
|
ServerType: "cx22",
|
||||||
|
Image: "ubuntu-24.04",
|
||||||
|
Location: "fsn1",
|
||||||
|
SSHKeys: []int64{1},
|
||||||
|
Labels: map[string]string{"orama-sandbox": "test"},
|
||||||
|
})
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateServer() error = %v", err)
|
||||||
|
}
|
||||||
|
if server.ID != 12345 {
|
||||||
|
t.Errorf("server ID = %d, want 12345", server.ID)
|
||||||
|
}
|
||||||
|
if server.Name != "sbx-test-1" {
|
||||||
|
t.Errorf("server name = %s, want sbx-test-1", server.Name)
|
||||||
|
}
|
||||||
|
if server.PublicNet.IPv4.IP != "1.2.3.4" {
|
||||||
|
t.Errorf("server IP = %s, want 1.2.3.4", server.PublicNet.IPv4.IP)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteServer(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != "DELETE" || r.URL.Path != "/v1/servers/12345" {
|
||||||
|
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||||
|
}
|
||||||
|
w.WriteHeader(200)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
if err := client.DeleteServer(12345); err != nil {
|
||||||
|
t.Errorf("DeleteServer() error = %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListServersByLabel(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Query().Get("label_selector") != "orama-sandbox=test" {
|
||||||
|
t.Errorf("unexpected label_selector: %s", r.URL.Query().Get("label_selector"))
|
||||||
|
}
|
||||||
|
w.WriteHeader(200)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||||
|
"servers": []map[string]interface{}{
|
||||||
|
{"id": 1, "name": "sbx-test-1", "status": "running", "public_net": map[string]interface{}{"ipv4": map[string]string{"ip": "1.1.1.1"}}, "server_type": map[string]string{"name": "cx22"}},
|
||||||
|
{"id": 2, "name": "sbx-test-2", "status": "running", "public_net": map[string]interface{}{"ipv4": map[string]string{"ip": "2.2.2.2"}}, "server_type": map[string]string{"name": "cx22"}},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
servers, err := client.ListServersByLabel("orama-sandbox=test")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ListServersByLabel() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(servers) != 2 {
|
||||||
|
t.Errorf("got %d servers, want 2", len(servers))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestWaitForServer_AlreadyRunning(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(200)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||||
|
"server": map[string]interface{}{
|
||||||
|
"id": 1,
|
||||||
|
"name": "test",
|
||||||
|
"status": "running",
|
||||||
|
"public_net": map[string]interface{}{
|
||||||
|
"ipv4": map[string]string{"ip": "1.1.1.1"},
|
||||||
|
},
|
||||||
|
"server_type": map[string]string{"name": "cx22"},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
server, err := client.WaitForServer(1, 5*time.Second)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("WaitForServer() error = %v", err)
|
||||||
|
}
|
||||||
|
if server.Status != "running" {
|
||||||
|
t.Errorf("server status = %s, want running", server.Status)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAssignFloatingIP(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != "POST" || r.URL.Path != "/v1/floating_ips/100/actions/assign" {
|
||||||
|
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||||
|
}
|
||||||
|
|
||||||
|
var body map[string]int64
|
||||||
|
json.NewDecoder(r.Body).Decode(&body)
|
||||||
|
if body["server"] != 200 {
|
||||||
|
t.Errorf("unexpected server ID: %d", body["server"])
|
||||||
|
}
|
||||||
|
|
||||||
|
w.WriteHeader(200)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{"action": map[string]interface{}{"id": 1, "status": "running"}})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
if err := client.AssignFloatingIP(100, 200); err != nil {
|
||||||
|
t.Errorf("AssignFloatingIP() error = %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestUploadSSHKey(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != "POST" || r.URL.Path != "/v1/ssh_keys" {
|
||||||
|
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||||
|
}
|
||||||
|
w.WriteHeader(201)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||||
|
"ssh_key": map[string]interface{}{
|
||||||
|
"id": 42,
|
||||||
|
"name": "orama-sandbox",
|
||||||
|
"fingerprint": "aa:bb:cc:dd",
|
||||||
|
"public_key": "ssh-ed25519 AAAA...",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
key, err := client.UploadSSHKey("orama-sandbox", "ssh-ed25519 AAAA...")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("UploadSSHKey() error = %v", err)
|
||||||
|
}
|
||||||
|
if key.ID != 42 {
|
||||||
|
t.Errorf("key ID = %d, want 42", key.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCreateFirewall(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.Method != "POST" || r.URL.Path != "/v1/firewalls" {
|
||||||
|
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||||
|
}
|
||||||
|
w.WriteHeader(201)
|
||||||
|
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||||
|
"firewall": map[string]interface{}{
|
||||||
|
"id": 99,
|
||||||
|
"name": "orama-sandbox",
|
||||||
|
},
|
||||||
|
})
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
client := newTestClient(srv, "test-token")
|
||||||
|
fw, err := client.CreateFirewall("orama-sandbox", SandboxFirewallRules(), map[string]string{"orama-sandbox": "infra"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CreateFirewall() error = %v", err)
|
||||||
|
}
|
||||||
|
if fw.ID != 99 {
|
||||||
|
t.Errorf("firewall ID = %d, want 99", fw.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSandboxFirewallRules(t *testing.T) {
|
||||||
|
rules := SandboxFirewallRules()
|
||||||
|
if len(rules) != 6 {
|
||||||
|
t.Errorf("got %d rules, want 6", len(rules))
|
||||||
|
}
|
||||||
|
|
||||||
|
expectedPorts := map[string]bool{"22": false, "53": false, "80": false, "443": false, "51820": false}
|
||||||
|
for _, r := range rules {
|
||||||
|
expectedPorts[r.Port] = true
|
||||||
|
if r.Direction != "in" {
|
||||||
|
t.Errorf("rule %s direction = %s, want in", r.Port, r.Direction)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for port, seen := range expectedPorts {
|
||||||
|
if !seen {
|
||||||
|
t.Errorf("missing firewall rule for port %s", port)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseHetznerError(t *testing.T) {
|
||||||
|
body := `{"error":{"code":"uniqueness_error","message":"server name already used"}}`
|
||||||
|
err := parseHetznerError([]byte(body), 409)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error")
|
||||||
|
}
|
||||||
|
expected := "hetzner API error (HTTP 409): uniqueness_error — server name already used"
|
||||||
|
if err.Error() != expected {
|
||||||
|
t.Errorf("error = %q, want %q", err.Error(), expected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// newTestClient creates a HetznerClient pointing at a test server.
|
||||||
|
func newTestClient(ts *httptest.Server, token string) *HetznerClient {
|
||||||
|
client := NewHetznerClient(token)
|
||||||
|
// Override the base URL by using a custom transport
|
||||||
|
client.httpClient = ts.Client()
|
||||||
|
// We need to override the base URL — wrap the transport
|
||||||
|
origTransport := client.httpClient.Transport
|
||||||
|
client.httpClient.Transport = &testTransport{
|
||||||
|
base: origTransport,
|
||||||
|
testURL: ts.URL,
|
||||||
|
}
|
||||||
|
return client
|
||||||
|
}
|
||||||
|
|
||||||
|
// testTransport rewrites requests to point at the test server.
|
||||||
|
type testTransport struct {
|
||||||
|
base http.RoundTripper
|
||||||
|
testURL string
|
||||||
|
}
|
||||||
|
|
||||||
|
func (t *testTransport) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||||
|
// Rewrite the URL to point at the test server
|
||||||
|
req.URL.Scheme = "http"
|
||||||
|
req.URL.Host = t.testURL[len("http://"):]
|
||||||
|
if t.base != nil {
|
||||||
|
return t.base.RoundTrip(req)
|
||||||
|
}
|
||||||
|
return http.DefaultTransport.RoundTrip(req)
|
||||||
|
}
|
||||||
26
pkg/cli/sandbox/names.go
Normal file
26
pkg/cli/sandbox/names.go
Normal file
@ -0,0 +1,26 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math/rand"
|
||||||
|
)
|
||||||
|
|
||||||
|
var adjectives = []string{
|
||||||
|
"swift", "bright", "calm", "dark", "eager",
|
||||||
|
"fair", "gold", "hazy", "iron", "jade",
|
||||||
|
"keen", "lush", "mild", "neat", "opal",
|
||||||
|
"pure", "raw", "sage", "teal", "warm",
|
||||||
|
}
|
||||||
|
|
||||||
|
var nouns = []string{
|
||||||
|
"falcon", "beacon", "cedar", "delta", "ember",
|
||||||
|
"frost", "grove", "haven", "ivory", "jewel",
|
||||||
|
"knot", "latch", "maple", "nexus", "orbit",
|
||||||
|
"prism", "reef", "spark", "tide", "vault",
|
||||||
|
}
|
||||||
|
|
||||||
|
// GenerateName produces a random adjective-noun name like "swift-falcon".
|
||||||
|
func GenerateName() string {
|
||||||
|
adj := adjectives[rand.Intn(len(adjectives))]
|
||||||
|
noun := nouns[rand.Intn(len(nouns))]
|
||||||
|
return adj + "-" + noun
|
||||||
|
}
|
||||||
137
pkg/cli/sandbox/rollout.go
Normal file
137
pkg/cli/sandbox/rollout.go
Normal file
@ -0,0 +1,137 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/cli/remotessh"
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Rollout builds, pushes, and performs a rolling upgrade on a sandbox cluster.
|
||||||
|
func Rollout(name string) error {
|
||||||
|
cfg, err := LoadConfig()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
state, err := resolveSandbox(name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
fmt.Printf("Rolling out to sandbox %q (%d nodes)\n\n", state.Name, len(state.Servers))
|
||||||
|
|
||||||
|
// Step 1: Find or require binary archive
|
||||||
|
archivePath := findNewestArchive()
|
||||||
|
if archivePath == "" {
|
||||||
|
return fmt.Errorf("no binary archive found in /tmp/ (run `orama build` first)")
|
||||||
|
}
|
||||||
|
|
||||||
|
info, _ := os.Stat(archivePath)
|
||||||
|
fmt.Printf("Archive: %s (%s)\n\n", filepath.Base(archivePath), formatBytes(info.Size()))
|
||||||
|
|
||||||
|
// Step 2: Push archive to all nodes
|
||||||
|
fmt.Println("Pushing archive to all nodes...")
|
||||||
|
remotePath := "/tmp/" + filepath.Base(archivePath)
|
||||||
|
|
||||||
|
for i, srv := range state.Servers {
|
||||||
|
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||||
|
|
||||||
|
fmt.Printf(" [%d/%d] Uploading to %s...\n", i+1, len(state.Servers), srv.Name)
|
||||||
|
if err := remotessh.UploadFile(node, archivePath, remotePath); err != nil {
|
||||||
|
return fmt.Errorf("upload to %s: %w", srv.Name, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Extract archive
|
||||||
|
extractCmd := fmt.Sprintf("mkdir -p /opt/orama && tar xzf %s -C /opt/orama && rm -f %s",
|
||||||
|
remotePath, remotePath)
|
||||||
|
if err := remotessh.RunSSHStreaming(node, extractCmd); err != nil {
|
||||||
|
return fmt.Errorf("extract on %s: %w", srv.Name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Step 3: Rolling upgrade — followers first, leader last
|
||||||
|
fmt.Println("\nRolling upgrade (followers first, leader last)...")
|
||||||
|
|
||||||
|
// Find the leader
|
||||||
|
leaderIdx := findLeaderIndex(state, sshKeyPath)
|
||||||
|
if leaderIdx < 0 {
|
||||||
|
fmt.Fprintf(os.Stderr, " Warning: could not detect RQLite leader, upgrading in order\n")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Upgrade non-leaders first
|
||||||
|
for i, srv := range state.Servers {
|
||||||
|
if i == leaderIdx {
|
||||||
|
continue // skip leader, do it last
|
||||||
|
}
|
||||||
|
if err := upgradeNode(srv, sshKeyPath, i+1, len(state.Servers)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// Wait between nodes
|
||||||
|
if i < len(state.Servers)-1 {
|
||||||
|
fmt.Printf(" Waiting 15s before next node...\n")
|
||||||
|
time.Sleep(15 * time.Second)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Upgrade leader last
|
||||||
|
if leaderIdx >= 0 {
|
||||||
|
srv := state.Servers[leaderIdx]
|
||||||
|
if err := upgradeNode(srv, sshKeyPath, len(state.Servers), len(state.Servers)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Printf("\nRollout complete for sandbox %q\n", state.Name)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// findLeaderIndex returns the index of the RQLite leader node, or -1 if unknown.
|
||||||
|
func findLeaderIndex(state *SandboxState, sshKeyPath string) int {
|
||||||
|
for i, srv := range state.Servers {
|
||||||
|
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||||
|
out, err := runSSHOutput(node, "curl -sf http://localhost:5001/status 2>/dev/null | grep -o '\"state\":\"[^\"]*\"'")
|
||||||
|
if err == nil && contains(out, "Leader") {
|
||||||
|
return i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
|
||||||
|
// upgradeNode performs `orama node upgrade --restart` on a single node.
|
||||||
|
func upgradeNode(srv ServerState, sshKeyPath string, current, total int) error {
|
||||||
|
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||||
|
|
||||||
|
fmt.Printf(" [%d/%d] Upgrading %s (%s)...\n", current, total, srv.Name, srv.IP)
|
||||||
|
if err := remotessh.RunSSHStreaming(node, "orama node upgrade --restart"); err != nil {
|
||||||
|
return fmt.Errorf("upgrade %s: %w", srv.Name, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for health
|
||||||
|
fmt.Printf(" Checking health...")
|
||||||
|
if err := waitForRQLiteHealth(node, 2*time.Minute); err != nil {
|
||||||
|
fmt.Printf(" WARN: %v\n", err)
|
||||||
|
} else {
|
||||||
|
fmt.Println(" OK")
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// contains checks if s contains substr.
|
||||||
|
func contains(s, substr string) bool {
|
||||||
|
return len(s) >= len(substr) && findSubstring(s, substr)
|
||||||
|
}
|
||||||
|
|
||||||
|
func findSubstring(s, substr string) bool {
|
||||||
|
for i := 0; i <= len(s)-len(substr); i++ {
|
||||||
|
if s[i:i+len(substr)] == substr {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
319
pkg/cli/sandbox/setup.go
Normal file
319
pkg/cli/sandbox/setup.go
Normal file
@ -0,0 +1,319 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"crypto/ed25519"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/pem"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"golang.org/x/crypto/ssh"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Setup runs the interactive sandbox setup wizard.
|
||||||
|
func Setup() error {
|
||||||
|
fmt.Println("Orama Sandbox Setup")
|
||||||
|
fmt.Println("====================")
|
||||||
|
fmt.Println()
|
||||||
|
|
||||||
|
reader := bufio.NewReader(os.Stdin)
|
||||||
|
|
||||||
|
// Step 1: Hetzner API token
|
||||||
|
fmt.Print("Hetzner Cloud API token: ")
|
||||||
|
token, err := reader.ReadString('\n')
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("read token: %w", err)
|
||||||
|
}
|
||||||
|
token = strings.TrimSpace(token)
|
||||||
|
if token == "" {
|
||||||
|
return fmt.Errorf("API token is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Print(" Validating token... ")
|
||||||
|
client := NewHetznerClient(token)
|
||||||
|
if err := client.ValidateToken(); err != nil {
|
||||||
|
fmt.Println("FAILED")
|
||||||
|
return fmt.Errorf("invalid token: %w", err)
|
||||||
|
}
|
||||||
|
fmt.Println("OK")
|
||||||
|
fmt.Println()
|
||||||
|
|
||||||
|
// Step 2: Domain
|
||||||
|
fmt.Print("Sandbox domain (e.g., sbx.dbrs.space): ")
|
||||||
|
domain, err := reader.ReadString('\n')
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("read domain: %w", err)
|
||||||
|
}
|
||||||
|
domain = strings.TrimSpace(domain)
|
||||||
|
if domain == "" {
|
||||||
|
return fmt.Errorf("domain is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg := &Config{
|
||||||
|
HetznerAPIToken: token,
|
||||||
|
Domain: domain,
|
||||||
|
}
|
||||||
|
cfg.Defaults()
|
||||||
|
|
||||||
|
// Step 3: Floating IPs
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("Checking floating IPs...")
|
||||||
|
floatingIPs, err := setupFloatingIPs(client, cfg.Location)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
cfg.FloatingIPs = floatingIPs
|
||||||
|
|
||||||
|
// Step 4: Firewall
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("Checking firewall...")
|
||||||
|
fwID, err := setupFirewall(client)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
cfg.FirewallID = fwID
|
||||||
|
|
||||||
|
// Step 5: SSH key
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("Setting up SSH key...")
|
||||||
|
sshKeyConfig, err := setupSSHKey(client)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
cfg.SSHKey = sshKeyConfig
|
||||||
|
|
||||||
|
// Step 6: Display DNS instructions
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("DNS Configuration")
|
||||||
|
fmt.Println("-----------------")
|
||||||
|
fmt.Println("Configure the following at your domain registrar:")
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Printf(" 1. Add glue records (Personal DNS Servers):\n")
|
||||||
|
fmt.Printf(" ns1.%s -> %s\n", domain, cfg.FloatingIPs[0].IP)
|
||||||
|
fmt.Printf(" ns2.%s -> %s\n", domain, cfg.FloatingIPs[1].IP)
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Printf(" 2. Set custom nameservers for %s:\n", domain)
|
||||||
|
fmt.Printf(" ns1.%s\n", domain)
|
||||||
|
fmt.Printf(" ns2.%s\n", domain)
|
||||||
|
fmt.Println()
|
||||||
|
|
||||||
|
// Step 7: Verify DNS (optional)
|
||||||
|
fmt.Print("Verify DNS now? [y/N]: ")
|
||||||
|
verifyChoice, _ := reader.ReadString('\n')
|
||||||
|
verifyChoice = strings.TrimSpace(strings.ToLower(verifyChoice))
|
||||||
|
if verifyChoice == "y" || verifyChoice == "yes" {
|
||||||
|
verifyDNS(domain)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Save config
|
||||||
|
if err := SaveConfig(cfg); err != nil {
|
||||||
|
return fmt.Errorf("save config: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("Setup complete! Config saved to ~/.orama/sandbox.yaml")
|
||||||
|
fmt.Println()
|
||||||
|
fmt.Println("Next: orama sandbox create")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// setupFloatingIPs checks for existing floating IPs or creates new ones.
|
||||||
|
func setupFloatingIPs(client *HetznerClient, location string) ([]FloatIP, error) {
|
||||||
|
existing, err := client.ListFloatingIPsByLabel("orama-sandbox-dns=true")
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("list floating IPs: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(existing) >= 2 {
|
||||||
|
fmt.Printf(" Found %d existing floating IPs:\n", len(existing))
|
||||||
|
result := make([]FloatIP, 2)
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
fmt.Printf(" ns%d: %s (ID: %d)\n", i+1, existing[i].IP, existing[i].ID)
|
||||||
|
result[i] = FloatIP{ID: existing[i].ID, IP: existing[i].IP}
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Need to create missing floating IPs
|
||||||
|
needed := 2 - len(existing)
|
||||||
|
fmt.Printf(" Need to create %d floating IP(s)...\n", needed)
|
||||||
|
|
||||||
|
reader := bufio.NewReader(os.Stdin)
|
||||||
|
fmt.Printf(" Create %d floating IP(s) in %s? (~$0.005/hr each) [Y/n]: ", needed, location)
|
||||||
|
choice, _ := reader.ReadString('\n')
|
||||||
|
choice = strings.TrimSpace(strings.ToLower(choice))
|
||||||
|
if choice == "n" || choice == "no" {
|
||||||
|
return nil, fmt.Errorf("floating IPs required, aborting setup")
|
||||||
|
}
|
||||||
|
|
||||||
|
result := make([]FloatIP, 0, 2)
|
||||||
|
for _, fip := range existing {
|
||||||
|
result = append(result, FloatIP{ID: fip.ID, IP: fip.IP})
|
||||||
|
}
|
||||||
|
|
||||||
|
for i := len(existing); i < 2; i++ {
|
||||||
|
desc := fmt.Sprintf("orama-sandbox-ns%d", i+1)
|
||||||
|
labels := map[string]string{"orama-sandbox-dns": "true"}
|
||||||
|
fip, err := client.CreateFloatingIP(location, desc, labels)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("create floating IP %d: %w", i+1, err)
|
||||||
|
}
|
||||||
|
fmt.Printf(" Created ns%d: %s (ID: %d)\n", i+1, fip.IP, fip.ID)
|
||||||
|
result = append(result, FloatIP{ID: fip.ID, IP: fip.IP})
|
||||||
|
}
|
||||||
|
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// setupFirewall ensures a sandbox firewall exists.
|
||||||
|
func setupFirewall(client *HetznerClient) (int64, error) {
|
||||||
|
existing, err := client.ListFirewallsByLabel("orama-sandbox=infra")
|
||||||
|
if err != nil {
|
||||||
|
return 0, fmt.Errorf("list firewalls: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(existing) > 0 {
|
||||||
|
fmt.Printf(" Found existing firewall: %s (ID: %d)\n", existing[0].Name, existing[0].ID)
|
||||||
|
return existing[0].ID, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Print(" Creating sandbox firewall... ")
|
||||||
|
fw, err := client.CreateFirewall(
|
||||||
|
"orama-sandbox",
|
||||||
|
SandboxFirewallRules(),
|
||||||
|
map[string]string{"orama-sandbox": "infra"},
|
||||||
|
)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Println("FAILED")
|
||||||
|
return 0, fmt.Errorf("create firewall: %w", err)
|
||||||
|
}
|
||||||
|
fmt.Printf("OK (ID: %d)\n", fw.ID)
|
||||||
|
return fw.ID, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// setupSSHKey generates an SSH keypair and uploads it to Hetzner.
|
||||||
|
func setupSSHKey(client *HetznerClient) (SSHKeyConfig, error) {
|
||||||
|
dir, err := configDir()
|
||||||
|
if err != nil {
|
||||||
|
return SSHKeyConfig{}, err
|
||||||
|
}
|
||||||
|
|
||||||
|
privPath := dir + "/sandbox_key"
|
||||||
|
pubPath := privPath + ".pub"
|
||||||
|
|
||||||
|
// Check for existing key
|
||||||
|
if _, err := os.Stat(privPath); err == nil {
|
||||||
|
fmt.Printf(" SSH key already exists: %s\n", privPath)
|
||||||
|
|
||||||
|
// Read public key and check if it's on Hetzner
|
||||||
|
pubData, err := os.ReadFile(pubPath)
|
||||||
|
if err != nil {
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("read public key: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Try to upload (will fail with uniqueness error if already exists)
|
||||||
|
key, err := client.UploadSSHKey("orama-sandbox", strings.TrimSpace(string(pubData)))
|
||||||
|
if err != nil {
|
||||||
|
// Key likely already exists on Hetzner — find it by listing
|
||||||
|
fmt.Printf(" SSH key may already be on Hetzner (upload: %v)\n", err)
|
||||||
|
fmt.Print(" Enter the Hetzner SSH key ID (or 0 to re-upload): ")
|
||||||
|
reader := bufio.NewReader(os.Stdin)
|
||||||
|
idStr, _ := reader.ReadString('\n')
|
||||||
|
idStr = strings.TrimSpace(idStr)
|
||||||
|
var hetznerID int64
|
||||||
|
fmt.Sscanf(idStr, "%d", &hetznerID)
|
||||||
|
|
||||||
|
if hetznerID == 0 {
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("could not resolve SSH key on Hetzner, try deleting and re-running setup")
|
||||||
|
}
|
||||||
|
|
||||||
|
return SSHKeyConfig{
|
||||||
|
HetznerID: hetznerID,
|
||||||
|
PrivateKeyPath: "~/.orama/sandbox_key",
|
||||||
|
PublicKeyPath: "~/.orama/sandbox_key.pub",
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Printf(" Uploaded to Hetzner (ID: %d)\n", key.ID)
|
||||||
|
return SSHKeyConfig{
|
||||||
|
HetznerID: key.ID,
|
||||||
|
PrivateKeyPath: "~/.orama/sandbox_key",
|
||||||
|
PublicKeyPath: "~/.orama/sandbox_key.pub",
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Generate new ed25519 keypair
|
||||||
|
fmt.Print(" Generating ed25519 keypair... ")
|
||||||
|
pubKey, privKey, err := ed25519.GenerateKey(rand.Reader)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Println("FAILED")
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("generate key: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Marshal private key to OpenSSH format
|
||||||
|
pemBlock, err := ssh.MarshalPrivateKey(privKey, "")
|
||||||
|
if err != nil {
|
||||||
|
fmt.Println("FAILED")
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("marshal private key: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
privPEM := pem.EncodeToMemory(pemBlock)
|
||||||
|
if err := os.WriteFile(privPath, privPEM, 0600); err != nil {
|
||||||
|
fmt.Println("FAILED")
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("write private key: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Marshal public key to authorized_keys format
|
||||||
|
sshPubKey, err := ssh.NewPublicKey(pubKey)
|
||||||
|
if err != nil {
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("convert public key: %w", err)
|
||||||
|
}
|
||||||
|
pubStr := strings.TrimSpace(string(ssh.MarshalAuthorizedKey(sshPubKey)))
|
||||||
|
|
||||||
|
if err := os.WriteFile(pubPath, []byte(pubStr+"\n"), 0644); err != nil {
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("write public key: %w", err)
|
||||||
|
}
|
||||||
|
fmt.Println("OK")
|
||||||
|
|
||||||
|
// Upload to Hetzner
|
||||||
|
fmt.Print(" Uploading to Hetzner... ")
|
||||||
|
key, err := client.UploadSSHKey("orama-sandbox", pubStr)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Println("FAILED")
|
||||||
|
return SSHKeyConfig{}, fmt.Errorf("upload SSH key: %w", err)
|
||||||
|
}
|
||||||
|
fmt.Printf("OK (ID: %d)\n", key.ID)
|
||||||
|
|
||||||
|
return SSHKeyConfig{
|
||||||
|
HetznerID: key.ID,
|
||||||
|
PrivateKeyPath: "~/.orama/sandbox_key",
|
||||||
|
PublicKeyPath: "~/.orama/sandbox_key.pub",
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// verifyDNS checks if the sandbox domain resolves.
|
||||||
|
func verifyDNS(domain string) {
|
||||||
|
fmt.Printf(" Checking NS records for %s...\n", domain)
|
||||||
|
out, err := exec.Command("dig", "+short", "NS", domain, "@8.8.8.8").Output()
|
||||||
|
if err != nil {
|
||||||
|
fmt.Printf(" Warning: dig failed: %v\n", err)
|
||||||
|
fmt.Println(" DNS verification skipped. You can verify later with:")
|
||||||
|
fmt.Printf(" dig NS %s @8.8.8.8\n", domain)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
result := strings.TrimSpace(string(out))
|
||||||
|
if result == "" {
|
||||||
|
fmt.Println(" Warning: No NS records found yet.")
|
||||||
|
fmt.Println(" DNS propagation can take up to 48 hours.")
|
||||||
|
fmt.Println(" The sandbox will still work once DNS is configured.")
|
||||||
|
} else {
|
||||||
|
fmt.Printf(" NS records:\n")
|
||||||
|
for _, line := range strings.Split(result, "\n") {
|
||||||
|
fmt.Printf(" %s\n", line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
56
pkg/cli/sandbox/ssh_cmd.go
Normal file
56
pkg/cli/sandbox/ssh_cmd.go
Normal file
@ -0,0 +1,56 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SSHInto opens an interactive SSH session to a sandbox node.
|
||||||
|
func SSHInto(name string, nodeNum int) error {
|
||||||
|
cfg, err := LoadConfig()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
state, err := resolveSandbox(name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if nodeNum < 1 || nodeNum > len(state.Servers) {
|
||||||
|
return fmt.Errorf("node number must be between 1 and %d", len(state.Servers))
|
||||||
|
}
|
||||||
|
|
||||||
|
srv := state.Servers[nodeNum-1]
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
|
||||||
|
fmt.Printf("Connecting to %s (%s, %s)...\n", srv.Name, srv.IP, srv.Role)
|
||||||
|
|
||||||
|
// Find ssh binary
|
||||||
|
sshBin, err := findSSHBinary()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Replace current process with SSH
|
||||||
|
args := []string{
|
||||||
|
"ssh",
|
||||||
|
"-o", "StrictHostKeyChecking=accept-new",
|
||||||
|
"-i", sshKeyPath,
|
||||||
|
fmt.Sprintf("root@%s", srv.IP),
|
||||||
|
}
|
||||||
|
|
||||||
|
return syscall.Exec(sshBin, args, os.Environ())
|
||||||
|
}
|
||||||
|
|
||||||
|
// findSSHBinary locates the ssh binary in PATH.
|
||||||
|
func findSSHBinary() (string, error) {
|
||||||
|
paths := []string{"/usr/bin/ssh", "/usr/local/bin/ssh", "/opt/homebrew/bin/ssh"}
|
||||||
|
for _, p := range paths {
|
||||||
|
if _, err := os.Stat(p); err == nil {
|
||||||
|
return p, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("ssh binary not found")
|
||||||
|
}
|
||||||
211
pkg/cli/sandbox/state.go
Normal file
211
pkg/cli/sandbox/state.go
Normal file
@ -0,0 +1,211 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SandboxStatus represents the lifecycle state of a sandbox.
|
||||||
|
type SandboxStatus string
|
||||||
|
|
||||||
|
const (
|
||||||
|
StatusCreating SandboxStatus = "creating"
|
||||||
|
StatusRunning SandboxStatus = "running"
|
||||||
|
StatusDestroying SandboxStatus = "destroying"
|
||||||
|
StatusError SandboxStatus = "error"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SandboxState holds the full state of an active sandbox cluster.
|
||||||
|
type SandboxState struct {
|
||||||
|
Name string `yaml:"name"`
|
||||||
|
CreatedAt time.Time `yaml:"created_at"`
|
||||||
|
Domain string `yaml:"domain"`
|
||||||
|
Status SandboxStatus `yaml:"status"`
|
||||||
|
Servers []ServerState `yaml:"servers"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// ServerState holds the state of a single server in the sandbox.
|
||||||
|
type ServerState struct {
|
||||||
|
ID int64 `yaml:"id"` // Hetzner server ID
|
||||||
|
Name string `yaml:"name"` // e.g., sbx-feature-webrtc-1
|
||||||
|
IP string `yaml:"ip"` // Public IPv4
|
||||||
|
Role string `yaml:"role"` // "nameserver" or "node"
|
||||||
|
FloatingIP string `yaml:"floating_ip,omitempty"` // Only for nameserver nodes
|
||||||
|
WgIP string `yaml:"wg_ip,omitempty"` // WireGuard IP (populated after install)
|
||||||
|
}
|
||||||
|
|
||||||
|
// sandboxesDir returns ~/.orama/sandboxes/, creating it if needed.
|
||||||
|
func sandboxesDir() (string, error) {
|
||||||
|
dir, err := configDir()
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
sbxDir := filepath.Join(dir, "sandboxes")
|
||||||
|
if err := os.MkdirAll(sbxDir, 0700); err != nil {
|
||||||
|
return "", fmt.Errorf("create sandboxes directory: %w", err)
|
||||||
|
}
|
||||||
|
return sbxDir, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// statePath returns the path for a sandbox's state file.
|
||||||
|
func statePath(name string) (string, error) {
|
||||||
|
dir, err := sandboxesDir()
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return filepath.Join(dir, name+".yaml"), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SaveState persists the sandbox state to disk.
|
||||||
|
func SaveState(state *SandboxState) error {
|
||||||
|
path, err := statePath(state.Name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
data, err := yaml.Marshal(state)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("marshal state: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||||
|
return fmt.Errorf("write state: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LoadState reads a sandbox state from disk.
|
||||||
|
func LoadState(name string) (*SandboxState, error) {
|
||||||
|
path, err := statePath(name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
if os.IsNotExist(err) {
|
||||||
|
return nil, fmt.Errorf("sandbox %q not found", name)
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("read state: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var state SandboxState
|
||||||
|
if err := yaml.Unmarshal(data, &state); err != nil {
|
||||||
|
return nil, fmt.Errorf("parse state: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return &state, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DeleteState removes the sandbox state file.
|
||||||
|
func DeleteState(name string) error {
|
||||||
|
path, err := statePath(name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
|
||||||
|
return fmt.Errorf("delete state: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ListStates returns all sandbox states from disk.
|
||||||
|
func ListStates() ([]*SandboxState, error) {
|
||||||
|
dir, err := sandboxesDir()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
entries, err := os.ReadDir(dir)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("read sandboxes directory: %w", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var states []*SandboxState
|
||||||
|
for _, entry := range entries {
|
||||||
|
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".yaml") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
name := strings.TrimSuffix(entry.Name(), ".yaml")
|
||||||
|
state, err := LoadState(name)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintf(os.Stderr, "Warning: could not load sandbox %q: %v\n", name, err)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
states = append(states, state)
|
||||||
|
}
|
||||||
|
|
||||||
|
return states, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// FindActiveSandbox returns the first sandbox in running or creating state.
|
||||||
|
// Returns nil if no active sandbox exists.
|
||||||
|
func FindActiveSandbox() (*SandboxState, error) {
|
||||||
|
states, err := ListStates()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, s := range states {
|
||||||
|
if s.Status == StatusRunning || s.Status == StatusCreating {
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ToNodes converts sandbox servers to inspector.Node structs for SSH operations.
|
||||||
|
// Sets SSHKey to the provided key path on each node.
|
||||||
|
func (s *SandboxState) ToNodes(sshKeyPath string) []inspector.Node {
|
||||||
|
nodes := make([]inspector.Node, len(s.Servers))
|
||||||
|
for i, srv := range s.Servers {
|
||||||
|
nodes[i] = inspector.Node{
|
||||||
|
Environment: "sandbox",
|
||||||
|
User: "root",
|
||||||
|
Host: srv.IP,
|
||||||
|
Role: srv.Role,
|
||||||
|
SSHKey: sshKeyPath,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nodes
|
||||||
|
}
|
||||||
|
|
||||||
|
// NameserverNodes returns only the nameserver nodes.
|
||||||
|
func (s *SandboxState) NameserverNodes() []ServerState {
|
||||||
|
var ns []ServerState
|
||||||
|
for _, srv := range s.Servers {
|
||||||
|
if srv.Role == "nameserver" {
|
||||||
|
ns = append(ns, srv)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ns
|
||||||
|
}
|
||||||
|
|
||||||
|
// RegularNodes returns only the non-nameserver nodes.
|
||||||
|
func (s *SandboxState) RegularNodes() []ServerState {
|
||||||
|
var nodes []ServerState
|
||||||
|
for _, srv := range s.Servers {
|
||||||
|
if srv.Role == "node" {
|
||||||
|
nodes = append(nodes, srv)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nodes
|
||||||
|
}
|
||||||
|
|
||||||
|
// GenesisServer returns the first server (genesis node).
|
||||||
|
func (s *SandboxState) GenesisServer() ServerState {
|
||||||
|
if len(s.Servers) == 0 {
|
||||||
|
return ServerState{}
|
||||||
|
}
|
||||||
|
return s.Servers[0]
|
||||||
|
}
|
||||||
214
pkg/cli/sandbox/state_test.go
Normal file
214
pkg/cli/sandbox/state_test.go
Normal file
@ -0,0 +1,214 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSaveAndLoadState(t *testing.T) {
|
||||||
|
// Use temp dir for test
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
origHome := os.Getenv("HOME")
|
||||||
|
os.Setenv("HOME", tmpDir)
|
||||||
|
defer os.Setenv("HOME", origHome)
|
||||||
|
|
||||||
|
state := &SandboxState{
|
||||||
|
Name: "test-sandbox",
|
||||||
|
CreatedAt: time.Date(2026, 2, 25, 10, 0, 0, 0, time.UTC),
|
||||||
|
Domain: "test.example.com",
|
||||||
|
Status: StatusRunning,
|
||||||
|
Servers: []ServerState{
|
||||||
|
{ID: 1, Name: "sbx-test-1", IP: "1.1.1.1", Role: "nameserver", FloatingIP: "10.0.0.1", WgIP: "10.0.0.1"},
|
||||||
|
{ID: 2, Name: "sbx-test-2", IP: "2.2.2.2", Role: "nameserver", FloatingIP: "10.0.0.2", WgIP: "10.0.0.2"},
|
||||||
|
{ID: 3, Name: "sbx-test-3", IP: "3.3.3.3", Role: "node", WgIP: "10.0.0.3"},
|
||||||
|
{ID: 4, Name: "sbx-test-4", IP: "4.4.4.4", Role: "node", WgIP: "10.0.0.4"},
|
||||||
|
{ID: 5, Name: "sbx-test-5", IP: "5.5.5.5", Role: "node", WgIP: "10.0.0.5"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := SaveState(state); err != nil {
|
||||||
|
t.Fatalf("SaveState() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify file exists
|
||||||
|
expected := filepath.Join(tmpDir, ".orama", "sandboxes", "test-sandbox.yaml")
|
||||||
|
if _, err := os.Stat(expected); err != nil {
|
||||||
|
t.Fatalf("state file not created at %s: %v", expected, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load back
|
||||||
|
loaded, err := LoadState("test-sandbox")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("LoadState() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if loaded.Name != "test-sandbox" {
|
||||||
|
t.Errorf("name = %s, want test-sandbox", loaded.Name)
|
||||||
|
}
|
||||||
|
if loaded.Domain != "test.example.com" {
|
||||||
|
t.Errorf("domain = %s, want test.example.com", loaded.Domain)
|
||||||
|
}
|
||||||
|
if loaded.Status != StatusRunning {
|
||||||
|
t.Errorf("status = %s, want running", loaded.Status)
|
||||||
|
}
|
||||||
|
if len(loaded.Servers) != 5 {
|
||||||
|
t.Errorf("servers = %d, want 5", len(loaded.Servers))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLoadState_NotFound(t *testing.T) {
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
origHome := os.Getenv("HOME")
|
||||||
|
os.Setenv("HOME", tmpDir)
|
||||||
|
defer os.Setenv("HOME", origHome)
|
||||||
|
|
||||||
|
_, err := LoadState("nonexistent")
|
||||||
|
if err == nil {
|
||||||
|
t.Error("LoadState() expected error for nonexistent sandbox")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDeleteState(t *testing.T) {
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
origHome := os.Getenv("HOME")
|
||||||
|
os.Setenv("HOME", tmpDir)
|
||||||
|
defer os.Setenv("HOME", origHome)
|
||||||
|
|
||||||
|
state := &SandboxState{
|
||||||
|
Name: "to-delete",
|
||||||
|
Status: StatusRunning,
|
||||||
|
}
|
||||||
|
if err := SaveState(state); err != nil {
|
||||||
|
t.Fatalf("SaveState() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := DeleteState("to-delete"); err != nil {
|
||||||
|
t.Fatalf("DeleteState() error = %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_, err := LoadState("to-delete")
|
||||||
|
if err == nil {
|
||||||
|
t.Error("LoadState() should fail after DeleteState()")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestListStates(t *testing.T) {
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
origHome := os.Getenv("HOME")
|
||||||
|
os.Setenv("HOME", tmpDir)
|
||||||
|
defer os.Setenv("HOME", origHome)
|
||||||
|
|
||||||
|
// Create 2 sandboxes
|
||||||
|
for _, name := range []string{"sandbox-a", "sandbox-b"} {
|
||||||
|
if err := SaveState(&SandboxState{Name: name, Status: StatusRunning}); err != nil {
|
||||||
|
t.Fatalf("SaveState(%s) error = %v", name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
states, err := ListStates()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ListStates() error = %v", err)
|
||||||
|
}
|
||||||
|
if len(states) != 2 {
|
||||||
|
t.Errorf("ListStates() returned %d, want 2", len(states))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestFindActiveSandbox(t *testing.T) {
|
||||||
|
tmpDir := t.TempDir()
|
||||||
|
origHome := os.Getenv("HOME")
|
||||||
|
os.Setenv("HOME", tmpDir)
|
||||||
|
defer os.Setenv("HOME", origHome)
|
||||||
|
|
||||||
|
// No sandboxes
|
||||||
|
active, err := FindActiveSandbox()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("FindActiveSandbox() error = %v", err)
|
||||||
|
}
|
||||||
|
if active != nil {
|
||||||
|
t.Error("expected nil when no sandboxes exist")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Add one running sandbox
|
||||||
|
if err := SaveState(&SandboxState{Name: "active-one", Status: StatusRunning}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := SaveState(&SandboxState{Name: "errored-one", Status: StatusError}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
active, err = FindActiveSandbox()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("FindActiveSandbox() error = %v", err)
|
||||||
|
}
|
||||||
|
if active == nil || active.Name != "active-one" {
|
||||||
|
t.Errorf("FindActiveSandbox() = %v, want active-one", active)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestToNodes(t *testing.T) {
|
||||||
|
state := &SandboxState{
|
||||||
|
Servers: []ServerState{
|
||||||
|
{IP: "1.1.1.1", Role: "nameserver"},
|
||||||
|
{IP: "2.2.2.2", Role: "node"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
nodes := state.ToNodes("/tmp/key")
|
||||||
|
if len(nodes) != 2 {
|
||||||
|
t.Fatalf("ToNodes() returned %d nodes, want 2", len(nodes))
|
||||||
|
}
|
||||||
|
if nodes[0].Host != "1.1.1.1" {
|
||||||
|
t.Errorf("node[0].Host = %s, want 1.1.1.1", nodes[0].Host)
|
||||||
|
}
|
||||||
|
if nodes[0].User != "root" {
|
||||||
|
t.Errorf("node[0].User = %s, want root", nodes[0].User)
|
||||||
|
}
|
||||||
|
if nodes[0].SSHKey != "/tmp/key" {
|
||||||
|
t.Errorf("node[0].SSHKey = %s, want /tmp/key", nodes[0].SSHKey)
|
||||||
|
}
|
||||||
|
if nodes[0].Environment != "sandbox" {
|
||||||
|
t.Errorf("node[0].Environment = %s, want sandbox", nodes[0].Environment)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNameserverAndRegularNodes(t *testing.T) {
|
||||||
|
state := &SandboxState{
|
||||||
|
Servers: []ServerState{
|
||||||
|
{Role: "nameserver"},
|
||||||
|
{Role: "nameserver"},
|
||||||
|
{Role: "node"},
|
||||||
|
{Role: "node"},
|
||||||
|
{Role: "node"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
ns := state.NameserverNodes()
|
||||||
|
if len(ns) != 2 {
|
||||||
|
t.Errorf("NameserverNodes() = %d, want 2", len(ns))
|
||||||
|
}
|
||||||
|
|
||||||
|
regular := state.RegularNodes()
|
||||||
|
if len(regular) != 3 {
|
||||||
|
t.Errorf("RegularNodes() = %d, want 3", len(regular))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGenesisServer(t *testing.T) {
|
||||||
|
state := &SandboxState{
|
||||||
|
Servers: []ServerState{
|
||||||
|
{Name: "first"},
|
||||||
|
{Name: "second"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
if state.GenesisServer().Name != "first" {
|
||||||
|
t.Errorf("GenesisServer().Name = %s, want first", state.GenesisServer().Name)
|
||||||
|
}
|
||||||
|
|
||||||
|
empty := &SandboxState{}
|
||||||
|
if empty.GenesisServer().Name != "" {
|
||||||
|
t.Error("GenesisServer() on empty state should return zero value")
|
||||||
|
}
|
||||||
|
}
|
||||||
160
pkg/cli/sandbox/status.go
Normal file
160
pkg/cli/sandbox/status.go
Normal file
@ -0,0 +1,160 @@
|
|||||||
|
package sandbox
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||||
|
)
|
||||||
|
|
||||||
|
// List prints all sandbox clusters.
|
||||||
|
func List() error {
|
||||||
|
states, err := ListStates()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(states) == 0 {
|
||||||
|
fmt.Println("No sandboxes found.")
|
||||||
|
fmt.Println("Create one: orama sandbox create")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Printf("%-20s %-10s %-5s %-25s %s\n", "NAME", "STATUS", "NODES", "CREATED", "DOMAIN")
|
||||||
|
for _, s := range states {
|
||||||
|
fmt.Printf("%-20s %-10s %-5d %-25s %s\n",
|
||||||
|
s.Name, s.Status, len(s.Servers), s.CreatedAt.Format("2006-01-02 15:04"), s.Domain)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check for orphaned servers on Hetzner
|
||||||
|
cfg, err := LoadConfig()
|
||||||
|
if err != nil {
|
||||||
|
return nil // Config not set up, skip orphan check
|
||||||
|
}
|
||||||
|
|
||||||
|
client := NewHetznerClient(cfg.HetznerAPIToken)
|
||||||
|
hetznerServers, err := client.ListServersByLabel("orama-sandbox")
|
||||||
|
if err != nil {
|
||||||
|
return nil // API error, skip orphan check
|
||||||
|
}
|
||||||
|
|
||||||
|
// Build set of known server IDs
|
||||||
|
known := make(map[int64]bool)
|
||||||
|
for _, s := range states {
|
||||||
|
for _, srv := range s.Servers {
|
||||||
|
known[srv.ID] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
var orphans []string
|
||||||
|
for _, srv := range hetznerServers {
|
||||||
|
if !known[srv.ID] {
|
||||||
|
orphans = append(orphans, fmt.Sprintf("%s (ID: %d, IP: %s)", srv.Name, srv.ID, srv.PublicNet.IPv4.IP))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(orphans) > 0 {
|
||||||
|
fmt.Printf("\nWarning: %d orphaned server(s) on Hetzner (no state file):\n", len(orphans))
|
||||||
|
for _, o := range orphans {
|
||||||
|
fmt.Printf(" %s\n", o)
|
||||||
|
}
|
||||||
|
fmt.Println("Delete manually at https://console.hetzner.cloud")
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Status prints the health report for a sandbox cluster.
|
||||||
|
func Status(name string) error {
|
||||||
|
cfg, err := LoadConfig()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
state, err := resolveSandbox(name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||||
|
fmt.Printf("Sandbox: %s (status: %s)\n\n", state.Name, state.Status)
|
||||||
|
|
||||||
|
for _, srv := range state.Servers {
|
||||||
|
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||||
|
|
||||||
|
fmt.Printf("%s (%s) — %s\n", srv.Name, srv.IP, srv.Role)
|
||||||
|
|
||||||
|
// Get node report
|
||||||
|
out, err := runSSHOutput(node, "orama node report --json 2>/dev/null")
|
||||||
|
if err != nil {
|
||||||
|
fmt.Printf(" Status: UNREACHABLE (%v)\n", err)
|
||||||
|
fmt.Println()
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
printNodeReport(out)
|
||||||
|
fmt.Println()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Cluster summary
|
||||||
|
fmt.Println("Cluster Summary")
|
||||||
|
fmt.Println("---------------")
|
||||||
|
genesis := state.GenesisServer()
|
||||||
|
genesisNode := inspector.Node{User: "root", Host: genesis.IP, SSHKey: sshKeyPath}
|
||||||
|
|
||||||
|
out, err := runSSHOutput(genesisNode, "curl -sf http://localhost:5001/status 2>/dev/null")
|
||||||
|
if err != nil {
|
||||||
|
fmt.Println(" RQLite: UNREACHABLE")
|
||||||
|
} else {
|
||||||
|
var status map[string]interface{}
|
||||||
|
if err := json.Unmarshal([]byte(out), &status); err == nil {
|
||||||
|
if store, ok := status["store"].(map[string]interface{}); ok {
|
||||||
|
if raft, ok := store["raft"].(map[string]interface{}); ok {
|
||||||
|
fmt.Printf(" RQLite state: %v\n", raft["state"])
|
||||||
|
fmt.Printf(" Commit index: %v\n", raft["commit_index"])
|
||||||
|
if nodes, ok := raft["nodes"].([]interface{}); ok {
|
||||||
|
fmt.Printf(" Nodes: %d\n", len(nodes))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// printNodeReport parses and prints a node report JSON.
|
||||||
|
func printNodeReport(jsonStr string) {
|
||||||
|
var report map[string]interface{}
|
||||||
|
if err := json.Unmarshal([]byte(jsonStr), &report); err != nil {
|
||||||
|
fmt.Printf(" Report: (parse error)\n")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Print key fields
|
||||||
|
if services, ok := report["services"].(map[string]interface{}); ok {
|
||||||
|
var active, inactive []string
|
||||||
|
for name, info := range services {
|
||||||
|
if svc, ok := info.(map[string]interface{}); ok {
|
||||||
|
if state, ok := svc["active"].(bool); ok && state {
|
||||||
|
active = append(active, name)
|
||||||
|
} else {
|
||||||
|
inactive = append(inactive, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(active) > 0 {
|
||||||
|
fmt.Printf(" Active: %s\n", strings.Join(active, ", "))
|
||||||
|
}
|
||||||
|
if len(inactive) > 0 {
|
||||||
|
fmt.Printf(" Inactive: %s\n", strings.Join(inactive, ", "))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if rqlite, ok := report["rqlite"].(map[string]interface{}); ok {
|
||||||
|
if state, ok := rqlite["state"].(string); ok {
|
||||||
|
fmt.Printf(" RQLite: %s\n", state)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Loading…
x
Reference in New Issue
Block a user