mirror of
https://github.com/DeBrosOfficial/orama.git
synced 2026-03-17 04:33:00 +00:00
Added hatzhner support for clustering cli orama to spin up clusters
This commit is contained in:
parent
6898f47e2e
commit
fade8f89ed
208
docs/SANDBOX.md
Normal file
208
docs/SANDBOX.md
Normal file
@ -0,0 +1,208 @@
|
||||
# Sandbox: Ephemeral Hetzner Cloud Clusters
|
||||
|
||||
Spin up temporary 5-node Orama clusters on Hetzner Cloud for development and testing. Total cost: ~€0.04/hour.
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# One-time setup (API key, domain, floating IPs, SSH key)
|
||||
orama sandbox setup
|
||||
|
||||
# Create a cluster (~5 minutes)
|
||||
orama sandbox create --name my-feature
|
||||
|
||||
# Check health
|
||||
orama sandbox status
|
||||
|
||||
# SSH into a node
|
||||
orama sandbox ssh 1
|
||||
|
||||
# Deploy code changes
|
||||
orama sandbox rollout
|
||||
|
||||
# Tear it down
|
||||
orama sandbox destroy
|
||||
```
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### 1. Hetzner Cloud Account
|
||||
|
||||
Create a project at [console.hetzner.cloud](https://console.hetzner.cloud) and generate an API token with read/write permissions under **Security > API Tokens**.
|
||||
|
||||
### 2. Domain with Glue Records
|
||||
|
||||
You need a domain (or subdomain) that points to Hetzner Floating IPs. The `orama sandbox setup` wizard will guide you through this.
|
||||
|
||||
**Example:** Using `sbx.dbrs.space`
|
||||
|
||||
At your domain registrar:
|
||||
1. Create glue records (Personal DNS Servers):
|
||||
- `ns1.sbx.dbrs.space` → `<floating-ip-1>`
|
||||
- `ns2.sbx.dbrs.space` → `<floating-ip-2>`
|
||||
2. Set custom nameservers for `sbx.dbrs.space`:
|
||||
- `ns1.sbx.dbrs.space`
|
||||
- `ns2.sbx.dbrs.space`
|
||||
|
||||
DNS propagation can take up to 48 hours.
|
||||
|
||||
### 3. Binary Archive
|
||||
|
||||
Build the binary archive before creating a cluster:
|
||||
|
||||
```bash
|
||||
orama build
|
||||
```
|
||||
|
||||
This creates `/tmp/orama-<version>-linux-amd64.tar.gz` with all pre-compiled binaries.
|
||||
|
||||
## Setup
|
||||
|
||||
Run the interactive setup wizard:
|
||||
|
||||
```bash
|
||||
orama sandbox setup
|
||||
```
|
||||
|
||||
This will:
|
||||
1. Prompt for your Hetzner API token and validate it
|
||||
2. Ask for your sandbox domain
|
||||
3. Create or reuse 2 Hetzner Floating IPs (~$0.005/hr each)
|
||||
4. Create a firewall with sandbox rules
|
||||
5. Generate an SSH keypair at `~/.orama/sandbox_key`
|
||||
6. Upload the public key to Hetzner
|
||||
7. Display DNS configuration instructions
|
||||
|
||||
Config is saved to `~/.orama/sandbox.yaml`.
|
||||
|
||||
## Commands
|
||||
|
||||
### `orama sandbox create [--name <name>]`
|
||||
|
||||
Creates a new 5-node cluster. If `--name` is omitted, a random name is generated (e.g., "swift-falcon").
|
||||
|
||||
**Cluster layout:**
|
||||
- Nodes 1-2: Nameservers (CoreDNS + Caddy + all services)
|
||||
- Nodes 3-5: Regular nodes (all services except CoreDNS)
|
||||
|
||||
**Phases:**
|
||||
1. Provision 5 CX22 servers on Hetzner (parallel, ~90s)
|
||||
2. Assign floating IPs to nameserver nodes (~10s)
|
||||
3. Upload binary archive to all nodes (parallel, ~60s)
|
||||
4. Install genesis node + generate invite tokens (~120s)
|
||||
5. Join remaining 4 nodes (serial with health checks, ~180s)
|
||||
6. Verify cluster health (~15s)
|
||||
|
||||
**One sandbox at a time.** Since the floating IPs are shared, only one sandbox can own the nameservers. Destroy the active sandbox before creating a new one.
|
||||
|
||||
### `orama sandbox destroy [--name <name>] [--force]`
|
||||
|
||||
Tears down a cluster:
|
||||
1. Unassigns floating IPs
|
||||
2. Deletes all 5 servers (parallel)
|
||||
3. Removes state file
|
||||
|
||||
Use `--force` to skip confirmation.
|
||||
|
||||
### `orama sandbox list`
|
||||
|
||||
Lists all sandboxes with their status. Also checks Hetzner for orphaned servers that don't have a corresponding state file.
|
||||
|
||||
### `orama sandbox status [--name <name>]`
|
||||
|
||||
Shows per-node health including:
|
||||
- Service status (active/inactive)
|
||||
- RQLite role (Leader/Follower)
|
||||
- Cluster summary (commit index, voter count)
|
||||
|
||||
### `orama sandbox rollout [--name <name>]`
|
||||
|
||||
Deploys code changes:
|
||||
1. Uses the latest binary archive from `/tmp/` (run `orama build` first)
|
||||
2. Pushes to all nodes
|
||||
3. Rolling upgrade: followers first, leader last, 15s between nodes
|
||||
|
||||
### `orama sandbox ssh <node-number>`
|
||||
|
||||
Opens an interactive SSH session to a sandbox node (1-5).
|
||||
|
||||
```bash
|
||||
orama sandbox ssh 1 # SSH into node 1 (genesis/ns1)
|
||||
orama sandbox ssh 3 # SSH into node 3 (regular node)
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
### Floating IPs
|
||||
|
||||
Hetzner Floating IPs are persistent IPv4 addresses that can be reassigned between servers. They solve the DNS chicken-and-egg problem:
|
||||
|
||||
- Glue records at the registrar point to 2 Floating IPs (configured once)
|
||||
- Each new sandbox assigns the Floating IPs to its nameserver nodes
|
||||
- DNS works instantly — no propagation delay between clusters
|
||||
|
||||
### SSH Authentication
|
||||
|
||||
Sandbox uses a standalone ed25519 keypair at `~/.orama/sandbox_key`, separate from the production wallet-derived keys. The public key is uploaded to Hetzner during setup and injected into every server at creation time.
|
||||
|
||||
### Server Naming
|
||||
|
||||
Servers: `sbx-<name>-<N>` (e.g., `sbx-swift-falcon-1` through `sbx-swift-falcon-5`)
|
||||
|
||||
### State Files
|
||||
|
||||
Sandbox state is stored at `~/.orama/sandboxes/<name>.yaml`. This tracks server IDs, IPs, roles, and cluster status.
|
||||
|
||||
## Cost
|
||||
|
||||
| Resource | Cost | Qty | Total |
|
||||
|----------|------|-----|-------|
|
||||
| CX22 (2 vCPU, 4GB) | €0.006/hr | 5 | €0.03/hr |
|
||||
| Floating IPv4 | €0.005/hr | 2 | €0.01/hr |
|
||||
| **Total** | | | **~€0.04/hr** |
|
||||
|
||||
Servers are billed per hour. Floating IPs are billed as long as they exist (even unassigned). Destroy the sandbox when not in use to save on server costs.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "sandbox not configured"
|
||||
|
||||
Run `orama sandbox setup` first.
|
||||
|
||||
### "no binary archive found"
|
||||
|
||||
Run `orama build` to create the binary archive.
|
||||
|
||||
### "sandbox X is already active"
|
||||
|
||||
Only one sandbox can be active at a time. Destroy it first:
|
||||
```bash
|
||||
orama sandbox destroy --name <name>
|
||||
```
|
||||
|
||||
### Server creation fails
|
||||
|
||||
Check:
|
||||
- Hetzner API token is valid and has read/write permissions
|
||||
- You haven't hit Hetzner's server limit (default: 10 per project)
|
||||
- The selected location has CX22 capacity
|
||||
|
||||
### Genesis install fails
|
||||
|
||||
SSH into the node to debug:
|
||||
```bash
|
||||
orama sandbox ssh 1
|
||||
journalctl -u orama-node -f
|
||||
```
|
||||
|
||||
The sandbox will be left in "error" state. You can destroy and recreate it.
|
||||
|
||||
### DNS not resolving
|
||||
|
||||
1. Verify glue records are configured at your registrar
|
||||
2. Check propagation: `dig NS sbx.dbrs.space @8.8.8.8`
|
||||
3. Propagation can take 24-48 hours for new domains
|
||||
|
||||
### Orphaned servers
|
||||
|
||||
If `orama sandbox list` shows orphaned servers, delete them manually at [console.hetzner.cloud](https://console.hetzner.cloud). Sandbox servers are labeled `orama-sandbox=<name>` for easy identification.
|
||||
121
pkg/cli/cmd/sandboxcmd/sandbox.go
Normal file
121
pkg/cli/cmd/sandboxcmd/sandbox.go
Normal file
@ -0,0 +1,121 @@
|
||||
package sandboxcmd
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
|
||||
"github.com/DeBrosOfficial/network/pkg/cli/sandbox"
|
||||
"github.com/spf13/cobra"
|
||||
)
|
||||
|
||||
// Cmd is the root command for sandbox operations.
|
||||
var Cmd = &cobra.Command{
|
||||
Use: "sandbox",
|
||||
Short: "Manage ephemeral Hetzner Cloud clusters for testing",
|
||||
Long: `Spin up temporary 5-node Orama clusters on Hetzner Cloud for development and testing.
|
||||
|
||||
Setup (one-time):
|
||||
orama sandbox setup
|
||||
|
||||
Usage:
|
||||
orama sandbox create [--name <name>] Create a new 5-node cluster
|
||||
orama sandbox destroy [--name <name>] Tear down a cluster
|
||||
orama sandbox list List active sandboxes
|
||||
orama sandbox status [--name <name>] Show cluster health
|
||||
orama sandbox rollout [--name <name>] Build + push + rolling upgrade
|
||||
orama sandbox ssh <node-number> SSH into a sandbox node (1-5)`,
|
||||
}
|
||||
|
||||
var setupCmd = &cobra.Command{
|
||||
Use: "setup",
|
||||
Short: "Interactive setup: Hetzner API key, domain, floating IPs, SSH key",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return sandbox.Setup()
|
||||
},
|
||||
}
|
||||
|
||||
var createCmd = &cobra.Command{
|
||||
Use: "create",
|
||||
Short: "Create a new 5-node sandbox cluster (~5 min)",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
name, _ := cmd.Flags().GetString("name")
|
||||
return sandbox.Create(name)
|
||||
},
|
||||
}
|
||||
|
||||
var destroyCmd = &cobra.Command{
|
||||
Use: "destroy",
|
||||
Short: "Destroy a sandbox cluster and release resources",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
name, _ := cmd.Flags().GetString("name")
|
||||
force, _ := cmd.Flags().GetBool("force")
|
||||
return sandbox.Destroy(name, force)
|
||||
},
|
||||
}
|
||||
|
||||
var listCmd = &cobra.Command{
|
||||
Use: "list",
|
||||
Short: "List active sandbox clusters",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
return sandbox.List()
|
||||
},
|
||||
}
|
||||
|
||||
var statusCmd = &cobra.Command{
|
||||
Use: "status",
|
||||
Short: "Show cluster health report",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
name, _ := cmd.Flags().GetString("name")
|
||||
return sandbox.Status(name)
|
||||
},
|
||||
}
|
||||
|
||||
var rolloutCmd = &cobra.Command{
|
||||
Use: "rollout",
|
||||
Short: "Build + push + rolling upgrade to sandbox cluster",
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
name, _ := cmd.Flags().GetString("name")
|
||||
return sandbox.Rollout(name)
|
||||
},
|
||||
}
|
||||
|
||||
var sshCmd = &cobra.Command{
|
||||
Use: "ssh <node-number>",
|
||||
Short: "SSH into a sandbox node (1-5)",
|
||||
Args: cobra.ExactArgs(1),
|
||||
RunE: func(cmd *cobra.Command, args []string) error {
|
||||
name, _ := cmd.Flags().GetString("name")
|
||||
var nodeNum int
|
||||
if _, err := fmt.Sscanf(args[0], "%d", &nodeNum); err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Invalid node number: %s (expected 1-5)\n", args[0])
|
||||
os.Exit(1)
|
||||
}
|
||||
return sandbox.SSHInto(name, nodeNum)
|
||||
},
|
||||
}
|
||||
|
||||
func init() {
|
||||
// create flags
|
||||
createCmd.Flags().String("name", "", "Sandbox name (random if not specified)")
|
||||
|
||||
// destroy flags
|
||||
destroyCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||
destroyCmd.Flags().Bool("force", false, "Skip confirmation")
|
||||
|
||||
// status flags
|
||||
statusCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||
|
||||
// rollout flags
|
||||
rolloutCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||
|
||||
// ssh flags
|
||||
sshCmd.Flags().String("name", "", "Sandbox name (uses active if not specified)")
|
||||
|
||||
Cmd.AddCommand(setupCmd)
|
||||
Cmd.AddCommand(createCmd)
|
||||
Cmd.AddCommand(destroyCmd)
|
||||
Cmd.AddCommand(listCmd)
|
||||
Cmd.AddCommand(statusCmd)
|
||||
Cmd.AddCommand(rolloutCmd)
|
||||
Cmd.AddCommand(sshCmd)
|
||||
}
|
||||
153
pkg/cli/sandbox/config.go
Normal file
153
pkg/cli/sandbox/config.go
Normal file
@ -0,0 +1,153 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// Config holds sandbox configuration, stored at ~/.orama/sandbox.yaml.
|
||||
type Config struct {
|
||||
HetznerAPIToken string `yaml:"hetzner_api_token"`
|
||||
Domain string `yaml:"domain"`
|
||||
Location string `yaml:"location"` // Hetzner datacenter (default: fsn1)
|
||||
ServerType string `yaml:"server_type"` // Hetzner server type (default: cx22)
|
||||
FloatingIPs []FloatIP `yaml:"floating_ips"`
|
||||
SSHKey SSHKeyConfig `yaml:"ssh_key"`
|
||||
FirewallID int64 `yaml:"firewall_id,omitempty"` // Hetzner firewall resource ID
|
||||
}
|
||||
|
||||
// FloatIP holds a Hetzner floating IP reference.
|
||||
type FloatIP struct {
|
||||
ID int64 `yaml:"id"`
|
||||
IP string `yaml:"ip"`
|
||||
}
|
||||
|
||||
// SSHKeyConfig holds SSH key paths and the Hetzner resource ID.
|
||||
type SSHKeyConfig struct {
|
||||
HetznerID int64 `yaml:"hetzner_id"`
|
||||
PrivateKeyPath string `yaml:"private_key_path"`
|
||||
PublicKeyPath string `yaml:"public_key_path"`
|
||||
}
|
||||
|
||||
// configDir returns ~/.orama/, creating it if needed.
|
||||
func configDir() (string, error) {
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("get home directory: %w", err)
|
||||
}
|
||||
dir := filepath.Join(home, ".orama")
|
||||
if err := os.MkdirAll(dir, 0700); err != nil {
|
||||
return "", fmt.Errorf("create config directory: %w", err)
|
||||
}
|
||||
return dir, nil
|
||||
}
|
||||
|
||||
// configPath returns the full path to ~/.orama/sandbox.yaml.
|
||||
func configPath() (string, error) {
|
||||
dir, err := configDir()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return filepath.Join(dir, "sandbox.yaml"), nil
|
||||
}
|
||||
|
||||
// LoadConfig reads the sandbox config from ~/.orama/sandbox.yaml.
|
||||
// Returns an error if the file doesn't exist (user must run setup first).
|
||||
func LoadConfig() (*Config, error) {
|
||||
path, err := configPath()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, fmt.Errorf("sandbox not configured, run: orama sandbox setup")
|
||||
}
|
||||
return nil, fmt.Errorf("read config: %w", err)
|
||||
}
|
||||
|
||||
var cfg Config
|
||||
if err := yaml.Unmarshal(data, &cfg); err != nil {
|
||||
return nil, fmt.Errorf("parse config %s: %w", path, err)
|
||||
}
|
||||
|
||||
if err := cfg.validate(); err != nil {
|
||||
return nil, fmt.Errorf("invalid config: %w", err)
|
||||
}
|
||||
|
||||
cfg.Defaults()
|
||||
|
||||
return &cfg, nil
|
||||
}
|
||||
|
||||
// SaveConfig writes the sandbox config to ~/.orama/sandbox.yaml.
|
||||
func SaveConfig(cfg *Config) error {
|
||||
path, err := configPath()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
data, err := yaml.Marshal(cfg)
|
||||
if err != nil {
|
||||
return fmt.Errorf("marshal config: %w", err)
|
||||
}
|
||||
|
||||
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||
return fmt.Errorf("write config: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// validate checks that required fields are present.
|
||||
func (c *Config) validate() error {
|
||||
if c.HetznerAPIToken == "" {
|
||||
return fmt.Errorf("hetzner_api_token is required")
|
||||
}
|
||||
if c.Domain == "" {
|
||||
return fmt.Errorf("domain is required")
|
||||
}
|
||||
if len(c.FloatingIPs) < 2 {
|
||||
return fmt.Errorf("2 floating IPs required, got %d", len(c.FloatingIPs))
|
||||
}
|
||||
if c.SSHKey.PrivateKeyPath == "" {
|
||||
return fmt.Errorf("ssh_key.private_key_path is required")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Defaults fills in default values for optional fields.
|
||||
func (c *Config) Defaults() {
|
||||
if c.Location == "" {
|
||||
c.Location = "fsn1"
|
||||
}
|
||||
if c.ServerType == "" {
|
||||
c.ServerType = "cx22"
|
||||
}
|
||||
}
|
||||
|
||||
// ExpandedPrivateKeyPath returns the absolute path to the SSH private key.
|
||||
func (c *Config) ExpandedPrivateKeyPath() string {
|
||||
return expandHome(c.SSHKey.PrivateKeyPath)
|
||||
}
|
||||
|
||||
// ExpandedPublicKeyPath returns the absolute path to the SSH public key.
|
||||
func (c *Config) ExpandedPublicKeyPath() string {
|
||||
return expandHome(c.SSHKey.PublicKeyPath)
|
||||
}
|
||||
|
||||
// expandHome replaces a leading ~ with the user's home directory.
|
||||
func expandHome(path string) string {
|
||||
if len(path) < 2 || path[:2] != "~/" {
|
||||
return path
|
||||
}
|
||||
home, err := os.UserHomeDir()
|
||||
if err != nil {
|
||||
return path
|
||||
}
|
||||
return filepath.Join(home, path[2:])
|
||||
}
|
||||
554
pkg/cli/sandbox/create.go
Normal file
554
pkg/cli/sandbox/create.go
Normal file
@ -0,0 +1,554 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/DeBrosOfficial/network/pkg/cli/remotessh"
|
||||
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||
)
|
||||
|
||||
// Create orchestrates the creation of a new sandbox cluster.
|
||||
func Create(name string) error {
|
||||
cfg, err := LoadConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Check for existing active sandbox
|
||||
active, err := FindActiveSandbox()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if active != nil {
|
||||
return fmt.Errorf("sandbox %q is already active (status: %s)\nDestroy it first: orama sandbox destroy --name %s",
|
||||
active.Name, active.Status, active.Name)
|
||||
}
|
||||
|
||||
// Generate name if not provided
|
||||
if name == "" {
|
||||
name = GenerateName()
|
||||
}
|
||||
|
||||
fmt.Printf("Creating sandbox %q (%s, %d nodes)\n\n", name, cfg.Domain, 5)
|
||||
|
||||
client := NewHetznerClient(cfg.HetznerAPIToken)
|
||||
|
||||
state := &SandboxState{
|
||||
Name: name,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
Domain: cfg.Domain,
|
||||
Status: StatusCreating,
|
||||
}
|
||||
|
||||
// Phase 1: Provision servers
|
||||
fmt.Println("Phase 1: Provisioning servers...")
|
||||
if err := phase1ProvisionServers(client, cfg, state); err != nil {
|
||||
cleanupFailedCreate(client, state)
|
||||
return fmt.Errorf("provision servers: %w", err)
|
||||
}
|
||||
SaveState(state)
|
||||
|
||||
// Phase 2: Assign floating IPs
|
||||
fmt.Println("\nPhase 2: Assigning floating IPs...")
|
||||
if err := phase2AssignFloatingIPs(client, cfg, state); err != nil {
|
||||
return fmt.Errorf("assign floating IPs: %w", err)
|
||||
}
|
||||
SaveState(state)
|
||||
|
||||
// Phase 3: Upload binary archive
|
||||
fmt.Println("\nPhase 3: Uploading binary archive...")
|
||||
if err := phase3UploadArchive(cfg, state); err != nil {
|
||||
return fmt.Errorf("upload archive: %w", err)
|
||||
}
|
||||
|
||||
// Phase 4: Install genesis node
|
||||
fmt.Println("\nPhase 4: Installing genesis node...")
|
||||
tokens, err := phase4InstallGenesis(cfg, state)
|
||||
if err != nil {
|
||||
state.Status = StatusError
|
||||
SaveState(state)
|
||||
return fmt.Errorf("install genesis: %w", err)
|
||||
}
|
||||
|
||||
// Phase 5: Join remaining nodes
|
||||
fmt.Println("\nPhase 5: Joining remaining nodes...")
|
||||
if err := phase5JoinNodes(cfg, state, tokens); err != nil {
|
||||
state.Status = StatusError
|
||||
SaveState(state)
|
||||
return fmt.Errorf("join nodes: %w", err)
|
||||
}
|
||||
|
||||
// Phase 6: Verify cluster
|
||||
fmt.Println("\nPhase 6: Verifying cluster...")
|
||||
phase6Verify(cfg, state)
|
||||
|
||||
state.Status = StatusRunning
|
||||
SaveState(state)
|
||||
|
||||
printCreateSummary(cfg, state)
|
||||
return nil
|
||||
}
|
||||
|
||||
// phase1ProvisionServers creates 5 Hetzner servers in parallel.
|
||||
func phase1ProvisionServers(client *HetznerClient, cfg *Config, state *SandboxState) error {
|
||||
type serverResult struct {
|
||||
index int
|
||||
server *HetznerServer
|
||||
err error
|
||||
}
|
||||
|
||||
results := make(chan serverResult, 5)
|
||||
|
||||
for i := 0; i < 5; i++ {
|
||||
go func(idx int) {
|
||||
role := "node"
|
||||
if idx < 2 {
|
||||
role = "nameserver"
|
||||
}
|
||||
|
||||
serverName := fmt.Sprintf("sbx-%s-%d", state.Name, idx+1)
|
||||
labels := map[string]string{
|
||||
"orama-sandbox": state.Name,
|
||||
"orama-sandbox-role": role,
|
||||
}
|
||||
|
||||
req := CreateServerRequest{
|
||||
Name: serverName,
|
||||
ServerType: cfg.ServerType,
|
||||
Image: "ubuntu-24.04",
|
||||
Location: cfg.Location,
|
||||
SSHKeys: []int64{cfg.SSHKey.HetznerID},
|
||||
Labels: labels,
|
||||
}
|
||||
if cfg.FirewallID > 0 {
|
||||
req.Firewalls = []struct {
|
||||
Firewall int64 `json:"firewall"`
|
||||
}{{Firewall: cfg.FirewallID}}
|
||||
}
|
||||
|
||||
srv, err := client.CreateServer(req)
|
||||
results <- serverResult{index: idx, server: srv, err: err}
|
||||
}(i)
|
||||
}
|
||||
|
||||
servers := make([]ServerState, 5)
|
||||
for i := 0; i < 5; i++ {
|
||||
r := <-results
|
||||
if r.err != nil {
|
||||
return fmt.Errorf("server %d: %w", r.index+1, r.err)
|
||||
}
|
||||
fmt.Printf(" Created %s (ID: %d, initializing...)\n", r.server.Name, r.server.ID)
|
||||
role := "node"
|
||||
if r.index < 2 {
|
||||
role = "nameserver"
|
||||
}
|
||||
servers[r.index] = ServerState{
|
||||
ID: r.server.ID,
|
||||
Name: r.server.Name,
|
||||
Role: role,
|
||||
}
|
||||
}
|
||||
|
||||
// Wait for all servers to reach "running"
|
||||
fmt.Print(" Waiting for servers to boot...")
|
||||
for i := range servers {
|
||||
srv, err := client.WaitForServer(servers[i].ID, 3*time.Minute)
|
||||
if err != nil {
|
||||
return fmt.Errorf("wait for %s: %w", servers[i].Name, err)
|
||||
}
|
||||
servers[i].IP = srv.PublicNet.IPv4.IP
|
||||
fmt.Print(".")
|
||||
}
|
||||
fmt.Println(" OK")
|
||||
|
||||
// Assign floating IPs to nameserver entries
|
||||
if len(cfg.FloatingIPs) >= 2 {
|
||||
servers[0].FloatingIP = cfg.FloatingIPs[0].IP
|
||||
servers[1].FloatingIP = cfg.FloatingIPs[1].IP
|
||||
}
|
||||
|
||||
state.Servers = servers
|
||||
|
||||
for _, srv := range servers {
|
||||
fmt.Printf(" %s: %s (%s)\n", srv.Name, srv.IP, srv.Role)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// phase2AssignFloatingIPs assigns floating IPs and configures loopback.
|
||||
func phase2AssignFloatingIPs(client *HetznerClient, cfg *Config, state *SandboxState) error {
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
|
||||
for i := 0; i < 2 && i < len(cfg.FloatingIPs) && i < len(state.Servers); i++ {
|
||||
fip := cfg.FloatingIPs[i]
|
||||
srv := state.Servers[i]
|
||||
|
||||
// Unassign if currently assigned elsewhere (ignore "not assigned" errors)
|
||||
fmt.Printf(" Assigning %s to %s...\n", fip.IP, srv.Name)
|
||||
if err := client.UnassignFloatingIP(fip.ID); err != nil {
|
||||
// Log but continue — may fail if not currently assigned, which is fine
|
||||
fmt.Printf(" Note: unassign %s: %v (continuing)\n", fip.IP, err)
|
||||
}
|
||||
|
||||
if err := client.AssignFloatingIP(fip.ID, srv.ID); err != nil {
|
||||
return fmt.Errorf("assign %s to %s: %w", fip.IP, srv.Name, err)
|
||||
}
|
||||
|
||||
// Configure floating IP on the server's loopback interface
|
||||
// Hetzner floating IPs require this: ip addr add <floating_ip>/32 dev lo
|
||||
node := inspector.Node{
|
||||
User: "root",
|
||||
Host: srv.IP,
|
||||
SSHKey: sshKeyPath,
|
||||
}
|
||||
|
||||
// Wait for SSH to be ready on freshly booted servers
|
||||
if err := waitForSSH(node, 2*time.Minute); err != nil {
|
||||
return fmt.Errorf("SSH not ready on %s: %w", srv.Name, err)
|
||||
}
|
||||
|
||||
cmd := fmt.Sprintf("ip addr add %s/32 dev lo 2>/dev/null || true", fip.IP)
|
||||
if err := remotessh.RunSSHStreaming(node, cmd); err != nil {
|
||||
return fmt.Errorf("configure loopback on %s: %w", srv.Name, err)
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// waitForSSH polls until SSH is responsive on the node.
|
||||
func waitForSSH(node inspector.Node, timeout time.Duration) error {
|
||||
deadline := time.Now().Add(timeout)
|
||||
for time.Now().Before(deadline) {
|
||||
_, err := runSSHOutput(node, "echo ok")
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
time.Sleep(3 * time.Second)
|
||||
}
|
||||
return fmt.Errorf("timeout after %s", timeout)
|
||||
}
|
||||
|
||||
// phase3UploadArchive builds (if needed) and uploads the binary archive to all nodes.
|
||||
func phase3UploadArchive(cfg *Config, state *SandboxState) error {
|
||||
// Find existing archive
|
||||
archivePath := findNewestArchive()
|
||||
if archivePath == "" {
|
||||
fmt.Println(" No binary archive found, run `orama build` first")
|
||||
return fmt.Errorf("no binary archive found in /tmp/ (run `orama build` first)")
|
||||
}
|
||||
|
||||
info, _ := os.Stat(archivePath)
|
||||
fmt.Printf(" Archive: %s (%s)\n", filepath.Base(archivePath), formatBytes(info.Size()))
|
||||
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
remotePath := "/tmp/" + filepath.Base(archivePath)
|
||||
|
||||
// Upload to all 5 nodes in parallel
|
||||
var wg sync.WaitGroup
|
||||
errs := make([]error, len(state.Servers))
|
||||
|
||||
for i, srv := range state.Servers {
|
||||
wg.Add(1)
|
||||
go func(idx int, srv ServerState) {
|
||||
defer wg.Done()
|
||||
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||
|
||||
if err := remotessh.UploadFile(node, archivePath, remotePath); err != nil {
|
||||
errs[idx] = fmt.Errorf("upload to %s: %w", srv.Name, err)
|
||||
return
|
||||
}
|
||||
|
||||
// Extract + install CLI
|
||||
extractCmd := fmt.Sprintf("mkdir -p /opt/orama && tar xzf %s -C /opt/orama && rm -f %s && cp /opt/orama/bin/orama /usr/local/bin/orama && chmod +x /usr/local/bin/orama",
|
||||
remotePath, remotePath)
|
||||
if err := remotessh.RunSSHStreaming(node, extractCmd); err != nil {
|
||||
errs[idx] = fmt.Errorf("extract on %s: %w", srv.Name, err)
|
||||
return
|
||||
}
|
||||
fmt.Printf(" Uploaded to %s\n", srv.Name)
|
||||
}(i, srv)
|
||||
}
|
||||
wg.Wait()
|
||||
|
||||
for _, err := range errs {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// phase4InstallGenesis installs the genesis node and generates invite tokens.
|
||||
func phase4InstallGenesis(cfg *Config, state *SandboxState) ([]string, error) {
|
||||
genesis := state.GenesisServer()
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
node := inspector.Node{User: "root", Host: genesis.IP, SSHKey: sshKeyPath}
|
||||
|
||||
// Install genesis
|
||||
installCmd := fmt.Sprintf("orama node install --vps-ip %s --domain %s --base-domain %s --nameserver --skip-checks",
|
||||
genesis.IP, cfg.Domain, cfg.Domain)
|
||||
fmt.Printf(" Installing on %s (%s)...\n", genesis.Name, genesis.IP)
|
||||
if err := remotessh.RunSSHStreaming(node, installCmd); err != nil {
|
||||
return nil, fmt.Errorf("install genesis: %w", err)
|
||||
}
|
||||
|
||||
// Wait for RQLite leader
|
||||
fmt.Print(" Waiting for RQLite leader...")
|
||||
if err := waitForRQLiteHealth(node, 3*time.Minute); err != nil {
|
||||
return nil, fmt.Errorf("genesis health: %w", err)
|
||||
}
|
||||
fmt.Println(" OK")
|
||||
|
||||
// Generate invite tokens (one per remaining node)
|
||||
fmt.Print(" Generating invite tokens...")
|
||||
remaining := len(state.Servers) - 1
|
||||
tokens := make([]string, remaining)
|
||||
|
||||
for i := 0; i < remaining; i++ {
|
||||
token, err := generateInviteToken(node)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("generate invite token %d: %w", i+1, err)
|
||||
}
|
||||
tokens[i] = token
|
||||
fmt.Print(".")
|
||||
}
|
||||
fmt.Println(" OK")
|
||||
|
||||
return tokens, nil
|
||||
}
|
||||
|
||||
// phase5JoinNodes joins the remaining 4 nodes to the cluster (serial).
|
||||
func phase5JoinNodes(cfg *Config, state *SandboxState, tokens []string) error {
|
||||
genesisIP := state.GenesisServer().IP
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
|
||||
for i := 1; i < len(state.Servers); i++ {
|
||||
srv := state.Servers[i]
|
||||
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||
token := tokens[i-1]
|
||||
|
||||
var installCmd string
|
||||
if srv.Role == "nameserver" {
|
||||
installCmd = fmt.Sprintf("orama node install --join http://%s --token %s --vps-ip %s --domain %s --base-domain %s --nameserver --skip-checks",
|
||||
genesisIP, token, srv.IP, cfg.Domain, cfg.Domain)
|
||||
} else {
|
||||
installCmd = fmt.Sprintf("orama node install --join http://%s --token %s --vps-ip %s --base-domain %s --skip-checks",
|
||||
genesisIP, token, srv.IP, cfg.Domain)
|
||||
}
|
||||
|
||||
fmt.Printf(" [%d/%d] Joining %s (%s, %s)...\n", i, len(state.Servers)-1, srv.Name, srv.IP, srv.Role)
|
||||
if err := remotessh.RunSSHStreaming(node, installCmd); err != nil {
|
||||
return fmt.Errorf("join %s: %w", srv.Name, err)
|
||||
}
|
||||
|
||||
// Wait for node health before proceeding
|
||||
fmt.Printf(" Waiting for %s health...", srv.Name)
|
||||
if err := waitForRQLiteHealth(node, 3*time.Minute); err != nil {
|
||||
fmt.Printf(" WARN: %v\n", err)
|
||||
} else {
|
||||
fmt.Println(" OK")
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// phase6Verify runs a basic cluster health check.
|
||||
func phase6Verify(cfg *Config, state *SandboxState) {
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
genesis := state.GenesisServer()
|
||||
node := inspector.Node{User: "root", Host: genesis.IP, SSHKey: sshKeyPath}
|
||||
|
||||
// Check RQLite cluster
|
||||
out, err := runSSHOutput(node, "curl -s http://localhost:5001/status | grep -o '\"state\":\"[^\"]*\"' | head -1")
|
||||
if err == nil {
|
||||
fmt.Printf(" RQLite: %s\n", strings.TrimSpace(out))
|
||||
}
|
||||
|
||||
// Check DNS (if floating IPs configured, only with safe domain names)
|
||||
if len(cfg.FloatingIPs) > 0 && isSafeDNSName(cfg.Domain) {
|
||||
out, err = runSSHOutput(node, fmt.Sprintf("dig +short @%s test.%s 2>/dev/null || echo 'DNS not responding'",
|
||||
cfg.FloatingIPs[0].IP, cfg.Domain))
|
||||
if err == nil {
|
||||
fmt.Printf(" DNS: %s\n", strings.TrimSpace(out))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// waitForRQLiteHealth polls RQLite until it reports Leader or Follower state.
|
||||
func waitForRQLiteHealth(node inspector.Node, timeout time.Duration) error {
|
||||
deadline := time.Now().Add(timeout)
|
||||
for time.Now().Before(deadline) {
|
||||
out, err := runSSHOutput(node, "curl -sf http://localhost:5001/status 2>/dev/null | grep -o '\"state\":\"[^\"]*\"'")
|
||||
if err == nil {
|
||||
result := strings.TrimSpace(out)
|
||||
if strings.Contains(result, "Leader") || strings.Contains(result, "Follower") {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
time.Sleep(5 * time.Second)
|
||||
}
|
||||
return fmt.Errorf("timeout waiting for RQLite health after %s", timeout)
|
||||
}
|
||||
|
||||
// generateInviteToken runs `orama node invite` on the node and parses the token.
|
||||
func generateInviteToken(node inspector.Node) (string, error) {
|
||||
out, err := runSSHOutput(node, "orama node invite --expiry 1h 2>&1")
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("invite command failed: %w", err)
|
||||
}
|
||||
|
||||
// Parse token from output — the invite command outputs:
|
||||
// "sudo orama install --join https://... --token <64-char-hex> --vps-ip ..."
|
||||
// Look for the --token flag value first
|
||||
fields := strings.Fields(out)
|
||||
for i, field := range fields {
|
||||
if field == "--token" && i+1 < len(fields) {
|
||||
candidate := fields[i+1]
|
||||
if len(candidate) == 64 && isHex(candidate) {
|
||||
return candidate, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: look for any standalone 64-char hex string
|
||||
for _, word := range fields {
|
||||
if len(word) == 64 && isHex(word) {
|
||||
return word, nil
|
||||
}
|
||||
}
|
||||
|
||||
return "", fmt.Errorf("could not parse token from invite output:\n%s", out)
|
||||
}
|
||||
|
||||
// isSafeDNSName returns true if the string is safe to use in shell commands.
|
||||
func isSafeDNSName(s string) bool {
|
||||
for _, c := range s {
|
||||
if !((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '.' || c == '-') {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return len(s) > 0
|
||||
}
|
||||
|
||||
// isHex returns true if s contains only hex characters.
|
||||
func isHex(s string) bool {
|
||||
for _, c := range s {
|
||||
if !((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// runSSHOutput runs a command via SSH and returns stdout as a string.
|
||||
func runSSHOutput(node inspector.Node, command string) (string, error) {
|
||||
args := []string{
|
||||
"ssh", "-n",
|
||||
"-o", "StrictHostKeyChecking=accept-new",
|
||||
"-o", "ConnectTimeout=10",
|
||||
"-o", "BatchMode=yes",
|
||||
"-i", node.SSHKey,
|
||||
fmt.Sprintf("%s@%s", node.User, node.Host),
|
||||
command,
|
||||
}
|
||||
|
||||
out, err := execCommand(args[0], args[1:]...)
|
||||
return string(out), err
|
||||
}
|
||||
|
||||
// execCommand runs a command and returns its output.
|
||||
func execCommand(name string, args ...string) ([]byte, error) {
|
||||
return exec.Command(name, args...).Output()
|
||||
}
|
||||
|
||||
// findNewestArchive finds the newest binary archive in /tmp/.
|
||||
func findNewestArchive() string {
|
||||
entries, err := os.ReadDir("/tmp")
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
|
||||
var best string
|
||||
var bestMod int64
|
||||
for _, entry := range entries {
|
||||
name := entry.Name()
|
||||
if strings.HasPrefix(name, "orama-") && strings.Contains(name, "-linux-") && strings.HasSuffix(name, ".tar.gz") {
|
||||
info, err := entry.Info()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
if info.ModTime().Unix() > bestMod {
|
||||
best = filepath.Join("/tmp", name)
|
||||
bestMod = info.ModTime().Unix()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return best
|
||||
}
|
||||
|
||||
// formatBytes formats a byte count as human-readable.
|
||||
func formatBytes(b int64) string {
|
||||
const unit = 1024
|
||||
if b < unit {
|
||||
return fmt.Sprintf("%d B", b)
|
||||
}
|
||||
div, exp := int64(unit), 0
|
||||
for n := b / unit; n >= unit; n /= unit {
|
||||
div *= unit
|
||||
exp++
|
||||
}
|
||||
return fmt.Sprintf("%.1f %cB", float64(b)/float64(div), "KMGTPE"[exp])
|
||||
}
|
||||
|
||||
// printCreateSummary prints the cluster summary after creation.
|
||||
func printCreateSummary(cfg *Config, state *SandboxState) {
|
||||
fmt.Printf("\nSandbox %q ready (%d nodes)\n", state.Name, len(state.Servers))
|
||||
fmt.Println()
|
||||
|
||||
fmt.Println("Nameservers:")
|
||||
for _, srv := range state.NameserverNodes() {
|
||||
floating := ""
|
||||
if srv.FloatingIP != "" {
|
||||
floating = fmt.Sprintf(" (floating: %s)", srv.FloatingIP)
|
||||
}
|
||||
fmt.Printf(" %s: %s%s\n", srv.Name, srv.IP, floating)
|
||||
}
|
||||
|
||||
fmt.Println("Nodes:")
|
||||
for _, srv := range state.RegularNodes() {
|
||||
fmt.Printf(" %s: %s\n", srv.Name, srv.IP)
|
||||
}
|
||||
|
||||
fmt.Println()
|
||||
fmt.Printf("Domain: %s\n", cfg.Domain)
|
||||
fmt.Printf("Gateway: https://%s\n", cfg.Domain)
|
||||
fmt.Println()
|
||||
fmt.Println("SSH: orama sandbox ssh 1")
|
||||
fmt.Println("Destroy: orama sandbox destroy")
|
||||
}
|
||||
|
||||
// cleanupFailedCreate deletes any servers that were created during a failed provision.
|
||||
func cleanupFailedCreate(client *HetznerClient, state *SandboxState) {
|
||||
if len(state.Servers) == 0 {
|
||||
return
|
||||
}
|
||||
fmt.Println("\nCleaning up failed creation...")
|
||||
for _, srv := range state.Servers {
|
||||
if srv.ID > 0 {
|
||||
client.DeleteServer(srv.ID)
|
||||
fmt.Printf(" Deleted %s\n", srv.Name)
|
||||
}
|
||||
}
|
||||
DeleteState(state.Name)
|
||||
}
|
||||
122
pkg/cli/sandbox/destroy.go
Normal file
122
pkg/cli/sandbox/destroy.go
Normal file
@ -0,0 +1,122 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Destroy tears down a sandbox cluster.
|
||||
func Destroy(name string, force bool) error {
|
||||
cfg, err := LoadConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Resolve sandbox name
|
||||
state, err := resolveSandbox(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Confirm destruction
|
||||
if !force {
|
||||
reader := bufio.NewReader(os.Stdin)
|
||||
fmt.Printf("Destroy sandbox %q? This deletes %d servers. [y/N]: ", state.Name, len(state.Servers))
|
||||
choice, _ := reader.ReadString('\n')
|
||||
choice = strings.TrimSpace(strings.ToLower(choice))
|
||||
if choice != "y" && choice != "yes" {
|
||||
fmt.Println("Aborted.")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
state.Status = StatusDestroying
|
||||
SaveState(state) // best-effort status update
|
||||
|
||||
client := NewHetznerClient(cfg.HetznerAPIToken)
|
||||
|
||||
// Step 1: Unassign floating IPs from nameserver nodes
|
||||
fmt.Println("Unassigning floating IPs...")
|
||||
for _, srv := range state.NameserverNodes() {
|
||||
if srv.FloatingIP == "" {
|
||||
continue
|
||||
}
|
||||
// Find the floating IP ID from config
|
||||
for _, fip := range cfg.FloatingIPs {
|
||||
if fip.IP == srv.FloatingIP {
|
||||
if err := client.UnassignFloatingIP(fip.ID); err != nil {
|
||||
fmt.Fprintf(os.Stderr, " Warning: could not unassign floating IP %s: %v\n", fip.IP, err)
|
||||
} else {
|
||||
fmt.Printf(" Unassigned %s from %s\n", fip.IP, srv.Name)
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Step 2: Delete all servers in parallel
|
||||
fmt.Printf("Deleting %d servers...\n", len(state.Servers))
|
||||
var wg sync.WaitGroup
|
||||
var mu sync.Mutex
|
||||
var failed []string
|
||||
|
||||
for _, srv := range state.Servers {
|
||||
wg.Add(1)
|
||||
go func(srv ServerState) {
|
||||
defer wg.Done()
|
||||
if err := client.DeleteServer(srv.ID); err != nil {
|
||||
// Treat 404 as already deleted (idempotent)
|
||||
if strings.Contains(err.Error(), "404") || strings.Contains(err.Error(), "not found") {
|
||||
fmt.Printf(" %s (ID %d): already deleted\n", srv.Name, srv.ID)
|
||||
} else {
|
||||
mu.Lock()
|
||||
failed = append(failed, fmt.Sprintf("%s (ID %d): %v", srv.Name, srv.ID, err))
|
||||
mu.Unlock()
|
||||
fmt.Fprintf(os.Stderr, " Warning: failed to delete %s: %v\n", srv.Name, err)
|
||||
}
|
||||
} else {
|
||||
fmt.Printf(" Deleted %s (ID %d)\n", srv.Name, srv.ID)
|
||||
}
|
||||
}(srv)
|
||||
}
|
||||
wg.Wait()
|
||||
|
||||
if len(failed) > 0 {
|
||||
fmt.Fprintf(os.Stderr, "\nFailed to delete %d server(s):\n", len(failed))
|
||||
for _, f := range failed {
|
||||
fmt.Fprintf(os.Stderr, " %s\n", f)
|
||||
}
|
||||
fmt.Fprintf(os.Stderr, "\nManual cleanup: delete servers at https://console.hetzner.cloud\n")
|
||||
state.Status = StatusError
|
||||
SaveState(state)
|
||||
return fmt.Errorf("failed to delete %d server(s)", len(failed))
|
||||
}
|
||||
|
||||
// Step 3: Remove state file
|
||||
if err := DeleteState(state.Name); err != nil {
|
||||
return fmt.Errorf("delete state: %w", err)
|
||||
}
|
||||
|
||||
fmt.Printf("\nSandbox %q destroyed (%d servers deleted)\n", state.Name, len(state.Servers))
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolveSandbox finds a sandbox by name or returns the active one.
|
||||
func resolveSandbox(name string) (*SandboxState, error) {
|
||||
if name != "" {
|
||||
return LoadState(name)
|
||||
}
|
||||
|
||||
// Find the active sandbox
|
||||
active, err := FindActiveSandbox()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if active == nil {
|
||||
return nil, fmt.Errorf("no active sandbox found, specify --name")
|
||||
}
|
||||
return active, nil
|
||||
}
|
||||
438
pkg/cli/sandbox/hetzner.go
Normal file
438
pkg/cli/sandbox/hetzner.go
Normal file
@ -0,0 +1,438 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
const hetznerBaseURL = "https://api.hetzner.cloud/v1"
|
||||
|
||||
// HetznerClient is a minimal Hetzner Cloud API client.
|
||||
type HetznerClient struct {
|
||||
token string
|
||||
httpClient *http.Client
|
||||
}
|
||||
|
||||
// NewHetznerClient creates a new Hetzner API client.
|
||||
func NewHetznerClient(token string) *HetznerClient {
|
||||
return &HetznerClient{
|
||||
token: token,
|
||||
httpClient: &http.Client{
|
||||
Timeout: 30 * time.Second,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// --- Request helpers ---
|
||||
|
||||
func (c *HetznerClient) doRequest(method, path string, body interface{}) ([]byte, int, error) {
|
||||
var bodyReader io.Reader
|
||||
if body != nil {
|
||||
data, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("marshal request body: %w", err)
|
||||
}
|
||||
bodyReader = bytes.NewReader(data)
|
||||
}
|
||||
|
||||
req, err := http.NewRequest(method, hetznerBaseURL+path, bodyReader)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("create request: %w", err)
|
||||
}
|
||||
|
||||
req.Header.Set("Authorization", "Bearer "+c.token)
|
||||
if body != nil {
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
|
||||
resp, err := c.httpClient.Do(req)
|
||||
if err != nil {
|
||||
return nil, 0, fmt.Errorf("request %s %s: %w", method, path, err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
respBody, err := io.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, resp.StatusCode, fmt.Errorf("read response: %w", err)
|
||||
}
|
||||
|
||||
return respBody, resp.StatusCode, nil
|
||||
}
|
||||
|
||||
func (c *HetznerClient) get(path string) ([]byte, error) {
|
||||
body, status, err := c.doRequest("GET", path, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if status < 200 || status >= 300 {
|
||||
return nil, parseHetznerError(body, status)
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
|
||||
func (c *HetznerClient) post(path string, payload interface{}) ([]byte, error) {
|
||||
body, status, err := c.doRequest("POST", path, payload)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if status < 200 || status >= 300 {
|
||||
return nil, parseHetznerError(body, status)
|
||||
}
|
||||
return body, nil
|
||||
}
|
||||
|
||||
func (c *HetznerClient) delete(path string) error {
|
||||
_, status, err := c.doRequest("DELETE", path, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if status < 200 || status >= 300 {
|
||||
return fmt.Errorf("delete %s: HTTP %d", path, status)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- API types ---
|
||||
|
||||
// HetznerServer represents a Hetzner Cloud server.
|
||||
type HetznerServer struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Status string `json:"status"` // initializing, running, off, ...
|
||||
PublicNet HetznerPublicNet `json:"public_net"`
|
||||
Labels map[string]string `json:"labels"`
|
||||
ServerType struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"server_type"`
|
||||
}
|
||||
|
||||
// HetznerPublicNet holds public networking info for a server.
|
||||
type HetznerPublicNet struct {
|
||||
IPv4 struct {
|
||||
IP string `json:"ip"`
|
||||
} `json:"ipv4"`
|
||||
}
|
||||
|
||||
// HetznerFloatingIP represents a Hetzner floating IP.
|
||||
type HetznerFloatingIP struct {
|
||||
ID int64 `json:"id"`
|
||||
IP string `json:"ip"`
|
||||
Server *int64 `json:"server"` // nil if unassigned
|
||||
Labels map[string]string `json:"labels"`
|
||||
Description string `json:"description"`
|
||||
HomeLocation struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"home_location"`
|
||||
}
|
||||
|
||||
// HetznerSSHKey represents a Hetzner SSH key.
|
||||
type HetznerSSHKey struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
PublicKey string `json:"public_key"`
|
||||
}
|
||||
|
||||
// HetznerFirewall represents a Hetzner firewall.
|
||||
type HetznerFirewall struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Rules []HetznerFWRule `json:"rules"`
|
||||
Labels map[string]string `json:"labels"`
|
||||
}
|
||||
|
||||
// HetznerFWRule represents a firewall rule.
|
||||
type HetznerFWRule struct {
|
||||
Direction string `json:"direction"`
|
||||
Protocol string `json:"protocol"`
|
||||
Port string `json:"port"`
|
||||
SourceIPs []string `json:"source_ips"`
|
||||
Description string `json:"description,omitempty"`
|
||||
}
|
||||
|
||||
// HetznerError represents an API error response.
|
||||
type HetznerError struct {
|
||||
Error struct {
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
} `json:"error"`
|
||||
}
|
||||
|
||||
func parseHetznerError(body []byte, status int) error {
|
||||
var he HetznerError
|
||||
if err := json.Unmarshal(body, &he); err == nil && he.Error.Message != "" {
|
||||
return fmt.Errorf("hetzner API error (HTTP %d): %s — %s", status, he.Error.Code, he.Error.Message)
|
||||
}
|
||||
return fmt.Errorf("hetzner API error: HTTP %d", status)
|
||||
}
|
||||
|
||||
// --- Server operations ---
|
||||
|
||||
// CreateServerRequest holds parameters for server creation.
|
||||
type CreateServerRequest struct {
|
||||
Name string `json:"name"`
|
||||
ServerType string `json:"server_type"`
|
||||
Image string `json:"image"`
|
||||
Location string `json:"location"`
|
||||
SSHKeys []int64 `json:"ssh_keys"`
|
||||
Labels map[string]string `json:"labels"`
|
||||
Firewalls []struct {
|
||||
Firewall int64 `json:"firewall"`
|
||||
} `json:"firewalls,omitempty"`
|
||||
}
|
||||
|
||||
// CreateServer creates a new server and returns it.
|
||||
func (c *HetznerClient) CreateServer(req CreateServerRequest) (*HetznerServer, error) {
|
||||
body, err := c.post("/servers", req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create server %q: %w", req.Name, err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Server HetznerServer `json:"server"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse create server response: %w", err)
|
||||
}
|
||||
|
||||
return &resp.Server, nil
|
||||
}
|
||||
|
||||
// GetServer retrieves a server by ID.
|
||||
func (c *HetznerClient) GetServer(id int64) (*HetznerServer, error) {
|
||||
body, err := c.get("/servers/" + strconv.FormatInt(id, 10))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get server %d: %w", id, err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Server HetznerServer `json:"server"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse server response: %w", err)
|
||||
}
|
||||
|
||||
return &resp.Server, nil
|
||||
}
|
||||
|
||||
// DeleteServer deletes a server by ID.
|
||||
func (c *HetznerClient) DeleteServer(id int64) error {
|
||||
return c.delete("/servers/" + strconv.FormatInt(id, 10))
|
||||
}
|
||||
|
||||
// ListServersByLabel lists servers filtered by a label selector.
|
||||
func (c *HetznerClient) ListServersByLabel(selector string) ([]HetznerServer, error) {
|
||||
body, err := c.get("/servers?label_selector=" + selector)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list servers: %w", err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Servers []HetznerServer `json:"servers"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse servers response: %w", err)
|
||||
}
|
||||
|
||||
return resp.Servers, nil
|
||||
}
|
||||
|
||||
// WaitForServer polls until the server reaches "running" status.
|
||||
func (c *HetznerClient) WaitForServer(id int64, timeout time.Duration) (*HetznerServer, error) {
|
||||
deadline := time.Now().Add(timeout)
|
||||
for time.Now().Before(deadline) {
|
||||
srv, err := c.GetServer(id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if srv.Status == "running" {
|
||||
return srv, nil
|
||||
}
|
||||
time.Sleep(3 * time.Second)
|
||||
}
|
||||
return nil, fmt.Errorf("server %d did not reach running state within %s", id, timeout)
|
||||
}
|
||||
|
||||
// --- Floating IP operations ---
|
||||
|
||||
// CreateFloatingIP creates a new floating IP.
|
||||
func (c *HetznerClient) CreateFloatingIP(location, description string, labels map[string]string) (*HetznerFloatingIP, error) {
|
||||
payload := map[string]interface{}{
|
||||
"type": "ipv4",
|
||||
"home_location": location,
|
||||
"description": description,
|
||||
"labels": labels,
|
||||
}
|
||||
|
||||
body, err := c.post("/floating_ips", payload)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create floating IP: %w", err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
FloatingIP HetznerFloatingIP `json:"floating_ip"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse floating IP response: %w", err)
|
||||
}
|
||||
|
||||
return &resp.FloatingIP, nil
|
||||
}
|
||||
|
||||
// ListFloatingIPsByLabel lists floating IPs filtered by label.
|
||||
func (c *HetznerClient) ListFloatingIPsByLabel(selector string) ([]HetznerFloatingIP, error) {
|
||||
body, err := c.get("/floating_ips?label_selector=" + selector)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list floating IPs: %w", err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
FloatingIPs []HetznerFloatingIP `json:"floating_ips"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse floating IPs response: %w", err)
|
||||
}
|
||||
|
||||
return resp.FloatingIPs, nil
|
||||
}
|
||||
|
||||
// AssignFloatingIP assigns a floating IP to a server.
|
||||
func (c *HetznerClient) AssignFloatingIP(floatingIPID, serverID int64) error {
|
||||
payload := map[string]int64{"server": serverID}
|
||||
_, err := c.post("/floating_ips/"+strconv.FormatInt(floatingIPID, 10)+"/actions/assign", payload)
|
||||
if err != nil {
|
||||
return fmt.Errorf("assign floating IP %d to server %d: %w", floatingIPID, serverID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// UnassignFloatingIP removes a floating IP assignment.
|
||||
func (c *HetznerClient) UnassignFloatingIP(floatingIPID int64) error {
|
||||
_, err := c.post("/floating_ips/"+strconv.FormatInt(floatingIPID, 10)+"/actions/unassign", struct{}{})
|
||||
if err != nil {
|
||||
return fmt.Errorf("unassign floating IP %d: %w", floatingIPID, err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- SSH Key operations ---
|
||||
|
||||
// UploadSSHKey uploads a public key to Hetzner.
|
||||
func (c *HetznerClient) UploadSSHKey(name, publicKey string) (*HetznerSSHKey, error) {
|
||||
payload := map[string]string{
|
||||
"name": name,
|
||||
"public_key": publicKey,
|
||||
}
|
||||
|
||||
body, err := c.post("/ssh_keys", payload)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("upload SSH key: %w", err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
SSHKey HetznerSSHKey `json:"ssh_key"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse SSH key response: %w", err)
|
||||
}
|
||||
|
||||
return &resp.SSHKey, nil
|
||||
}
|
||||
|
||||
// GetSSHKey retrieves an SSH key by ID.
|
||||
func (c *HetznerClient) GetSSHKey(id int64) (*HetznerSSHKey, error) {
|
||||
body, err := c.get("/ssh_keys/" + strconv.FormatInt(id, 10))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get SSH key %d: %w", id, err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
SSHKey HetznerSSHKey `json:"ssh_key"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse SSH key response: %w", err)
|
||||
}
|
||||
|
||||
return &resp.SSHKey, nil
|
||||
}
|
||||
|
||||
// --- Firewall operations ---
|
||||
|
||||
// CreateFirewall creates a firewall with the given rules.
|
||||
func (c *HetznerClient) CreateFirewall(name string, rules []HetznerFWRule, labels map[string]string) (*HetznerFirewall, error) {
|
||||
payload := map[string]interface{}{
|
||||
"name": name,
|
||||
"rules": rules,
|
||||
"labels": labels,
|
||||
}
|
||||
|
||||
body, err := c.post("/firewalls", payload)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create firewall: %w", err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Firewall HetznerFirewall `json:"firewall"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse firewall response: %w", err)
|
||||
}
|
||||
|
||||
return &resp.Firewall, nil
|
||||
}
|
||||
|
||||
// ListFirewallsByLabel lists firewalls filtered by label.
|
||||
func (c *HetznerClient) ListFirewallsByLabel(selector string) ([]HetznerFirewall, error) {
|
||||
body, err := c.get("/firewalls?label_selector=" + selector)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list firewalls: %w", err)
|
||||
}
|
||||
|
||||
var resp struct {
|
||||
Firewalls []HetznerFirewall `json:"firewalls"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &resp); err != nil {
|
||||
return nil, fmt.Errorf("parse firewalls response: %w", err)
|
||||
}
|
||||
|
||||
return resp.Firewalls, nil
|
||||
}
|
||||
|
||||
// DeleteFirewall deletes a firewall by ID.
|
||||
func (c *HetznerClient) DeleteFirewall(id int64) error {
|
||||
return c.delete("/firewalls/" + strconv.FormatInt(id, 10))
|
||||
}
|
||||
|
||||
// --- Validation ---
|
||||
|
||||
// ValidateToken checks if the API token is valid by making a simple request.
|
||||
func (c *HetznerClient) ValidateToken() error {
|
||||
_, err := c.get("/servers?per_page=1")
|
||||
if err != nil {
|
||||
return fmt.Errorf("invalid Hetzner API token: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// --- Sandbox firewall rules ---
|
||||
|
||||
// SandboxFirewallRules returns the standard firewall rules for sandbox nodes.
|
||||
func SandboxFirewallRules() []HetznerFWRule {
|
||||
allIPv4 := []string{"0.0.0.0/0"}
|
||||
allIPv6 := []string{"::/0"}
|
||||
allIPs := append(allIPv4, allIPv6...)
|
||||
|
||||
return []HetznerFWRule{
|
||||
{Direction: "in", Protocol: "tcp", Port: "22", SourceIPs: allIPs, Description: "SSH"},
|
||||
{Direction: "in", Protocol: "tcp", Port: "53", SourceIPs: allIPs, Description: "DNS TCP"},
|
||||
{Direction: "in", Protocol: "udp", Port: "53", SourceIPs: allIPs, Description: "DNS UDP"},
|
||||
{Direction: "in", Protocol: "tcp", Port: "80", SourceIPs: allIPs, Description: "HTTP"},
|
||||
{Direction: "in", Protocol: "tcp", Port: "443", SourceIPs: allIPs, Description: "HTTPS"},
|
||||
{Direction: "in", Protocol: "udp", Port: "51820", SourceIPs: allIPs, Description: "WireGuard"},
|
||||
}
|
||||
}
|
||||
303
pkg/cli/sandbox/hetzner_test.go
Normal file
303
pkg/cli/sandbox/hetzner_test.go
Normal file
@ -0,0 +1,303 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestValidateToken_Success(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("Authorization") != "Bearer test-token" {
|
||||
t.Errorf("unexpected auth header: %s", r.Header.Get("Authorization"))
|
||||
}
|
||||
w.WriteHeader(200)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{"servers": []interface{}{}})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
if err := client.ValidateToken(); err != nil {
|
||||
t.Errorf("ValidateToken() error = %v, want nil", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateToken_InvalidToken(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(401)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||
"error": map[string]string{
|
||||
"code": "unauthorized",
|
||||
"message": "unable to authenticate",
|
||||
},
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "bad-token")
|
||||
if err := client.ValidateToken(); err == nil {
|
||||
t.Error("ValidateToken() expected error for invalid token")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateServer(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != "POST" || r.URL.Path != "/v1/servers" {
|
||||
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||
}
|
||||
|
||||
var req CreateServerRequest
|
||||
json.NewDecoder(r.Body).Decode(&req)
|
||||
|
||||
if req.Name != "sbx-test-1" {
|
||||
t.Errorf("unexpected server name: %s", req.Name)
|
||||
}
|
||||
if req.ServerType != "cx22" {
|
||||
t.Errorf("unexpected server type: %s", req.ServerType)
|
||||
}
|
||||
|
||||
w.WriteHeader(201)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||
"server": map[string]interface{}{
|
||||
"id": 12345,
|
||||
"name": req.Name,
|
||||
"status": "initializing",
|
||||
"public_net": map[string]interface{}{
|
||||
"ipv4": map[string]string{"ip": "1.2.3.4"},
|
||||
},
|
||||
"labels": req.Labels,
|
||||
"server_type": map[string]string{"name": "cx22"},
|
||||
},
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
server, err := client.CreateServer(CreateServerRequest{
|
||||
Name: "sbx-test-1",
|
||||
ServerType: "cx22",
|
||||
Image: "ubuntu-24.04",
|
||||
Location: "fsn1",
|
||||
SSHKeys: []int64{1},
|
||||
Labels: map[string]string{"orama-sandbox": "test"},
|
||||
})
|
||||
|
||||
if err != nil {
|
||||
t.Fatalf("CreateServer() error = %v", err)
|
||||
}
|
||||
if server.ID != 12345 {
|
||||
t.Errorf("server ID = %d, want 12345", server.ID)
|
||||
}
|
||||
if server.Name != "sbx-test-1" {
|
||||
t.Errorf("server name = %s, want sbx-test-1", server.Name)
|
||||
}
|
||||
if server.PublicNet.IPv4.IP != "1.2.3.4" {
|
||||
t.Errorf("server IP = %s, want 1.2.3.4", server.PublicNet.IPv4.IP)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteServer(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != "DELETE" || r.URL.Path != "/v1/servers/12345" {
|
||||
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||
}
|
||||
w.WriteHeader(200)
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
if err := client.DeleteServer(12345); err != nil {
|
||||
t.Errorf("DeleteServer() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestListServersByLabel(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Query().Get("label_selector") != "orama-sandbox=test" {
|
||||
t.Errorf("unexpected label_selector: %s", r.URL.Query().Get("label_selector"))
|
||||
}
|
||||
w.WriteHeader(200)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||
"servers": []map[string]interface{}{
|
||||
{"id": 1, "name": "sbx-test-1", "status": "running", "public_net": map[string]interface{}{"ipv4": map[string]string{"ip": "1.1.1.1"}}, "server_type": map[string]string{"name": "cx22"}},
|
||||
{"id": 2, "name": "sbx-test-2", "status": "running", "public_net": map[string]interface{}{"ipv4": map[string]string{"ip": "2.2.2.2"}}, "server_type": map[string]string{"name": "cx22"}},
|
||||
},
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
servers, err := client.ListServersByLabel("orama-sandbox=test")
|
||||
if err != nil {
|
||||
t.Fatalf("ListServersByLabel() error = %v", err)
|
||||
}
|
||||
if len(servers) != 2 {
|
||||
t.Errorf("got %d servers, want 2", len(servers))
|
||||
}
|
||||
}
|
||||
|
||||
func TestWaitForServer_AlreadyRunning(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.WriteHeader(200)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||
"server": map[string]interface{}{
|
||||
"id": 1,
|
||||
"name": "test",
|
||||
"status": "running",
|
||||
"public_net": map[string]interface{}{
|
||||
"ipv4": map[string]string{"ip": "1.1.1.1"},
|
||||
},
|
||||
"server_type": map[string]string{"name": "cx22"},
|
||||
},
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
server, err := client.WaitForServer(1, 5*time.Second)
|
||||
if err != nil {
|
||||
t.Fatalf("WaitForServer() error = %v", err)
|
||||
}
|
||||
if server.Status != "running" {
|
||||
t.Errorf("server status = %s, want running", server.Status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAssignFloatingIP(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != "POST" || r.URL.Path != "/v1/floating_ips/100/actions/assign" {
|
||||
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||
}
|
||||
|
||||
var body map[string]int64
|
||||
json.NewDecoder(r.Body).Decode(&body)
|
||||
if body["server"] != 200 {
|
||||
t.Errorf("unexpected server ID: %d", body["server"])
|
||||
}
|
||||
|
||||
w.WriteHeader(200)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{"action": map[string]interface{}{"id": 1, "status": "running"}})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
if err := client.AssignFloatingIP(100, 200); err != nil {
|
||||
t.Errorf("AssignFloatingIP() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUploadSSHKey(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != "POST" || r.URL.Path != "/v1/ssh_keys" {
|
||||
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||
}
|
||||
w.WriteHeader(201)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||
"ssh_key": map[string]interface{}{
|
||||
"id": 42,
|
||||
"name": "orama-sandbox",
|
||||
"fingerprint": "aa:bb:cc:dd",
|
||||
"public_key": "ssh-ed25519 AAAA...",
|
||||
},
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
key, err := client.UploadSSHKey("orama-sandbox", "ssh-ed25519 AAAA...")
|
||||
if err != nil {
|
||||
t.Fatalf("UploadSSHKey() error = %v", err)
|
||||
}
|
||||
if key.ID != 42 {
|
||||
t.Errorf("key ID = %d, want 42", key.ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateFirewall(t *testing.T) {
|
||||
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Method != "POST" || r.URL.Path != "/v1/firewalls" {
|
||||
t.Errorf("unexpected request: %s %s", r.Method, r.URL.Path)
|
||||
}
|
||||
w.WriteHeader(201)
|
||||
json.NewEncoder(w).Encode(map[string]interface{}{
|
||||
"firewall": map[string]interface{}{
|
||||
"id": 99,
|
||||
"name": "orama-sandbox",
|
||||
},
|
||||
})
|
||||
}))
|
||||
defer srv.Close()
|
||||
|
||||
client := newTestClient(srv, "test-token")
|
||||
fw, err := client.CreateFirewall("orama-sandbox", SandboxFirewallRules(), map[string]string{"orama-sandbox": "infra"})
|
||||
if err != nil {
|
||||
t.Fatalf("CreateFirewall() error = %v", err)
|
||||
}
|
||||
if fw.ID != 99 {
|
||||
t.Errorf("firewall ID = %d, want 99", fw.ID)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSandboxFirewallRules(t *testing.T) {
|
||||
rules := SandboxFirewallRules()
|
||||
if len(rules) != 6 {
|
||||
t.Errorf("got %d rules, want 6", len(rules))
|
||||
}
|
||||
|
||||
expectedPorts := map[string]bool{"22": false, "53": false, "80": false, "443": false, "51820": false}
|
||||
for _, r := range rules {
|
||||
expectedPorts[r.Port] = true
|
||||
if r.Direction != "in" {
|
||||
t.Errorf("rule %s direction = %s, want in", r.Port, r.Direction)
|
||||
}
|
||||
}
|
||||
for port, seen := range expectedPorts {
|
||||
if !seen {
|
||||
t.Errorf("missing firewall rule for port %s", port)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseHetznerError(t *testing.T) {
|
||||
body := `{"error":{"code":"uniqueness_error","message":"server name already used"}}`
|
||||
err := parseHetznerError([]byte(body), 409)
|
||||
if err == nil {
|
||||
t.Fatal("expected error")
|
||||
}
|
||||
expected := "hetzner API error (HTTP 409): uniqueness_error — server name already used"
|
||||
if err.Error() != expected {
|
||||
t.Errorf("error = %q, want %q", err.Error(), expected)
|
||||
}
|
||||
}
|
||||
|
||||
// newTestClient creates a HetznerClient pointing at a test server.
|
||||
func newTestClient(ts *httptest.Server, token string) *HetznerClient {
|
||||
client := NewHetznerClient(token)
|
||||
// Override the base URL by using a custom transport
|
||||
client.httpClient = ts.Client()
|
||||
// We need to override the base URL — wrap the transport
|
||||
origTransport := client.httpClient.Transport
|
||||
client.httpClient.Transport = &testTransport{
|
||||
base: origTransport,
|
||||
testURL: ts.URL,
|
||||
}
|
||||
return client
|
||||
}
|
||||
|
||||
// testTransport rewrites requests to point at the test server.
|
||||
type testTransport struct {
|
||||
base http.RoundTripper
|
||||
testURL string
|
||||
}
|
||||
|
||||
func (t *testTransport) RoundTrip(req *http.Request) (*http.Response, error) {
|
||||
// Rewrite the URL to point at the test server
|
||||
req.URL.Scheme = "http"
|
||||
req.URL.Host = t.testURL[len("http://"):]
|
||||
if t.base != nil {
|
||||
return t.base.RoundTrip(req)
|
||||
}
|
||||
return http.DefaultTransport.RoundTrip(req)
|
||||
}
|
||||
26
pkg/cli/sandbox/names.go
Normal file
26
pkg/cli/sandbox/names.go
Normal file
@ -0,0 +1,26 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"math/rand"
|
||||
)
|
||||
|
||||
var adjectives = []string{
|
||||
"swift", "bright", "calm", "dark", "eager",
|
||||
"fair", "gold", "hazy", "iron", "jade",
|
||||
"keen", "lush", "mild", "neat", "opal",
|
||||
"pure", "raw", "sage", "teal", "warm",
|
||||
}
|
||||
|
||||
var nouns = []string{
|
||||
"falcon", "beacon", "cedar", "delta", "ember",
|
||||
"frost", "grove", "haven", "ivory", "jewel",
|
||||
"knot", "latch", "maple", "nexus", "orbit",
|
||||
"prism", "reef", "spark", "tide", "vault",
|
||||
}
|
||||
|
||||
// GenerateName produces a random adjective-noun name like "swift-falcon".
|
||||
func GenerateName() string {
|
||||
adj := adjectives[rand.Intn(len(adjectives))]
|
||||
noun := nouns[rand.Intn(len(nouns))]
|
||||
return adj + "-" + noun
|
||||
}
|
||||
137
pkg/cli/sandbox/rollout.go
Normal file
137
pkg/cli/sandbox/rollout.go
Normal file
@ -0,0 +1,137 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"time"
|
||||
|
||||
"github.com/DeBrosOfficial/network/pkg/cli/remotessh"
|
||||
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||
)
|
||||
|
||||
// Rollout builds, pushes, and performs a rolling upgrade on a sandbox cluster.
|
||||
func Rollout(name string) error {
|
||||
cfg, err := LoadConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
state, err := resolveSandbox(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
fmt.Printf("Rolling out to sandbox %q (%d nodes)\n\n", state.Name, len(state.Servers))
|
||||
|
||||
// Step 1: Find or require binary archive
|
||||
archivePath := findNewestArchive()
|
||||
if archivePath == "" {
|
||||
return fmt.Errorf("no binary archive found in /tmp/ (run `orama build` first)")
|
||||
}
|
||||
|
||||
info, _ := os.Stat(archivePath)
|
||||
fmt.Printf("Archive: %s (%s)\n\n", filepath.Base(archivePath), formatBytes(info.Size()))
|
||||
|
||||
// Step 2: Push archive to all nodes
|
||||
fmt.Println("Pushing archive to all nodes...")
|
||||
remotePath := "/tmp/" + filepath.Base(archivePath)
|
||||
|
||||
for i, srv := range state.Servers {
|
||||
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||
|
||||
fmt.Printf(" [%d/%d] Uploading to %s...\n", i+1, len(state.Servers), srv.Name)
|
||||
if err := remotessh.UploadFile(node, archivePath, remotePath); err != nil {
|
||||
return fmt.Errorf("upload to %s: %w", srv.Name, err)
|
||||
}
|
||||
|
||||
// Extract archive
|
||||
extractCmd := fmt.Sprintf("mkdir -p /opt/orama && tar xzf %s -C /opt/orama && rm -f %s",
|
||||
remotePath, remotePath)
|
||||
if err := remotessh.RunSSHStreaming(node, extractCmd); err != nil {
|
||||
return fmt.Errorf("extract on %s: %w", srv.Name, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Step 3: Rolling upgrade — followers first, leader last
|
||||
fmt.Println("\nRolling upgrade (followers first, leader last)...")
|
||||
|
||||
// Find the leader
|
||||
leaderIdx := findLeaderIndex(state, sshKeyPath)
|
||||
if leaderIdx < 0 {
|
||||
fmt.Fprintf(os.Stderr, " Warning: could not detect RQLite leader, upgrading in order\n")
|
||||
}
|
||||
|
||||
// Upgrade non-leaders first
|
||||
for i, srv := range state.Servers {
|
||||
if i == leaderIdx {
|
||||
continue // skip leader, do it last
|
||||
}
|
||||
if err := upgradeNode(srv, sshKeyPath, i+1, len(state.Servers)); err != nil {
|
||||
return err
|
||||
}
|
||||
// Wait between nodes
|
||||
if i < len(state.Servers)-1 {
|
||||
fmt.Printf(" Waiting 15s before next node...\n")
|
||||
time.Sleep(15 * time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
// Upgrade leader last
|
||||
if leaderIdx >= 0 {
|
||||
srv := state.Servers[leaderIdx]
|
||||
if err := upgradeNode(srv, sshKeyPath, len(state.Servers), len(state.Servers)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Printf("\nRollout complete for sandbox %q\n", state.Name)
|
||||
return nil
|
||||
}
|
||||
|
||||
// findLeaderIndex returns the index of the RQLite leader node, or -1 if unknown.
|
||||
func findLeaderIndex(state *SandboxState, sshKeyPath string) int {
|
||||
for i, srv := range state.Servers {
|
||||
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||
out, err := runSSHOutput(node, "curl -sf http://localhost:5001/status 2>/dev/null | grep -o '\"state\":\"[^\"]*\"'")
|
||||
if err == nil && contains(out, "Leader") {
|
||||
return i
|
||||
}
|
||||
}
|
||||
return -1
|
||||
}
|
||||
|
||||
// upgradeNode performs `orama node upgrade --restart` on a single node.
|
||||
func upgradeNode(srv ServerState, sshKeyPath string, current, total int) error {
|
||||
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||
|
||||
fmt.Printf(" [%d/%d] Upgrading %s (%s)...\n", current, total, srv.Name, srv.IP)
|
||||
if err := remotessh.RunSSHStreaming(node, "orama node upgrade --restart"); err != nil {
|
||||
return fmt.Errorf("upgrade %s: %w", srv.Name, err)
|
||||
}
|
||||
|
||||
// Wait for health
|
||||
fmt.Printf(" Checking health...")
|
||||
if err := waitForRQLiteHealth(node, 2*time.Minute); err != nil {
|
||||
fmt.Printf(" WARN: %v\n", err)
|
||||
} else {
|
||||
fmt.Println(" OK")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// contains checks if s contains substr.
|
||||
func contains(s, substr string) bool {
|
||||
return len(s) >= len(substr) && findSubstring(s, substr)
|
||||
}
|
||||
|
||||
func findSubstring(s, substr string) bool {
|
||||
for i := 0; i <= len(s)-len(substr); i++ {
|
||||
if s[i:i+len(substr)] == substr {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
319
pkg/cli/sandbox/setup.go
Normal file
319
pkg/cli/sandbox/setup.go
Normal file
@ -0,0 +1,319 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"encoding/pem"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
|
||||
"golang.org/x/crypto/ssh"
|
||||
)
|
||||
|
||||
// Setup runs the interactive sandbox setup wizard.
|
||||
func Setup() error {
|
||||
fmt.Println("Orama Sandbox Setup")
|
||||
fmt.Println("====================")
|
||||
fmt.Println()
|
||||
|
||||
reader := bufio.NewReader(os.Stdin)
|
||||
|
||||
// Step 1: Hetzner API token
|
||||
fmt.Print("Hetzner Cloud API token: ")
|
||||
token, err := reader.ReadString('\n')
|
||||
if err != nil {
|
||||
return fmt.Errorf("read token: %w", err)
|
||||
}
|
||||
token = strings.TrimSpace(token)
|
||||
if token == "" {
|
||||
return fmt.Errorf("API token is required")
|
||||
}
|
||||
|
||||
fmt.Print(" Validating token... ")
|
||||
client := NewHetznerClient(token)
|
||||
if err := client.ValidateToken(); err != nil {
|
||||
fmt.Println("FAILED")
|
||||
return fmt.Errorf("invalid token: %w", err)
|
||||
}
|
||||
fmt.Println("OK")
|
||||
fmt.Println()
|
||||
|
||||
// Step 2: Domain
|
||||
fmt.Print("Sandbox domain (e.g., sbx.dbrs.space): ")
|
||||
domain, err := reader.ReadString('\n')
|
||||
if err != nil {
|
||||
return fmt.Errorf("read domain: %w", err)
|
||||
}
|
||||
domain = strings.TrimSpace(domain)
|
||||
if domain == "" {
|
||||
return fmt.Errorf("domain is required")
|
||||
}
|
||||
|
||||
cfg := &Config{
|
||||
HetznerAPIToken: token,
|
||||
Domain: domain,
|
||||
}
|
||||
cfg.Defaults()
|
||||
|
||||
// Step 3: Floating IPs
|
||||
fmt.Println()
|
||||
fmt.Println("Checking floating IPs...")
|
||||
floatingIPs, err := setupFloatingIPs(client, cfg.Location)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cfg.FloatingIPs = floatingIPs
|
||||
|
||||
// Step 4: Firewall
|
||||
fmt.Println()
|
||||
fmt.Println("Checking firewall...")
|
||||
fwID, err := setupFirewall(client)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cfg.FirewallID = fwID
|
||||
|
||||
// Step 5: SSH key
|
||||
fmt.Println()
|
||||
fmt.Println("Setting up SSH key...")
|
||||
sshKeyConfig, err := setupSSHKey(client)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cfg.SSHKey = sshKeyConfig
|
||||
|
||||
// Step 6: Display DNS instructions
|
||||
fmt.Println()
|
||||
fmt.Println("DNS Configuration")
|
||||
fmt.Println("-----------------")
|
||||
fmt.Println("Configure the following at your domain registrar:")
|
||||
fmt.Println()
|
||||
fmt.Printf(" 1. Add glue records (Personal DNS Servers):\n")
|
||||
fmt.Printf(" ns1.%s -> %s\n", domain, cfg.FloatingIPs[0].IP)
|
||||
fmt.Printf(" ns2.%s -> %s\n", domain, cfg.FloatingIPs[1].IP)
|
||||
fmt.Println()
|
||||
fmt.Printf(" 2. Set custom nameservers for %s:\n", domain)
|
||||
fmt.Printf(" ns1.%s\n", domain)
|
||||
fmt.Printf(" ns2.%s\n", domain)
|
||||
fmt.Println()
|
||||
|
||||
// Step 7: Verify DNS (optional)
|
||||
fmt.Print("Verify DNS now? [y/N]: ")
|
||||
verifyChoice, _ := reader.ReadString('\n')
|
||||
verifyChoice = strings.TrimSpace(strings.ToLower(verifyChoice))
|
||||
if verifyChoice == "y" || verifyChoice == "yes" {
|
||||
verifyDNS(domain)
|
||||
}
|
||||
|
||||
// Save config
|
||||
if err := SaveConfig(cfg); err != nil {
|
||||
return fmt.Errorf("save config: %w", err)
|
||||
}
|
||||
|
||||
fmt.Println()
|
||||
fmt.Println("Setup complete! Config saved to ~/.orama/sandbox.yaml")
|
||||
fmt.Println()
|
||||
fmt.Println("Next: orama sandbox create")
|
||||
return nil
|
||||
}
|
||||
|
||||
// setupFloatingIPs checks for existing floating IPs or creates new ones.
|
||||
func setupFloatingIPs(client *HetznerClient, location string) ([]FloatIP, error) {
|
||||
existing, err := client.ListFloatingIPsByLabel("orama-sandbox-dns=true")
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list floating IPs: %w", err)
|
||||
}
|
||||
|
||||
if len(existing) >= 2 {
|
||||
fmt.Printf(" Found %d existing floating IPs:\n", len(existing))
|
||||
result := make([]FloatIP, 2)
|
||||
for i := 0; i < 2; i++ {
|
||||
fmt.Printf(" ns%d: %s (ID: %d)\n", i+1, existing[i].IP, existing[i].ID)
|
||||
result[i] = FloatIP{ID: existing[i].ID, IP: existing[i].IP}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// Need to create missing floating IPs
|
||||
needed := 2 - len(existing)
|
||||
fmt.Printf(" Need to create %d floating IP(s)...\n", needed)
|
||||
|
||||
reader := bufio.NewReader(os.Stdin)
|
||||
fmt.Printf(" Create %d floating IP(s) in %s? (~$0.005/hr each) [Y/n]: ", needed, location)
|
||||
choice, _ := reader.ReadString('\n')
|
||||
choice = strings.TrimSpace(strings.ToLower(choice))
|
||||
if choice == "n" || choice == "no" {
|
||||
return nil, fmt.Errorf("floating IPs required, aborting setup")
|
||||
}
|
||||
|
||||
result := make([]FloatIP, 0, 2)
|
||||
for _, fip := range existing {
|
||||
result = append(result, FloatIP{ID: fip.ID, IP: fip.IP})
|
||||
}
|
||||
|
||||
for i := len(existing); i < 2; i++ {
|
||||
desc := fmt.Sprintf("orama-sandbox-ns%d", i+1)
|
||||
labels := map[string]string{"orama-sandbox-dns": "true"}
|
||||
fip, err := client.CreateFloatingIP(location, desc, labels)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create floating IP %d: %w", i+1, err)
|
||||
}
|
||||
fmt.Printf(" Created ns%d: %s (ID: %d)\n", i+1, fip.IP, fip.ID)
|
||||
result = append(result, FloatIP{ID: fip.ID, IP: fip.IP})
|
||||
}
|
||||
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// setupFirewall ensures a sandbox firewall exists.
|
||||
func setupFirewall(client *HetznerClient) (int64, error) {
|
||||
existing, err := client.ListFirewallsByLabel("orama-sandbox=infra")
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("list firewalls: %w", err)
|
||||
}
|
||||
|
||||
if len(existing) > 0 {
|
||||
fmt.Printf(" Found existing firewall: %s (ID: %d)\n", existing[0].Name, existing[0].ID)
|
||||
return existing[0].ID, nil
|
||||
}
|
||||
|
||||
fmt.Print(" Creating sandbox firewall... ")
|
||||
fw, err := client.CreateFirewall(
|
||||
"orama-sandbox",
|
||||
SandboxFirewallRules(),
|
||||
map[string]string{"orama-sandbox": "infra"},
|
||||
)
|
||||
if err != nil {
|
||||
fmt.Println("FAILED")
|
||||
return 0, fmt.Errorf("create firewall: %w", err)
|
||||
}
|
||||
fmt.Printf("OK (ID: %d)\n", fw.ID)
|
||||
return fw.ID, nil
|
||||
}
|
||||
|
||||
// setupSSHKey generates an SSH keypair and uploads it to Hetzner.
|
||||
func setupSSHKey(client *HetznerClient) (SSHKeyConfig, error) {
|
||||
dir, err := configDir()
|
||||
if err != nil {
|
||||
return SSHKeyConfig{}, err
|
||||
}
|
||||
|
||||
privPath := dir + "/sandbox_key"
|
||||
pubPath := privPath + ".pub"
|
||||
|
||||
// Check for existing key
|
||||
if _, err := os.Stat(privPath); err == nil {
|
||||
fmt.Printf(" SSH key already exists: %s\n", privPath)
|
||||
|
||||
// Read public key and check if it's on Hetzner
|
||||
pubData, err := os.ReadFile(pubPath)
|
||||
if err != nil {
|
||||
return SSHKeyConfig{}, fmt.Errorf("read public key: %w", err)
|
||||
}
|
||||
|
||||
// Try to upload (will fail with uniqueness error if already exists)
|
||||
key, err := client.UploadSSHKey("orama-sandbox", strings.TrimSpace(string(pubData)))
|
||||
if err != nil {
|
||||
// Key likely already exists on Hetzner — find it by listing
|
||||
fmt.Printf(" SSH key may already be on Hetzner (upload: %v)\n", err)
|
||||
fmt.Print(" Enter the Hetzner SSH key ID (or 0 to re-upload): ")
|
||||
reader := bufio.NewReader(os.Stdin)
|
||||
idStr, _ := reader.ReadString('\n')
|
||||
idStr = strings.TrimSpace(idStr)
|
||||
var hetznerID int64
|
||||
fmt.Sscanf(idStr, "%d", &hetznerID)
|
||||
|
||||
if hetznerID == 0 {
|
||||
return SSHKeyConfig{}, fmt.Errorf("could not resolve SSH key on Hetzner, try deleting and re-running setup")
|
||||
}
|
||||
|
||||
return SSHKeyConfig{
|
||||
HetznerID: hetznerID,
|
||||
PrivateKeyPath: "~/.orama/sandbox_key",
|
||||
PublicKeyPath: "~/.orama/sandbox_key.pub",
|
||||
}, nil
|
||||
}
|
||||
|
||||
fmt.Printf(" Uploaded to Hetzner (ID: %d)\n", key.ID)
|
||||
return SSHKeyConfig{
|
||||
HetznerID: key.ID,
|
||||
PrivateKeyPath: "~/.orama/sandbox_key",
|
||||
PublicKeyPath: "~/.orama/sandbox_key.pub",
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Generate new ed25519 keypair
|
||||
fmt.Print(" Generating ed25519 keypair... ")
|
||||
pubKey, privKey, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
fmt.Println("FAILED")
|
||||
return SSHKeyConfig{}, fmt.Errorf("generate key: %w", err)
|
||||
}
|
||||
|
||||
// Marshal private key to OpenSSH format
|
||||
pemBlock, err := ssh.MarshalPrivateKey(privKey, "")
|
||||
if err != nil {
|
||||
fmt.Println("FAILED")
|
||||
return SSHKeyConfig{}, fmt.Errorf("marshal private key: %w", err)
|
||||
}
|
||||
|
||||
privPEM := pem.EncodeToMemory(pemBlock)
|
||||
if err := os.WriteFile(privPath, privPEM, 0600); err != nil {
|
||||
fmt.Println("FAILED")
|
||||
return SSHKeyConfig{}, fmt.Errorf("write private key: %w", err)
|
||||
}
|
||||
|
||||
// Marshal public key to authorized_keys format
|
||||
sshPubKey, err := ssh.NewPublicKey(pubKey)
|
||||
if err != nil {
|
||||
return SSHKeyConfig{}, fmt.Errorf("convert public key: %w", err)
|
||||
}
|
||||
pubStr := strings.TrimSpace(string(ssh.MarshalAuthorizedKey(sshPubKey)))
|
||||
|
||||
if err := os.WriteFile(pubPath, []byte(pubStr+"\n"), 0644); err != nil {
|
||||
return SSHKeyConfig{}, fmt.Errorf("write public key: %w", err)
|
||||
}
|
||||
fmt.Println("OK")
|
||||
|
||||
// Upload to Hetzner
|
||||
fmt.Print(" Uploading to Hetzner... ")
|
||||
key, err := client.UploadSSHKey("orama-sandbox", pubStr)
|
||||
if err != nil {
|
||||
fmt.Println("FAILED")
|
||||
return SSHKeyConfig{}, fmt.Errorf("upload SSH key: %w", err)
|
||||
}
|
||||
fmt.Printf("OK (ID: %d)\n", key.ID)
|
||||
|
||||
return SSHKeyConfig{
|
||||
HetznerID: key.ID,
|
||||
PrivateKeyPath: "~/.orama/sandbox_key",
|
||||
PublicKeyPath: "~/.orama/sandbox_key.pub",
|
||||
}, nil
|
||||
}
|
||||
|
||||
// verifyDNS checks if the sandbox domain resolves.
|
||||
func verifyDNS(domain string) {
|
||||
fmt.Printf(" Checking NS records for %s...\n", domain)
|
||||
out, err := exec.Command("dig", "+short", "NS", domain, "@8.8.8.8").Output()
|
||||
if err != nil {
|
||||
fmt.Printf(" Warning: dig failed: %v\n", err)
|
||||
fmt.Println(" DNS verification skipped. You can verify later with:")
|
||||
fmt.Printf(" dig NS %s @8.8.8.8\n", domain)
|
||||
return
|
||||
}
|
||||
|
||||
result := strings.TrimSpace(string(out))
|
||||
if result == "" {
|
||||
fmt.Println(" Warning: No NS records found yet.")
|
||||
fmt.Println(" DNS propagation can take up to 48 hours.")
|
||||
fmt.Println(" The sandbox will still work once DNS is configured.")
|
||||
} else {
|
||||
fmt.Printf(" NS records:\n")
|
||||
for _, line := range strings.Split(result, "\n") {
|
||||
fmt.Printf(" %s\n", line)
|
||||
}
|
||||
}
|
||||
}
|
||||
56
pkg/cli/sandbox/ssh_cmd.go
Normal file
56
pkg/cli/sandbox/ssh_cmd.go
Normal file
@ -0,0 +1,56 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// SSHInto opens an interactive SSH session to a sandbox node.
|
||||
func SSHInto(name string, nodeNum int) error {
|
||||
cfg, err := LoadConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
state, err := resolveSandbox(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if nodeNum < 1 || nodeNum > len(state.Servers) {
|
||||
return fmt.Errorf("node number must be between 1 and %d", len(state.Servers))
|
||||
}
|
||||
|
||||
srv := state.Servers[nodeNum-1]
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
|
||||
fmt.Printf("Connecting to %s (%s, %s)...\n", srv.Name, srv.IP, srv.Role)
|
||||
|
||||
// Find ssh binary
|
||||
sshBin, err := findSSHBinary()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Replace current process with SSH
|
||||
args := []string{
|
||||
"ssh",
|
||||
"-o", "StrictHostKeyChecking=accept-new",
|
||||
"-i", sshKeyPath,
|
||||
fmt.Sprintf("root@%s", srv.IP),
|
||||
}
|
||||
|
||||
return syscall.Exec(sshBin, args, os.Environ())
|
||||
}
|
||||
|
||||
// findSSHBinary locates the ssh binary in PATH.
|
||||
func findSSHBinary() (string, error) {
|
||||
paths := []string{"/usr/bin/ssh", "/usr/local/bin/ssh", "/opt/homebrew/bin/ssh"}
|
||||
for _, p := range paths {
|
||||
if _, err := os.Stat(p); err == nil {
|
||||
return p, nil
|
||||
}
|
||||
}
|
||||
return "", fmt.Errorf("ssh binary not found")
|
||||
}
|
||||
211
pkg/cli/sandbox/state.go
Normal file
211
pkg/cli/sandbox/state.go
Normal file
@ -0,0 +1,211 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// SandboxStatus represents the lifecycle state of a sandbox.
|
||||
type SandboxStatus string
|
||||
|
||||
const (
|
||||
StatusCreating SandboxStatus = "creating"
|
||||
StatusRunning SandboxStatus = "running"
|
||||
StatusDestroying SandboxStatus = "destroying"
|
||||
StatusError SandboxStatus = "error"
|
||||
)
|
||||
|
||||
// SandboxState holds the full state of an active sandbox cluster.
|
||||
type SandboxState struct {
|
||||
Name string `yaml:"name"`
|
||||
CreatedAt time.Time `yaml:"created_at"`
|
||||
Domain string `yaml:"domain"`
|
||||
Status SandboxStatus `yaml:"status"`
|
||||
Servers []ServerState `yaml:"servers"`
|
||||
}
|
||||
|
||||
// ServerState holds the state of a single server in the sandbox.
|
||||
type ServerState struct {
|
||||
ID int64 `yaml:"id"` // Hetzner server ID
|
||||
Name string `yaml:"name"` // e.g., sbx-feature-webrtc-1
|
||||
IP string `yaml:"ip"` // Public IPv4
|
||||
Role string `yaml:"role"` // "nameserver" or "node"
|
||||
FloatingIP string `yaml:"floating_ip,omitempty"` // Only for nameserver nodes
|
||||
WgIP string `yaml:"wg_ip,omitempty"` // WireGuard IP (populated after install)
|
||||
}
|
||||
|
||||
// sandboxesDir returns ~/.orama/sandboxes/, creating it if needed.
|
||||
func sandboxesDir() (string, error) {
|
||||
dir, err := configDir()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
sbxDir := filepath.Join(dir, "sandboxes")
|
||||
if err := os.MkdirAll(sbxDir, 0700); err != nil {
|
||||
return "", fmt.Errorf("create sandboxes directory: %w", err)
|
||||
}
|
||||
return sbxDir, nil
|
||||
}
|
||||
|
||||
// statePath returns the path for a sandbox's state file.
|
||||
func statePath(name string) (string, error) {
|
||||
dir, err := sandboxesDir()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return filepath.Join(dir, name+".yaml"), nil
|
||||
}
|
||||
|
||||
// SaveState persists the sandbox state to disk.
|
||||
func SaveState(state *SandboxState) error {
|
||||
path, err := statePath(state.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
data, err := yaml.Marshal(state)
|
||||
if err != nil {
|
||||
return fmt.Errorf("marshal state: %w", err)
|
||||
}
|
||||
|
||||
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||
return fmt.Errorf("write state: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// LoadState reads a sandbox state from disk.
|
||||
func LoadState(name string) (*SandboxState, error) {
|
||||
path, err := statePath(name)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
data, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, fmt.Errorf("sandbox %q not found", name)
|
||||
}
|
||||
return nil, fmt.Errorf("read state: %w", err)
|
||||
}
|
||||
|
||||
var state SandboxState
|
||||
if err := yaml.Unmarshal(data, &state); err != nil {
|
||||
return nil, fmt.Errorf("parse state: %w", err)
|
||||
}
|
||||
|
||||
return &state, nil
|
||||
}
|
||||
|
||||
// DeleteState removes the sandbox state file.
|
||||
func DeleteState(name string) error {
|
||||
path, err := statePath(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if err := os.Remove(path); err != nil && !os.IsNotExist(err) {
|
||||
return fmt.Errorf("delete state: %w", err)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ListStates returns all sandbox states from disk.
|
||||
func ListStates() ([]*SandboxState, error) {
|
||||
dir, err := sandboxesDir()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read sandboxes directory: %w", err)
|
||||
}
|
||||
|
||||
var states []*SandboxState
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".yaml") {
|
||||
continue
|
||||
}
|
||||
name := strings.TrimSuffix(entry.Name(), ".yaml")
|
||||
state, err := LoadState(name)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Warning: could not load sandbox %q: %v\n", name, err)
|
||||
continue
|
||||
}
|
||||
states = append(states, state)
|
||||
}
|
||||
|
||||
return states, nil
|
||||
}
|
||||
|
||||
// FindActiveSandbox returns the first sandbox in running or creating state.
|
||||
// Returns nil if no active sandbox exists.
|
||||
func FindActiveSandbox() (*SandboxState, error) {
|
||||
states, err := ListStates()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
for _, s := range states {
|
||||
if s.Status == StatusRunning || s.Status == StatusCreating {
|
||||
return s, nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// ToNodes converts sandbox servers to inspector.Node structs for SSH operations.
|
||||
// Sets SSHKey to the provided key path on each node.
|
||||
func (s *SandboxState) ToNodes(sshKeyPath string) []inspector.Node {
|
||||
nodes := make([]inspector.Node, len(s.Servers))
|
||||
for i, srv := range s.Servers {
|
||||
nodes[i] = inspector.Node{
|
||||
Environment: "sandbox",
|
||||
User: "root",
|
||||
Host: srv.IP,
|
||||
Role: srv.Role,
|
||||
SSHKey: sshKeyPath,
|
||||
}
|
||||
}
|
||||
return nodes
|
||||
}
|
||||
|
||||
// NameserverNodes returns only the nameserver nodes.
|
||||
func (s *SandboxState) NameserverNodes() []ServerState {
|
||||
var ns []ServerState
|
||||
for _, srv := range s.Servers {
|
||||
if srv.Role == "nameserver" {
|
||||
ns = append(ns, srv)
|
||||
}
|
||||
}
|
||||
return ns
|
||||
}
|
||||
|
||||
// RegularNodes returns only the non-nameserver nodes.
|
||||
func (s *SandboxState) RegularNodes() []ServerState {
|
||||
var nodes []ServerState
|
||||
for _, srv := range s.Servers {
|
||||
if srv.Role == "node" {
|
||||
nodes = append(nodes, srv)
|
||||
}
|
||||
}
|
||||
return nodes
|
||||
}
|
||||
|
||||
// GenesisServer returns the first server (genesis node).
|
||||
func (s *SandboxState) GenesisServer() ServerState {
|
||||
if len(s.Servers) == 0 {
|
||||
return ServerState{}
|
||||
}
|
||||
return s.Servers[0]
|
||||
}
|
||||
214
pkg/cli/sandbox/state_test.go
Normal file
214
pkg/cli/sandbox/state_test.go
Normal file
@ -0,0 +1,214 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSaveAndLoadState(t *testing.T) {
|
||||
// Use temp dir for test
|
||||
tmpDir := t.TempDir()
|
||||
origHome := os.Getenv("HOME")
|
||||
os.Setenv("HOME", tmpDir)
|
||||
defer os.Setenv("HOME", origHome)
|
||||
|
||||
state := &SandboxState{
|
||||
Name: "test-sandbox",
|
||||
CreatedAt: time.Date(2026, 2, 25, 10, 0, 0, 0, time.UTC),
|
||||
Domain: "test.example.com",
|
||||
Status: StatusRunning,
|
||||
Servers: []ServerState{
|
||||
{ID: 1, Name: "sbx-test-1", IP: "1.1.1.1", Role: "nameserver", FloatingIP: "10.0.0.1", WgIP: "10.0.0.1"},
|
||||
{ID: 2, Name: "sbx-test-2", IP: "2.2.2.2", Role: "nameserver", FloatingIP: "10.0.0.2", WgIP: "10.0.0.2"},
|
||||
{ID: 3, Name: "sbx-test-3", IP: "3.3.3.3", Role: "node", WgIP: "10.0.0.3"},
|
||||
{ID: 4, Name: "sbx-test-4", IP: "4.4.4.4", Role: "node", WgIP: "10.0.0.4"},
|
||||
{ID: 5, Name: "sbx-test-5", IP: "5.5.5.5", Role: "node", WgIP: "10.0.0.5"},
|
||||
},
|
||||
}
|
||||
|
||||
if err := SaveState(state); err != nil {
|
||||
t.Fatalf("SaveState() error = %v", err)
|
||||
}
|
||||
|
||||
// Verify file exists
|
||||
expected := filepath.Join(tmpDir, ".orama", "sandboxes", "test-sandbox.yaml")
|
||||
if _, err := os.Stat(expected); err != nil {
|
||||
t.Fatalf("state file not created at %s: %v", expected, err)
|
||||
}
|
||||
|
||||
// Load back
|
||||
loaded, err := LoadState("test-sandbox")
|
||||
if err != nil {
|
||||
t.Fatalf("LoadState() error = %v", err)
|
||||
}
|
||||
|
||||
if loaded.Name != "test-sandbox" {
|
||||
t.Errorf("name = %s, want test-sandbox", loaded.Name)
|
||||
}
|
||||
if loaded.Domain != "test.example.com" {
|
||||
t.Errorf("domain = %s, want test.example.com", loaded.Domain)
|
||||
}
|
||||
if loaded.Status != StatusRunning {
|
||||
t.Errorf("status = %s, want running", loaded.Status)
|
||||
}
|
||||
if len(loaded.Servers) != 5 {
|
||||
t.Errorf("servers = %d, want 5", len(loaded.Servers))
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadState_NotFound(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
origHome := os.Getenv("HOME")
|
||||
os.Setenv("HOME", tmpDir)
|
||||
defer os.Setenv("HOME", origHome)
|
||||
|
||||
_, err := LoadState("nonexistent")
|
||||
if err == nil {
|
||||
t.Error("LoadState() expected error for nonexistent sandbox")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteState(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
origHome := os.Getenv("HOME")
|
||||
os.Setenv("HOME", tmpDir)
|
||||
defer os.Setenv("HOME", origHome)
|
||||
|
||||
state := &SandboxState{
|
||||
Name: "to-delete",
|
||||
Status: StatusRunning,
|
||||
}
|
||||
if err := SaveState(state); err != nil {
|
||||
t.Fatalf("SaveState() error = %v", err)
|
||||
}
|
||||
|
||||
if err := DeleteState("to-delete"); err != nil {
|
||||
t.Fatalf("DeleteState() error = %v", err)
|
||||
}
|
||||
|
||||
_, err := LoadState("to-delete")
|
||||
if err == nil {
|
||||
t.Error("LoadState() should fail after DeleteState()")
|
||||
}
|
||||
}
|
||||
|
||||
func TestListStates(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
origHome := os.Getenv("HOME")
|
||||
os.Setenv("HOME", tmpDir)
|
||||
defer os.Setenv("HOME", origHome)
|
||||
|
||||
// Create 2 sandboxes
|
||||
for _, name := range []string{"sandbox-a", "sandbox-b"} {
|
||||
if err := SaveState(&SandboxState{Name: name, Status: StatusRunning}); err != nil {
|
||||
t.Fatalf("SaveState(%s) error = %v", name, err)
|
||||
}
|
||||
}
|
||||
|
||||
states, err := ListStates()
|
||||
if err != nil {
|
||||
t.Fatalf("ListStates() error = %v", err)
|
||||
}
|
||||
if len(states) != 2 {
|
||||
t.Errorf("ListStates() returned %d, want 2", len(states))
|
||||
}
|
||||
}
|
||||
|
||||
func TestFindActiveSandbox(t *testing.T) {
|
||||
tmpDir := t.TempDir()
|
||||
origHome := os.Getenv("HOME")
|
||||
os.Setenv("HOME", tmpDir)
|
||||
defer os.Setenv("HOME", origHome)
|
||||
|
||||
// No sandboxes
|
||||
active, err := FindActiveSandbox()
|
||||
if err != nil {
|
||||
t.Fatalf("FindActiveSandbox() error = %v", err)
|
||||
}
|
||||
if active != nil {
|
||||
t.Error("expected nil when no sandboxes exist")
|
||||
}
|
||||
|
||||
// Add one running sandbox
|
||||
if err := SaveState(&SandboxState{Name: "active-one", Status: StatusRunning}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := SaveState(&SandboxState{Name: "errored-one", Status: StatusError}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
active, err = FindActiveSandbox()
|
||||
if err != nil {
|
||||
t.Fatalf("FindActiveSandbox() error = %v", err)
|
||||
}
|
||||
if active == nil || active.Name != "active-one" {
|
||||
t.Errorf("FindActiveSandbox() = %v, want active-one", active)
|
||||
}
|
||||
}
|
||||
|
||||
func TestToNodes(t *testing.T) {
|
||||
state := &SandboxState{
|
||||
Servers: []ServerState{
|
||||
{IP: "1.1.1.1", Role: "nameserver"},
|
||||
{IP: "2.2.2.2", Role: "node"},
|
||||
},
|
||||
}
|
||||
|
||||
nodes := state.ToNodes("/tmp/key")
|
||||
if len(nodes) != 2 {
|
||||
t.Fatalf("ToNodes() returned %d nodes, want 2", len(nodes))
|
||||
}
|
||||
if nodes[0].Host != "1.1.1.1" {
|
||||
t.Errorf("node[0].Host = %s, want 1.1.1.1", nodes[0].Host)
|
||||
}
|
||||
if nodes[0].User != "root" {
|
||||
t.Errorf("node[0].User = %s, want root", nodes[0].User)
|
||||
}
|
||||
if nodes[0].SSHKey != "/tmp/key" {
|
||||
t.Errorf("node[0].SSHKey = %s, want /tmp/key", nodes[0].SSHKey)
|
||||
}
|
||||
if nodes[0].Environment != "sandbox" {
|
||||
t.Errorf("node[0].Environment = %s, want sandbox", nodes[0].Environment)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNameserverAndRegularNodes(t *testing.T) {
|
||||
state := &SandboxState{
|
||||
Servers: []ServerState{
|
||||
{Role: "nameserver"},
|
||||
{Role: "nameserver"},
|
||||
{Role: "node"},
|
||||
{Role: "node"},
|
||||
{Role: "node"},
|
||||
},
|
||||
}
|
||||
|
||||
ns := state.NameserverNodes()
|
||||
if len(ns) != 2 {
|
||||
t.Errorf("NameserverNodes() = %d, want 2", len(ns))
|
||||
}
|
||||
|
||||
regular := state.RegularNodes()
|
||||
if len(regular) != 3 {
|
||||
t.Errorf("RegularNodes() = %d, want 3", len(regular))
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenesisServer(t *testing.T) {
|
||||
state := &SandboxState{
|
||||
Servers: []ServerState{
|
||||
{Name: "first"},
|
||||
{Name: "second"},
|
||||
},
|
||||
}
|
||||
if state.GenesisServer().Name != "first" {
|
||||
t.Errorf("GenesisServer().Name = %s, want first", state.GenesisServer().Name)
|
||||
}
|
||||
|
||||
empty := &SandboxState{}
|
||||
if empty.GenesisServer().Name != "" {
|
||||
t.Error("GenesisServer() on empty state should return zero value")
|
||||
}
|
||||
}
|
||||
160
pkg/cli/sandbox/status.go
Normal file
160
pkg/cli/sandbox/status.go
Normal file
@ -0,0 +1,160 @@
|
||||
package sandbox
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/DeBrosOfficial/network/pkg/inspector"
|
||||
)
|
||||
|
||||
// List prints all sandbox clusters.
|
||||
func List() error {
|
||||
states, err := ListStates()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(states) == 0 {
|
||||
fmt.Println("No sandboxes found.")
|
||||
fmt.Println("Create one: orama sandbox create")
|
||||
return nil
|
||||
}
|
||||
|
||||
fmt.Printf("%-20s %-10s %-5s %-25s %s\n", "NAME", "STATUS", "NODES", "CREATED", "DOMAIN")
|
||||
for _, s := range states {
|
||||
fmt.Printf("%-20s %-10s %-5d %-25s %s\n",
|
||||
s.Name, s.Status, len(s.Servers), s.CreatedAt.Format("2006-01-02 15:04"), s.Domain)
|
||||
}
|
||||
|
||||
// Check for orphaned servers on Hetzner
|
||||
cfg, err := LoadConfig()
|
||||
if err != nil {
|
||||
return nil // Config not set up, skip orphan check
|
||||
}
|
||||
|
||||
client := NewHetznerClient(cfg.HetznerAPIToken)
|
||||
hetznerServers, err := client.ListServersByLabel("orama-sandbox")
|
||||
if err != nil {
|
||||
return nil // API error, skip orphan check
|
||||
}
|
||||
|
||||
// Build set of known server IDs
|
||||
known := make(map[int64]bool)
|
||||
for _, s := range states {
|
||||
for _, srv := range s.Servers {
|
||||
known[srv.ID] = true
|
||||
}
|
||||
}
|
||||
|
||||
var orphans []string
|
||||
for _, srv := range hetznerServers {
|
||||
if !known[srv.ID] {
|
||||
orphans = append(orphans, fmt.Sprintf("%s (ID: %d, IP: %s)", srv.Name, srv.ID, srv.PublicNet.IPv4.IP))
|
||||
}
|
||||
}
|
||||
|
||||
if len(orphans) > 0 {
|
||||
fmt.Printf("\nWarning: %d orphaned server(s) on Hetzner (no state file):\n", len(orphans))
|
||||
for _, o := range orphans {
|
||||
fmt.Printf(" %s\n", o)
|
||||
}
|
||||
fmt.Println("Delete manually at https://console.hetzner.cloud")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// Status prints the health report for a sandbox cluster.
|
||||
func Status(name string) error {
|
||||
cfg, err := LoadConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
state, err := resolveSandbox(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
sshKeyPath := cfg.ExpandedPrivateKeyPath()
|
||||
fmt.Printf("Sandbox: %s (status: %s)\n\n", state.Name, state.Status)
|
||||
|
||||
for _, srv := range state.Servers {
|
||||
node := inspector.Node{User: "root", Host: srv.IP, SSHKey: sshKeyPath}
|
||||
|
||||
fmt.Printf("%s (%s) — %s\n", srv.Name, srv.IP, srv.Role)
|
||||
|
||||
// Get node report
|
||||
out, err := runSSHOutput(node, "orama node report --json 2>/dev/null")
|
||||
if err != nil {
|
||||
fmt.Printf(" Status: UNREACHABLE (%v)\n", err)
|
||||
fmt.Println()
|
||||
continue
|
||||
}
|
||||
|
||||
printNodeReport(out)
|
||||
fmt.Println()
|
||||
}
|
||||
|
||||
// Cluster summary
|
||||
fmt.Println("Cluster Summary")
|
||||
fmt.Println("---------------")
|
||||
genesis := state.GenesisServer()
|
||||
genesisNode := inspector.Node{User: "root", Host: genesis.IP, SSHKey: sshKeyPath}
|
||||
|
||||
out, err := runSSHOutput(genesisNode, "curl -sf http://localhost:5001/status 2>/dev/null")
|
||||
if err != nil {
|
||||
fmt.Println(" RQLite: UNREACHABLE")
|
||||
} else {
|
||||
var status map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(out), &status); err == nil {
|
||||
if store, ok := status["store"].(map[string]interface{}); ok {
|
||||
if raft, ok := store["raft"].(map[string]interface{}); ok {
|
||||
fmt.Printf(" RQLite state: %v\n", raft["state"])
|
||||
fmt.Printf(" Commit index: %v\n", raft["commit_index"])
|
||||
if nodes, ok := raft["nodes"].([]interface{}); ok {
|
||||
fmt.Printf(" Nodes: %d\n", len(nodes))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// printNodeReport parses and prints a node report JSON.
|
||||
func printNodeReport(jsonStr string) {
|
||||
var report map[string]interface{}
|
||||
if err := json.Unmarshal([]byte(jsonStr), &report); err != nil {
|
||||
fmt.Printf(" Report: (parse error)\n")
|
||||
return
|
||||
}
|
||||
|
||||
// Print key fields
|
||||
if services, ok := report["services"].(map[string]interface{}); ok {
|
||||
var active, inactive []string
|
||||
for name, info := range services {
|
||||
if svc, ok := info.(map[string]interface{}); ok {
|
||||
if state, ok := svc["active"].(bool); ok && state {
|
||||
active = append(active, name)
|
||||
} else {
|
||||
inactive = append(inactive, name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(active) > 0 {
|
||||
fmt.Printf(" Active: %s\n", strings.Join(active, ", "))
|
||||
}
|
||||
if len(inactive) > 0 {
|
||||
fmt.Printf(" Inactive: %s\n", strings.Join(inactive, ", "))
|
||||
}
|
||||
}
|
||||
|
||||
if rqlite, ok := report["rqlite"].(map[string]interface{}); ok {
|
||||
if state, ok := rqlite["state"].(string); ok {
|
||||
fmt.Printf(" RQLite: %s\n", state)
|
||||
}
|
||||
}
|
||||
}
|
||||
Loading…
x
Reference in New Issue
Block a user