mirror of
https://github.com/DeBrosOfficial/orama.git
synced 2026-03-27 09:24:12 +00:00
Compare commits
No commits in common. "main" and "v0.82.0-nightly" have entirely different histories.
main
...
v0.82.0-ni
92
.cursor/rules/network.mdc
Normal file
92
.cursor/rules/network.mdc
Normal file
@ -0,0 +1,92 @@
|
|||||||
|
---
|
||||||
|
alwaysApply: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# AI Instructions
|
||||||
|
|
||||||
|
You have access to the **network** MCP (Model Context Protocol) server for this project. This MCP provides deep, pre-analyzed context about the codebase that is far more accurate than default file searching.
|
||||||
|
|
||||||
|
## IMPORTANT: Always Use MCP First
|
||||||
|
|
||||||
|
**Before making any code changes or answering questions about this codebase, ALWAYS consult the MCP tools first.**
|
||||||
|
|
||||||
|
The MCP has pre-indexed the entire codebase with semantic understanding, embeddings, and structural analysis. While you can use your own file search capabilities, the MCP provides much better context because:
|
||||||
|
- It understands code semantics, not just text matching
|
||||||
|
- It has pre-analyzed the architecture, patterns, and relationships
|
||||||
|
- It can answer questions about intent and purpose, not just content
|
||||||
|
|
||||||
|
## Available MCP Tools
|
||||||
|
|
||||||
|
### Code Understanding
|
||||||
|
- `network_ask_question` - Ask natural language questions about the codebase. Use this for "how does X work?", "where is Y implemented?", "what does Z do?" questions. The MCP will search relevant code and provide informed answers.
|
||||||
|
- `network_search_code` - Semantic code search. Find code by meaning, not just text. Great for finding implementations, patterns, or related functionality.
|
||||||
|
- `network_get_architecture` - Get the full project architecture overview including tech stack, design patterns, domain entities, and API endpoints.
|
||||||
|
- `network_get_file_summary` - Get a detailed summary of what a specific file does, its purpose, exports, and responsibilities.
|
||||||
|
- `network_find_function` - Find a specific function or method definition by name across the codebase.
|
||||||
|
- `network_list_functions` - List all functions defined in a specific file.
|
||||||
|
|
||||||
|
### Skills (Learned Procedures)
|
||||||
|
Skills are reusable procedures that the agent has learned about this specific project (e.g., "how to deploy", "how to run tests", "how to add a new API endpoint").
|
||||||
|
|
||||||
|
- `network_list_skills` - List all learned skills for this project.
|
||||||
|
- `network_get_skill` - Get detailed information about a specific skill including its step-by-step procedure.
|
||||||
|
- `network_execute_skill` - Get the procedure for a learned skill so you can execute it step by step. Returns prerequisites, warnings, and commands to run.
|
||||||
|
- `network_learn_skill` - Teach the agent a new skill. The agent will explore, discover, and memorize how to perform this task.
|
||||||
|
- `network_get_learning_status` - Check the status of an ongoing skill learning session.
|
||||||
|
- `network_answer_question` - Answer a question that the learning agent asked during skill learning.
|
||||||
|
- `network_cancel_learning` - Cancel an active learning session.
|
||||||
|
- `network_forget_skill` - Delete a learned skill.
|
||||||
|
- `network_update_skill` - Update a learned skill with corrections or new information (e.g., 'Use .env.prod instead of .env', 'Add step to backup database first', 'The port should be 3000 not 8080').
|
||||||
|
|
||||||
|
#### Skill Learning Workflow (IMPORTANT)
|
||||||
|
|
||||||
|
When learning a skill, follow this **collaborative, goal-oriented workflow**. You (Cursor) are the executor, the MCP agent provides guidance:
|
||||||
|
|
||||||
|
**Goal-Oriented Learning**: The agent identifies specific GOALS (pieces of information to gather) and tracks progress by goal completion, not by iterations.
|
||||||
|
|
||||||
|
1. **Start Learning**: Call `learn_skill` with name and detailed description
|
||||||
|
2. **Monitor Progress**: Call `get_learning_status` to check progress
|
||||||
|
3. **Handle Status Responses**:
|
||||||
|
- `active` → Learning in progress, check again in a few seconds
|
||||||
|
- `waiting_input` → The agent has a question. Read it and call `answer_question` with your response
|
||||||
|
- `waiting_execution` → **IMPORTANT**: The agent needs you to run a command!
|
||||||
|
- Read the `pendingExecution.command` from the response
|
||||||
|
- **Execute the command yourself** using your terminal access
|
||||||
|
- Call `answer_question` with the command output
|
||||||
|
- `completed` → Skill learned successfully!
|
||||||
|
- `failed` → Check errors and try again
|
||||||
|
4. **Repeat** steps 2-3 until status is `completed`
|
||||||
|
|
||||||
|
**Key Insight**: The MCP agent runs on the server and cannot SSH to remote servers directly. When it needs remote access, it generates the SSH command for YOU to execute. You have terminal access - use it!
|
||||||
|
|
||||||
|
**User Override Commands**: If the agent gets stuck, you can include these keywords in your answer:
|
||||||
|
- `COMPLETE` or `SKIP` - Skip to synthesis phase and generate the skill from current data
|
||||||
|
- `PHASE:synthesizing` - Force transition to drafting phase
|
||||||
|
- `GOAL:goal_id=value` - Directly provide a goal's value (e.g., `GOAL:cluster_secret=abc123`)
|
||||||
|
- `I have provided X` - Tell the agent it already has certain information
|
||||||
|
|
||||||
|
**Example for `waiting_execution`**:
|
||||||
|
```
|
||||||
|
// Status response shows:
|
||||||
|
// pendingExecution: { command: "ssh root@192.168.1.1 'ls -la /home/user/.orama'" }
|
||||||
|
//
|
||||||
|
// You should:
|
||||||
|
// 1. Run the command in your terminal
|
||||||
|
// 2. Get the output
|
||||||
|
// 3. Call answer_question with the output
|
||||||
|
```
|
||||||
|
|
||||||
|
## Recommended Workflow
|
||||||
|
|
||||||
|
1. **For questions:** Use `network_ask_question` or `network_search_code` to understand the codebase.
|
||||||
|
---
|
||||||
|
|
||||||
|
# DeBros Network Gateway
|
||||||
|
|
||||||
|
This project is a high-performance, edge-focused API gateway and reverse proxy designed to bridge decentralized web services with standard HTTP clients. It serves as a comprehensive middleware layer that facilitates wallet-based authentication, distributed caching via Olric, IPFS storage interaction, and serverless execution of WebAssembly (Wasm) functions. Additionally, it provides specialized utility services such as push notifications and an anonymizing proxy, ensuring secure and monitored communication between users and decentralized infrastructure.
|
||||||
|
|
||||||
|
**Architecture:** API Gateway / Edge Middleware
|
||||||
|
|
||||||
|
## Tech Stack
|
||||||
|
- **backend:** Go
|
||||||
|
|
||||||
@ -8,7 +8,7 @@ NOCOLOR='\033[0m'
|
|||||||
|
|
||||||
# Run tests before push
|
# Run tests before push
|
||||||
echo -e "\n${CYAN}Running tests...${NOCOLOR}"
|
echo -e "\n${CYAN}Running tests...${NOCOLOR}"
|
||||||
cd "$(git rev-parse --show-toplevel)/core" && go test ./...
|
go test ./... # Runs all tests in your repo
|
||||||
status=$?
|
status=$?
|
||||||
if [ $status -ne 0 ]; then
|
if [ $status -ne 0 ]; then
|
||||||
echo -e "${RED}Push aborted: some tests failed.${NOCOLOR}"
|
echo -e "${RED}Push aborted: some tests failed.${NOCOLOR}"
|
||||||
91
.github/ISSUE_TEMPLATE/bug_report.yml
vendored
91
.github/ISSUE_TEMPLATE/bug_report.yml
vendored
@ -1,91 +0,0 @@
|
|||||||
name: Bug Report
|
|
||||||
description: Report a bug in Orama Network
|
|
||||||
labels: ["bug"]
|
|
||||||
body:
|
|
||||||
- type: markdown
|
|
||||||
attributes:
|
|
||||||
value: |
|
|
||||||
Thanks for reporting a bug! Please fill out the sections below.
|
|
||||||
|
|
||||||
**Security issues:** If this is a security vulnerability, do NOT open an issue. Email security@orama.io instead.
|
|
||||||
|
|
||||||
- type: input
|
|
||||||
id: version
|
|
||||||
attributes:
|
|
||||||
label: Orama version
|
|
||||||
description: "Run `orama version` to find this"
|
|
||||||
placeholder: "v0.18.0-beta"
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: dropdown
|
|
||||||
id: component
|
|
||||||
attributes:
|
|
||||||
label: Component
|
|
||||||
options:
|
|
||||||
- Gateway / API
|
|
||||||
- CLI (orama command)
|
|
||||||
- WireGuard / Networking
|
|
||||||
- RQLite / Storage
|
|
||||||
- Olric / Caching
|
|
||||||
- IPFS / Pinning
|
|
||||||
- CoreDNS
|
|
||||||
- OramaOS
|
|
||||||
- Other
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: description
|
|
||||||
attributes:
|
|
||||||
label: Description
|
|
||||||
description: A clear description of the bug
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: steps
|
|
||||||
attributes:
|
|
||||||
label: Steps to reproduce
|
|
||||||
description: Minimal steps to reproduce the behavior
|
|
||||||
placeholder: |
|
|
||||||
1. Run `orama ...`
|
|
||||||
2. See error
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: expected
|
|
||||||
attributes:
|
|
||||||
label: Expected behavior
|
|
||||||
description: What you expected to happen
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: actual
|
|
||||||
attributes:
|
|
||||||
label: Actual behavior
|
|
||||||
description: What actually happened (include error messages and logs if any)
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: environment
|
|
||||||
attributes:
|
|
||||||
label: Environment
|
|
||||||
description: OS, Go version, deployment environment, etc.
|
|
||||||
placeholder: |
|
|
||||||
- OS: Ubuntu 22.04
|
|
||||||
- Go: 1.23
|
|
||||||
- Environment: sandbox
|
|
||||||
validations:
|
|
||||||
required: false
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: context
|
|
||||||
attributes:
|
|
||||||
label: Additional context
|
|
||||||
description: Logs, screenshots, monitor reports, or anything else that might help
|
|
||||||
validations:
|
|
||||||
required: false
|
|
||||||
49
.github/ISSUE_TEMPLATE/feature_request.yml
vendored
49
.github/ISSUE_TEMPLATE/feature_request.yml
vendored
@ -1,49 +0,0 @@
|
|||||||
name: Feature Request
|
|
||||||
description: Suggest a new feature or improvement
|
|
||||||
labels: ["enhancement"]
|
|
||||||
body:
|
|
||||||
- type: markdown
|
|
||||||
attributes:
|
|
||||||
value: |
|
|
||||||
Thanks for the suggestion! Please describe what you'd like to see.
|
|
||||||
|
|
||||||
- type: dropdown
|
|
||||||
id: component
|
|
||||||
attributes:
|
|
||||||
label: Component
|
|
||||||
options:
|
|
||||||
- Gateway / API
|
|
||||||
- CLI (orama command)
|
|
||||||
- WireGuard / Networking
|
|
||||||
- RQLite / Storage
|
|
||||||
- Olric / Caching
|
|
||||||
- IPFS / Pinning
|
|
||||||
- CoreDNS
|
|
||||||
- OramaOS
|
|
||||||
- Other
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: problem
|
|
||||||
attributes:
|
|
||||||
label: Problem
|
|
||||||
description: What problem does this solve? Why do you need it?
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: solution
|
|
||||||
attributes:
|
|
||||||
label: Proposed solution
|
|
||||||
description: How do you think this should work?
|
|
||||||
validations:
|
|
||||||
required: true
|
|
||||||
|
|
||||||
- type: textarea
|
|
||||||
id: alternatives
|
|
||||||
attributes:
|
|
||||||
label: Alternatives considered
|
|
||||||
description: Any workarounds or alternative approaches you've thought of
|
|
||||||
validations:
|
|
||||||
required: false
|
|
||||||
31
.github/PULL_REQUEST_TEMPLATE.md
vendored
31
.github/PULL_REQUEST_TEMPLATE.md
vendored
@ -1,31 +0,0 @@
|
|||||||
## Summary
|
|
||||||
|
|
||||||
<!-- What does this PR do? Keep it to 1-3 bullet points. -->
|
|
||||||
|
|
||||||
## Motivation
|
|
||||||
|
|
||||||
<!-- Why is this change needed? Link to an issue if applicable. -->
|
|
||||||
|
|
||||||
## Test plan
|
|
||||||
|
|
||||||
<!-- How did you verify this works? -->
|
|
||||||
|
|
||||||
- [ ] `make test` passes
|
|
||||||
- [ ] Tested on sandbox/staging environment
|
|
||||||
|
|
||||||
## Distributed system impact
|
|
||||||
|
|
||||||
<!-- Does this change affect any of the following? If yes, explain. -->
|
|
||||||
|
|
||||||
- [ ] Raft quorum / RQLite
|
|
||||||
- [ ] WireGuard mesh / networking
|
|
||||||
- [ ] Olric gossip / caching
|
|
||||||
- [ ] Service startup ordering
|
|
||||||
- [ ] Rolling upgrade compatibility
|
|
||||||
|
|
||||||
## Checklist
|
|
||||||
|
|
||||||
- [ ] Tests added for new functionality or bug fix
|
|
||||||
- [ ] No debug code (`fmt.Println`, `log.Println`) left behind
|
|
||||||
- [ ] Docs updated (if user-facing behavior changed)
|
|
||||||
- [ ] Errors wrapped with context (`fmt.Errorf("...: %w", err)`)
|
|
||||||
80
.github/workflows/publish-sdk.yml
vendored
80
.github/workflows/publish-sdk.yml
vendored
@ -1,80 +0,0 @@
|
|||||||
name: Publish SDK to npm
|
|
||||||
|
|
||||||
on:
|
|
||||||
workflow_dispatch:
|
|
||||||
inputs:
|
|
||||||
version:
|
|
||||||
description: "Version to publish (e.g., 1.0.0). Leave empty to use package.json version."
|
|
||||||
required: false
|
|
||||||
dry-run:
|
|
||||||
description: "Dry run (don't actually publish)"
|
|
||||||
type: boolean
|
|
||||||
default: false
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: write
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
publish:
|
|
||||||
name: Build & Publish @debros/orama
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
defaults:
|
|
||||||
run:
|
|
||||||
working-directory: sdk
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout code
|
|
||||||
uses: actions/checkout@v4
|
|
||||||
|
|
||||||
- name: Set up Node.js
|
|
||||||
uses: actions/setup-node@v4
|
|
||||||
with:
|
|
||||||
node-version: "20"
|
|
||||||
registry-url: "https://registry.npmjs.org"
|
|
||||||
|
|
||||||
- name: Install pnpm
|
|
||||||
uses: pnpm/action-setup@v4
|
|
||||||
with:
|
|
||||||
version: 9
|
|
||||||
|
|
||||||
- name: Install dependencies
|
|
||||||
run: pnpm install --frozen-lockfile
|
|
||||||
|
|
||||||
- name: Bump version
|
|
||||||
if: inputs.version != ''
|
|
||||||
run: npm version ${{ inputs.version }} --no-git-tag-version
|
|
||||||
|
|
||||||
- name: Typecheck
|
|
||||||
run: pnpm typecheck
|
|
||||||
|
|
||||||
- name: Build
|
|
||||||
run: pnpm build
|
|
||||||
|
|
||||||
- name: Run unit tests
|
|
||||||
run: pnpm vitest run tests/unit
|
|
||||||
|
|
||||||
- name: Publish (dry run)
|
|
||||||
if: inputs.dry-run == true
|
|
||||||
run: npm publish --access public --dry-run
|
|
||||||
env:
|
|
||||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
|
||||||
|
|
||||||
- name: Publish
|
|
||||||
if: inputs.dry-run == false
|
|
||||||
run: npm publish --access public
|
|
||||||
env:
|
|
||||||
NODE_AUTH_TOKEN: ${{ secrets.NPM_TOKEN }}
|
|
||||||
|
|
||||||
- name: Get published version
|
|
||||||
if: inputs.dry-run == false
|
|
||||||
id: version
|
|
||||||
run: echo "version=$(node -p "require('./package.json').version")" >> $GITHUB_OUTPUT
|
|
||||||
|
|
||||||
- name: Create git tag
|
|
||||||
if: inputs.dry-run == false
|
|
||||||
working-directory: .
|
|
||||||
run: |
|
|
||||||
git config user.name "github-actions[bot]"
|
|
||||||
git config user.email "github-actions[bot]@users.noreply.github.com"
|
|
||||||
git tag "sdk/v${{ steps.version.outputs.version }}"
|
|
||||||
git push origin "sdk/v${{ steps.version.outputs.version }}"
|
|
||||||
12
.github/workflows/release-apt.yml
vendored
12
.github/workflows/release-apt.yml
vendored
@ -28,8 +28,7 @@ jobs:
|
|||||||
- name: Set up Go
|
- name: Set up Go
|
||||||
uses: actions/setup-go@v5
|
uses: actions/setup-go@v5
|
||||||
with:
|
with:
|
||||||
go-version: "1.24"
|
go-version: "1.23"
|
||||||
cache-dependency-path: core/go.sum
|
|
||||||
|
|
||||||
- name: Get version
|
- name: Get version
|
||||||
id: version
|
id: version
|
||||||
@ -47,7 +46,6 @@ jobs:
|
|||||||
uses: docker/setup-qemu-action@v3
|
uses: docker/setup-qemu-action@v3
|
||||||
|
|
||||||
- name: Build binary
|
- name: Build binary
|
||||||
working-directory: core
|
|
||||||
env:
|
env:
|
||||||
GOARCH: ${{ matrix.arch }}
|
GOARCH: ${{ matrix.arch }}
|
||||||
CGO_ENABLED: 0
|
CGO_ENABLED: 0
|
||||||
@ -59,9 +57,9 @@ jobs:
|
|||||||
|
|
||||||
mkdir -p build/usr/local/bin
|
mkdir -p build/usr/local/bin
|
||||||
go build -ldflags "$LDFLAGS" -o build/usr/local/bin/orama cmd/cli/main.go
|
go build -ldflags "$LDFLAGS" -o build/usr/local/bin/orama cmd/cli/main.go
|
||||||
go build -ldflags "$LDFLAGS" -o build/usr/local/bin/orama-node cmd/node/main.go
|
go build -ldflags "$LDFLAGS" -o build/usr/local/bin/debros-node cmd/node/main.go
|
||||||
# Build the entire gateway package so helper files (e.g., config parsing) are included
|
# Build the entire gateway package so helper files (e.g., config parsing) are included
|
||||||
go build -ldflags "$LDFLAGS" -o build/usr/local/bin/orama-gateway ./cmd/gateway
|
go build -ldflags "$LDFLAGS" -o build/usr/local/bin/debros-gateway ./cmd/gateway
|
||||||
|
|
||||||
- name: Create Debian package structure
|
- name: Create Debian package structure
|
||||||
run: |
|
run: |
|
||||||
@ -73,7 +71,7 @@ jobs:
|
|||||||
mkdir -p ${PKG_NAME}/usr/local/bin
|
mkdir -p ${PKG_NAME}/usr/local/bin
|
||||||
|
|
||||||
# Copy binaries
|
# Copy binaries
|
||||||
cp core/build/usr/local/bin/* ${PKG_NAME}/usr/local/bin/
|
cp build/usr/local/bin/* ${PKG_NAME}/usr/local/bin/
|
||||||
chmod 755 ${PKG_NAME}/usr/local/bin/*
|
chmod 755 ${PKG_NAME}/usr/local/bin/*
|
||||||
|
|
||||||
# Create control file
|
# Create control file
|
||||||
@ -84,7 +82,7 @@ jobs:
|
|||||||
Priority: optional
|
Priority: optional
|
||||||
Architecture: ${ARCH}
|
Architecture: ${ARCH}
|
||||||
Depends: libc6
|
Depends: libc6
|
||||||
Maintainer: DeBros Team <team@orama.network>
|
Maintainer: DeBros Team <team@debros.network>
|
||||||
Description: Orama Network - Distributed P2P Database System
|
Description: Orama Network - Distributed P2P Database System
|
||||||
Orama is a distributed peer-to-peer network that combines
|
Orama is a distributed peer-to-peer network that combines
|
||||||
RQLite for distributed SQL, IPFS for content-addressed storage,
|
RQLite for distributed SQL, IPFS for content-addressed storage,
|
||||||
|
|||||||
33
.github/workflows/release.yaml
vendored
33
.github/workflows/release.yaml
vendored
@ -23,8 +23,8 @@ jobs:
|
|||||||
- name: Set up Go
|
- name: Set up Go
|
||||||
uses: actions/setup-go@v4
|
uses: actions/setup-go@v4
|
||||||
with:
|
with:
|
||||||
go-version: '1.24'
|
go-version: '1.21'
|
||||||
cache-dependency-path: core/go.sum
|
cache: true
|
||||||
|
|
||||||
- name: Run GoReleaser
|
- name: Run GoReleaser
|
||||||
uses: goreleaser/goreleaser-action@v5
|
uses: goreleaser/goreleaser-action@v5
|
||||||
@ -34,7 +34,6 @@ jobs:
|
|||||||
args: release --clean
|
args: release --clean
|
||||||
env:
|
env:
|
||||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
HOMEBREW_TAP_TOKEN: ${{ secrets.HOMEBREW_TAP_TOKEN }}
|
|
||||||
|
|
||||||
- name: Upload artifacts
|
- name: Upload artifacts
|
||||||
uses: actions/upload-artifact@v4
|
uses: actions/upload-artifact@v4
|
||||||
@ -43,26 +42,32 @@ jobs:
|
|||||||
path: dist/
|
path: dist/
|
||||||
retention-days: 5
|
retention-days: 5
|
||||||
|
|
||||||
# Verify release artifacts
|
# Optional: Publish to GitHub Packages (requires additional setup)
|
||||||
verify-release:
|
publish-packages:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
needs: build-release
|
needs: build-release
|
||||||
if: startsWith(github.ref, 'refs/tags/')
|
if: startsWith(github.ref, 'refs/tags/')
|
||||||
|
|
||||||
steps:
|
steps:
|
||||||
|
- name: Checkout code
|
||||||
|
uses: actions/checkout@v4
|
||||||
|
|
||||||
- name: Download artifacts
|
- name: Download artifacts
|
||||||
uses: actions/download-artifact@v4
|
uses: actions/download-artifact@v4
|
||||||
with:
|
with:
|
||||||
name: release-artifacts
|
name: release-artifacts
|
||||||
path: dist/
|
path: dist/
|
||||||
|
|
||||||
- name: List release artifacts
|
- name: Publish to GitHub Packages
|
||||||
|
env:
|
||||||
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
run: |
|
run: |
|
||||||
echo "=== Release Artifacts ==="
|
echo "Publishing Debian packages to GitHub Packages..."
|
||||||
ls -la dist/
|
for deb in dist/*.deb; do
|
||||||
echo ""
|
if [ -f "$deb" ]; then
|
||||||
echo "=== .deb packages ==="
|
curl -H "Authorization: token $GITHUB_TOKEN" \
|
||||||
ls -la dist/*.deb 2>/dev/null || echo "No .deb files found"
|
-H "Content-Type: application/octet-stream" \
|
||||||
echo ""
|
--data-binary @"$deb" \
|
||||||
echo "=== Archives ==="
|
"https://uploads.github.com/repos/${{ github.repository }}/releases/upload?name=$(basename "$deb")"
|
||||||
ls -la dist/*.tar.gz 2>/dev/null || echo "No .tar.gz files found"
|
fi
|
||||||
|
done
|
||||||
|
|||||||
131
.gitignore
vendored
131
.gitignore
vendored
@ -1,4 +1,34 @@
|
|||||||
# === Global ===
|
# Binaries for programs and plugins
|
||||||
|
*.exe
|
||||||
|
*.exe~
|
||||||
|
*.dll
|
||||||
|
*.so
|
||||||
|
*.dylib
|
||||||
|
|
||||||
|
# Test binary, built with `go test -c`
|
||||||
|
*.test
|
||||||
|
|
||||||
|
# Output of the go coverage tool, specifically when used with LiteIDE
|
||||||
|
*.out
|
||||||
|
|
||||||
|
# Dependency directories (remove the comment below to include it)
|
||||||
|
# vendor/
|
||||||
|
|
||||||
|
# Go workspace file
|
||||||
|
go.work
|
||||||
|
|
||||||
|
# Built binaries
|
||||||
|
bin/
|
||||||
|
dist/
|
||||||
|
|
||||||
|
# IDE and editor files
|
||||||
|
.vscode/
|
||||||
|
.idea/
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
|
*~
|
||||||
|
|
||||||
|
# OS generated files
|
||||||
.DS_Store
|
.DS_Store
|
||||||
.DS_Store?
|
.DS_Store?
|
||||||
._*
|
._*
|
||||||
@ -6,85 +36,44 @@
|
|||||||
.Trashes
|
.Trashes
|
||||||
ehthumbs.db
|
ehthumbs.db
|
||||||
Thumbs.db
|
Thumbs.db
|
||||||
*.swp
|
|
||||||
*.swo
|
|
||||||
*~
|
|
||||||
|
|
||||||
# IDE
|
# Log files
|
||||||
.vscode/
|
*.log
|
||||||
.idea/
|
|
||||||
.cursor/
|
|
||||||
|
|
||||||
# Environment & credentials
|
# Environment variables
|
||||||
.env
|
.env
|
||||||
.env.*
|
.env.local
|
||||||
!.env.example
|
.env.*.local
|
||||||
.mcp.json
|
|
||||||
.claude/
|
|
||||||
.codex/
|
|
||||||
|
|
||||||
# === Core (Go) ===
|
# Temporary files
|
||||||
core/phantom-auth/
|
tmp/
|
||||||
core/bin/
|
temp/
|
||||||
core/bin-linux/
|
*.tmp
|
||||||
core/dist/
|
|
||||||
core/orama-cli-linux
|
|
||||||
core/keys_backup/
|
|
||||||
core/.gocache/
|
|
||||||
core/configs/
|
|
||||||
core/data/*
|
|
||||||
core/tmp/
|
|
||||||
core/temp/
|
|
||||||
core/results/
|
|
||||||
core/rnd/
|
|
||||||
core/vps.txt
|
|
||||||
core/coverage.txt
|
|
||||||
core/coverage.html
|
|
||||||
core/profile.out
|
|
||||||
core/e2e/config.yaml
|
|
||||||
core/scripts/remote-nodes.conf
|
|
||||||
|
|
||||||
# Go build artifacts
|
# Coverage reports
|
||||||
*.exe
|
coverage.txt
|
||||||
*.exe~
|
coverage.html
|
||||||
*.dll
|
profile.out
|
||||||
*.so
|
|
||||||
*.dylib
|
# Build artifacts
|
||||||
*.test
|
|
||||||
*.out
|
|
||||||
*.deb
|
*.deb
|
||||||
*.rpm
|
*.rpm
|
||||||
*.tar.gz
|
*.tar.gz
|
||||||
*.zip
|
*.zip
|
||||||
go.work
|
|
||||||
|
|
||||||
# Logs
|
# Local development files
|
||||||
*.log
|
|
||||||
|
|
||||||
# Databases
|
|
||||||
*.db
|
|
||||||
|
|
||||||
# === Website ===
|
|
||||||
website/node_modules/
|
|
||||||
website/dist/
|
|
||||||
website/invest-api/invest-api
|
|
||||||
website/invest-api/*.db
|
|
||||||
website/invest-api/*.db-shm
|
|
||||||
website/invest-api/*.db-wal
|
|
||||||
|
|
||||||
# === SDK (TypeScript) ===
|
|
||||||
sdk/node_modules/
|
|
||||||
sdk/dist/
|
|
||||||
sdk/coverage/
|
|
||||||
|
|
||||||
# === Vault (Zig) ===
|
|
||||||
vault/.zig-cache/
|
|
||||||
vault/zig-out/
|
|
||||||
|
|
||||||
# === OS ===
|
|
||||||
os/output/
|
|
||||||
|
|
||||||
# === Local development ===
|
|
||||||
.dev/
|
|
||||||
.local/
|
.local/
|
||||||
local/
|
local/
|
||||||
|
|
||||||
|
data/*
|
||||||
|
./bootstrap
|
||||||
|
./node
|
||||||
|
data/bootstrap/rqlite/
|
||||||
|
|
||||||
|
.env.*
|
||||||
|
|
||||||
|
configs/
|
||||||
|
|
||||||
|
.dev/
|
||||||
|
|
||||||
|
.gocache/
|
||||||
121
.goreleaser.yaml
121
.goreleaser.yaml
@ -1,23 +1,17 @@
|
|||||||
# GoReleaser Configuration for DeBros Network
|
# GoReleaser Configuration for DeBros Network
|
||||||
# Builds and releases orama (CLI) and orama-node binaries
|
# Builds and releases the dbn binary for multiple platforms
|
||||||
# Publishes to: GitHub Releases, Homebrew, and apt (.deb packages)
|
# Other binaries (node, gateway, identity) are installed via: dbn setup
|
||||||
|
|
||||||
project_name: orama-network
|
project_name: debros-network
|
||||||
|
|
||||||
env:
|
env:
|
||||||
- GO111MODULE=on
|
- GO111MODULE=on
|
||||||
|
|
||||||
before:
|
|
||||||
hooks:
|
|
||||||
- cmd: go mod tidy
|
|
||||||
dir: core
|
|
||||||
|
|
||||||
builds:
|
builds:
|
||||||
# orama CLI binary
|
# dbn binary - only build the CLI
|
||||||
- id: orama
|
- id: dbn
|
||||||
dir: core
|
|
||||||
main: ./cmd/cli
|
main: ./cmd/cli
|
||||||
binary: orama
|
binary: dbn
|
||||||
goos:
|
goos:
|
||||||
- linux
|
- linux
|
||||||
- darwin
|
- darwin
|
||||||
@ -31,107 +25,18 @@ builds:
|
|||||||
- -X main.date={{.Date}}
|
- -X main.date={{.Date}}
|
||||||
mod_timestamp: "{{ .CommitTimestamp }}"
|
mod_timestamp: "{{ .CommitTimestamp }}"
|
||||||
|
|
||||||
# orama-node binary (Linux only for apt)
|
|
||||||
- id: orama-node
|
|
||||||
dir: core
|
|
||||||
main: ./cmd/node
|
|
||||||
binary: orama-node
|
|
||||||
goos:
|
|
||||||
- linux
|
|
||||||
goarch:
|
|
||||||
- amd64
|
|
||||||
- arm64
|
|
||||||
ldflags:
|
|
||||||
- -s -w
|
|
||||||
- -X main.version={{.Version}}
|
|
||||||
- -X main.commit={{.ShortCommit}}
|
|
||||||
- -X main.date={{.Date}}
|
|
||||||
mod_timestamp: "{{ .CommitTimestamp }}"
|
|
||||||
|
|
||||||
archives:
|
archives:
|
||||||
# Tar.gz archives for orama CLI
|
# Tar.gz archives for dbn
|
||||||
- id: orama-archives
|
- id: binaries
|
||||||
builds:
|
|
||||||
- orama
|
|
||||||
format: tar.gz
|
format: tar.gz
|
||||||
name_template: "orama_{{ .Version }}_{{ .Os }}_{{ .Arch }}"
|
name_template: "{{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}"
|
||||||
files:
|
files:
|
||||||
- README.md
|
- README.md
|
||||||
- LICENSE
|
- LICENSE
|
||||||
|
- CHANGELOG.md
|
||||||
# Tar.gz archives for orama-node
|
format_overrides:
|
||||||
- id: orama-node-archives
|
- goos: windows
|
||||||
builds:
|
format: zip
|
||||||
- orama-node
|
|
||||||
format: tar.gz
|
|
||||||
name_template: "orama-node_{{ .Version }}_{{ .Os }}_{{ .Arch }}"
|
|
||||||
files:
|
|
||||||
- README.md
|
|
||||||
- LICENSE
|
|
||||||
|
|
||||||
# Debian packages for apt
|
|
||||||
nfpms:
|
|
||||||
# orama CLI .deb package
|
|
||||||
- id: orama-deb
|
|
||||||
package_name: orama
|
|
||||||
builds:
|
|
||||||
- orama
|
|
||||||
vendor: DeBros
|
|
||||||
homepage: https://github.com/DeBrosOfficial/network
|
|
||||||
maintainer: DeBros <dev@debros.io>
|
|
||||||
description: CLI tool for the Orama decentralized network
|
|
||||||
license: MIT
|
|
||||||
formats:
|
|
||||||
- deb
|
|
||||||
bindir: /usr/bin
|
|
||||||
section: utils
|
|
||||||
priority: optional
|
|
||||||
contents:
|
|
||||||
- src: ./core/README.md
|
|
||||||
dst: /usr/share/doc/orama/README.md
|
|
||||||
deb:
|
|
||||||
lintian_overrides:
|
|
||||||
- statically-linked-binary
|
|
||||||
|
|
||||||
# orama-node .deb package
|
|
||||||
- id: orama-node-deb
|
|
||||||
package_name: orama-node
|
|
||||||
builds:
|
|
||||||
- orama-node
|
|
||||||
vendor: DeBros
|
|
||||||
homepage: https://github.com/DeBrosOfficial/network
|
|
||||||
maintainer: DeBros <dev@debros.io>
|
|
||||||
description: Node daemon for the Orama decentralized network
|
|
||||||
license: MIT
|
|
||||||
formats:
|
|
||||||
- deb
|
|
||||||
bindir: /usr/bin
|
|
||||||
section: net
|
|
||||||
priority: optional
|
|
||||||
contents:
|
|
||||||
- src: ./core/README.md
|
|
||||||
dst: /usr/share/doc/orama-node/README.md
|
|
||||||
deb:
|
|
||||||
lintian_overrides:
|
|
||||||
- statically-linked-binary
|
|
||||||
|
|
||||||
# Homebrew tap for macOS (orama CLI only)
|
|
||||||
brews:
|
|
||||||
- name: orama
|
|
||||||
ids:
|
|
||||||
- orama-archives
|
|
||||||
repository:
|
|
||||||
owner: DeBrosOfficial
|
|
||||||
name: homebrew-tap
|
|
||||||
token: "{{ .Env.HOMEBREW_TAP_TOKEN }}"
|
|
||||||
folder: Formula
|
|
||||||
homepage: https://github.com/DeBrosOfficial/network
|
|
||||||
description: CLI tool for the Orama decentralized network
|
|
||||||
license: MIT
|
|
||||||
install: |
|
|
||||||
bin.install "orama"
|
|
||||||
test: |
|
|
||||||
system "#{bin}/orama", "--version"
|
|
||||||
|
|
||||||
checksum:
|
checksum:
|
||||||
name_template: "checksums.txt"
|
name_template: "checksums.txt"
|
||||||
|
|||||||
1698
CHANGELOG.md
Normal file
1698
CHANGELOG.md
Normal file
File diff suppressed because it is too large
Load Diff
@ -32,7 +32,7 @@ This Code applies within all project spaces and when an individual is officially
|
|||||||
|
|
||||||
## Enforcement
|
## Enforcement
|
||||||
|
|
||||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the maintainers at: security@orama.io
|
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported to the maintainers at: security@debros.io
|
||||||
|
|
||||||
All complaints will be reviewed and investigated promptly and fairly.
|
All complaints will be reviewed and investigated promptly and fairly.
|
||||||
|
|
||||||
|
|||||||
@ -1,78 +1,47 @@
|
|||||||
# Contributing to Orama Network
|
# Contributing to DeBros Network
|
||||||
|
|
||||||
Thanks for helping improve the network! This monorepo contains multiple projects — pick the one relevant to your contribution.
|
Thanks for helping improve the network! This guide covers setup, local dev, tests, and PR guidelines.
|
||||||
|
|
||||||
## Repository Structure
|
## Requirements
|
||||||
|
|
||||||
| Package | Language | Build |
|
- Go 1.22+ (1.23 recommended)
|
||||||
|---------|----------|-------|
|
- RQLite (optional for local runs; the Makefile starts nodes with embedded setup)
|
||||||
| `core/` | Go 1.24+ | `make core-build` |
|
- Make (optional)
|
||||||
| `website/` | TypeScript (pnpm) | `make website-build` |
|
|
||||||
| `vault/` | Zig 0.14+ | `make vault-build` |
|
|
||||||
| `os/` | Go + Buildroot | `make os-build` |
|
|
||||||
|
|
||||||
## Setup
|
## Setup
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/DeBrosOfficial/network.git
|
git clone https://github.com/DeBrosOfficial/network.git
|
||||||
cd network
|
cd network
|
||||||
```
|
|
||||||
|
|
||||||
### Core (Go)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd core
|
|
||||||
make deps
|
make deps
|
||||||
make build
|
|
||||||
make test
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Website
|
## Build, Test, Lint
|
||||||
|
|
||||||
|
- Build: `make build`
|
||||||
|
- Test: `make test`
|
||||||
|
- Format/Vet: `make fmt vet` (or `make lint`)
|
||||||
|
|
||||||
|
````
|
||||||
|
|
||||||
|
Useful CLI commands:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd website
|
./bin/orama health
|
||||||
pnpm install
|
./bin/orama peers
|
||||||
pnpm dev
|
./bin/orama status
|
||||||
```
|
````
|
||||||
|
|
||||||
### Vault (Zig)
|
## Versioning
|
||||||
|
|
||||||
```bash
|
- The CLI reports its version via `orama version`.
|
||||||
cd vault
|
- Releases are tagged (e.g., `v0.18.0-beta`) and published via GoReleaser.
|
||||||
zig build
|
|
||||||
zig build test
|
|
||||||
```
|
|
||||||
|
|
||||||
## Pull Requests
|
## Pull Requests
|
||||||
|
|
||||||
1. Fork and create a topic branch from `main`.
|
1. Fork and create a topic branch.
|
||||||
2. Ensure `make test` passes for affected packages.
|
2. Ensure `make build test` passes; include tests for new functionality.
|
||||||
3. Include tests for new functionality or bug fixes.
|
3. Keep PRs focused and well-described (motivation, approach, testing).
|
||||||
4. Keep PRs focused — one concern per PR.
|
4. Update README/docs for behavior changes.
|
||||||
5. Write a clear description: motivation, approach, and how you tested it.
|
|
||||||
6. Update docs if you're changing user-facing behavior.
|
|
||||||
|
|
||||||
## Code Style
|
|
||||||
|
|
||||||
### Go (core/, os/)
|
|
||||||
|
|
||||||
- Follow standard Go conventions
|
|
||||||
- Run `make lint` before submitting
|
|
||||||
- Wrap errors with context: `fmt.Errorf("failed to X: %w", err)`
|
|
||||||
- No magic values — use named constants
|
|
||||||
|
|
||||||
### TypeScript (website/)
|
|
||||||
|
|
||||||
- TypeScript strict mode
|
|
||||||
- Follow existing patterns in the codebase
|
|
||||||
|
|
||||||
### Zig (vault/)
|
|
||||||
|
|
||||||
- Follow standard Zig conventions
|
|
||||||
- Run `zig build test` before submitting
|
|
||||||
|
|
||||||
## Security
|
|
||||||
|
|
||||||
If you find a security vulnerability, **do not open a public issue**. Email security@debros.io instead.
|
|
||||||
|
|
||||||
Thank you for contributing!
|
Thank you for contributing!
|
||||||
|
|||||||
150
Makefile
150
Makefile
@ -1,66 +1,122 @@
|
|||||||
# Orama Monorepo
|
TEST?=./...
|
||||||
# Delegates to sub-project Makefiles
|
|
||||||
|
|
||||||
.PHONY: help build test clean
|
.PHONY: test
|
||||||
|
test:
|
||||||
|
@echo Running tests...
|
||||||
|
go test -v $(TEST)
|
||||||
|
|
||||||
# === Core (Go network) ===
|
# Gateway-focused E2E tests assume gateway and nodes are already running
|
||||||
.PHONY: core core-build core-test core-clean core-lint
|
# Auto-discovers configuration from ~/.orama and queries database for API key
|
||||||
core: core-build
|
# No environment variables required
|
||||||
|
.PHONY: test-e2e
|
||||||
|
test-e2e:
|
||||||
|
@echo "Running comprehensive E2E tests..."
|
||||||
|
@echo "Auto-discovering configuration from ~/.orama..."
|
||||||
|
go test -v -tags e2e ./e2e
|
||||||
|
|
||||||
core-build:
|
# Network - Distributed P2P Database System
|
||||||
$(MAKE) -C core build
|
# Makefile for development and build tasks
|
||||||
|
|
||||||
core-test:
|
.PHONY: build clean test run-node run-node2 run-node3 run-example deps tidy fmt vet lint clear-ports install-hooks kill
|
||||||
$(MAKE) -C core test
|
|
||||||
|
|
||||||
core-lint:
|
VERSION := 0.82.0
|
||||||
$(MAKE) -C core lint
|
COMMIT ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo unknown)
|
||||||
|
DATE ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
|
||||||
|
LDFLAGS := -X 'main.version=$(VERSION)' -X 'main.commit=$(COMMIT)' -X 'main.date=$(DATE)'
|
||||||
|
|
||||||
core-clean:
|
# Build targets
|
||||||
$(MAKE) -C core clean
|
build: deps
|
||||||
|
@echo "Building network executables (version=$(VERSION))..."
|
||||||
|
@mkdir -p bin
|
||||||
|
go build -ldflags "$(LDFLAGS)" -o bin/identity ./cmd/identity
|
||||||
|
go build -ldflags "$(LDFLAGS)" -o bin/orama-node ./cmd/node
|
||||||
|
go build -ldflags "$(LDFLAGS)" -o bin/orama cmd/cli/main.go
|
||||||
|
go build -ldflags "$(LDFLAGS)" -o bin/rqlite-mcp ./cmd/rqlite-mcp
|
||||||
|
# Inject gateway build metadata via pkg path variables
|
||||||
|
go build -ldflags "$(LDFLAGS) -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildVersion=$(VERSION)' -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildCommit=$(COMMIT)' -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildTime=$(DATE)'" -o bin/gateway ./cmd/gateway
|
||||||
|
@echo "Build complete! Run ./bin/orama version"
|
||||||
|
|
||||||
# === Website ===
|
# Install git hooks
|
||||||
.PHONY: website website-dev website-build
|
install-hooks:
|
||||||
website-dev:
|
@echo "Installing git hooks..."
|
||||||
cd website && pnpm dev
|
@bash scripts/install-hooks.sh
|
||||||
|
|
||||||
website-build:
|
# Clean build artifacts
|
||||||
cd website && pnpm build
|
clean:
|
||||||
|
@echo "Cleaning build artifacts..."
|
||||||
|
rm -rf bin/
|
||||||
|
rm -rf data/
|
||||||
|
@echo "Clean complete!"
|
||||||
|
|
||||||
# === SDK (TypeScript) ===
|
# Run bootstrap node (auto-selects identity and data dir)
|
||||||
.PHONY: sdk sdk-build sdk-test
|
run-node:
|
||||||
sdk: sdk-build
|
@echo "Starting node..."
|
||||||
|
@echo "Config: ~/.orama/node.yaml"
|
||||||
|
go run ./cmd/orama-node --config node.yaml
|
||||||
|
|
||||||
sdk-build:
|
# Run second node - requires join address
|
||||||
cd sdk && pnpm install && pnpm build
|
run-node2:
|
||||||
|
@echo "Starting second node..."
|
||||||
|
@echo "Config: ~/.orama/node2.yaml"
|
||||||
|
go run ./cmd/orama-node --config node2.yaml
|
||||||
|
|
||||||
sdk-test:
|
# Run third node - requires join address
|
||||||
cd sdk && pnpm test
|
run-node3:
|
||||||
|
@echo "Starting third node..."
|
||||||
|
@echo "Config: ~/.orama/node3.yaml"
|
||||||
|
go run ./cmd/orama-node --config node3.yaml
|
||||||
|
|
||||||
# === Vault (Zig) ===
|
# Run gateway HTTP server
|
||||||
.PHONY: vault vault-build vault-test
|
run-gateway:
|
||||||
vault-build:
|
@echo "Starting gateway HTTP server..."
|
||||||
cd vault && zig build
|
@echo "Note: Config must be in ~/.orama/data/gateway.yaml"
|
||||||
|
go run ./cmd/orama-gateway
|
||||||
|
|
||||||
vault-test:
|
# Development environment target
|
||||||
cd vault && zig build test
|
# Uses orama dev up to start full stack with dependency and port checking
|
||||||
|
dev: build
|
||||||
|
@./bin/orama dev up
|
||||||
|
|
||||||
# === OS ===
|
# Graceful shutdown of all dev services
|
||||||
.PHONY: os os-build
|
stop:
|
||||||
os-build:
|
@if [ -f ./bin/orama ]; then \
|
||||||
$(MAKE) -C os
|
./bin/orama dev down || true; \
|
||||||
|
fi
|
||||||
|
@bash scripts/dev-kill-all.sh
|
||||||
|
|
||||||
# === Aggregate ===
|
# Force kill all processes (immediate termination)
|
||||||
build: core-build
|
kill:
|
||||||
test: core-test
|
@bash scripts/dev-kill-all.sh
|
||||||
clean: core-clean
|
|
||||||
|
|
||||||
|
# Help
|
||||||
help:
|
help:
|
||||||
@echo "Orama Monorepo"
|
@echo "Available targets:"
|
||||||
|
@echo " build - Build all executables"
|
||||||
|
@echo " clean - Clean build artifacts"
|
||||||
|
@echo " test - Run tests"
|
||||||
@echo ""
|
@echo ""
|
||||||
@echo " Core (Go): make core-build | core-test | core-lint | core-clean"
|
@echo "Local Development (Recommended):"
|
||||||
@echo " Website: make website-dev | website-build"
|
@echo " make dev - Start full development stack with one command"
|
||||||
@echo " Vault (Zig): make vault-build | vault-test"
|
@echo " - Checks dependencies and available ports"
|
||||||
@echo " OS: make os-build"
|
@echo " - Generates configs and starts all services"
|
||||||
|
@echo " - Validates cluster health"
|
||||||
|
@echo " make stop - Gracefully stop all development services"
|
||||||
|
@echo " make kill - Force kill all development services (use if stop fails)"
|
||||||
@echo ""
|
@echo ""
|
||||||
@echo " Aggregate: make build | test | clean (delegates to core)"
|
@echo "Development Management (via orama):"
|
||||||
|
@echo " ./bin/orama dev status - Show status of all dev services"
|
||||||
|
@echo " ./bin/orama dev logs <component> [--follow]"
|
||||||
|
@echo ""
|
||||||
|
@echo "Individual Node Targets (advanced):"
|
||||||
|
@echo " run-node - Start first node directly"
|
||||||
|
@echo " run-node2 - Start second node directly"
|
||||||
|
@echo " run-node3 - Start third node directly"
|
||||||
|
@echo " run-gateway - Start HTTP gateway directly"
|
||||||
|
@echo ""
|
||||||
|
@echo "Maintenance:"
|
||||||
|
@echo " deps - Download dependencies"
|
||||||
|
@echo " tidy - Tidy dependencies"
|
||||||
|
@echo " fmt - Format code"
|
||||||
|
@echo " vet - Vet code"
|
||||||
|
@echo " lint - Lint code (fmt + vet)"
|
||||||
|
@echo " help - Show this help"
|
||||||
|
|||||||
348
README.md
348
README.md
@ -1,50 +1,324 @@
|
|||||||
# Orama Network
|
# DeBros Network - Distributed P2P Database System
|
||||||
|
|
||||||
A decentralized infrastructure platform combining distributed SQL, IPFS storage, caching, serverless WASM execution, and privacy relay — all managed through a unified API gateway.
|
A decentralized peer-to-peer data platform built in Go. Combines distributed SQL (RQLite), pub/sub messaging, and resilient peer discovery so applications can share state without central infrastructure.
|
||||||
|
|
||||||
## Packages
|
|
||||||
|
|
||||||
| Package | Language | Description |
|
|
||||||
|---------|----------|-------------|
|
|
||||||
| [core/](core/) | Go | API gateway, distributed node, CLI, and client SDK |
|
|
||||||
| [sdk/](sdk/) | TypeScript | `@debros/orama` — JavaScript/TypeScript SDK ([npm](https://www.npmjs.com/package/@debros/orama)) |
|
|
||||||
| [website/](website/) | TypeScript | Marketing website and invest portal |
|
|
||||||
| [vault/](vault/) | Zig | Distributed secrets vault (Shamir's Secret Sharing) |
|
|
||||||
| [os/](os/) | Go + Buildroot | OramaOS — hardened minimal Linux for network nodes |
|
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
|
### Local Development
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Build the core network binaries
|
# Build the project
|
||||||
make core-build
|
make build
|
||||||
|
|
||||||
# Run tests
|
# Start 5-node development cluster
|
||||||
make core-test
|
make dev
|
||||||
|
|
||||||
# Start website dev server
|
|
||||||
make website-dev
|
|
||||||
|
|
||||||
# Build vault
|
|
||||||
make vault-build
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Documentation
|
The cluster automatically performs health checks before declaring success.
|
||||||
|
|
||||||
| Document | Description |
|
### Stop Development Environment
|
||||||
|----------|-------------|
|
|
||||||
| [Architecture](core/docs/ARCHITECTURE.md) | System architecture and design patterns |
|
|
||||||
| [Deployment Guide](core/docs/DEPLOYMENT_GUIDE.md) | Deploy apps, databases, and domains |
|
|
||||||
| [Dev & Deploy](core/docs/DEV_DEPLOY.md) | Building, deploying to VPS, rolling upgrades |
|
|
||||||
| [Security](core/docs/SECURITY.md) | Security hardening and threat model |
|
|
||||||
| [Monitoring](core/docs/MONITORING.md) | Cluster health monitoring |
|
|
||||||
| [Client SDK](core/docs/CLIENT_SDK.md) | Go SDK documentation |
|
|
||||||
| [Serverless](core/docs/SERVERLESS.md) | WASM serverless functions |
|
|
||||||
| [Common Problems](core/docs/COMMON_PROBLEMS.md) | Troubleshooting known issues |
|
|
||||||
|
|
||||||
## Contributing
|
```bash
|
||||||
|
make stop
|
||||||
|
```
|
||||||
|
|
||||||
See [CONTRIBUTING.md](CONTRIBUTING.md) for setup, development, and PR guidelines.
|
## Testing Services
|
||||||
|
|
||||||
## License
|
After running `make dev`, test service health using these curl requests:
|
||||||
|
|
||||||
[AGPL-3.0](LICENSE)
|
### Node Unified Gateways
|
||||||
|
|
||||||
|
Each node is accessible via a single unified gateway port:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Node-1 (port 6001)
|
||||||
|
curl http://localhost:6001/health
|
||||||
|
|
||||||
|
# Node-2 (port 6002)
|
||||||
|
curl http://localhost:6002/health
|
||||||
|
|
||||||
|
# Node-3 (port 6003)
|
||||||
|
curl http://localhost:6003/health
|
||||||
|
|
||||||
|
# Node-4 (port 6004)
|
||||||
|
curl http://localhost:6004/health
|
||||||
|
|
||||||
|
# Node-5 (port 6005)
|
||||||
|
curl http://localhost:6005/health
|
||||||
|
```
|
||||||
|
|
||||||
|
## Network Architecture
|
||||||
|
|
||||||
|
### Unified Gateway Ports
|
||||||
|
|
||||||
|
```
|
||||||
|
Node-1: localhost:6001 → /rqlite/http, /rqlite/raft, /cluster, /ipfs/api
|
||||||
|
Node-2: localhost:6002 → Same routes
|
||||||
|
Node-3: localhost:6003 → Same routes
|
||||||
|
Node-4: localhost:6004 → Same routes
|
||||||
|
Node-5: localhost:6005 → Same routes
|
||||||
|
```
|
||||||
|
|
||||||
|
### Direct Service Ports (for debugging)
|
||||||
|
|
||||||
|
```
|
||||||
|
RQLite HTTP: 5001, 5002, 5003, 5004, 5005 (one per node)
|
||||||
|
RQLite Raft: 7001, 7002, 7003, 7004, 7005
|
||||||
|
IPFS API: 4501, 4502, 4503, 4504, 4505
|
||||||
|
IPFS Swarm: 4101, 4102, 4103, 4104, 4105
|
||||||
|
Cluster API: 9094, 9104, 9114, 9124, 9134
|
||||||
|
Internal Gateway: 6000
|
||||||
|
Olric Cache: 3320
|
||||||
|
Anon SOCKS: 9050
|
||||||
|
```
|
||||||
|
|
||||||
|
## Development Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Start full cluster (5 nodes + gateway)
|
||||||
|
make dev
|
||||||
|
|
||||||
|
# Check service status
|
||||||
|
orama dev status
|
||||||
|
|
||||||
|
# View logs
|
||||||
|
orama dev logs node-1 # Node-1 logs
|
||||||
|
orama dev logs node-1 --follow # Follow logs in real-time
|
||||||
|
orama dev logs gateway --follow # Gateway logs
|
||||||
|
|
||||||
|
# Stop all services
|
||||||
|
orama stop
|
||||||
|
|
||||||
|
# Build binaries
|
||||||
|
make build
|
||||||
|
```
|
||||||
|
|
||||||
|
## CLI Commands
|
||||||
|
|
||||||
|
### Network Status
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/orama health # Cluster health check
|
||||||
|
./bin/orama peers # List connected peers
|
||||||
|
./bin/orama status # Network status
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Operations
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/orama query "SELECT * FROM users"
|
||||||
|
./bin/orama query "CREATE TABLE users (id INTEGER PRIMARY KEY)"
|
||||||
|
./bin/orama transaction --file ops.json
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pub/Sub
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/orama pubsub publish <topic> <message>
|
||||||
|
./bin/orama pubsub subscribe <topic> 30s
|
||||||
|
./bin/orama pubsub topics
|
||||||
|
```
|
||||||
|
|
||||||
|
### Authentication
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./bin/orama auth login
|
||||||
|
./bin/orama auth status
|
||||||
|
./bin/orama auth logout
|
||||||
|
```
|
||||||
|
|
||||||
|
## Serverless Functions (WASM)
|
||||||
|
|
||||||
|
Orama supports high-performance serverless function execution using WebAssembly (WASM). Functions are isolated, secure, and can interact with network services like the distributed cache.
|
||||||
|
|
||||||
|
### 1. Build Functions
|
||||||
|
|
||||||
|
Functions must be compiled to WASM. We recommend using [TinyGo](https://tinygo.org/).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build example functions to examples/functions/bin/
|
||||||
|
./examples/functions/build.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Deployment
|
||||||
|
|
||||||
|
Deploy your compiled `.wasm` file to the network via the Gateway.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Deploy a function
|
||||||
|
curl -X POST http://localhost:6001/v1/functions \
|
||||||
|
-H "Authorization: Bearer <your_api_key>" \
|
||||||
|
-F "name=hello-world" \
|
||||||
|
-F "namespace=default" \
|
||||||
|
-F "wasm=@./examples/functions/bin/hello.wasm"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Invocation
|
||||||
|
|
||||||
|
Trigger your function with a JSON payload. The function receives the payload via `stdin` and returns its response via `stdout`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Invoke via HTTP
|
||||||
|
curl -X POST http://localhost:6001/v1/functions/hello-world/invoke \
|
||||||
|
-H "Authorization: Bearer <your_api_key>" \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"name": "Developer"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Management
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List all functions in a namespace
|
||||||
|
curl http://localhost:6001/v1/functions?namespace=default
|
||||||
|
|
||||||
|
# Delete a function
|
||||||
|
curl -X DELETE http://localhost:6001/v1/functions/hello-world?namespace=default
|
||||||
|
```
|
||||||
|
|
||||||
|
## Production Deployment
|
||||||
|
|
||||||
|
### Prerequisites
|
||||||
|
|
||||||
|
- Ubuntu 22.04+ or Debian 12+
|
||||||
|
- `amd64` or `arm64` architecture
|
||||||
|
- 4GB RAM, 50GB SSD, 2 CPU cores
|
||||||
|
|
||||||
|
### Required Ports
|
||||||
|
|
||||||
|
**External (must be open in firewall):**
|
||||||
|
|
||||||
|
- **80** - HTTP (ACME/Let's Encrypt certificate challenges)
|
||||||
|
- **443** - HTTPS (Main gateway API endpoint)
|
||||||
|
- **4101** - IPFS Swarm (peer connections)
|
||||||
|
- **7001** - RQLite Raft (cluster consensus)
|
||||||
|
|
||||||
|
**Internal (bound to localhost, no firewall needed):**
|
||||||
|
|
||||||
|
- 4501 - IPFS API
|
||||||
|
- 5001 - RQLite HTTP API
|
||||||
|
- 6001 - Unified Gateway
|
||||||
|
- 8080 - IPFS Gateway
|
||||||
|
- 9050 - Anyone Client SOCKS5 proxy
|
||||||
|
- 9094 - IPFS Cluster API
|
||||||
|
- 3320/3322 - Olric Cache
|
||||||
|
|
||||||
|
### Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Install via APT
|
||||||
|
echo "deb https://debrosficial.github.io/network/apt stable main" | sudo tee /etc/apt/sources.list.d/debros.list
|
||||||
|
|
||||||
|
sudo apt update && sudo apt install orama
|
||||||
|
|
||||||
|
sudo orama install --interactive
|
||||||
|
```
|
||||||
|
|
||||||
|
### Service Management
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Status
|
||||||
|
orama status
|
||||||
|
|
||||||
|
# Control services
|
||||||
|
sudo orama start
|
||||||
|
sudo orama stop
|
||||||
|
sudo orama restart
|
||||||
|
|
||||||
|
# View logs
|
||||||
|
orama logs node --follow
|
||||||
|
orama logs gateway --follow
|
||||||
|
orama logs ipfs --follow
|
||||||
|
```
|
||||||
|
|
||||||
|
### Upgrade
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Upgrade to latest version
|
||||||
|
sudo orama upgrade --interactive
|
||||||
|
```
|
||||||
|
|
||||||
|
## Configuration
|
||||||
|
|
||||||
|
All configuration lives in `~/.orama/`:
|
||||||
|
|
||||||
|
- `configs/node.yaml` - Node configuration
|
||||||
|
- `configs/gateway.yaml` - Gateway configuration
|
||||||
|
- `configs/olric.yaml` - Cache configuration
|
||||||
|
- `secrets/` - Keys and certificates
|
||||||
|
- `data/` - Service data directories
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Services Not Starting
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check status
|
||||||
|
systemctl status debros-node
|
||||||
|
|
||||||
|
# View logs
|
||||||
|
journalctl -u debros-node -f
|
||||||
|
|
||||||
|
# Check log files
|
||||||
|
tail -f /home/debros/.orama/logs/node.log
|
||||||
|
```
|
||||||
|
|
||||||
|
### Port Conflicts
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check what's using specific ports
|
||||||
|
sudo lsof -i :443 # HTTPS Gateway
|
||||||
|
sudo lsof -i :7001 # TCP/SNI Gateway
|
||||||
|
sudo lsof -i :6001 # Internal Gateway
|
||||||
|
```
|
||||||
|
|
||||||
|
### RQLite Cluster Issues
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Connect to RQLite CLI
|
||||||
|
rqlite -H localhost -p 5001
|
||||||
|
|
||||||
|
# Check cluster status
|
||||||
|
.nodes
|
||||||
|
.status
|
||||||
|
.ready
|
||||||
|
|
||||||
|
# Check consistency level
|
||||||
|
.consistency
|
||||||
|
```
|
||||||
|
|
||||||
|
### Reset Installation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Production reset (⚠️ DESTROYS DATA)
|
||||||
|
sudo orama uninstall
|
||||||
|
sudo rm -rf /home/debros/.orama
|
||||||
|
sudo orama install
|
||||||
|
```
|
||||||
|
|
||||||
|
## HTTP Gateway API
|
||||||
|
|
||||||
|
### Main Gateway Endpoints
|
||||||
|
|
||||||
|
- `GET /health` - Health status
|
||||||
|
- `GET /v1/status` - Full status
|
||||||
|
- `GET /v1/version` - Version info
|
||||||
|
- `POST /v1/rqlite/exec` - Execute SQL
|
||||||
|
- `POST /v1/rqlite/query` - Query database
|
||||||
|
- `GET /v1/rqlite/schema` - Get schema
|
||||||
|
- `POST /v1/pubsub/publish` - Publish message
|
||||||
|
- `GET /v1/pubsub/topics` - List topics
|
||||||
|
- `GET /v1/pubsub/ws?topic=<name>` - WebSocket subscribe
|
||||||
|
- `POST /v1/functions` - Deploy function (multipart/form-data)
|
||||||
|
- `POST /v1/functions/{name}/invoke` - Invoke function
|
||||||
|
- `GET /v1/functions` - List functions
|
||||||
|
- `DELETE /v1/functions/{name}` - Delete function
|
||||||
|
- `GET /v1/functions/{name}/logs` - Get function logs
|
||||||
|
|
||||||
|
See `openapi/gateway.yaml` for complete API specification.
|
||||||
|
|
||||||
|
## Resources
|
||||||
|
|
||||||
|
- [RQLite Documentation](https://rqlite.io/docs/)
|
||||||
|
- [LibP2P Documentation](https://docs.libp2p.io/)
|
||||||
|
- [GitHub Repository](https://github.com/DeBrosOfficial/network)
|
||||||
|
- [Issue Tracker](https://github.com/DeBrosOfficial/network/issues)
|
||||||
|
|||||||
151
cmd/cli/main.go
Normal file
151
cmd/cli/main.go
Normal file
@ -0,0 +1,151 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/DeBrosOfficial/network/pkg/cli"
|
||||||
|
)
|
||||||
|
|
||||||
|
var (
|
||||||
|
timeout = 30 * time.Second
|
||||||
|
format = "table"
|
||||||
|
)
|
||||||
|
|
||||||
|
// version metadata populated via -ldflags at build time
|
||||||
|
var (
|
||||||
|
version = "dev"
|
||||||
|
commit = ""
|
||||||
|
date = ""
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
if len(os.Args) < 2 {
|
||||||
|
showHelp()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
command := os.Args[1]
|
||||||
|
args := os.Args[2:]
|
||||||
|
|
||||||
|
// Parse global flags
|
||||||
|
parseGlobalFlags(args)
|
||||||
|
|
||||||
|
switch command {
|
||||||
|
case "version":
|
||||||
|
fmt.Printf("orama %s", version)
|
||||||
|
if commit != "" {
|
||||||
|
fmt.Printf(" (commit %s)", commit)
|
||||||
|
}
|
||||||
|
if date != "" {
|
||||||
|
fmt.Printf(" built %s", date)
|
||||||
|
}
|
||||||
|
fmt.Println()
|
||||||
|
return
|
||||||
|
|
||||||
|
// Development environment commands
|
||||||
|
case "dev":
|
||||||
|
cli.HandleDevCommand(args)
|
||||||
|
|
||||||
|
// Production environment commands (legacy with 'prod' prefix)
|
||||||
|
case "prod":
|
||||||
|
cli.HandleProdCommand(args)
|
||||||
|
|
||||||
|
// Direct production commands (new simplified interface)
|
||||||
|
case "install":
|
||||||
|
cli.HandleProdCommand(append([]string{"install"}, args...))
|
||||||
|
case "upgrade":
|
||||||
|
cli.HandleProdCommand(append([]string{"upgrade"}, args...))
|
||||||
|
case "migrate":
|
||||||
|
cli.HandleProdCommand(append([]string{"migrate"}, args...))
|
||||||
|
case "status":
|
||||||
|
cli.HandleProdCommand(append([]string{"status"}, args...))
|
||||||
|
case "start":
|
||||||
|
cli.HandleProdCommand(append([]string{"start"}, args...))
|
||||||
|
case "stop":
|
||||||
|
cli.HandleProdCommand(append([]string{"stop"}, args...))
|
||||||
|
case "restart":
|
||||||
|
cli.HandleProdCommand(append([]string{"restart"}, args...))
|
||||||
|
case "logs":
|
||||||
|
cli.HandleProdCommand(append([]string{"logs"}, args...))
|
||||||
|
case "uninstall":
|
||||||
|
cli.HandleProdCommand(append([]string{"uninstall"}, args...))
|
||||||
|
|
||||||
|
// Authentication commands
|
||||||
|
case "auth":
|
||||||
|
cli.HandleAuthCommand(args)
|
||||||
|
|
||||||
|
// Help
|
||||||
|
case "help", "--help", "-h":
|
||||||
|
showHelp()
|
||||||
|
|
||||||
|
default:
|
||||||
|
fmt.Fprintf(os.Stderr, "Unknown command: %s\n", command)
|
||||||
|
showHelp()
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseGlobalFlags(args []string) {
|
||||||
|
for i, arg := range args {
|
||||||
|
switch arg {
|
||||||
|
case "-f", "--format":
|
||||||
|
if i+1 < len(args) {
|
||||||
|
format = args[i+1]
|
||||||
|
}
|
||||||
|
case "-t", "--timeout":
|
||||||
|
if i+1 < len(args) {
|
||||||
|
if d, err := time.ParseDuration(args[i+1]); err == nil {
|
||||||
|
timeout = d
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func showHelp() {
|
||||||
|
fmt.Printf("Orama CLI - Distributed P2P Network Management Tool\n\n")
|
||||||
|
fmt.Printf("Usage: orama <command> [args...]\n\n")
|
||||||
|
|
||||||
|
fmt.Printf("💻 Local Development:\n")
|
||||||
|
fmt.Printf(" dev up - Start full local dev environment\n")
|
||||||
|
fmt.Printf(" dev down - Stop all dev services\n")
|
||||||
|
fmt.Printf(" dev status - Show status of dev services\n")
|
||||||
|
fmt.Printf(" dev logs <component> - View dev component logs\n")
|
||||||
|
fmt.Printf(" dev help - Show dev command help\n\n")
|
||||||
|
|
||||||
|
fmt.Printf("🚀 Production Deployment:\n")
|
||||||
|
fmt.Printf(" install - Install production node (requires root/sudo)\n")
|
||||||
|
fmt.Printf(" upgrade - Upgrade existing installation\n")
|
||||||
|
fmt.Printf(" status - Show production service status\n")
|
||||||
|
fmt.Printf(" start - Start all production services (requires root/sudo)\n")
|
||||||
|
fmt.Printf(" stop - Stop all production services (requires root/sudo)\n")
|
||||||
|
fmt.Printf(" restart - Restart all production services (requires root/sudo)\n")
|
||||||
|
fmt.Printf(" logs <service> - View production service logs\n")
|
||||||
|
fmt.Printf(" uninstall - Remove production services (requires root/sudo)\n\n")
|
||||||
|
|
||||||
|
fmt.Printf("🔐 Authentication:\n")
|
||||||
|
fmt.Printf(" auth login - Authenticate with wallet\n")
|
||||||
|
fmt.Printf(" auth logout - Clear stored credentials\n")
|
||||||
|
fmt.Printf(" auth whoami - Show current authentication\n")
|
||||||
|
fmt.Printf(" auth status - Show detailed auth info\n")
|
||||||
|
fmt.Printf(" auth help - Show auth command help\n\n")
|
||||||
|
|
||||||
|
fmt.Printf("Global Flags:\n")
|
||||||
|
fmt.Printf(" -f, --format <format> - Output format: table, json (default: table)\n")
|
||||||
|
fmt.Printf(" -t, --timeout <duration> - Operation timeout (default: 30s)\n")
|
||||||
|
fmt.Printf(" --help, -h - Show this help message\n\n")
|
||||||
|
|
||||||
|
fmt.Printf("Examples:\n")
|
||||||
|
fmt.Printf(" # First node (creates new cluster)\n")
|
||||||
|
fmt.Printf(" sudo orama install --vps-ip 203.0.113.1 --domain node-1.orama.network\n\n")
|
||||||
|
|
||||||
|
fmt.Printf(" # Join existing cluster\n")
|
||||||
|
fmt.Printf(" sudo orama install --vps-ip 203.0.113.2 --domain node-2.orama.network \\\n")
|
||||||
|
fmt.Printf(" --peers /ip4/203.0.113.1/tcp/4001/p2p/12D3KooW... --cluster-secret <hex>\n\n")
|
||||||
|
|
||||||
|
fmt.Printf(" # Service management\n")
|
||||||
|
fmt.Printf(" orama status\n")
|
||||||
|
fmt.Printf(" orama logs node --follow\n")
|
||||||
|
}
|
||||||
@ -14,6 +14,10 @@ import (
|
|||||||
"go.uber.org/zap"
|
"go.uber.org/zap"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// For transition, alias main.GatewayConfig to pkg/gateway.Config
|
||||||
|
// server.go will be removed; this keeps compatibility until then.
|
||||||
|
type GatewayConfig = gateway.Config
|
||||||
|
|
||||||
func getEnvDefault(key, def string) string {
|
func getEnvDefault(key, def string) string {
|
||||||
if v := os.Getenv(key); strings.TrimSpace(v) != "" {
|
if v := os.Getenv(key); strings.TrimSpace(v) != "" {
|
||||||
return v
|
return v
|
||||||
@ -69,18 +73,10 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Load YAML
|
// Load YAML
|
||||||
type yamlWebRTCCfg struct {
|
|
||||||
Enabled bool `yaml:"enabled"`
|
|
||||||
SFUPort int `yaml:"sfu_port"`
|
|
||||||
TURNDomain string `yaml:"turn_domain"`
|
|
||||||
TURNSecret string `yaml:"turn_secret"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type yamlCfg struct {
|
type yamlCfg struct {
|
||||||
ListenAddr string `yaml:"listen_addr"`
|
ListenAddr string `yaml:"listen_addr"`
|
||||||
ClientNamespace string `yaml:"client_namespace"`
|
ClientNamespace string `yaml:"client_namespace"`
|
||||||
RQLiteDSN string `yaml:"rqlite_dsn"`
|
RQLiteDSN string `yaml:"rqlite_dsn"`
|
||||||
GlobalRQLiteDSN string `yaml:"global_rqlite_dsn"`
|
|
||||||
Peers []string `yaml:"bootstrap_peers"`
|
Peers []string `yaml:"bootstrap_peers"`
|
||||||
EnableHTTPS bool `yaml:"enable_https"`
|
EnableHTTPS bool `yaml:"enable_https"`
|
||||||
DomainName string `yaml:"domain_name"`
|
DomainName string `yaml:"domain_name"`
|
||||||
@ -91,7 +87,6 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config {
|
|||||||
IPFSAPIURL string `yaml:"ipfs_api_url"`
|
IPFSAPIURL string `yaml:"ipfs_api_url"`
|
||||||
IPFSTimeout string `yaml:"ipfs_timeout"`
|
IPFSTimeout string `yaml:"ipfs_timeout"`
|
||||||
IPFSReplicationFactor int `yaml:"ipfs_replication_factor"`
|
IPFSReplicationFactor int `yaml:"ipfs_replication_factor"`
|
||||||
WebRTC yamlWebRTCCfg `yaml:"webrtc"`
|
|
||||||
}
|
}
|
||||||
|
|
||||||
data, err := os.ReadFile(configPath)
|
data, err := os.ReadFile(configPath)
|
||||||
@ -100,7 +95,7 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config {
|
|||||||
zap.String("path", configPath),
|
zap.String("path", configPath),
|
||||||
zap.Error(err))
|
zap.Error(err))
|
||||||
fmt.Fprintf(os.Stderr, "\nConfig file not found at %s\n", configPath)
|
fmt.Fprintf(os.Stderr, "\nConfig file not found at %s\n", configPath)
|
||||||
fmt.Fprintf(os.Stderr, "Generate it using: orama config init --type gateway\n")
|
fmt.Fprintf(os.Stderr, "Generate it using: dbn config init --type gateway\n")
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -118,7 +113,6 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config {
|
|||||||
ClientNamespace: "default",
|
ClientNamespace: "default",
|
||||||
BootstrapPeers: nil,
|
BootstrapPeers: nil,
|
||||||
RQLiteDSN: "",
|
RQLiteDSN: "",
|
||||||
GlobalRQLiteDSN: "",
|
|
||||||
EnableHTTPS: false,
|
EnableHTTPS: false,
|
||||||
DomainName: "",
|
DomainName: "",
|
||||||
TLSCacheDir: "",
|
TLSCacheDir: "",
|
||||||
@ -139,9 +133,6 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config {
|
|||||||
if v := strings.TrimSpace(y.RQLiteDSN); v != "" {
|
if v := strings.TrimSpace(y.RQLiteDSN); v != "" {
|
||||||
cfg.RQLiteDSN = v
|
cfg.RQLiteDSN = v
|
||||||
}
|
}
|
||||||
if v := strings.TrimSpace(y.GlobalRQLiteDSN); v != "" {
|
|
||||||
cfg.GlobalRQLiteDSN = v
|
|
||||||
}
|
|
||||||
if len(y.Peers) > 0 {
|
if len(y.Peers) > 0 {
|
||||||
var peers []string
|
var peers []string
|
||||||
for _, p := range y.Peers {
|
for _, p := range y.Peers {
|
||||||
@ -200,18 +191,6 @@ func parseGatewayConfig(logger *logging.ColoredLogger) *gateway.Config {
|
|||||||
cfg.IPFSReplicationFactor = y.IPFSReplicationFactor
|
cfg.IPFSReplicationFactor = y.IPFSReplicationFactor
|
||||||
}
|
}
|
||||||
|
|
||||||
// WebRTC configuration
|
|
||||||
cfg.WebRTCEnabled = y.WebRTC.Enabled
|
|
||||||
if y.WebRTC.SFUPort > 0 {
|
|
||||||
cfg.SFUPort = y.WebRTC.SFUPort
|
|
||||||
}
|
|
||||||
if v := strings.TrimSpace(y.WebRTC.TURNDomain); v != "" {
|
|
||||||
cfg.TURNDomain = v
|
|
||||||
}
|
|
||||||
if v := strings.TrimSpace(y.WebRTC.TURNSecret); v != "" {
|
|
||||||
cfg.TURNSecret = v
|
|
||||||
}
|
|
||||||
|
|
||||||
// Validate configuration
|
// Validate configuration
|
||||||
if errs := cfg.ValidateConfig(); len(errs) > 0 {
|
if errs := cfg.ValidateConfig(); len(errs) > 0 {
|
||||||
fmt.Fprintf(os.Stderr, "\nGateway configuration errors (%d):\n", len(errs))
|
fmt.Fprintf(os.Stderr, "\nGateway configuration errors (%d):\n", len(errs))
|
||||||
@ -68,11 +68,6 @@ func main() {
|
|||||||
httpServer := &http.Server{
|
httpServer := &http.Server{
|
||||||
Addr: ":80",
|
Addr: ":80",
|
||||||
Handler: manager.HTTPHandler(nil), // Redirects all HTTP traffic to HTTPS except ACME challenge
|
Handler: manager.HTTPHandler(nil), // Redirects all HTTP traffic to HTTPS except ACME challenge
|
||||||
ReadHeaderTimeout: 10 * time.Second,
|
|
||||||
ReadTimeout: 60 * time.Second,
|
|
||||||
WriteTimeout: 120 * time.Second,
|
|
||||||
IdleTimeout: 120 * time.Second,
|
|
||||||
MaxHeaderBytes: 1 << 20, // 1MB
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Create HTTPS server (port 443)
|
// Create HTTPS server (port 443)
|
||||||
@ -80,11 +75,6 @@ func main() {
|
|||||||
Addr: ":443",
|
Addr: ":443",
|
||||||
Handler: gw.Routes(),
|
Handler: gw.Routes(),
|
||||||
TLSConfig: manager.TLSConfig(),
|
TLSConfig: manager.TLSConfig(),
|
||||||
ReadHeaderTimeout: 10 * time.Second,
|
|
||||||
ReadTimeout: 60 * time.Second,
|
|
||||||
WriteTimeout: 120 * time.Second,
|
|
||||||
IdleTimeout: 120 * time.Second,
|
|
||||||
MaxHeaderBytes: 1 << 20, // 1MB
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Start HTTP server for ACME challenge
|
// Start HTTP server for ACME challenge
|
||||||
@ -173,11 +163,6 @@ func main() {
|
|||||||
server := &http.Server{
|
server := &http.Server{
|
||||||
Addr: cfg.ListenAddr,
|
Addr: cfg.ListenAddr,
|
||||||
Handler: gw.Routes(),
|
Handler: gw.Routes(),
|
||||||
ReadHeaderTimeout: 10 * time.Second,
|
|
||||||
ReadTimeout: 60 * time.Second,
|
|
||||||
WriteTimeout: 120 * time.Second,
|
|
||||||
IdleTimeout: 120 * time.Second,
|
|
||||||
MaxHeaderBytes: 1 << 20, // 1MB
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Try to bind listener explicitly so binding failures are visible immediately.
|
// Try to bind listener explicitly so binding failures are visible immediately.
|
||||||
320
cmd/rqlite-mcp/main.go
Normal file
320
cmd/rqlite-mcp/main.go
Normal file
@ -0,0 +1,320 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/rqlite/gorqlite"
|
||||||
|
)
|
||||||
|
|
||||||
|
// MCP JSON-RPC types
|
||||||
|
type JSONRPCRequest struct {
|
||||||
|
JSONRPC string `json:"jsonrpc"`
|
||||||
|
ID any `json:"id,omitempty"`
|
||||||
|
Method string `json:"method"`
|
||||||
|
Params json.RawMessage `json:"params,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type JSONRPCResponse struct {
|
||||||
|
JSONRPC string `json:"jsonrpc"`
|
||||||
|
ID any `json:"id"`
|
||||||
|
Result any `json:"result,omitempty"`
|
||||||
|
Error *ResponseError `json:"error,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type ResponseError struct {
|
||||||
|
Code int `json:"code"`
|
||||||
|
Message string `json:"message"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tool definition
|
||||||
|
type Tool struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Description string `json:"description"`
|
||||||
|
InputSchema any `json:"inputSchema"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Tool call types
|
||||||
|
type CallToolRequest struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Arguments json.RawMessage `json:"arguments"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type TextContent struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
Text string `json:"text"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type CallToolResult struct {
|
||||||
|
Content []TextContent `json:"content"`
|
||||||
|
IsError bool `json:"isError,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type MCPServer struct {
|
||||||
|
conn *gorqlite.Connection
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewMCPServer(rqliteURL string) (*MCPServer, error) {
|
||||||
|
conn, err := gorqlite.Open(rqliteURL)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &MCPServer{
|
||||||
|
conn: conn,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *MCPServer) handleRequest(req JSONRPCRequest) JSONRPCResponse {
|
||||||
|
var resp JSONRPCResponse
|
||||||
|
resp.JSONRPC = "2.0"
|
||||||
|
resp.ID = req.ID
|
||||||
|
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
// log.Printf("Received method: %s", req.Method)
|
||||||
|
|
||||||
|
switch req.Method {
|
||||||
|
case "initialize":
|
||||||
|
resp.Result = map[string]any{
|
||||||
|
"protocolVersion": "2024-11-05",
|
||||||
|
"capabilities": map[string]any{
|
||||||
|
"tools": map[string]any{},
|
||||||
|
},
|
||||||
|
"serverInfo": map[string]any{
|
||||||
|
"name": "rqlite-mcp",
|
||||||
|
"version": "0.1.0",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
case "notifications/initialized":
|
||||||
|
// This is a notification, no response needed
|
||||||
|
return JSONRPCResponse{}
|
||||||
|
|
||||||
|
case "tools/list":
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
tools := []Tool{
|
||||||
|
{
|
||||||
|
Name: "list_tables",
|
||||||
|
Description: "List all tables in the Rqlite database",
|
||||||
|
InputSchema: map[string]any{
|
||||||
|
"type": "object",
|
||||||
|
"properties": map[string]any{},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Name: "query",
|
||||||
|
Description: "Run a SELECT query on the Rqlite database",
|
||||||
|
InputSchema: map[string]any{
|
||||||
|
"type": "object",
|
||||||
|
"properties": map[string]any{
|
||||||
|
"sql": map[string]any{
|
||||||
|
"type": "string",
|
||||||
|
"description": "The SQL SELECT query to run",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": []string{"sql"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Name: "execute",
|
||||||
|
Description: "Run an INSERT, UPDATE, or DELETE statement on the Rqlite database",
|
||||||
|
InputSchema: map[string]any{
|
||||||
|
"type": "object",
|
||||||
|
"properties": map[string]any{
|
||||||
|
"sql": map[string]any{
|
||||||
|
"type": "string",
|
||||||
|
"description": "The SQL statement (INSERT, UPDATE, DELETE) to run",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"required": []string{"sql"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
resp.Result = map[string]any{"tools": tools}
|
||||||
|
|
||||||
|
case "tools/call":
|
||||||
|
var callReq CallToolRequest
|
||||||
|
if err := json.Unmarshal(req.Params, &callReq); err != nil {
|
||||||
|
resp.Error = &ResponseError{Code: -32700, Message: "Parse error"}
|
||||||
|
return resp
|
||||||
|
}
|
||||||
|
resp.Result = s.handleToolCall(callReq)
|
||||||
|
|
||||||
|
default:
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
resp.Error = &ResponseError{Code: -32601, Message: "Method not found"}
|
||||||
|
}
|
||||||
|
|
||||||
|
return resp
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *MCPServer) handleToolCall(req CallToolRequest) CallToolResult {
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
// log.Printf("Tool call: %s", req.Name)
|
||||||
|
|
||||||
|
switch req.Name {
|
||||||
|
case "list_tables":
|
||||||
|
rows, err := s.conn.QueryOne("SELECT name FROM sqlite_master WHERE type='table' ORDER BY name")
|
||||||
|
if err != nil {
|
||||||
|
return errorResult(fmt.Sprintf("Error listing tables: %v", err))
|
||||||
|
}
|
||||||
|
var tables []string
|
||||||
|
for rows.Next() {
|
||||||
|
slice, err := rows.Slice()
|
||||||
|
if err == nil && len(slice) > 0 {
|
||||||
|
tables = append(tables, fmt.Sprint(slice[0]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(tables) == 0 {
|
||||||
|
return textResult("No tables found")
|
||||||
|
}
|
||||||
|
return textResult(strings.Join(tables, "\n"))
|
||||||
|
|
||||||
|
case "query":
|
||||||
|
var args struct {
|
||||||
|
SQL string `json:"sql"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(req.Arguments, &args); err != nil {
|
||||||
|
return errorResult(fmt.Sprintf("Invalid arguments: %v", err))
|
||||||
|
}
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
rows, err := s.conn.QueryOne(args.SQL)
|
||||||
|
if err != nil {
|
||||||
|
return errorResult(fmt.Sprintf("Query error: %v", err))
|
||||||
|
}
|
||||||
|
|
||||||
|
var result strings.Builder
|
||||||
|
cols := rows.Columns()
|
||||||
|
result.WriteString(strings.Join(cols, " | ") + "\n")
|
||||||
|
result.WriteString(strings.Repeat("-", len(cols)*10) + "\n")
|
||||||
|
|
||||||
|
rowCount := 0
|
||||||
|
for rows.Next() {
|
||||||
|
vals, err := rows.Slice()
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rowCount++
|
||||||
|
for i, v := range vals {
|
||||||
|
if i > 0 {
|
||||||
|
result.WriteString(" | ")
|
||||||
|
}
|
||||||
|
result.WriteString(fmt.Sprint(v))
|
||||||
|
}
|
||||||
|
result.WriteString("\n")
|
||||||
|
}
|
||||||
|
result.WriteString(fmt.Sprintf("\n(%d rows)", rowCount))
|
||||||
|
return textResult(result.String())
|
||||||
|
|
||||||
|
case "execute":
|
||||||
|
var args struct {
|
||||||
|
SQL string `json:"sql"`
|
||||||
|
}
|
||||||
|
if err := json.Unmarshal(req.Arguments, &args); err != nil {
|
||||||
|
return errorResult(fmt.Sprintf("Invalid arguments: %v", err))
|
||||||
|
}
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
res, err := s.conn.WriteOne(args.SQL)
|
||||||
|
if err != nil {
|
||||||
|
return errorResult(fmt.Sprintf("Execution error: %v", err))
|
||||||
|
}
|
||||||
|
return textResult(fmt.Sprintf("Rows affected: %d", res.RowsAffected))
|
||||||
|
|
||||||
|
default:
|
||||||
|
return errorResult(fmt.Sprintf("Unknown tool: %s", req.Name))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func textResult(text string) CallToolResult {
|
||||||
|
return CallToolResult{
|
||||||
|
Content: []TextContent{
|
||||||
|
{
|
||||||
|
Type: "text",
|
||||||
|
Text: text,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func errorResult(text string) CallToolResult {
|
||||||
|
return CallToolResult{
|
||||||
|
Content: []TextContent{
|
||||||
|
{
|
||||||
|
Type: "text",
|
||||||
|
Text: text,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
IsError: true,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
// Log to stderr so stdout is clean for JSON-RPC
|
||||||
|
log.SetOutput(os.Stderr)
|
||||||
|
|
||||||
|
rqliteURL := "http://localhost:5001"
|
||||||
|
if u := os.Getenv("RQLITE_URL"); u != "" {
|
||||||
|
rqliteURL = u
|
||||||
|
}
|
||||||
|
|
||||||
|
var server *MCPServer
|
||||||
|
var err error
|
||||||
|
|
||||||
|
// Retry connecting to rqlite
|
||||||
|
maxRetries := 30
|
||||||
|
for i := 0; i < maxRetries; i++ {
|
||||||
|
server, err = NewMCPServer(rqliteURL)
|
||||||
|
if err == nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if i%5 == 0 {
|
||||||
|
log.Printf("Waiting for Rqlite at %s... (%d/%d)", rqliteURL, i+1, maxRetries)
|
||||||
|
}
|
||||||
|
time.Sleep(1 * time.Second)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
log.Fatalf("Failed to connect to Rqlite after %d retries: %v", maxRetries, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
log.Printf("MCP Rqlite server started (stdio transport)")
|
||||||
|
log.Printf("Connected to Rqlite at %s", rqliteURL)
|
||||||
|
|
||||||
|
// Read JSON-RPC requests from stdin, write responses to stdout
|
||||||
|
scanner := bufio.NewScanner(os.Stdin)
|
||||||
|
for scanner.Scan() {
|
||||||
|
line := scanner.Text()
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
var req JSONRPCRequest
|
||||||
|
if err := json.Unmarshal([]byte(line), &req); err != nil {
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
resp := server.handleRequest(req)
|
||||||
|
|
||||||
|
// Don't send response for notifications (no ID)
|
||||||
|
if req.ID == nil && strings.HasPrefix(req.Method, "notifications/") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
respData, err := json.Marshal(resp)
|
||||||
|
if err != nil {
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
fmt.Println(string(respData))
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := scanner.Err(); err != nil {
|
||||||
|
// Debug logging disabled to prevent excessive disk writes
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -1,8 +0,0 @@
|
|||||||
# OpenRouter API Key for changelog generation
|
|
||||||
# Get your API key from https://openrouter.ai/keys
|
|
||||||
OPENROUTER_API_KEY=your-api-key-here
|
|
||||||
|
|
||||||
# ZeroSSL API Key for TLS certificates (alternative to Let's Encrypt)
|
|
||||||
# Get your free API key from https://app.zerossl.com/developer
|
|
||||||
# If not set, Caddy will use Let's Encrypt as the default CA
|
|
||||||
ZEROSSL_API_KEY=
|
|
||||||
181
core/Makefile
181
core/Makefile
@ -1,181 +0,0 @@
|
|||||||
TEST?=./...
|
|
||||||
|
|
||||||
.PHONY: test
|
|
||||||
test:
|
|
||||||
@echo Running tests...
|
|
||||||
go test -v $(TEST)
|
|
||||||
|
|
||||||
# Gateway-focused E2E tests assume gateway and nodes are already running
|
|
||||||
# Auto-discovers configuration from ~/.orama and queries database for API key
|
|
||||||
# No environment variables required
|
|
||||||
.PHONY: test-e2e test-e2e-deployments test-e2e-fullstack test-e2e-https test-e2e-quick test-e2e-prod test-e2e-shared test-e2e-cluster test-e2e-integration test-e2e-production
|
|
||||||
|
|
||||||
# Production E2E tests - includes production-only tests
|
|
||||||
test-e2e-prod:
|
|
||||||
@if [ -z "$$ORAMA_GATEWAY_URL" ]; then \
|
|
||||||
echo "❌ ORAMA_GATEWAY_URL not set"; \
|
|
||||||
echo "Usage: ORAMA_GATEWAY_URL=https://dbrs.space make test-e2e-prod"; \
|
|
||||||
exit 1; \
|
|
||||||
fi
|
|
||||||
@echo "Running E2E tests (including production-only) against $$ORAMA_GATEWAY_URL..."
|
|
||||||
go test -v -tags "e2e production" -timeout 30m ./e2e/...
|
|
||||||
|
|
||||||
# Generic e2e target
|
|
||||||
test-e2e:
|
|
||||||
@echo "Running comprehensive E2E tests..."
|
|
||||||
@echo "Auto-discovering configuration from ~/.orama..."
|
|
||||||
go test -v -tags e2e -timeout 30m ./e2e/...
|
|
||||||
|
|
||||||
test-e2e-deployments:
|
|
||||||
@echo "Running deployment E2E tests..."
|
|
||||||
go test -v -tags e2e -timeout 15m ./e2e/deployments/...
|
|
||||||
|
|
||||||
test-e2e-fullstack:
|
|
||||||
@echo "Running fullstack E2E tests..."
|
|
||||||
go test -v -tags e2e -timeout 20m -run "TestFullStack" ./e2e/...
|
|
||||||
|
|
||||||
test-e2e-https:
|
|
||||||
@echo "Running HTTPS/external access E2E tests..."
|
|
||||||
go test -v -tags e2e -timeout 10m -run "TestHTTPS" ./e2e/...
|
|
||||||
|
|
||||||
test-e2e-shared:
|
|
||||||
@echo "Running shared E2E tests..."
|
|
||||||
go test -v -tags e2e -timeout 10m ./e2e/shared/...
|
|
||||||
|
|
||||||
test-e2e-cluster:
|
|
||||||
@echo "Running cluster E2E tests..."
|
|
||||||
go test -v -tags e2e -timeout 15m ./e2e/cluster/...
|
|
||||||
|
|
||||||
test-e2e-integration:
|
|
||||||
@echo "Running integration E2E tests..."
|
|
||||||
go test -v -tags e2e -timeout 20m ./e2e/integration/...
|
|
||||||
|
|
||||||
test-e2e-production:
|
|
||||||
@echo "Running production-only E2E tests..."
|
|
||||||
go test -v -tags "e2e production" -timeout 15m ./e2e/production/...
|
|
||||||
|
|
||||||
test-e2e-quick:
|
|
||||||
@echo "Running quick E2E smoke tests..."
|
|
||||||
go test -v -tags e2e -timeout 5m -run "TestStatic|TestHealth" ./e2e/...
|
|
||||||
|
|
||||||
# Network - Distributed P2P Database System
|
|
||||||
# Makefile for development and build tasks
|
|
||||||
|
|
||||||
.PHONY: build clean test deps tidy fmt vet lint install-hooks push-devnet push-testnet rollout-devnet rollout-testnet release
|
|
||||||
|
|
||||||
VERSION := 0.120.0
|
|
||||||
COMMIT ?= $(shell git rev-parse --short HEAD 2>/dev/null || echo unknown)
|
|
||||||
DATE ?= $(shell date -u +%Y-%m-%dT%H:%M:%SZ)
|
|
||||||
LDFLAGS := -X 'main.version=$(VERSION)' -X 'main.commit=$(COMMIT)' -X 'main.date=$(DATE)'
|
|
||||||
LDFLAGS_LINUX := -s -w $(LDFLAGS)
|
|
||||||
|
|
||||||
# Build targets
|
|
||||||
build: deps
|
|
||||||
@echo "Building network executables (version=$(VERSION))..."
|
|
||||||
@mkdir -p bin
|
|
||||||
go build -ldflags "$(LDFLAGS)" -o bin/identity ./cmd/identity
|
|
||||||
go build -ldflags "$(LDFLAGS)" -o bin/orama-node ./cmd/node
|
|
||||||
go build -ldflags "$(LDFLAGS)" -o bin/orama ./cmd/cli/
|
|
||||||
# Inject gateway build metadata via pkg path variables
|
|
||||||
go build -ldflags "$(LDFLAGS) -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildVersion=$(VERSION)' -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildCommit=$(COMMIT)' -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildTime=$(DATE)'" -o bin/gateway ./cmd/gateway
|
|
||||||
go build -ldflags "$(LDFLAGS)" -o bin/sfu ./cmd/sfu
|
|
||||||
go build -ldflags "$(LDFLAGS)" -o bin/turn ./cmd/turn
|
|
||||||
@echo "Build complete! Run ./bin/orama version"
|
|
||||||
|
|
||||||
# Cross-compile CLI for Linux (only binary needed locally; VPS builds everything else from source)
|
|
||||||
build-linux: deps
|
|
||||||
@echo "Cross-compiling CLI for linux/amd64 (version=$(VERSION))..."
|
|
||||||
@mkdir -p bin-linux
|
|
||||||
GOOS=linux GOARCH=amd64 go build -ldflags "$(LDFLAGS_LINUX)" -trimpath -o bin-linux/orama ./cmd/cli/
|
|
||||||
@echo "✓ CLI built at bin-linux/orama"
|
|
||||||
@echo ""
|
|
||||||
@echo "Prefer 'make build-archive' for full pre-built binary archive."
|
|
||||||
|
|
||||||
# Build pre-compiled binary archive for deployment (all binaries + deps)
|
|
||||||
build-archive: deps
|
|
||||||
@echo "Building binary archive (version=$(VERSION))..."
|
|
||||||
go build -ldflags "$(LDFLAGS)" -o bin/orama ./cmd/cli/
|
|
||||||
./bin/orama build --output /tmp/orama-$(VERSION)-linux-amd64.tar.gz
|
|
||||||
|
|
||||||
# Install git hooks
|
|
||||||
install-hooks:
|
|
||||||
@echo "Installing git hooks..."
|
|
||||||
@bash scripts/install-hooks.sh
|
|
||||||
|
|
||||||
# Install orama CLI to ~/.local/bin and configure PATH
|
|
||||||
install: build
|
|
||||||
@bash scripts/install.sh
|
|
||||||
|
|
||||||
# Clean build artifacts
|
|
||||||
clean:
|
|
||||||
@echo "Cleaning build artifacts..."
|
|
||||||
rm -rf bin/
|
|
||||||
rm -rf data/
|
|
||||||
@echo "Clean complete!"
|
|
||||||
|
|
||||||
# Push binary archive to devnet nodes (fanout distribution)
|
|
||||||
push-devnet:
|
|
||||||
./bin/orama node push --env devnet
|
|
||||||
|
|
||||||
# Push binary archive to testnet nodes (fanout distribution)
|
|
||||||
push-testnet:
|
|
||||||
./bin/orama node push --env testnet
|
|
||||||
|
|
||||||
# Full rollout to devnet (build + push + rolling upgrade)
|
|
||||||
rollout-devnet:
|
|
||||||
./bin/orama node rollout --env devnet --yes
|
|
||||||
|
|
||||||
# Full rollout to testnet (build + push + rolling upgrade)
|
|
||||||
rollout-testnet:
|
|
||||||
./bin/orama node rollout --env testnet --yes
|
|
||||||
|
|
||||||
# Interactive release workflow (tag + push)
|
|
||||||
release:
|
|
||||||
@bash scripts/release.sh
|
|
||||||
|
|
||||||
# Check health of all nodes in an environment
|
|
||||||
# Usage: make health ENV=devnet
|
|
||||||
health:
|
|
||||||
@if [ -z "$(ENV)" ]; then \
|
|
||||||
echo "Usage: make health ENV=devnet|testnet"; \
|
|
||||||
exit 1; \
|
|
||||||
fi
|
|
||||||
./bin/orama monitor report --env $(ENV)
|
|
||||||
|
|
||||||
# Help
|
|
||||||
help:
|
|
||||||
@echo "Available targets:"
|
|
||||||
@echo " build - Build all executables"
|
|
||||||
@echo " install - Build and install 'orama' CLI to ~/.local/bin"
|
|
||||||
@echo " clean - Clean build artifacts"
|
|
||||||
@echo " test - Run unit tests"
|
|
||||||
@echo ""
|
|
||||||
@echo "E2E Testing:"
|
|
||||||
@echo " make test-e2e-prod - Run all E2E tests incl. production-only (needs ORAMA_GATEWAY_URL)"
|
|
||||||
@echo " make test-e2e-shared - Run shared E2E tests (cache, storage, pubsub, auth)"
|
|
||||||
@echo " make test-e2e-cluster - Run cluster E2E tests (libp2p, olric, rqlite, namespace)"
|
|
||||||
@echo " make test-e2e-integration - Run integration E2E tests (fullstack, persistence, concurrency)"
|
|
||||||
@echo " make test-e2e-deployments - Run deployment E2E tests"
|
|
||||||
@echo " make test-e2e-production - Run production-only E2E tests (DNS, HTTPS, cross-node)"
|
|
||||||
@echo " make test-e2e-quick - Quick smoke tests (static deploys, health checks)"
|
|
||||||
@echo " make test-e2e - Generic E2E tests (auto-discovers config)"
|
|
||||||
@echo ""
|
|
||||||
@echo " Example:"
|
|
||||||
@echo " ORAMA_GATEWAY_URL=https://orama-devnet.network make test-e2e-prod"
|
|
||||||
@echo ""
|
|
||||||
@echo "Deployment:"
|
|
||||||
@echo " make build-archive - Build pre-compiled binary archive for deployment"
|
|
||||||
@echo " make push-devnet - Push binary archive to devnet nodes"
|
|
||||||
@echo " make push-testnet - Push binary archive to testnet nodes"
|
|
||||||
@echo " make rollout-devnet - Full rollout: build + push + rolling upgrade (devnet)"
|
|
||||||
@echo " make rollout-testnet - Full rollout: build + push + rolling upgrade (testnet)"
|
|
||||||
@echo " make health ENV=devnet - Check health of all nodes in an environment"
|
|
||||||
@echo " make release - Interactive release workflow (tag + push)"
|
|
||||||
@echo ""
|
|
||||||
@echo "Maintenance:"
|
|
||||||
@echo " deps - Download dependencies"
|
|
||||||
@echo " tidy - Tidy dependencies"
|
|
||||||
@echo " fmt - Format code"
|
|
||||||
@echo " vet - Vet code"
|
|
||||||
@echo " lint - Lint code (fmt + vet)"
|
|
||||||
@echo " help - Show this help"
|
|
||||||
@ -1,5 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
runCLI()
|
|
||||||
}
|
|
||||||
@ -1,103 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
|
|
||||||
"github.com/spf13/cobra"
|
|
||||||
|
|
||||||
// Command groups
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/app"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/authcmd"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/buildcmd"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/dbcmd"
|
|
||||||
deploycmd "github.com/DeBrosOfficial/network/pkg/cli/cmd/deploy"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/envcmd"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/functioncmd"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/inspectcmd"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/monitorcmd"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/namespacecmd"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/node"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli/cmd/sandboxcmd"
|
|
||||||
)
|
|
||||||
|
|
||||||
// version metadata populated via -ldflags at build time
|
|
||||||
// Must match Makefile: -X 'main.version=...' -X 'main.commit=...' -X 'main.date=...'
|
|
||||||
var (
|
|
||||||
version = "dev"
|
|
||||||
commit = ""
|
|
||||||
date = ""
|
|
||||||
)
|
|
||||||
|
|
||||||
func newRootCmd() *cobra.Command {
|
|
||||||
rootCmd := &cobra.Command{
|
|
||||||
Use: "orama",
|
|
||||||
Short: "Orama CLI - Distributed P2P Network Management Tool",
|
|
||||||
Long: `Orama CLI is a tool for managing nodes, deploying applications,
|
|
||||||
and interacting with the Orama distributed network.`,
|
|
||||||
SilenceUsage: true,
|
|
||||||
SilenceErrors: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
// Version command
|
|
||||||
rootCmd.AddCommand(&cobra.Command{
|
|
||||||
Use: "version",
|
|
||||||
Short: "Show version information",
|
|
||||||
Run: func(cmd *cobra.Command, args []string) {
|
|
||||||
fmt.Printf("orama %s", version)
|
|
||||||
if commit != "" {
|
|
||||||
fmt.Printf(" (commit %s)", commit)
|
|
||||||
}
|
|
||||||
if date != "" {
|
|
||||||
fmt.Printf(" built %s", date)
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
},
|
|
||||||
})
|
|
||||||
|
|
||||||
// Node operator commands (was "prod")
|
|
||||||
rootCmd.AddCommand(node.Cmd)
|
|
||||||
|
|
||||||
// Deploy command (top-level, upsert)
|
|
||||||
rootCmd.AddCommand(deploycmd.Cmd)
|
|
||||||
|
|
||||||
// App management (was "deployments")
|
|
||||||
rootCmd.AddCommand(app.Cmd)
|
|
||||||
|
|
||||||
// Database commands
|
|
||||||
rootCmd.AddCommand(dbcmd.Cmd)
|
|
||||||
|
|
||||||
// Namespace commands
|
|
||||||
rootCmd.AddCommand(namespacecmd.Cmd)
|
|
||||||
|
|
||||||
// Environment commands
|
|
||||||
rootCmd.AddCommand(envcmd.Cmd)
|
|
||||||
|
|
||||||
// Auth commands
|
|
||||||
rootCmd.AddCommand(authcmd.Cmd)
|
|
||||||
|
|
||||||
// Inspect command
|
|
||||||
rootCmd.AddCommand(inspectcmd.Cmd)
|
|
||||||
|
|
||||||
// Monitor command
|
|
||||||
rootCmd.AddCommand(monitorcmd.Cmd)
|
|
||||||
|
|
||||||
// Serverless function commands
|
|
||||||
rootCmd.AddCommand(functioncmd.Cmd)
|
|
||||||
|
|
||||||
// Build command (cross-compile binary archive)
|
|
||||||
rootCmd.AddCommand(buildcmd.Cmd)
|
|
||||||
|
|
||||||
// Sandbox command (ephemeral Hetzner Cloud clusters)
|
|
||||||
rootCmd.AddCommand(sandboxcmd.Cmd)
|
|
||||||
|
|
||||||
return rootCmd
|
|
||||||
}
|
|
||||||
|
|
||||||
func runCLI() {
|
|
||||||
rootCmd := newRootCmd()
|
|
||||||
if err := rootCmd.Execute(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,11 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"os"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/cli"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
cli.HandleInspectCommand(os.Args[1:])
|
|
||||||
}
|
|
||||||
@ -1,118 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/config"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/logging"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/sfu"
|
|
||||||
"go.uber.org/zap"
|
|
||||||
)
|
|
||||||
|
|
||||||
// newSFUServer creates a new SFU server from config and logger.
|
|
||||||
// Wrapper to keep main.go clean and avoid importing sfu in main.
|
|
||||||
func newSFUServer(cfg *sfu.Config, logger *zap.Logger) (*sfu.Server, error) {
|
|
||||||
return sfu.NewServer(cfg, logger)
|
|
||||||
}
|
|
||||||
|
|
||||||
func parseSFUConfig(logger *logging.ColoredLogger) *sfu.Config {
|
|
||||||
configFlag := flag.String("config", "", "Config file path (absolute path or filename in ~/.orama)")
|
|
||||||
flag.Parse()
|
|
||||||
|
|
||||||
var configPath string
|
|
||||||
var err error
|
|
||||||
if *configFlag != "" {
|
|
||||||
if filepath.IsAbs(*configFlag) {
|
|
||||||
configPath = *configFlag
|
|
||||||
} else {
|
|
||||||
configPath, err = config.DefaultPath(*configFlag)
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentSFU, "Failed to determine config path", zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "Configuration error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
configPath, err = config.DefaultPath("sfu.yaml")
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentSFU, "Failed to determine config path", zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "Configuration error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type yamlTURNServer struct {
|
|
||||||
Host string `yaml:"host"`
|
|
||||||
Port int `yaml:"port"`
|
|
||||||
Secure bool `yaml:"secure"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type yamlCfg struct {
|
|
||||||
ListenAddr string `yaml:"listen_addr"`
|
|
||||||
Namespace string `yaml:"namespace"`
|
|
||||||
MediaPortStart int `yaml:"media_port_start"`
|
|
||||||
MediaPortEnd int `yaml:"media_port_end"`
|
|
||||||
TURNServers []yamlTURNServer `yaml:"turn_servers"`
|
|
||||||
TURNSecret string `yaml:"turn_secret"`
|
|
||||||
TURNCredentialTTL int `yaml:"turn_credential_ttl"`
|
|
||||||
RQLiteDSN string `yaml:"rqlite_dsn"`
|
|
||||||
}
|
|
||||||
|
|
||||||
data, err := os.ReadFile(configPath)
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentSFU, "Config file not found",
|
|
||||||
zap.String("path", configPath), zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "\nConfig file not found at %s\n", configPath)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
var y yamlCfg
|
|
||||||
if err := config.DecodeStrict(strings.NewReader(string(data)), &y); err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentSFU, "Failed to parse SFU config", zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "Configuration parse error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
var turnServers []sfu.TURNServerConfig
|
|
||||||
for _, ts := range y.TURNServers {
|
|
||||||
turnServers = append(turnServers, sfu.TURNServerConfig{
|
|
||||||
Host: ts.Host,
|
|
||||||
Port: ts.Port,
|
|
||||||
Secure: ts.Secure,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
cfg := &sfu.Config{
|
|
||||||
ListenAddr: y.ListenAddr,
|
|
||||||
Namespace: y.Namespace,
|
|
||||||
MediaPortStart: y.MediaPortStart,
|
|
||||||
MediaPortEnd: y.MediaPortEnd,
|
|
||||||
TURNServers: turnServers,
|
|
||||||
TURNSecret: y.TURNSecret,
|
|
||||||
TURNCredentialTTL: y.TURNCredentialTTL,
|
|
||||||
RQLiteDSN: y.RQLiteDSN,
|
|
||||||
}
|
|
||||||
|
|
||||||
if errs := cfg.Validate(); len(errs) > 0 {
|
|
||||||
fmt.Fprintf(os.Stderr, "\nSFU configuration errors (%d):\n", len(errs))
|
|
||||||
for _, e := range errs {
|
|
||||||
fmt.Fprintf(os.Stderr, " - %s\n", e)
|
|
||||||
}
|
|
||||||
fmt.Fprintf(os.Stderr, "\nPlease fix the configuration and try again.\n")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentSFU, "Loaded SFU configuration",
|
|
||||||
zap.String("path", configPath),
|
|
||||||
zap.String("listen_addr", cfg.ListenAddr),
|
|
||||||
zap.String("namespace", cfg.Namespace),
|
|
||||||
zap.Int("media_ports", cfg.MediaPortEnd-cfg.MediaPortStart),
|
|
||||||
zap.Int("turn_servers", len(cfg.TURNServers)),
|
|
||||||
)
|
|
||||||
|
|
||||||
return cfg
|
|
||||||
}
|
|
||||||
@ -1,61 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"errors"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/signal"
|
|
||||||
"syscall"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/logging"
|
|
||||||
"go.uber.org/zap"
|
|
||||||
)
|
|
||||||
|
|
||||||
var (
|
|
||||||
version = "dev"
|
|
||||||
commit = "unknown"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
logger, err := logging.NewColoredLogger(logging.ComponentSFU, true)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentSFU, "Starting SFU server",
|
|
||||||
zap.String("version", version),
|
|
||||||
zap.String("commit", commit))
|
|
||||||
|
|
||||||
cfg := parseSFUConfig(logger)
|
|
||||||
|
|
||||||
server, err := newSFUServer(cfg, logger.Logger)
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentSFU, "Failed to create SFU server", zap.Error(err))
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Start HTTP server in background
|
|
||||||
go func() {
|
|
||||||
if err := server.ListenAndServe(); err != nil && !errors.Is(err, http.ErrServerClosed) {
|
|
||||||
logger.ComponentError(logging.ComponentSFU, "SFU server error", zap.Error(err))
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for termination signal
|
|
||||||
quit := make(chan os.Signal, 1)
|
|
||||||
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
|
|
||||||
sig := <-quit
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentSFU, "Shutdown signal received", zap.String("signal", sig.String()))
|
|
||||||
|
|
||||||
// Graceful drain: notify peers and wait
|
|
||||||
server.Drain(30 * time.Second)
|
|
||||||
|
|
||||||
if err := server.Close(); err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentSFU, "Error during shutdown", zap.Error(err))
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentSFU, "SFU server shutdown complete")
|
|
||||||
}
|
|
||||||
@ -1,100 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/config"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/logging"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/turn"
|
|
||||||
"go.uber.org/zap"
|
|
||||||
)
|
|
||||||
|
|
||||||
func parseTURNConfig(logger *logging.ColoredLogger) *turn.Config {
|
|
||||||
configFlag := flag.String("config", "", "Config file path (absolute path or filename in ~/.orama)")
|
|
||||||
flag.Parse()
|
|
||||||
|
|
||||||
var configPath string
|
|
||||||
var err error
|
|
||||||
if *configFlag != "" {
|
|
||||||
if filepath.IsAbs(*configFlag) {
|
|
||||||
configPath = *configFlag
|
|
||||||
} else {
|
|
||||||
configPath, err = config.DefaultPath(*configFlag)
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentTURN, "Failed to determine config path", zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "Configuration error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
configPath, err = config.DefaultPath("turn.yaml")
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentTURN, "Failed to determine config path", zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "Configuration error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
type yamlCfg struct {
|
|
||||||
ListenAddr string `yaml:"listen_addr"`
|
|
||||||
TURNSListenAddr string `yaml:"turns_listen_addr"`
|
|
||||||
PublicIP string `yaml:"public_ip"`
|
|
||||||
Realm string `yaml:"realm"`
|
|
||||||
AuthSecret string `yaml:"auth_secret"`
|
|
||||||
RelayPortStart int `yaml:"relay_port_start"`
|
|
||||||
RelayPortEnd int `yaml:"relay_port_end"`
|
|
||||||
Namespace string `yaml:"namespace"`
|
|
||||||
TLSCertPath string `yaml:"tls_cert_path"`
|
|
||||||
TLSKeyPath string `yaml:"tls_key_path"`
|
|
||||||
}
|
|
||||||
|
|
||||||
data, err := os.ReadFile(configPath)
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentTURN, "Config file not found",
|
|
||||||
zap.String("path", configPath), zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "\nConfig file not found at %s\n", configPath)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
var y yamlCfg
|
|
||||||
if err := config.DecodeStrict(strings.NewReader(string(data)), &y); err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentTURN, "Failed to parse TURN config", zap.Error(err))
|
|
||||||
fmt.Fprintf(os.Stderr, "Configuration parse error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
cfg := &turn.Config{
|
|
||||||
ListenAddr: y.ListenAddr,
|
|
||||||
TURNSListenAddr: y.TURNSListenAddr,
|
|
||||||
PublicIP: y.PublicIP,
|
|
||||||
Realm: y.Realm,
|
|
||||||
AuthSecret: y.AuthSecret,
|
|
||||||
RelayPortStart: y.RelayPortStart,
|
|
||||||
RelayPortEnd: y.RelayPortEnd,
|
|
||||||
Namespace: y.Namespace,
|
|
||||||
TLSCertPath: y.TLSCertPath,
|
|
||||||
TLSKeyPath: y.TLSKeyPath,
|
|
||||||
}
|
|
||||||
|
|
||||||
if errs := cfg.Validate(); len(errs) > 0 {
|
|
||||||
fmt.Fprintf(os.Stderr, "\nTURN configuration errors (%d):\n", len(errs))
|
|
||||||
for _, e := range errs {
|
|
||||||
fmt.Fprintf(os.Stderr, " - %s\n", e)
|
|
||||||
}
|
|
||||||
fmt.Fprintf(os.Stderr, "\nPlease fix the configuration and try again.\n")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentTURN, "Loaded TURN configuration",
|
|
||||||
zap.String("path", configPath),
|
|
||||||
zap.String("listen_addr", cfg.ListenAddr),
|
|
||||||
zap.String("namespace", cfg.Namespace),
|
|
||||||
zap.String("realm", cfg.Realm),
|
|
||||||
)
|
|
||||||
|
|
||||||
return cfg
|
|
||||||
}
|
|
||||||
@ -1,48 +0,0 @@
|
|||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"os"
|
|
||||||
"os/signal"
|
|
||||||
"syscall"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/logging"
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/turn"
|
|
||||||
"go.uber.org/zap"
|
|
||||||
)
|
|
||||||
|
|
||||||
var (
|
|
||||||
version = "dev"
|
|
||||||
commit = "unknown"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
logger, err := logging.NewColoredLogger(logging.ComponentTURN, true)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentTURN, "Starting TURN server",
|
|
||||||
zap.String("version", version),
|
|
||||||
zap.String("commit", commit))
|
|
||||||
|
|
||||||
cfg := parseTURNConfig(logger)
|
|
||||||
|
|
||||||
server, err := turn.NewServer(cfg, logger.Logger)
|
|
||||||
if err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentTURN, "Failed to start TURN server", zap.Error(err))
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Wait for termination signal
|
|
||||||
quit := make(chan os.Signal, 1)
|
|
||||||
signal.Notify(quit, os.Interrupt, syscall.SIGTERM)
|
|
||||||
sig := <-quit
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentTURN, "Shutdown signal received", zap.String("signal", sig.String()))
|
|
||||||
|
|
||||||
if err := server.Close(); err != nil {
|
|
||||||
logger.ComponentError(logging.ComponentTURN, "Error during shutdown", zap.Error(err))
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.ComponentInfo(logging.ComponentTURN, "TURN server shutdown complete")
|
|
||||||
}
|
|
||||||
@ -1,571 +0,0 @@
|
|||||||
# Orama Network Architecture
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
Orama Network is a high-performance API Gateway and Reverse Proxy designed for a decentralized ecosystem. It serves as a unified entry point that orchestrates traffic between clients and various backend services.
|
|
||||||
|
|
||||||
## Architecture Pattern
|
|
||||||
|
|
||||||
**Modular Gateway / Edge Proxy Architecture**
|
|
||||||
|
|
||||||
The system follows a clean, layered architecture with clear separation of concerns:
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ Clients │
|
|
||||||
│ (Web, Mobile, CLI, SDKs) │
|
|
||||||
└────────────────────────┬────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
│ HTTPS/WSS
|
|
||||||
▼
|
|
||||||
┌─────────────────────────────────────────────────────────────┐
|
|
||||||
│ API Gateway (Port 443) │
|
|
||||||
│ ┌──────────────────────────────────────────────────────┐ │
|
|
||||||
│ │ Handlers Layer (HTTP/WebSocket) │ │
|
|
||||||
│ │ - Auth handlers - Storage handlers │ │
|
|
||||||
│ │ - Cache handlers - PubSub handlers │ │
|
|
||||||
│ │ - Serverless - Database handlers │ │
|
|
||||||
│ └──────────────────────┬───────────────────────────────┘ │
|
|
||||||
│ │ │
|
|
||||||
│ ┌──────────────────────▼───────────────────────────────┐ │
|
|
||||||
│ │ Middleware (Security, Auth, Logging) │ │
|
|
||||||
│ └──────────────────────┬───────────────────────────────┘ │
|
|
||||||
│ │ │
|
|
||||||
│ ┌──────────────────────▼───────────────────────────────┐ │
|
|
||||||
│ │ Service Coordination (Gateway Core) │ │
|
|
||||||
│ └──────────────────────┬───────────────────────────────┘ │
|
|
||||||
└─────────────────────────┼────────────────────────────────────┘
|
|
||||||
│
|
|
||||||
┌─────────────────┼─────────────────┐
|
|
||||||
│ │ │
|
|
||||||
▼ ▼ ▼
|
|
||||||
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
|
|
||||||
│ RQLite │ │ Olric │ │ IPFS │
|
|
||||||
│ (Database) │ │ (Cache) │ │ (Storage) │
|
|
||||||
│ │ │ │ │ │
|
|
||||||
│ Port 5001 │ │ Port 3320 │ │ Port 4501 │
|
|
||||||
└──────────────┘ └──────────────┘ └──────────────┘
|
|
||||||
|
|
||||||
┌─────────────────┐ ┌──────────────┐
|
|
||||||
│ IPFS Cluster │ │ Serverless │
|
|
||||||
│ (Pinning) │ │ (WASM) │
|
|
||||||
│ │ │ │
|
|
||||||
│ Port 9094 │ │ In-Process │
|
|
||||||
└─────────────────┘ └──────────────┘
|
|
||||||
|
|
||||||
┌─────────────────┐
|
|
||||||
│ Anyone │
|
|
||||||
│ (Anonymity) │
|
|
||||||
│ │
|
|
||||||
│ Port 9050 │
|
|
||||||
└─────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
## Core Components
|
|
||||||
|
|
||||||
### 1. API Gateway (`pkg/gateway/`)
|
|
||||||
|
|
||||||
The gateway is the main entry point for all client requests. It coordinates between various backend services.
|
|
||||||
|
|
||||||
**Key Files:**
|
|
||||||
- `gateway.go` - Core gateway struct and routing
|
|
||||||
- `dependencies.go` - Service initialization and dependency injection
|
|
||||||
- `lifecycle.go` - Start/stop/health lifecycle management
|
|
||||||
- `middleware.go` - Authentication, logging, error handling
|
|
||||||
- `routes.go` - HTTP route registration
|
|
||||||
|
|
||||||
**Handler Packages:**
|
|
||||||
- `handlers/auth/` - Authentication (JWT, API keys, wallet signatures)
|
|
||||||
- `handlers/storage/` - IPFS storage operations
|
|
||||||
- `handlers/cache/` - Distributed cache operations
|
|
||||||
- `handlers/pubsub/` - Pub/sub messaging
|
|
||||||
- `handlers/serverless/` - Serverless function deployment and execution
|
|
||||||
|
|
||||||
### 2. Client SDK (`pkg/client/`)
|
|
||||||
|
|
||||||
Provides a clean Go SDK for interacting with the Orama Network.
|
|
||||||
|
|
||||||
**Architecture:**
|
|
||||||
```go
|
|
||||||
// Main client interface
|
|
||||||
type NetworkClient interface {
|
|
||||||
Storage() StorageClient
|
|
||||||
Cache() CacheClient
|
|
||||||
Database() DatabaseClient
|
|
||||||
PubSub() PubSubClient
|
|
||||||
Serverless() ServerlessClient
|
|
||||||
Auth() AuthClient
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key Files:**
|
|
||||||
- `client.go` - Main client orchestration
|
|
||||||
- `config.go` - Client configuration
|
|
||||||
- `storage_client.go` - IPFS storage client
|
|
||||||
- `cache_client.go` - Olric cache client
|
|
||||||
- `database_client.go` - RQLite database client
|
|
||||||
- `pubsub_bridge.go` - Pub/sub messaging client
|
|
||||||
- `transport.go` - HTTP transport layer
|
|
||||||
- `errors.go` - Client-specific errors
|
|
||||||
|
|
||||||
**Usage Example:**
|
|
||||||
```go
|
|
||||||
import "github.com/DeBrosOfficial/network/pkg/client"
|
|
||||||
|
|
||||||
// Create client
|
|
||||||
cfg := client.DefaultClientConfig()
|
|
||||||
cfg.GatewayURL = "https://api.orama.network"
|
|
||||||
cfg.APIKey = "your-api-key"
|
|
||||||
|
|
||||||
c := client.NewNetworkClient(cfg)
|
|
||||||
|
|
||||||
// Use storage
|
|
||||||
resp, err := c.Storage().Upload(ctx, data, "file.txt")
|
|
||||||
|
|
||||||
// Use cache
|
|
||||||
err = c.Cache().Set(ctx, "key", value, 0)
|
|
||||||
|
|
||||||
// Query database
|
|
||||||
rows, err := c.Database().Query(ctx, "SELECT * FROM users")
|
|
||||||
|
|
||||||
// Publish message
|
|
||||||
err = c.PubSub().Publish(ctx, "chat", []byte("hello"))
|
|
||||||
|
|
||||||
// Deploy function
|
|
||||||
fn, err := c.Serverless().Deploy(ctx, def, wasmBytes)
|
|
||||||
|
|
||||||
// Invoke function
|
|
||||||
result, err := c.Serverless().Invoke(ctx, "function-name", input)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Database Layer (`pkg/rqlite/`)
|
|
||||||
|
|
||||||
ORM-like interface over RQLite distributed SQL database.
|
|
||||||
|
|
||||||
**Key Files:**
|
|
||||||
- `client.go` - Main ORM client
|
|
||||||
- `orm_types.go` - Interfaces (Client, Tx, Repository[T])
|
|
||||||
- `query_builder.go` - Fluent query builder
|
|
||||||
- `repository.go` - Generic repository pattern
|
|
||||||
- `scanner.go` - Reflection-based row scanning
|
|
||||||
- `transaction.go` - Transaction support
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
- Fluent query builder
|
|
||||||
- Generic repository pattern with type safety
|
|
||||||
- Automatic struct mapping
|
|
||||||
- Transaction support
|
|
||||||
- Connection pooling with retry
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
```go
|
|
||||||
// Query builder
|
|
||||||
users, err := client.CreateQueryBuilder("users").
|
|
||||||
Select("id", "name", "email").
|
|
||||||
Where("age > ?", 18).
|
|
||||||
OrderBy("name ASC").
|
|
||||||
Limit(10).
|
|
||||||
GetMany(ctx, &users)
|
|
||||||
|
|
||||||
// Repository pattern
|
|
||||||
type User struct {
|
|
||||||
ID int `db:"id"`
|
|
||||||
Name string `db:"name"`
|
|
||||||
Email string `db:"email"`
|
|
||||||
}
|
|
||||||
|
|
||||||
repo := client.Repository("users")
|
|
||||||
user := &User{Name: "Alice", Email: "alice@example.com"}
|
|
||||||
err := repo.Save(ctx, user)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Serverless Engine (`pkg/serverless/`)
|
|
||||||
|
|
||||||
WebAssembly (WASM) function execution engine with host functions.
|
|
||||||
|
|
||||||
**Architecture:**
|
|
||||||
```
|
|
||||||
pkg/serverless/
|
|
||||||
├── engine.go - Core WASM engine
|
|
||||||
├── execution/ - Function execution
|
|
||||||
│ ├── executor.go
|
|
||||||
│ └── lifecycle.go
|
|
||||||
├── cache/ - Module caching
|
|
||||||
│ └── module_cache.go
|
|
||||||
├── registry/ - Function metadata
|
|
||||||
│ ├── registry.go
|
|
||||||
│ ├── function_store.go
|
|
||||||
│ ├── ipfs_store.go
|
|
||||||
│ └── invocation_logger.go
|
|
||||||
└── hostfunctions/ - Host functions by domain
|
|
||||||
├── cache.go - Cache operations
|
|
||||||
├── storage.go - Storage operations
|
|
||||||
├── database.go - Database queries
|
|
||||||
├── pubsub.go - Messaging
|
|
||||||
├── http.go - HTTP requests
|
|
||||||
└── logging.go - Logging
|
|
||||||
```
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
- Secure WASM execution sandbox
|
|
||||||
- Memory and CPU limits
|
|
||||||
- Host function injection (cache, storage, DB, HTTP)
|
|
||||||
- Function versioning
|
|
||||||
- Invocation logging
|
|
||||||
- Hot module reloading
|
|
||||||
|
|
||||||
### 5. Configuration System (`pkg/config/`)
|
|
||||||
|
|
||||||
Domain-specific configuration with validation.
|
|
||||||
|
|
||||||
**Structure:**
|
|
||||||
```
|
|
||||||
pkg/config/
|
|
||||||
├── config.go - Main config aggregator
|
|
||||||
├── loader.go - YAML loading
|
|
||||||
├── node_config.go - Node settings
|
|
||||||
├── database_config.go - Database settings
|
|
||||||
├── gateway_config.go - Gateway settings
|
|
||||||
└── validate/ - Validation
|
|
||||||
├── validators.go
|
|
||||||
├── node.go
|
|
||||||
├── database.go
|
|
||||||
└── gateway.go
|
|
||||||
```
|
|
||||||
|
|
||||||
### 6. Anyone Integration (`pkg/anyoneproxy/`)
|
|
||||||
|
|
||||||
Integration with the Anyone Protocol for anonymous routing.
|
|
||||||
|
|
||||||
**Modes:**
|
|
||||||
|
|
||||||
| Mode | Purpose | Port | Rewards |
|
|
||||||
|------|---------|------|---------|
|
|
||||||
| Client | Route traffic anonymously | 9050 (SOCKS5) | No |
|
|
||||||
| Relay | Provide bandwidth to network | 9001 (ORPort) + 9050 | Yes ($ANYONE) |
|
|
||||||
|
|
||||||
**Key Files:**
|
|
||||||
- `pkg/anyoneproxy/socks.go` - SOCKS5 proxy client interface
|
|
||||||
- `pkg/gateway/anon_proxy_handler.go` - Anonymous proxy API endpoint
|
|
||||||
- `pkg/environments/production/installers/anyone_relay.go` - Relay installation
|
|
||||||
|
|
||||||
**Features:**
|
|
||||||
- Smart routing (bypasses proxy for local/private addresses)
|
|
||||||
- Automatic detection of existing Anyone installations
|
|
||||||
- Migration support for existing relay operators
|
|
||||||
- Exit relay mode with legal warnings
|
|
||||||
|
|
||||||
**API Endpoint:**
|
|
||||||
- `POST /v1/proxy/anon` - Route HTTP requests through Anyone network
|
|
||||||
|
|
||||||
**Relay Requirements:**
|
|
||||||
- Linux OS (Debian/Ubuntu)
|
|
||||||
- 100 $ANYONE tokens in wallet
|
|
||||||
- ORPort accessible from internet
|
|
||||||
- Registration at dashboard.anyone.io
|
|
||||||
|
|
||||||
### 7. Shared Utilities
|
|
||||||
|
|
||||||
**HTTP Utilities (`pkg/httputil/`):**
|
|
||||||
- Request parsing and validation
|
|
||||||
- JSON response writers
|
|
||||||
- Error handling
|
|
||||||
- Authentication extraction
|
|
||||||
|
|
||||||
**Error Handling (`pkg/errors/`):**
|
|
||||||
- Typed errors (ValidationError, NotFoundError, etc.)
|
|
||||||
- HTTP status code mapping
|
|
||||||
- Error wrapping with context
|
|
||||||
- Stack traces
|
|
||||||
|
|
||||||
**Contracts (`pkg/contracts/`):**
|
|
||||||
- Interface definitions for all services
|
|
||||||
- Enables dependency injection
|
|
||||||
- Clean abstractions
|
|
||||||
|
|
||||||
## Data Flow
|
|
||||||
|
|
||||||
### 1. HTTP Request Flow
|
|
||||||
|
|
||||||
```
|
|
||||||
Client Request
|
|
||||||
↓
|
|
||||||
[HTTPS Termination]
|
|
||||||
↓
|
|
||||||
[Authentication Middleware]
|
|
||||||
↓
|
|
||||||
[Route Handler]
|
|
||||||
↓
|
|
||||||
[Service Layer]
|
|
||||||
↓
|
|
||||||
[Backend Service] (RQLite/Olric/IPFS)
|
|
||||||
↓
|
|
||||||
[Response Formatting]
|
|
||||||
↓
|
|
||||||
Client Response
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. WebSocket Flow (Pub/Sub)
|
|
||||||
|
|
||||||
```
|
|
||||||
Client WebSocket Connect
|
|
||||||
↓
|
|
||||||
[Upgrade to WebSocket]
|
|
||||||
↓
|
|
||||||
[Authentication]
|
|
||||||
↓
|
|
||||||
[Subscribe to Topic]
|
|
||||||
↓
|
|
||||||
[LibP2P PubSub] ←→ [Local Subscribers]
|
|
||||||
↓
|
|
||||||
[Message Broadcasting]
|
|
||||||
↓
|
|
||||||
Client Receives Messages
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Serverless Invocation Flow
|
|
||||||
|
|
||||||
```
|
|
||||||
Function Deployment:
|
|
||||||
Upload WASM → Store in IPFS → Save Metadata (RQLite) → Compile Module
|
|
||||||
|
|
||||||
Function Invocation:
|
|
||||||
Request → Load Metadata → Get WASM from IPFS →
|
|
||||||
Execute in Sandbox → Return Result → Log Invocation
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Architecture
|
|
||||||
|
|
||||||
### Authentication Methods
|
|
||||||
|
|
||||||
1. **Wallet Signatures** (Ethereum-style)
|
|
||||||
- Challenge/response flow
|
|
||||||
- Nonce-based to prevent replay attacks
|
|
||||||
- Issues JWT tokens after verification
|
|
||||||
|
|
||||||
2. **API Keys**
|
|
||||||
- Long-lived credentials
|
|
||||||
- Stored in RQLite
|
|
||||||
- Namespace-scoped
|
|
||||||
|
|
||||||
3. **JWT Tokens**
|
|
||||||
- Short-lived (15 min default)
|
|
||||||
- Refresh token support
|
|
||||||
- Claims-based authorization
|
|
||||||
|
|
||||||
### Network Security (WireGuard Mesh)
|
|
||||||
|
|
||||||
All inter-node communication is encrypted via a WireGuard VPN mesh:
|
|
||||||
|
|
||||||
- **WireGuard IPs:** Each node gets a private IP (10.0.0.x/24) used for all cluster traffic
|
|
||||||
- **UFW Firewall:** Only public ports are exposed: 22 (SSH), 53 (DNS, nameservers only), 80/443 (HTTP/HTTPS), 51820 (WireGuard UDP)
|
|
||||||
- **IPv6 disabled:** System-wide via sysctl to prevent bypass of IPv4 firewall rules
|
|
||||||
- **Internal services** (RQLite 5001/7001, IPFS 4001/4501, Olric 3320/3322, Gateway 6001) are only accessible via WireGuard or localhost
|
|
||||||
- **Invite tokens:** Single-use, time-limited tokens for secure node joining. No shared secrets on the CLI
|
|
||||||
- **Join flow:** New nodes authenticate via HTTPS (443) with TOFU certificate pinning, establish WireGuard tunnel, then join all services over the encrypted mesh
|
|
||||||
|
|
||||||
### Service Authentication
|
|
||||||
|
|
||||||
- **RQLite:** HTTP basic auth on all queries/executions — credentials generated at genesis, distributed via join response
|
|
||||||
- **Olric:** Memberlist gossip encrypted with a shared 32-byte key
|
|
||||||
- **IPFS Cluster:** TrustedPeers restricted to known cluster peer IDs (not `*`)
|
|
||||||
- **Internal endpoints:** `/v1/internal/wg/peers` and `/v1/internal/wg/peer/remove` require cluster secret
|
|
||||||
- **Vault:** V1 push/pull endpoints require session token authentication when guardian is configured
|
|
||||||
- **WebSockets:** Origin header validated against the node's configured domain
|
|
||||||
|
|
||||||
### Token & Key Security
|
|
||||||
|
|
||||||
- **Refresh tokens:** Stored as SHA-256 hashes (never plaintext)
|
|
||||||
- **API keys:** Stored as HMAC-SHA256 hashes with a server-side secret
|
|
||||||
- **TURN secrets:** Encrypted at rest with AES-256-GCM (key derived from cluster secret)
|
|
||||||
- **Binary signing:** Build archives signed with rootwallet EVM signature, verified on install
|
|
||||||
|
|
||||||
### Process Isolation
|
|
||||||
|
|
||||||
- **Dedicated user:** All services run as `orama` user (not root)
|
|
||||||
- **systemd hardening:** `ProtectSystem=strict`, `NoNewPrivileges=yes`, `PrivateDevices=yes`, etc.
|
|
||||||
- **Capabilities:** Caddy and CoreDNS get `CAP_NET_BIND_SERVICE` for privileged ports
|
|
||||||
|
|
||||||
See [SECURITY.md](SECURITY.md) for the full security hardening reference.
|
|
||||||
|
|
||||||
### TLS/HTTPS
|
|
||||||
|
|
||||||
- Automatic ACME (Let's Encrypt) certificate management via Caddy
|
|
||||||
- TLS 1.3 support
|
|
||||||
- HTTP/2 enabled
|
|
||||||
- On-demand TLS for deployment custom domains
|
|
||||||
|
|
||||||
### Middleware Stack
|
|
||||||
|
|
||||||
1. **Logger** - Request/response logging
|
|
||||||
2. **CORS** - Cross-origin resource sharing
|
|
||||||
3. **Authentication** - JWT/API key validation
|
|
||||||
4. **Authorization** - Namespace access control
|
|
||||||
5. **Rate Limiting** - Per-client rate limits
|
|
||||||
6. **Error Handling** - Consistent error responses
|
|
||||||
|
|
||||||
## Scalability
|
|
||||||
|
|
||||||
### Horizontal Scaling
|
|
||||||
|
|
||||||
- **Gateway:** Stateless, can run multiple instances behind load balancer
|
|
||||||
- **RQLite:** Multi-node cluster with Raft consensus
|
|
||||||
- **IPFS:** Distributed storage across nodes
|
|
||||||
- **Olric:** Distributed cache with consistent hashing
|
|
||||||
|
|
||||||
### Caching Strategy
|
|
||||||
|
|
||||||
1. **WASM Module Cache** - Compiled modules cached in memory
|
|
||||||
2. **Olric Distributed Cache** - Shared cache across nodes
|
|
||||||
3. **Local Cache** - Per-gateway request caching
|
|
||||||
|
|
||||||
### High Availability
|
|
||||||
|
|
||||||
- **Database:** RQLite cluster with automatic leader election
|
|
||||||
- **Storage:** IPFS replication factor configurable
|
|
||||||
- **Cache:** Olric replication and eventual consistency
|
|
||||||
- **Gateway:** Stateless, multiple replicas supported
|
|
||||||
|
|
||||||
## Monitoring & Observability
|
|
||||||
|
|
||||||
### Health Checks
|
|
||||||
|
|
||||||
- `/health` - Liveness probe
|
|
||||||
- `/v1/status` - Detailed status with service checks
|
|
||||||
|
|
||||||
### Metrics
|
|
||||||
|
|
||||||
- Prometheus-compatible metrics endpoint
|
|
||||||
- Request counts, latencies, error rates
|
|
||||||
- Service-specific metrics (cache hit ratio, DB query times)
|
|
||||||
|
|
||||||
### Logging
|
|
||||||
|
|
||||||
- Structured logging (JSON format)
|
|
||||||
- Log levels: DEBUG, INFO, WARN, ERROR
|
|
||||||
- Correlation IDs for request tracing
|
|
||||||
|
|
||||||
## Development Patterns
|
|
||||||
|
|
||||||
### SOLID Principles
|
|
||||||
|
|
||||||
- **Single Responsibility:** Each handler/service has one focus
|
|
||||||
- **Open/Closed:** Interface-based design for extensibility
|
|
||||||
- **Liskov Substitution:** All implementations conform to contracts
|
|
||||||
- **Interface Segregation:** Small, focused interfaces
|
|
||||||
- **Dependency Inversion:** Depend on abstractions, not implementations
|
|
||||||
|
|
||||||
### Code Organization
|
|
||||||
|
|
||||||
- **Average file size:** ~150 lines
|
|
||||||
- **Package structure:** Domain-driven, feature-focused
|
|
||||||
- **Testing:** Unit tests for logic, E2E tests for integration
|
|
||||||
- **Documentation:** Godoc comments on all public APIs
|
|
||||||
|
|
||||||
## Deployment
|
|
||||||
|
|
||||||
### Building & Testing
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make build # Build all binaries
|
|
||||||
make test # Run unit tests
|
|
||||||
make test-e2e # Run E2E tests
|
|
||||||
```
|
|
||||||
|
|
||||||
### Production
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# First node (genesis — creates cluster)
|
|
||||||
# Nameserver nodes use the base domain as --domain
|
|
||||||
sudo orama install --vps-ip <IP> --domain example.com --base-domain example.com --nameserver
|
|
||||||
|
|
||||||
# On the genesis node, generate an invite for a new node
|
|
||||||
orama invite
|
|
||||||
# Outputs: sudo orama install --join https://example.com --token <TOKEN> --vps-ip <NEW_IP>
|
|
||||||
|
|
||||||
# Additional nameserver nodes (join via invite token over HTTPS)
|
|
||||||
sudo orama install --join https://example.com --token <TOKEN> \
|
|
||||||
--vps-ip <IP> --domain example.com --base-domain example.com --nameserver
|
|
||||||
```
|
|
||||||
|
|
||||||
**Security:** Nodes join via single-use invite tokens over HTTPS. A WireGuard VPN tunnel
|
|
||||||
is established before any cluster services start. All inter-node traffic (RQLite, IPFS,
|
|
||||||
Olric, LibP2P) flows over the encrypted WireGuard mesh — no cluster ports are exposed
|
|
||||||
publicly. **Never use `http://<ip>:6001`** for joining — port 6001 is internal-only and
|
|
||||||
blocked by UFW. Use the domain (`https://node1.example.com`) or, if DNS is not yet
|
|
||||||
configured, use the IP over HTTP port 80 (`http://<ip>`) which goes through Caddy.
|
|
||||||
|
|
||||||
### Docker (Future)
|
|
||||||
|
|
||||||
Planned containerization with Docker Compose and Kubernetes support.
|
|
||||||
|
|
||||||
## WebRTC (Voice/Video/Data)
|
|
||||||
|
|
||||||
Namespaces can opt in to WebRTC support for real-time voice, video, and data channels.
|
|
||||||
|
|
||||||
### Components
|
|
||||||
|
|
||||||
- **SFU (Selective Forwarding Unit)** — Pion WebRTC server that handles signaling (WebSocket), SDP negotiation, and RTP forwarding. Runs on all 3 cluster nodes, binds only to WireGuard IPs.
|
|
||||||
- **TURN Server** — Pion TURN relay that provides NAT traversal. Runs on 2 of 3 nodes for redundancy. Public-facing (UDP 3478, 443, relay range 49152-65535).
|
|
||||||
|
|
||||||
### Security Model
|
|
||||||
|
|
||||||
- **TURN-shielded**: SFU binds only to WireGuard (10.0.0.x), never 0.0.0.0. All client media flows through TURN relay.
|
|
||||||
- **Forced relay**: `iceTransportPolicy: relay` enforced server-side — no direct peer connections.
|
|
||||||
- **HMAC credentials**: Per-namespace TURN shared secret with 10-minute TTL.
|
|
||||||
- **Namespace isolation**: Each namespace has its own TURN secret, port ranges, and rooms.
|
|
||||||
|
|
||||||
### Port Allocation
|
|
||||||
|
|
||||||
WebRTC uses a separate port allocation system from core namespace services:
|
|
||||||
|
|
||||||
| Service | Port Range |
|
|
||||||
|---------|-----------|
|
|
||||||
| SFU signaling | 30000-30099 |
|
|
||||||
| SFU media (RTP) | 20000-29999 |
|
|
||||||
| TURN listen | 3478/udp (standard) |
|
|
||||||
| TURN TLS | 443/udp |
|
|
||||||
| TURN relay | 49152-65535/udp |
|
|
||||||
|
|
||||||
See [docs/WEBRTC.md](WEBRTC.md) for full details including client integration, API reference, and debugging.
|
|
||||||
|
|
||||||
## OramaOS
|
|
||||||
|
|
||||||
For mainnet, devnet, and testnet environments, nodes run **OramaOS** — a custom minimal Linux image built with Buildroot.
|
|
||||||
|
|
||||||
**Key properties:**
|
|
||||||
- No SSH, no shell — operators cannot access the filesystem
|
|
||||||
- LUKS full-disk encryption with Shamir key distribution across peers
|
|
||||||
- Read-only rootfs (SquashFS + dm-verity)
|
|
||||||
- A/B partition updates with cryptographic signature verification
|
|
||||||
- Service sandboxing via Linux namespaces + seccomp
|
|
||||||
- Single root process: the **orama-agent**
|
|
||||||
|
|
||||||
**The orama-agent manages:**
|
|
||||||
- Boot sequence and LUKS key reconstruction
|
|
||||||
- WireGuard tunnel setup
|
|
||||||
- Service lifecycle in sandboxed namespaces
|
|
||||||
- Command reception from Gateway over WireGuard (port 9998)
|
|
||||||
- OS updates (download, verify, A/B swap, reboot with rollback)
|
|
||||||
|
|
||||||
**Node enrollment:** OramaOS nodes join via `orama node enroll` instead of `orama node install`. The enrollment flow uses a registration code + invite token + wallet verification.
|
|
||||||
|
|
||||||
See [ORAMAOS_DEPLOYMENT.md](ORAMAOS_DEPLOYMENT.md) for the full deployment guide.
|
|
||||||
|
|
||||||
Sandbox clusters remain on Ubuntu for development convenience.
|
|
||||||
|
|
||||||
## Future Enhancements
|
|
||||||
|
|
||||||
1. **GraphQL Support** - GraphQL gateway alongside REST
|
|
||||||
2. **gRPC Support** - gRPC protocol support
|
|
||||||
3. **Event Sourcing** - Event-driven architecture
|
|
||||||
4. **Kubernetes Operator** - Native K8s deployment
|
|
||||||
5. **Observability** - OpenTelemetry integration
|
|
||||||
6. **Multi-tenancy** - Enhanced namespace isolation
|
|
||||||
|
|
||||||
## Resources
|
|
||||||
|
|
||||||
- [RQLite Documentation](https://rqlite.io/docs/)
|
|
||||||
- [IPFS Documentation](https://docs.ipfs.tech/)
|
|
||||||
- [LibP2P Documentation](https://docs.libp2p.io/)
|
|
||||||
- [WebAssembly (WASM)](https://webassembly.org/)
|
|
||||||
@ -1,151 +0,0 @@
|
|||||||
# Clean Node — Full Reset Guide
|
|
||||||
|
|
||||||
How to completely remove all Orama Network state from a VPS so it can be reinstalled fresh.
|
|
||||||
|
|
||||||
> **OramaOS nodes:** This guide applies to Ubuntu-based nodes only. OramaOS has no SSH or shell access. To remove an OramaOS node: use `POST /v1/node/leave` via the Gateway API for graceful departure, or reflash the OramaOS image via your VPS provider's dashboard for a factory reset. See [ORAMAOS_DEPLOYMENT.md](ORAMAOS_DEPLOYMENT.md) for details.
|
|
||||||
|
|
||||||
## Quick Clean (Copy-Paste)
|
|
||||||
|
|
||||||
Run this as root or with sudo on the target VPS:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Stop and disable all services
|
|
||||||
sudo systemctl stop orama-node orama-ipfs orama-ipfs-cluster orama-olric orama-anyone-relay orama-anyone-client coredns caddy 2>/dev/null
|
|
||||||
sudo systemctl disable orama-node orama-ipfs orama-ipfs-cluster orama-olric orama-anyone-relay orama-anyone-client coredns caddy 2>/dev/null
|
|
||||||
|
|
||||||
# 1b. Kill leftover processes (binaries may run outside systemd)
|
|
||||||
sudo pkill -f orama-node 2>/dev/null; sudo pkill -f ipfs-cluster-service 2>/dev/null
|
|
||||||
sudo pkill -f "ipfs daemon" 2>/dev/null; sudo pkill -f olric-server 2>/dev/null
|
|
||||||
sudo pkill -f rqlited 2>/dev/null; sudo pkill -f coredns 2>/dev/null
|
|
||||||
sleep 1
|
|
||||||
|
|
||||||
# 2. Remove systemd service files
|
|
||||||
sudo rm -f /etc/systemd/system/orama-*.service
|
|
||||||
sudo rm -f /etc/systemd/system/coredns.service
|
|
||||||
sudo rm -f /etc/systemd/system/caddy.service
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
|
|
||||||
# 3. Tear down WireGuard
|
|
||||||
# Must stop the systemd unit first — wg-quick@wg0 is a oneshot with
|
|
||||||
# RemainAfterExit=yes, so it stays "active (exited)" even after the
|
|
||||||
# interface is removed. Without "stop", a future "systemctl start" is a no-op.
|
|
||||||
sudo systemctl stop wg-quick@wg0 2>/dev/null
|
|
||||||
sudo wg-quick down wg0 2>/dev/null
|
|
||||||
sudo systemctl disable wg-quick@wg0 2>/dev/null
|
|
||||||
sudo rm -f /etc/wireguard/wg0.conf
|
|
||||||
|
|
||||||
# 4. Reset UFW firewall
|
|
||||||
sudo ufw --force reset
|
|
||||||
sudo ufw allow 22/tcp
|
|
||||||
sudo ufw --force enable
|
|
||||||
|
|
||||||
# 5. Remove orama data directory
|
|
||||||
sudo rm -rf /opt/orama
|
|
||||||
|
|
||||||
# 6. Remove legacy orama user (if exists from old installs)
|
|
||||||
sudo userdel -r orama 2>/dev/null
|
|
||||||
sudo rm -rf /home/orama
|
|
||||||
sudo rm -f /etc/sudoers.d/orama-access
|
|
||||||
sudo rm -f /etc/sudoers.d/orama-deployments
|
|
||||||
sudo rm -f /etc/sudoers.d/orama-wireguard
|
|
||||||
|
|
||||||
# 7. Remove CoreDNS config
|
|
||||||
sudo rm -rf /etc/coredns
|
|
||||||
|
|
||||||
# 8. Remove Caddy config and data
|
|
||||||
sudo rm -rf /etc/caddy
|
|
||||||
sudo rm -rf /var/lib/caddy
|
|
||||||
|
|
||||||
# 9. Remove deployment systemd services (dynamic)
|
|
||||||
sudo rm -f /etc/systemd/system/orama-deploy-*.service
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
|
|
||||||
# 10. Clean temp files
|
|
||||||
sudo rm -f /tmp/orama /tmp/network-source.tar.gz /tmp/network-source.zip
|
|
||||||
sudo rm -rf /tmp/network-extract /tmp/coredns-build /tmp/caddy-build
|
|
||||||
|
|
||||||
echo "Node cleaned. Ready for fresh install."
|
|
||||||
```
|
|
||||||
|
|
||||||
## What This Removes
|
|
||||||
|
|
||||||
| Category | Paths |
|
|
||||||
|----------|-------|
|
|
||||||
| **App data** | `/opt/orama/.orama/` (configs, secrets, logs, IPFS, RQLite, Olric) |
|
|
||||||
| **Source code** | `/opt/orama/src/` |
|
|
||||||
| **Binaries** | `/opt/orama/bin/orama-node`, `/opt/orama/bin/gateway` |
|
|
||||||
| **Systemd** | `orama-*.service`, `coredns.service`, `caddy.service`, `orama-deploy-*.service` |
|
|
||||||
| **WireGuard** | `/etc/wireguard/wg0.conf`, `wg-quick@wg0` systemd unit |
|
|
||||||
| **Firewall** | All UFW rules (reset to default + SSH only) |
|
|
||||||
| **Legacy** | `orama` user, `/etc/sudoers.d/orama-*` (old installs only) |
|
|
||||||
| **CoreDNS** | `/etc/coredns/Corefile` |
|
|
||||||
| **Caddy** | `/etc/caddy/Caddyfile`, `/var/lib/caddy/` (TLS certs) |
|
|
||||||
| **Anyone Relay** | `orama-anyone-relay.service`, `orama-anyone-client.service` |
|
|
||||||
| **Temp files** | `/tmp/orama`, `/tmp/network-source.*`, build dirs |
|
|
||||||
|
|
||||||
## What This Does NOT Remove
|
|
||||||
|
|
||||||
These are shared system tools that may be used by other software. Remove manually if desired:
|
|
||||||
|
|
||||||
| Binary | Path | Remove Command |
|
|
||||||
|--------|------|----------------|
|
|
||||||
| RQLite | `/usr/local/bin/rqlited` | `sudo rm /usr/local/bin/rqlited` |
|
|
||||||
| IPFS | `/usr/local/bin/ipfs` | `sudo rm /usr/local/bin/ipfs` |
|
|
||||||
| IPFS Cluster | `/usr/local/bin/ipfs-cluster-service` | `sudo rm /usr/local/bin/ipfs-cluster-service` |
|
|
||||||
| Olric | `/usr/local/bin/olric-server` | `sudo rm /usr/local/bin/olric-server` |
|
|
||||||
| CoreDNS | `/usr/local/bin/coredns` | `sudo rm /usr/local/bin/coredns` |
|
|
||||||
| Caddy | `/usr/bin/caddy` | `sudo rm /usr/bin/caddy` |
|
|
||||||
| xcaddy | `/usr/local/bin/xcaddy` | `sudo rm /usr/local/bin/xcaddy` |
|
|
||||||
| Go | `/usr/local/go/` | `sudo rm -rf /usr/local/go` |
|
|
||||||
| Orama CLI | `/usr/local/bin/orama` | `sudo rm /usr/local/bin/orama` |
|
|
||||||
|
|
||||||
## Nuclear Clean (Remove Everything Including Binaries)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Run quick clean above first, then:
|
|
||||||
sudo rm -f /usr/local/bin/rqlited
|
|
||||||
sudo rm -f /usr/local/bin/ipfs
|
|
||||||
sudo rm -f /usr/local/bin/ipfs-cluster-service
|
|
||||||
sudo rm -f /usr/local/bin/olric-server
|
|
||||||
sudo rm -f /usr/local/bin/coredns
|
|
||||||
sudo rm -f /usr/local/bin/xcaddy
|
|
||||||
sudo rm -f /usr/bin/caddy
|
|
||||||
sudo rm -f /usr/local/bin/orama
|
|
||||||
```
|
|
||||||
|
|
||||||
## Multi-Node Clean
|
|
||||||
|
|
||||||
To clean all nodes at once from your local machine:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Define your nodes
|
|
||||||
NODES=(
|
|
||||||
"ubuntu@141.227.165.168:password1"
|
|
||||||
"ubuntu@141.227.165.154:password2"
|
|
||||||
"ubuntu@141.227.156.51:password3"
|
|
||||||
)
|
|
||||||
|
|
||||||
for entry in "${NODES[@]}"; do
|
|
||||||
IFS=: read -r userhost pass <<< "$entry"
|
|
||||||
echo "Cleaning $userhost..."
|
|
||||||
sshpass -p "$pass" ssh -o StrictHostKeyChecking=no "$userhost" 'bash -s' << 'CLEAN'
|
|
||||||
sudo systemctl stop orama-node orama-ipfs orama-ipfs-cluster orama-olric orama-anyone-relay orama-anyone-client coredns caddy 2>/dev/null
|
|
||||||
sudo systemctl disable orama-node orama-ipfs orama-ipfs-cluster orama-olric orama-anyone-relay orama-anyone-client coredns caddy 2>/dev/null
|
|
||||||
sudo rm -f /etc/systemd/system/orama-*.service /etc/systemd/system/coredns.service /etc/systemd/system/caddy.service /etc/systemd/system/orama-deploy-*.service
|
|
||||||
sudo systemctl daemon-reload
|
|
||||||
sudo systemctl stop wg-quick@wg0 2>/dev/null
|
|
||||||
sudo wg-quick down wg0 2>/dev/null
|
|
||||||
sudo systemctl disable wg-quick@wg0 2>/dev/null
|
|
||||||
sudo rm -f /etc/wireguard/wg0.conf
|
|
||||||
sudo ufw --force reset && sudo ufw allow 22/tcp && sudo ufw --force enable
|
|
||||||
sudo rm -rf /opt/orama
|
|
||||||
sudo userdel -r orama 2>/dev/null
|
|
||||||
sudo rm -rf /home/orama
|
|
||||||
sudo rm -f /etc/sudoers.d/orama-access /etc/sudoers.d/orama-deployments /etc/sudoers.d/orama-wireguard
|
|
||||||
sudo rm -rf /etc/coredns /etc/caddy /var/lib/caddy
|
|
||||||
sudo rm -f /tmp/orama /tmp/network-source.tar.gz
|
|
||||||
sudo rm -rf /tmp/network-extract /tmp/coredns-build /tmp/caddy-build
|
|
||||||
echo "Done"
|
|
||||||
CLEAN
|
|
||||||
done
|
|
||||||
```
|
|
||||||
@ -1,546 +0,0 @@
|
|||||||
# Orama Network Client SDK
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The Orama Network Client SDK provides a clean, type-safe Go interface for interacting with the Orama Network. It abstracts away the complexity of HTTP requests, authentication, and error handling.
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
go get github.com/DeBrosOfficial/network/pkg/client
|
|
||||||
```
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
```go
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"fmt"
|
|
||||||
"log"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/client"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
// Create client configuration
|
|
||||||
cfg := client.DefaultClientConfig()
|
|
||||||
cfg.GatewayURL = "https://api.orama.network"
|
|
||||||
cfg.APIKey = "your-api-key-here"
|
|
||||||
|
|
||||||
// Create client
|
|
||||||
c := client.NewNetworkClient(cfg)
|
|
||||||
|
|
||||||
// Use the client
|
|
||||||
ctx := context.Background()
|
|
||||||
|
|
||||||
// Upload to storage
|
|
||||||
data := []byte("Hello, Orama!")
|
|
||||||
resp, err := c.Storage().Upload(ctx, data, "hello.txt")
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Uploaded: CID=%s\n", resp.CID)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### ClientConfig
|
|
||||||
|
|
||||||
```go
|
|
||||||
type ClientConfig struct {
|
|
||||||
// Gateway URL (e.g., "https://api.orama.network")
|
|
||||||
GatewayURL string
|
|
||||||
|
|
||||||
// Authentication (choose one)
|
|
||||||
APIKey string // API key authentication
|
|
||||||
JWTToken string // JWT token authentication
|
|
||||||
|
|
||||||
// Client options
|
|
||||||
Timeout time.Duration // Request timeout (default: 30s)
|
|
||||||
UserAgent string // Custom user agent
|
|
||||||
|
|
||||||
// Network client namespace
|
|
||||||
Namespace string // Default namespace for operations
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Creating a Client
|
|
||||||
|
|
||||||
```go
|
|
||||||
// Default configuration
|
|
||||||
cfg := client.DefaultClientConfig()
|
|
||||||
cfg.GatewayURL = "https://api.orama.network"
|
|
||||||
cfg.APIKey = "your-api-key"
|
|
||||||
|
|
||||||
c := client.NewNetworkClient(cfg)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Authentication
|
|
||||||
|
|
||||||
### API Key Authentication
|
|
||||||
|
|
||||||
```go
|
|
||||||
cfg := client.DefaultClientConfig()
|
|
||||||
cfg.APIKey = "your-api-key-here"
|
|
||||||
c := client.NewNetworkClient(cfg)
|
|
||||||
```
|
|
||||||
|
|
||||||
### JWT Token Authentication
|
|
||||||
|
|
||||||
```go
|
|
||||||
cfg := client.DefaultClientConfig()
|
|
||||||
cfg.JWTToken = "your-jwt-token-here"
|
|
||||||
c := client.NewNetworkClient(cfg)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Obtaining Credentials
|
|
||||||
|
|
||||||
```go
|
|
||||||
// 1. Login with wallet signature (not yet implemented in SDK)
|
|
||||||
// Use the gateway API directly: POST /v1/auth/challenge + /v1/auth/verify
|
|
||||||
|
|
||||||
// 2. Issue API key after authentication
|
|
||||||
// POST /v1/auth/apikey with JWT token
|
|
||||||
```
|
|
||||||
|
|
||||||
## Storage Client
|
|
||||||
|
|
||||||
Upload, download, pin, and unpin files to IPFS.
|
|
||||||
|
|
||||||
### Upload File
|
|
||||||
|
|
||||||
```go
|
|
||||||
data := []byte("Hello, World!")
|
|
||||||
resp, err := c.Storage().Upload(ctx, data, "hello.txt")
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("CID: %s\n", resp.CID)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Upload with Options
|
|
||||||
|
|
||||||
```go
|
|
||||||
opts := &client.StorageUploadOptions{
|
|
||||||
Pin: true, // Pin after upload
|
|
||||||
Encrypt: true, // Encrypt before upload
|
|
||||||
ReplicationFactor: 3, // Number of replicas
|
|
||||||
}
|
|
||||||
resp, err := c.Storage().UploadWithOptions(ctx, data, "file.txt", opts)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Get File
|
|
||||||
|
|
||||||
```go
|
|
||||||
cid := "QmXxx..."
|
|
||||||
data, err := c.Storage().Get(ctx, cid)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Downloaded %d bytes\n", len(data))
|
|
||||||
```
|
|
||||||
|
|
||||||
### Pin File
|
|
||||||
|
|
||||||
```go
|
|
||||||
cid := "QmXxx..."
|
|
||||||
resp, err := c.Storage().Pin(ctx, cid)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Pinned: %s\n", resp.CID)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Unpin File
|
|
||||||
|
|
||||||
```go
|
|
||||||
cid := "QmXxx..."
|
|
||||||
err := c.Storage().Unpin(ctx, cid)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Println("Unpinned successfully")
|
|
||||||
```
|
|
||||||
|
|
||||||
### Check Pin Status
|
|
||||||
|
|
||||||
```go
|
|
||||||
cid := "QmXxx..."
|
|
||||||
status, err := c.Storage().Status(ctx, cid)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Status: %s, Replicas: %d\n", status.Status, status.Replicas)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Cache Client
|
|
||||||
|
|
||||||
Distributed key-value cache using Olric.
|
|
||||||
|
|
||||||
### Set Value
|
|
||||||
|
|
||||||
```go
|
|
||||||
key := "user:123"
|
|
||||||
value := map[string]interface{}{
|
|
||||||
"name": "Alice",
|
|
||||||
"email": "alice@example.com",
|
|
||||||
}
|
|
||||||
ttl := 5 * time.Minute
|
|
||||||
|
|
||||||
err := c.Cache().Set(ctx, key, value, ttl)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Get Value
|
|
||||||
|
|
||||||
```go
|
|
||||||
key := "user:123"
|
|
||||||
var user map[string]interface{}
|
|
||||||
err := c.Cache().Get(ctx, key, &user)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("User: %+v\n", user)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Delete Value
|
|
||||||
|
|
||||||
```go
|
|
||||||
key := "user:123"
|
|
||||||
err := c.Cache().Delete(ctx, key)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Multi-Get
|
|
||||||
|
|
||||||
```go
|
|
||||||
keys := []string{"user:1", "user:2", "user:3"}
|
|
||||||
results, err := c.Cache().MGet(ctx, keys)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
for key, value := range results {
|
|
||||||
fmt.Printf("%s: %v\n", key, value)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Database Client
|
|
||||||
|
|
||||||
Query RQLite distributed SQL database.
|
|
||||||
|
|
||||||
### Execute Query (Write)
|
|
||||||
|
|
||||||
```go
|
|
||||||
sql := "INSERT INTO users (name, email) VALUES (?, ?)"
|
|
||||||
args := []interface{}{"Alice", "alice@example.com"}
|
|
||||||
|
|
||||||
result, err := c.Database().Execute(ctx, sql, args...)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Inserted %d rows\n", result.RowsAffected)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Query (Read)
|
|
||||||
|
|
||||||
```go
|
|
||||||
sql := "SELECT id, name, email FROM users WHERE id = ?"
|
|
||||||
args := []interface{}{123}
|
|
||||||
|
|
||||||
rows, err := c.Database().Query(ctx, sql, args...)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
type User struct {
|
|
||||||
ID int `json:"id"`
|
|
||||||
Name string `json:"name"`
|
|
||||||
Email string `json:"email"`
|
|
||||||
}
|
|
||||||
|
|
||||||
var users []User
|
|
||||||
for _, row := range rows {
|
|
||||||
var user User
|
|
||||||
// Parse row into user struct
|
|
||||||
// (manual parsing required, or use ORM layer)
|
|
||||||
users = append(users, user)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Create Table
|
|
||||||
|
|
||||||
```go
|
|
||||||
schema := `CREATE TABLE IF NOT EXISTS users (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
email TEXT UNIQUE NOT NULL,
|
|
||||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
|
||||||
)`
|
|
||||||
|
|
||||||
_, err := c.Database().Execute(ctx, schema)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Transaction
|
|
||||||
|
|
||||||
```go
|
|
||||||
tx, err := c.Database().Begin(ctx)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = tx.Execute(ctx, "INSERT INTO users (name) VALUES (?)", "Alice")
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback(ctx)
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = tx.Execute(ctx, "INSERT INTO users (name) VALUES (?)", "Bob")
|
|
||||||
if err != nil {
|
|
||||||
tx.Rollback(ctx)
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
err = tx.Commit(ctx)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## PubSub Client
|
|
||||||
|
|
||||||
Publish and subscribe to topics.
|
|
||||||
|
|
||||||
### Publish Message
|
|
||||||
|
|
||||||
```go
|
|
||||||
topic := "chat"
|
|
||||||
message := []byte("Hello, everyone!")
|
|
||||||
|
|
||||||
err := c.PubSub().Publish(ctx, topic, message)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Subscribe to Topic
|
|
||||||
|
|
||||||
```go
|
|
||||||
topic := "chat"
|
|
||||||
handler := func(ctx context.Context, msg []byte) error {
|
|
||||||
fmt.Printf("Received: %s\n", string(msg))
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
unsubscribe, err := c.PubSub().Subscribe(ctx, topic, handler)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Later: unsubscribe
|
|
||||||
defer unsubscribe()
|
|
||||||
```
|
|
||||||
|
|
||||||
### List Topics
|
|
||||||
|
|
||||||
```go
|
|
||||||
topics, err := c.PubSub().ListTopics(ctx)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Topics: %v\n", topics)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Serverless Client
|
|
||||||
|
|
||||||
Deploy and invoke WebAssembly functions.
|
|
||||||
|
|
||||||
### Deploy Function
|
|
||||||
|
|
||||||
```go
|
|
||||||
// Read WASM file
|
|
||||||
wasmBytes, err := os.ReadFile("function.wasm")
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Function definition
|
|
||||||
def := &client.FunctionDefinition{
|
|
||||||
Name: "hello-world",
|
|
||||||
Namespace: "default",
|
|
||||||
Description: "Hello world function",
|
|
||||||
MemoryLimit: 64, // MB
|
|
||||||
Timeout: 30, // seconds
|
|
||||||
}
|
|
||||||
|
|
||||||
// Deploy
|
|
||||||
fn, err := c.Serverless().Deploy(ctx, def, wasmBytes)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Deployed: %s (CID: %s)\n", fn.Name, fn.WASMCID)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Invoke Function
|
|
||||||
|
|
||||||
```go
|
|
||||||
functionName := "hello-world"
|
|
||||||
input := map[string]interface{}{
|
|
||||||
"name": "Alice",
|
|
||||||
}
|
|
||||||
|
|
||||||
output, err := c.Serverless().Invoke(ctx, functionName, input)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
fmt.Printf("Result: %s\n", output)
|
|
||||||
```
|
|
||||||
|
|
||||||
### List Functions
|
|
||||||
|
|
||||||
```go
|
|
||||||
functions, err := c.Serverless().List(ctx)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
for _, fn := range functions {
|
|
||||||
fmt.Printf("- %s: %s\n", fn.Name, fn.Description)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Delete Function
|
|
||||||
|
|
||||||
```go
|
|
||||||
functionName := "hello-world"
|
|
||||||
err := c.Serverless().Delete(ctx, functionName)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Get Function Logs
|
|
||||||
|
|
||||||
```go
|
|
||||||
functionName := "hello-world"
|
|
||||||
logs, err := c.Serverless().GetLogs(ctx, functionName, 100)
|
|
||||||
if err != nil {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
for _, log := range logs {
|
|
||||||
fmt.Printf("[%s] %s: %s\n", log.Timestamp, log.Level, log.Message)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
All client methods return typed errors that can be checked:
|
|
||||||
|
|
||||||
```go
|
|
||||||
import "github.com/DeBrosOfficial/network/pkg/errors"
|
|
||||||
|
|
||||||
resp, err := c.Storage().Upload(ctx, data, "file.txt")
|
|
||||||
if err != nil {
|
|
||||||
if errors.IsNotFound(err) {
|
|
||||||
fmt.Println("Resource not found")
|
|
||||||
} else if errors.IsUnauthorized(err) {
|
|
||||||
fmt.Println("Authentication failed")
|
|
||||||
} else if errors.IsValidation(err) {
|
|
||||||
fmt.Println("Validation error")
|
|
||||||
} else {
|
|
||||||
log.Fatal(err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Advanced Usage
|
|
||||||
|
|
||||||
### Custom Timeout
|
|
||||||
|
|
||||||
```go
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
resp, err := c.Storage().Upload(ctx, data, "file.txt")
|
|
||||||
```
|
|
||||||
|
|
||||||
### Retry Logic
|
|
||||||
|
|
||||||
```go
|
|
||||||
import "github.com/DeBrosOfficial/network/pkg/errors"
|
|
||||||
|
|
||||||
maxRetries := 3
|
|
||||||
for i := 0; i < maxRetries; i++ {
|
|
||||||
resp, err := c.Storage().Upload(ctx, data, "file.txt")
|
|
||||||
if err == nil {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
if !errors.ShouldRetry(err) {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
time.Sleep(time.Second * time.Duration(i+1))
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Multiple Namespaces
|
|
||||||
|
|
||||||
```go
|
|
||||||
// Default namespace
|
|
||||||
c1 := client.NewNetworkClient(cfg)
|
|
||||||
c1.Storage().Upload(ctx, data, "file.txt") // Uses default namespace
|
|
||||||
|
|
||||||
// Override namespace per request
|
|
||||||
opts := &client.StorageUploadOptions{
|
|
||||||
Namespace: "custom-namespace",
|
|
||||||
}
|
|
||||||
c1.Storage().UploadWithOptions(ctx, data, "file.txt", opts)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
### Mock Client
|
|
||||||
|
|
||||||
```go
|
|
||||||
// Create a mock client for testing
|
|
||||||
mockClient := &MockNetworkClient{
|
|
||||||
StorageClient: &MockStorageClient{
|
|
||||||
UploadFunc: func(ctx context.Context, data []byte, filename string) (*UploadResponse, error) {
|
|
||||||
return &UploadResponse{CID: "QmMock"}, nil
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
// Use in tests
|
|
||||||
resp, err := mockClient.Storage().Upload(ctx, data, "test.txt")
|
|
||||||
assert.NoError(t, err)
|
|
||||||
assert.Equal(t, "QmMock", resp.CID)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
See the `examples/` directory for complete examples:
|
|
||||||
|
|
||||||
- `examples/storage/` - Storage upload/download examples
|
|
||||||
- `examples/cache/` - Cache operations
|
|
||||||
- `examples/database/` - Database queries
|
|
||||||
- `examples/pubsub/` - Pub/sub messaging
|
|
||||||
- `examples/serverless/` - Serverless functions
|
|
||||||
|
|
||||||
## API Reference
|
|
||||||
|
|
||||||
Complete API documentation is available at:
|
|
||||||
- GoDoc: https://pkg.go.dev/github.com/DeBrosOfficial/network/pkg/client
|
|
||||||
- OpenAPI: `openapi/gateway.yaml`
|
|
||||||
|
|
||||||
## Support
|
|
||||||
|
|
||||||
- GitHub Issues: https://github.com/DeBrosOfficial/network/issues
|
|
||||||
- Documentation: https://github.com/DeBrosOfficial/network/tree/main/docs
|
|
||||||
@ -1,217 +0,0 @@
|
|||||||
# Common Problems & Solutions
|
|
||||||
|
|
||||||
Troubleshooting guide for known issues in the Orama Network.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 1. Namespace Gateway: "Olric unavailable"
|
|
||||||
|
|
||||||
**Symptom:** `ns-<name>.orama-devnet.network/v1/health` returns `"olric": {"status": "unavailable"}`.
|
|
||||||
|
|
||||||
**Cause:** The Olric memberlist gossip between namespace nodes is broken. Olric uses UDP pings for health checks — if those fail, the cluster can't bootstrap and the gateway reports Olric as unavailable.
|
|
||||||
|
|
||||||
### Check 1: WireGuard packet loss between nodes
|
|
||||||
|
|
||||||
SSH into each node and ping the other namespace nodes over WireGuard:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ping -c 10 -W 2 10.0.0.X # replace with the WG IP of each peer
|
|
||||||
```
|
|
||||||
|
|
||||||
If you see packet loss over WireGuard but **not** over the public IP (`ping <public-ip>`), the WireGuard peer session is corrupted.
|
|
||||||
|
|
||||||
**Fix — Reset the WireGuard peer on both sides:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# On Node A — replace <pubkey> and <endpoint> with Node B's values
|
|
||||||
wg set wg0 peer <NodeB-pubkey> remove
|
|
||||||
wg set wg0 peer <NodeB-pubkey> endpoint <NodeB-public-ip>:51820 allowed-ips <NodeB-wg-ip>/32 persistent-keepalive 25
|
|
||||||
|
|
||||||
# On Node B — same but with Node A's values
|
|
||||||
wg set wg0 peer <NodeA-pubkey> remove
|
|
||||||
wg set wg0 peer <NodeA-pubkey> endpoint <NodeA-public-ip>:51820 allowed-ips <NodeA-wg-ip>/32 persistent-keepalive 25
|
|
||||||
```
|
|
||||||
|
|
||||||
Then restart services: `sudo orama node restart`
|
|
||||||
|
|
||||||
You can find peer public keys with `wg show wg0`.
|
|
||||||
|
|
||||||
### Check 2: Olric bound to 0.0.0.0 instead of WireGuard IP
|
|
||||||
|
|
||||||
Check the Olric config on each node:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cat /opt/orama/.orama/data/namespaces/<name>/configs/olric-*.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
If `bindAddr` is `0.0.0.0`, the node will try to bind to IPv6 on dual-stack hosts, breaking memberlist gossip.
|
|
||||||
|
|
||||||
**Fix:** Edit the YAML to use the node's WireGuard IP (run `ip addr show wg0` to find it), then restart: `sudo orama node restart`
|
|
||||||
|
|
||||||
This was fixed in code (BindAddr validation in `SpawnOlric`), so new namespaces won't have this issue.
|
|
||||||
|
|
||||||
### Check 3: Olric logs show "Failed UDP ping" constantly
|
|
||||||
|
|
||||||
```bash
|
|
||||||
journalctl -u orama-namespace-olric@<name>.service --no-pager -n 30
|
|
||||||
```
|
|
||||||
|
|
||||||
If every UDP ping fails but TCP stream connections succeed, it's the WireGuard packet loss issue (see Check 1).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2. Namespace Gateway: Missing config fields
|
|
||||||
|
|
||||||
**Symptom:** Gateway config YAML is missing `global_rqlite_dsn`, has `olric_timeout: 0s`, or `olric_servers` only lists `localhost`.
|
|
||||||
|
|
||||||
**Cause:** Before the spawn handler fix, `spawnGatewayRemote()` didn't send `global_rqlite_dsn` or `olric_timeout` to remote nodes.
|
|
||||||
|
|
||||||
**Fix:** Edit the gateway config manually:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vim /opt/orama/.orama/data/namespaces/<name>/configs/gateway-*.yaml
|
|
||||||
```
|
|
||||||
|
|
||||||
Add/fix:
|
|
||||||
```yaml
|
|
||||||
global_rqlite_dsn: "http://10.0.0.X:10001"
|
|
||||||
olric_timeout: 30s
|
|
||||||
olric_servers:
|
|
||||||
- "10.0.0.X:10002"
|
|
||||||
- "10.0.0.Y:10002"
|
|
||||||
- "10.0.0.Z:10002"
|
|
||||||
```
|
|
||||||
|
|
||||||
Then: `sudo orama node restart`
|
|
||||||
|
|
||||||
This was fixed in code, so new namespaces get the correct config.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3. Namespace not restoring after restart (missing cluster-state.json)
|
|
||||||
|
|
||||||
**Symptom:** After `orama node restart`, the namespace services don't come back because `RestoreLocalClustersFromDisk` has no state file.
|
|
||||||
|
|
||||||
**Check:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ls /opt/orama/.orama/data/namespaces/<name>/cluster-state.json
|
|
||||||
```
|
|
||||||
|
|
||||||
If the file doesn't exist, the node can't restore the namespace.
|
|
||||||
|
|
||||||
**Fix:** Create the file manually from another node that has it, or reconstruct it. The format is:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"namespace": "<name>",
|
|
||||||
"rqlite": { "http_port": 10001, "raft_port": 10000, ... },
|
|
||||||
"olric": { "http_port": 10002, "memberlist_port": 10003, ... },
|
|
||||||
"gateway": { "http_port": 10004, ... }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
This was fixed in code — `ProvisionCluster` now saves state to all nodes (including remote ones via the `save-cluster-state` spawn action).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 4. Namespace gateway processes not restarting after upgrade
|
|
||||||
|
|
||||||
**Symptom:** After `orama upgrade --restart` or `orama node restart`, namespace gateway/olric/rqlite services don't start.
|
|
||||||
|
|
||||||
**Cause:** `orama node stop` disables systemd template services (`orama-namespace-gateway@<name>.service`). They have `PartOf=orama-node.service`, but that only propagates restart to **enabled** services.
|
|
||||||
|
|
||||||
**Fix:** Re-enable the services before restarting:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
systemctl enable orama-namespace-rqlite@<name>.service
|
|
||||||
systemctl enable orama-namespace-olric@<name>.service
|
|
||||||
systemctl enable orama-namespace-gateway@<name>.service
|
|
||||||
sudo orama node restart
|
|
||||||
```
|
|
||||||
|
|
||||||
This was fixed in code — the upgrade orchestrator now re-enables `@` services before restarting.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 5. SSH commands eating stdin inside heredocs
|
|
||||||
|
|
||||||
**Symptom:** When running a script that SSHes into multiple nodes inside a heredoc (`<<'EOS'`), only the first SSH command runs — the rest are silently skipped.
|
|
||||||
|
|
||||||
**Cause:** `ssh` reads from stdin, consuming the rest of the heredoc.
|
|
||||||
|
|
||||||
**Fix:** Add `-n` flag to all `ssh` calls inside heredocs:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ssh -n user@host 'command'
|
|
||||||
```
|
|
||||||
|
|
||||||
`scp` is not affected (doesn't read stdin).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 6. RQLite returns 401 Unauthorized
|
|
||||||
|
|
||||||
**Symptom:** RQLite queries fail with HTTP 401 after security hardening.
|
|
||||||
|
|
||||||
**Cause:** RQLite now requires basic auth. The client isn't sending credentials.
|
|
||||||
|
|
||||||
**Fix:** Ensure the RQLite client is configured with the credentials from `/opt/orama/.orama/secrets/rqlite-auth.json`. The central RQLite client wrapper (`pkg/rqlite/client.go`) handles this automatically. If using a standalone client (e.g., CoreDNS plugin), ensure it's also configured.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 7. Olric cluster split after upgrade
|
|
||||||
|
|
||||||
**Symptom:** Olric nodes can't gossip after enabling memberlist encryption.
|
|
||||||
|
|
||||||
**Cause:** Olric memberlist encryption is all-or-nothing. Nodes with encryption can't communicate with nodes without it.
|
|
||||||
|
|
||||||
**Fix:** All nodes must be restarted simultaneously when enabling Olric encryption. The cache will be lost (it rebuilds from DB). This is expected — Olric is a cache, not persistent storage.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 8. OramaOS: LUKS unlock fails
|
|
||||||
|
|
||||||
**Symptom:** OramaOS node can't reconstruct its LUKS key after reboot.
|
|
||||||
|
|
||||||
**Cause:** Not enough peer vault-guardians are online to meet the Shamir threshold (K = max(3, N/3)).
|
|
||||||
|
|
||||||
**Fix:** Ensure enough cluster nodes are online and reachable over WireGuard. The agent retries with exponential backoff. For genesis nodes before 5+ peers exist, use:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
orama node unlock --genesis --node-ip <wg-ip>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 9. OramaOS: Enrollment timeout
|
|
||||||
|
|
||||||
**Symptom:** `orama node enroll` hangs or times out.
|
|
||||||
|
|
||||||
**Cause:** The OramaOS node's port 9999 isn't reachable, or the Gateway can't reach the node's WebSocket.
|
|
||||||
|
|
||||||
**Fix:** Check that port 9999 is open in your VPS provider's external firewall (Hetzner firewall, AWS security groups, etc.). OramaOS opens it internally, but provider-level firewalls must be configured separately.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 10. Binary signature verification fails
|
|
||||||
|
|
||||||
**Symptom:** `orama node install` rejects the binary archive with a signature error.
|
|
||||||
|
|
||||||
**Cause:** The archive was tampered with, or the manifest.sig file is missing/corrupted.
|
|
||||||
|
|
||||||
**Fix:** Rebuild the archive with `orama build` and re-sign with `make sign` (in the orama-os repo). Ensure you're using the rootwallet that matches the embedded signer address.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## General Debugging Tips
|
|
||||||
|
|
||||||
- **Always use `sudo orama node restart`** instead of raw `systemctl` commands
|
|
||||||
- **Namespace data lives at:** `/opt/orama/.orama/data/namespaces/<name>/`
|
|
||||||
- **Check service logs:** `journalctl -u orama-namespace-olric@<name>.service --no-pager -n 50`
|
|
||||||
- **Check WireGuard:** `wg show wg0` — look for recent handshakes and transfer bytes
|
|
||||||
- **Check gateway health:** `curl http://localhost:<port>/v1/health` from the node itself
|
|
||||||
- **Node IPs:** Check `scripts/remote-nodes.conf` for credentials, `wg show wg0` for WG IPs
|
|
||||||
- **OramaOS nodes:** No SSH access — use Gateway API endpoints (`/v1/node/status`, `/v1/node/logs`) for diagnostics
|
|
||||||
File diff suppressed because it is too large
Load Diff
@ -1,152 +0,0 @@
|
|||||||
# Devnet Installation Commands
|
|
||||||
|
|
||||||
This document contains example installation commands for a multi-node devnet cluster.
|
|
||||||
|
|
||||||
**Wallet:** `<YOUR_WALLET_ADDRESS>`
|
|
||||||
**Contact:** `@anon: <YOUR_WALLET_ADDRESS>`
|
|
||||||
|
|
||||||
## Node Configuration
|
|
||||||
|
|
||||||
| Node | Role | Nameserver | Anyone Relay |
|
|
||||||
|------|------|------------|--------------|
|
|
||||||
| ns1 | Genesis | Yes | No |
|
|
||||||
| ns2 | Nameserver | Yes | Yes (relay-1) |
|
|
||||||
| ns3 | Nameserver | Yes | Yes (relay-2) |
|
|
||||||
| node4 | Worker | No | Yes (relay-3) |
|
|
||||||
| node5 | Worker | No | Yes (relay-4) |
|
|
||||||
| node6 | Worker | No | No |
|
|
||||||
|
|
||||||
**Note:** Store credentials securely (not in version control).
|
|
||||||
|
|
||||||
## MyFamily Fingerprints
|
|
||||||
|
|
||||||
If running multiple Anyone relays, configure MyFamily with all your relay fingerprints:
|
|
||||||
```
|
|
||||||
<FINGERPRINT_1>,<FINGERPRINT_2>,<FINGERPRINT_3>,...
|
|
||||||
```
|
|
||||||
|
|
||||||
## Installation Order
|
|
||||||
|
|
||||||
Install nodes **one at a time**, waiting for each to complete before starting the next:
|
|
||||||
|
|
||||||
1. ns1 (genesis, no Anyone relay)
|
|
||||||
2. ns2 (nameserver + relay)
|
|
||||||
3. ns3 (nameserver + relay)
|
|
||||||
4. node4 (non-nameserver + relay)
|
|
||||||
5. node5 (non-nameserver + relay)
|
|
||||||
6. node6 (non-nameserver, no relay)
|
|
||||||
|
|
||||||
## ns1 - Genesis Node (No Anyone Relay)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SSH: <user>@<ns1-ip>
|
|
||||||
|
|
||||||
sudo orama node install \
|
|
||||||
--vps-ip <ns1-ip> \
|
|
||||||
--domain <your-domain.com> \
|
|
||||||
--base-domain <your-domain.com> \
|
|
||||||
--nameserver
|
|
||||||
```
|
|
||||||
|
|
||||||
After ns1 is installed, generate invite tokens:
|
|
||||||
```bash
|
|
||||||
sudo orama node invite --expiry 24h
|
|
||||||
```
|
|
||||||
|
|
||||||
## ns2 - Nameserver + Relay
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SSH: <user>@<ns2-ip>
|
|
||||||
|
|
||||||
sudo orama node install \
|
|
||||||
--join http://<ns1-ip> --token <TOKEN> \
|
|
||||||
--vps-ip <ns2-ip> \
|
|
||||||
--domain <your-domain.com> \
|
|
||||||
--base-domain <your-domain.com> \
|
|
||||||
--nameserver \
|
|
||||||
--anyone-relay --anyone-migrate \
|
|
||||||
--anyone-nickname <relay-name> \
|
|
||||||
--anyone-wallet <wallet-address> \
|
|
||||||
--anyone-contact "<contact-info>" \
|
|
||||||
--anyone-family "<fingerprint1>,<fingerprint2>,..."
|
|
||||||
```
|
|
||||||
|
|
||||||
## ns3 - Nameserver + Relay
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SSH: <user>@<ns3-ip>
|
|
||||||
|
|
||||||
sudo orama node install \
|
|
||||||
--join http://<ns1-ip> --token <TOKEN> \
|
|
||||||
--vps-ip <ns3-ip> \
|
|
||||||
--domain <your-domain.com> \
|
|
||||||
--base-domain <your-domain.com> \
|
|
||||||
--nameserver \
|
|
||||||
--anyone-relay --anyone-migrate \
|
|
||||||
--anyone-nickname <relay-name> \
|
|
||||||
--anyone-wallet <wallet-address> \
|
|
||||||
--anyone-contact "<contact-info>" \
|
|
||||||
--anyone-family "<fingerprint1>,<fingerprint2>,..."
|
|
||||||
```
|
|
||||||
|
|
||||||
## node4 - Non-Nameserver + Relay
|
|
||||||
|
|
||||||
Domain is auto-generated (e.g., `node-a3f8k2.<your-domain.com>`). No `--domain` flag needed.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SSH: <user>@<node4-ip>
|
|
||||||
|
|
||||||
sudo orama node install \
|
|
||||||
--join http://<ns1-ip> --token <TOKEN> \
|
|
||||||
--vps-ip <node4-ip> \
|
|
||||||
--base-domain <your-domain.com> \
|
|
||||||
--anyone-relay --anyone-migrate \
|
|
||||||
--anyone-nickname <relay-name> \
|
|
||||||
--anyone-wallet <wallet-address> \
|
|
||||||
--anyone-contact "<contact-info>" \
|
|
||||||
--anyone-family "<fingerprint1>,<fingerprint2>,..."
|
|
||||||
```
|
|
||||||
|
|
||||||
## node5 - Non-Nameserver + Relay
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SSH: <user>@<node5-ip>
|
|
||||||
|
|
||||||
sudo orama node install \
|
|
||||||
--join http://<ns1-ip> --token <TOKEN> \
|
|
||||||
--vps-ip <node5-ip> \
|
|
||||||
--base-domain <your-domain.com> \
|
|
||||||
--anyone-relay --anyone-migrate \
|
|
||||||
--anyone-nickname <relay-name> \
|
|
||||||
--anyone-wallet <wallet-address> \
|
|
||||||
--anyone-contact "<contact-info>" \
|
|
||||||
--anyone-family "<fingerprint1>,<fingerprint2>,..."
|
|
||||||
```
|
|
||||||
|
|
||||||
## node6 - Non-Nameserver (No Anyone Relay)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SSH: <user>@<node6-ip>
|
|
||||||
|
|
||||||
sudo orama node install \
|
|
||||||
--join http://<ns1-ip> --token <TOKEN> \
|
|
||||||
--vps-ip <node6-ip> \
|
|
||||||
--base-domain <your-domain.com>
|
|
||||||
```
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
After all nodes are installed, verify cluster health:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Full cluster report (from local machine)
|
|
||||||
./bin/orama monitor report --env devnet
|
|
||||||
|
|
||||||
# Single node health
|
|
||||||
./bin/orama monitor report --env devnet --node <ip>
|
|
||||||
|
|
||||||
# Or manually from any VPS:
|
|
||||||
curl -s http://localhost:5001/status | jq -r '.store.raft.state, .store.raft.num_peers'
|
|
||||||
curl -s http://localhost:6001/health
|
|
||||||
systemctl status orama-anyone-relay
|
|
||||||
```
|
|
||||||
@ -1,469 +0,0 @@
|
|||||||
# Development Guide
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Go 1.21+
|
|
||||||
- Node.js 18+ (for anyone-client in dev mode)
|
|
||||||
- macOS or Linux
|
|
||||||
|
|
||||||
## Building
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Build all binaries
|
|
||||||
make build
|
|
||||||
|
|
||||||
# Outputs:
|
|
||||||
# bin/orama-node — the node binary
|
|
||||||
# bin/orama — the CLI
|
|
||||||
# bin/gateway — standalone gateway (optional)
|
|
||||||
# bin/identity — identity tool
|
|
||||||
```
|
|
||||||
|
|
||||||
## Running Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make test
|
|
||||||
```
|
|
||||||
|
|
||||||
## Deploying to VPS
|
|
||||||
|
|
||||||
All binaries are pre-compiled locally and shipped as a binary archive. Zero compilation on the VPS.
|
|
||||||
|
|
||||||
### Deploy Workflow
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# One-command: build + push + rolling upgrade
|
|
||||||
orama node rollout --env testnet
|
|
||||||
|
|
||||||
# Or step by step:
|
|
||||||
|
|
||||||
# 1. Build binary archive (cross-compiles all binaries for linux/amd64)
|
|
||||||
orama build
|
|
||||||
# Creates: /tmp/orama-<version>-linux-amd64.tar.gz
|
|
||||||
|
|
||||||
# 2. Push archive to all nodes (fanout via hub node)
|
|
||||||
orama node push --env testnet
|
|
||||||
|
|
||||||
# 3. Rolling upgrade (one node at a time, followers first, leader last)
|
|
||||||
orama node upgrade --env testnet
|
|
||||||
```
|
|
||||||
|
|
||||||
### Fresh Node Install
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Build the archive first (if not already built)
|
|
||||||
orama build
|
|
||||||
|
|
||||||
# Install on a new VPS (auto-uploads binary archive, zero compilation)
|
|
||||||
orama node install --vps-ip <ip> --nameserver --domain <domain> --base-domain <domain>
|
|
||||||
```
|
|
||||||
|
|
||||||
The installer auto-detects the binary archive at `/opt/orama/manifest.json` and copies pre-built binaries instead of compiling from source.
|
|
||||||
|
|
||||||
### Upgrading a Multi-Node Cluster (CRITICAL)
|
|
||||||
|
|
||||||
**NEVER restart all nodes simultaneously.** RQLite uses Raft consensus and requires a majority (quorum) to function.
|
|
||||||
|
|
||||||
#### Safe Upgrade Procedure
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Full rollout (build + push + rolling upgrade, one command)
|
|
||||||
orama node rollout --env testnet
|
|
||||||
|
|
||||||
# Or with more control:
|
|
||||||
orama node push --env testnet # Push archive to all nodes
|
|
||||||
orama node upgrade --env testnet # Rolling upgrade (auto-detects leader)
|
|
||||||
orama node upgrade --env testnet --node 1.2.3.4 # Single node only
|
|
||||||
orama node upgrade --env testnet --delay 60 # 60s between nodes
|
|
||||||
```
|
|
||||||
|
|
||||||
The rolling upgrade automatically:
|
|
||||||
1. Upgrades **follower** nodes first
|
|
||||||
2. Upgrades the **leader** last
|
|
||||||
3. Waits a configurable delay between nodes (default: 30s)
|
|
||||||
|
|
||||||
After each node, verify health:
|
|
||||||
```bash
|
|
||||||
orama monitor report --env testnet
|
|
||||||
```
|
|
||||||
|
|
||||||
#### What NOT to Do
|
|
||||||
|
|
||||||
- **DON'T** stop all nodes, replace binaries, then start all nodes
|
|
||||||
- **DON'T** run `orama node upgrade --restart` on multiple nodes in parallel
|
|
||||||
- **DON'T** clear RQLite data directories unless doing a full cluster rebuild
|
|
||||||
- **DON'T** use `systemctl stop orama-node` on multiple nodes simultaneously
|
|
||||||
|
|
||||||
#### Recovery from Cluster Split
|
|
||||||
|
|
||||||
If nodes get stuck in "Candidate" state or show "leader not found" errors:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Recover the Raft cluster (specify the node with highest commit index as leader)
|
|
||||||
orama node recover-raft --env testnet --leader 1.2.3.4
|
|
||||||
```
|
|
||||||
|
|
||||||
This will:
|
|
||||||
1. Stop orama-node on ALL nodes
|
|
||||||
2. Backup + delete raft/ on non-leader nodes
|
|
||||||
3. Start the leader, wait for Leader state
|
|
||||||
4. Start remaining nodes in batches
|
|
||||||
5. Verify cluster health
|
|
||||||
|
|
||||||
### Cleaning Nodes for Reinstallation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Wipe all data and services (preserves Anyone relay keys)
|
|
||||||
orama node clean --env testnet --force
|
|
||||||
|
|
||||||
# Also remove shared binaries (rqlited, ipfs, caddy, etc.)
|
|
||||||
orama node clean --env testnet --nuclear --force
|
|
||||||
|
|
||||||
# Single node only
|
|
||||||
orama node clean --env testnet --node 1.2.3.4 --force
|
|
||||||
```
|
|
||||||
|
|
||||||
### Push Options
|
|
||||||
|
|
||||||
```bash
|
|
||||||
orama node push --env devnet # Fanout via hub (default, fastest)
|
|
||||||
orama node push --env testnet --node 1.2.3.4 # Single node
|
|
||||||
orama node push --env testnet --direct # Sequential, no fanout
|
|
||||||
```
|
|
||||||
|
|
||||||
### CLI Flags Reference
|
|
||||||
|
|
||||||
#### `orama node install`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--vps-ip <ip>` | VPS public IP address (required) |
|
|
||||||
| `--domain <domain>` | Domain for HTTPS certificates. Required for nameserver nodes (use the base domain, e.g., `example.com`). Auto-generated for non-nameserver nodes if omitted (e.g., `node-a3f8k2.example.com`) |
|
|
||||||
| `--base-domain <domain>` | Base domain for deployment routing (e.g., example.com) |
|
|
||||||
| `--nameserver` | Configure this node as a nameserver (CoreDNS + Caddy) |
|
|
||||||
| `--join <url>` | Join existing cluster via HTTPS URL (e.g., `https://node1.example.com`) |
|
|
||||||
| `--token <token>` | Invite token for joining (from `orama node invite` on existing node) |
|
|
||||||
| `--force` | Force reconfiguration even if already installed |
|
|
||||||
| `--skip-firewall` | Skip UFW firewall setup |
|
|
||||||
| `--skip-checks` | Skip minimum resource checks (RAM/CPU) |
|
|
||||||
| `--anyone-relay` | Install and configure an Anyone relay on this node |
|
|
||||||
| `--anyone-migrate` | Migrate existing Anyone relay installation (preserves keys/fingerprint) |
|
|
||||||
| `--anyone-nickname <name>` | Relay nickname (required for relay mode) |
|
|
||||||
| `--anyone-wallet <addr>` | Ethereum wallet for relay rewards (required for relay mode) |
|
|
||||||
| `--anyone-contact <info>` | Contact info for relay (required for relay mode) |
|
|
||||||
| `--anyone-family <fps>` | Comma-separated fingerprints of related relays (MyFamily) |
|
|
||||||
| `--anyone-orport <port>` | ORPort for relay (default: 9001) |
|
|
||||||
| `--anyone-exit` | Configure as an exit relay (default: non-exit) |
|
|
||||||
| `--anyone-bandwidth <pct>` | Limit relay to N% of VPS bandwidth (default: 30, 0=unlimited). Runs a speedtest during install to measure available bandwidth |
|
|
||||||
| `--anyone-accounting <GB>` | Monthly data cap for relay in GB (0=unlimited) |
|
|
||||||
|
|
||||||
#### `orama node invite`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--expiry <duration>` | Token expiry duration (default: 1h, e.g. `--expiry 24h`) |
|
|
||||||
|
|
||||||
**Important notes about invite tokens:**
|
|
||||||
|
|
||||||
- **Tokens are single-use.** Once a node consumes a token during the join handshake, it cannot be reused. Generate a separate token for each node you want to join.
|
|
||||||
- **Expiry is checked in UTC.** RQLite uses `datetime('now')` which is always UTC. If your local timezone differs, account for the offset when choosing expiry durations.
|
|
||||||
- **Use longer expiry for multi-node deployments.** When deploying multiple nodes, use `--expiry 24h` to avoid tokens expiring mid-deployment.
|
|
||||||
|
|
||||||
#### `orama node upgrade`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--restart` | Restart all services after upgrade (local mode) |
|
|
||||||
| `--env <env>` | Target environment for remote rolling upgrade |
|
|
||||||
| `--node <ip>` | Upgrade a single node only |
|
|
||||||
| `--delay <seconds>` | Delay between nodes during rolling upgrade (default: 30) |
|
|
||||||
| `--anyone-relay` | Enable Anyone relay (same flags as install) |
|
|
||||||
| `--anyone-bandwidth <pct>` | Limit relay to N% of VPS bandwidth (default: 30, 0=unlimited) |
|
|
||||||
| `--anyone-accounting <GB>` | Monthly data cap for relay in GB (0=unlimited) |
|
|
||||||
|
|
||||||
#### `orama build`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--arch <arch>` | Target architecture (default: amd64) |
|
|
||||||
| `--output <path>` | Output archive path |
|
|
||||||
| `--verbose` | Verbose build output |
|
|
||||||
|
|
||||||
#### `orama node push`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--env <env>` | Target environment (required) |
|
|
||||||
| `--node <ip>` | Push to a single node only |
|
|
||||||
| `--direct` | Sequential upload (no hub fanout) |
|
|
||||||
|
|
||||||
#### `orama node rollout`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--env <env>` | Target environment (required) |
|
|
||||||
| `--no-build` | Skip the build step |
|
|
||||||
| `--yes` | Skip confirmation |
|
|
||||||
| `--delay <seconds>` | Delay between nodes (default: 30) |
|
|
||||||
|
|
||||||
#### `orama node clean`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--env <env>` | Target environment (required) |
|
|
||||||
| `--node <ip>` | Clean a single node only |
|
|
||||||
| `--nuclear` | Also remove shared binaries |
|
|
||||||
| `--force` | Skip confirmation (DESTRUCTIVE) |
|
|
||||||
|
|
||||||
#### `orama node recover-raft`
|
|
||||||
|
|
||||||
| Flag | Description |
|
|
||||||
|------|-------------|
|
|
||||||
| `--env <env>` | Target environment (required) |
|
|
||||||
| `--leader <ip>` | Leader node IP — highest commit index (required) |
|
|
||||||
| `--force` | Skip confirmation (DESTRUCTIVE) |
|
|
||||||
|
|
||||||
#### `orama node` (Service Management)
|
|
||||||
|
|
||||||
Use these commands to manage services on production nodes:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Stop all services (orama-node, coredns, caddy)
|
|
||||||
sudo orama node stop
|
|
||||||
|
|
||||||
# Start all services
|
|
||||||
sudo orama node start
|
|
||||||
|
|
||||||
# Restart all services
|
|
||||||
sudo orama node restart
|
|
||||||
|
|
||||||
# Check service status
|
|
||||||
sudo orama node status
|
|
||||||
|
|
||||||
# Diagnose common issues
|
|
||||||
sudo orama node doctor
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note:** Always use `orama node stop` instead of manually running `systemctl stop`. The CLI ensures all related services (including CoreDNS and Caddy on nameserver nodes) are handled correctly.
|
|
||||||
|
|
||||||
#### `orama node report`
|
|
||||||
|
|
||||||
Outputs comprehensive health data as JSON. Used by `orama monitor` over SSH:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo orama node report --json
|
|
||||||
```
|
|
||||||
|
|
||||||
See [MONITORING.md](MONITORING.md) for full details.
|
|
||||||
|
|
||||||
#### `orama monitor`
|
|
||||||
|
|
||||||
Real-time cluster monitoring from your local machine:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Interactive TUI
|
|
||||||
orama monitor --env testnet
|
|
||||||
|
|
||||||
# Cluster overview
|
|
||||||
orama monitor cluster --env testnet
|
|
||||||
|
|
||||||
# Alerts only
|
|
||||||
orama monitor alerts --env testnet
|
|
||||||
|
|
||||||
# Full JSON for LLM analysis
|
|
||||||
orama monitor report --env testnet
|
|
||||||
```
|
|
||||||
|
|
||||||
See [MONITORING.md](MONITORING.md) for all subcommands and flags.
|
|
||||||
|
|
||||||
### Node Join Flow
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Genesis node (first node, creates cluster)
|
|
||||||
# Nameserver nodes use the base domain as --domain
|
|
||||||
sudo orama node install --vps-ip 1.2.3.4 --domain example.com \
|
|
||||||
--base-domain example.com --nameserver
|
|
||||||
|
|
||||||
# 2. On genesis node, generate an invite
|
|
||||||
orama node invite --expiry 24h
|
|
||||||
# Output: sudo orama node install --join https://example.com --token <TOKEN> --vps-ip <IP>
|
|
||||||
|
|
||||||
# 3a. Join as nameserver (requires --domain set to base domain)
|
|
||||||
sudo orama node install --join http://1.2.3.4 --token abc123... \
|
|
||||||
--vps-ip 5.6.7.8 --domain example.com --base-domain example.com --nameserver
|
|
||||||
|
|
||||||
# 3b. Join as regular node (domain auto-generated, no --domain needed)
|
|
||||||
sudo orama node install --join http://1.2.3.4 --token abc123... \
|
|
||||||
--vps-ip 5.6.7.8 --base-domain example.com
|
|
||||||
```
|
|
||||||
|
|
||||||
The join flow establishes a WireGuard VPN tunnel before starting cluster services.
|
|
||||||
All inter-node communication (RQLite, IPFS, Olric) uses WireGuard IPs (10.0.0.x).
|
|
||||||
No cluster ports are ever exposed publicly.
|
|
||||||
|
|
||||||
#### DNS Prerequisite
|
|
||||||
|
|
||||||
The `--join` URL should use the HTTPS domain of the genesis node (e.g., `https://node1.example.com`).
|
|
||||||
For this to work, the domain registrar for `example.com` must have NS records pointing to the genesis
|
|
||||||
node's IP so that `node1.example.com` resolves publicly.
|
|
||||||
|
|
||||||
**If DNS is not yet configured**, you can use the genesis node's public IP with HTTP as a fallback:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo orama node install --join http://1.2.3.4 --vps-ip 5.6.7.8 --token abc123... --nameserver
|
|
||||||
```
|
|
||||||
|
|
||||||
This works because Caddy's `:80` block proxies all HTTP traffic to the gateway. However, once DNS
|
|
||||||
is properly configured, always use the HTTPS domain URL.
|
|
||||||
|
|
||||||
**Important:** Never use `http://<ip>:6001` — port 6001 is the internal gateway and is blocked by
|
|
||||||
UFW from external access. The join request goes through Caddy on port 80 (HTTP) or 443 (HTTPS),
|
|
||||||
which proxies to the gateway internally.
|
|
||||||
|
|
||||||
## OramaOS Enrollment
|
|
||||||
|
|
||||||
For OramaOS nodes (mainnet, devnet, testnet), use the enrollment flow instead of `orama node install`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Flash OramaOS image to VPS (via provider dashboard)
|
|
||||||
# 2. Generate invite token on existing cluster node
|
|
||||||
orama node invite --expiry 24h
|
|
||||||
|
|
||||||
# 3. Enroll the OramaOS node
|
|
||||||
orama node enroll --node-ip <vps-public-ip> --token <invite-token> --gateway <gateway-url>
|
|
||||||
|
|
||||||
# 4. For genesis node reboots (before 5+ peers exist)
|
|
||||||
orama node unlock --genesis --node-ip <wg-ip>
|
|
||||||
```
|
|
||||||
|
|
||||||
OramaOS nodes have no SSH access. All management happens through the Gateway API:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Status, logs, commands — all via Gateway proxy
|
|
||||||
curl "https://gateway.example.com/v1/node/status?node_id=<id>"
|
|
||||||
curl "https://gateway.example.com/v1/node/logs?node_id=<id>&service=gateway"
|
|
||||||
```
|
|
||||||
|
|
||||||
See [ORAMAOS_DEPLOYMENT.md](ORAMAOS_DEPLOYMENT.md) for the full guide.
|
|
||||||
|
|
||||||
**Note:** `orama node clean` does not work on OramaOS nodes (no SSH). Use `orama node leave` for graceful departure, or reflash the image for a factory reset.
|
|
||||||
|
|
||||||
## Pre-Install Checklist (Ubuntu Only)
|
|
||||||
|
|
||||||
Before running `orama node install` on a VPS, ensure:
|
|
||||||
|
|
||||||
1. **Stop Docker if running.** Docker commonly binds ports 4001 and 8080 which conflict with IPFS. The installer checks for port conflicts and shows which process is using each port, but it's easier to stop Docker first:
|
|
||||||
```bash
|
|
||||||
sudo systemctl stop docker docker.socket
|
|
||||||
sudo systemctl disable docker docker.socket
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Stop any existing IPFS instance.**
|
|
||||||
```bash
|
|
||||||
sudo systemctl stop ipfs
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Stop any service on port 53** (for nameserver nodes). The installer handles `systemd-resolved` automatically, but other DNS services (like `bind9` or `dnsmasq`) must be stopped manually.
|
|
||||||
|
|
||||||
## Recovering from Failed Joins
|
|
||||||
|
|
||||||
If a node partially joins the cluster (registers in RQLite's Raft but then fails or gets cleaned), the remaining cluster can lose quorum permanently. This happens because RQLite thinks there are N voters but only N-1 are reachable.
|
|
||||||
|
|
||||||
**Symptoms:** RQLite stuck in "Candidate" state, no leader elected, all writes fail.
|
|
||||||
|
|
||||||
**Solution:** Do a full clean reinstall of all affected nodes. Use [CLEAN_NODE.md](CLEAN_NODE.md) to reset each node, then reinstall starting from the genesis node.
|
|
||||||
|
|
||||||
**Prevention:** Always ensure a joining node can complete the full installation before it joins. The installer validates port availability upfront to catch conflicts early.
|
|
||||||
|
|
||||||
## Debugging Production Issues
|
|
||||||
|
|
||||||
Always follow the local-first approach:
|
|
||||||
|
|
||||||
1. **Reproduce locally** — set up the same conditions on your machine
|
|
||||||
2. **Find the root cause** — understand why it's happening
|
|
||||||
3. **Fix in the codebase** — make changes to the source code
|
|
||||||
4. **Test locally** — run `make test` and verify
|
|
||||||
5. **Deploy** — only then deploy the fix to production
|
|
||||||
|
|
||||||
Never fix issues directly on the server — those fixes are lost on next deployment.
|
|
||||||
|
|
||||||
## Trusting the Self-Signed TLS Certificate
|
|
||||||
|
|
||||||
When Let's Encrypt is rate-limited, Caddy falls back to its internal CA (self-signed certificates). Browsers will show security warnings unless you install the root CA certificate.
|
|
||||||
|
|
||||||
### Downloading the Root CA Certificate
|
|
||||||
|
|
||||||
From VPS 1 (or any node), copy the certificate:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Copy the cert to an accessible location on the VPS
|
|
||||||
ssh ubuntu@<VPS_IP> "sudo cp /var/lib/caddy/.local/share/caddy/pki/authorities/local/root.crt /tmp/caddy-root-ca.crt && sudo chmod 644 /tmp/caddy-root-ca.crt"
|
|
||||||
|
|
||||||
# Download to your local machine
|
|
||||||
scp ubuntu@<VPS_IP>:/tmp/caddy-root-ca.crt ~/Downloads/caddy-root-ca.crt
|
|
||||||
```
|
|
||||||
|
|
||||||
### macOS
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/Downloads/caddy-root-ca.crt
|
|
||||||
```
|
|
||||||
|
|
||||||
This adds the cert system-wide. All browsers (Safari, Chrome, Arc, etc.) will trust it immediately. Firefox uses its own certificate store — go to **Settings > Privacy & Security > Certificates > View Certificates > Import** and import the `.crt` file there.
|
|
||||||
|
|
||||||
To remove it later:
|
|
||||||
```bash
|
|
||||||
sudo security remove-trusted-cert -d ~/Downloads/caddy-root-ca.crt
|
|
||||||
```
|
|
||||||
|
|
||||||
### iOS (iPhone/iPad)
|
|
||||||
|
|
||||||
1. Transfer `caddy-root-ca.crt` to your device (AirDrop, email attachment, or host it on a URL)
|
|
||||||
2. Open the file — iOS will show "Profile Downloaded"
|
|
||||||
3. Go to **Settings > General > VPN & Device Management** (or "Profiles" on older iOS)
|
|
||||||
4. Tap the "Caddy Local Authority" profile and tap **Install**
|
|
||||||
5. Go to **Settings > General > About > Certificate Trust Settings**
|
|
||||||
6. Enable **full trust** for "Caddy Local Authority - 2026 ECC Root"
|
|
||||||
|
|
||||||
### Android
|
|
||||||
|
|
||||||
1. Transfer `caddy-root-ca.crt` to your device
|
|
||||||
2. Go to **Settings > Security > Encryption & Credentials > Install a certificate > CA certificate**
|
|
||||||
3. Select the `caddy-root-ca.crt` file
|
|
||||||
4. Confirm the installation
|
|
||||||
|
|
||||||
Note: On Android 7+, user-installed CA certificates are only trusted by apps that explicitly opt in. Chrome will trust it, but some apps may not.
|
|
||||||
|
|
||||||
### Windows
|
|
||||||
|
|
||||||
```powershell
|
|
||||||
certutil -addstore -f "ROOT" caddy-root-ca.crt
|
|
||||||
```
|
|
||||||
|
|
||||||
Or double-click the `.crt` file > **Install Certificate** > **Local Machine** > **Place in "Trusted Root Certification Authorities"**.
|
|
||||||
|
|
||||||
### Linux
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo cp caddy-root-ca.crt /usr/local/share/ca-certificates/caddy-root-ca.crt
|
|
||||||
sudo update-ca-certificates
|
|
||||||
```
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
See [ARCHITECTURE.md](ARCHITECTURE.md) for the full architecture overview.
|
|
||||||
|
|
||||||
Key directories:
|
|
||||||
|
|
||||||
```
|
|
||||||
cmd/
|
|
||||||
cli/ — CLI entry point (orama command)
|
|
||||||
node/ — Node entry point (orama-node)
|
|
||||||
gateway/ — Standalone gateway entry point
|
|
||||||
pkg/
|
|
||||||
cli/ — CLI command implementations
|
|
||||||
gateway/ — HTTP gateway, routes, middleware
|
|
||||||
deployments/ — Deployment types, service, storage
|
|
||||||
environments/ — Production (systemd) and development (direct) modes
|
|
||||||
rqlite/ — Distributed SQLite via RQLite
|
|
||||||
```
|
|
||||||
@ -1,213 +0,0 @@
|
|||||||
# Inspector
|
|
||||||
|
|
||||||
The inspector is a cluster health check tool that SSHs into every node, collects subsystem data in parallel, runs deterministic checks, and optionally sends failures to an AI model for root-cause analysis.
|
|
||||||
|
|
||||||
## Pipeline
|
|
||||||
|
|
||||||
```
|
|
||||||
Collect (parallel SSH) → Check (deterministic Go) → Report (table/JSON) → Analyze (optional AI)
|
|
||||||
```
|
|
||||||
|
|
||||||
1. **Collect** — SSH into every node in parallel, run diagnostic commands, parse results into structured data.
|
|
||||||
2. **Check** — Run pure Go check functions against the collected data. Each check produces a pass/fail/warn/skip result with a severity level.
|
|
||||||
3. **Report** — Print results as a table (default) or JSON. Failures sort first, grouped by subsystem.
|
|
||||||
4. **Analyze** — If `--ai` is enabled and there are failures or warnings, send them to an LLM via OpenRouter for root-cause analysis.
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Inspect all subsystems on devnet
|
|
||||||
orama inspect --env devnet
|
|
||||||
|
|
||||||
# Inspect only RQLite
|
|
||||||
orama inspect --env devnet --subsystem rqlite
|
|
||||||
|
|
||||||
# JSON output
|
|
||||||
orama inspect --env devnet --format json
|
|
||||||
|
|
||||||
# With AI analysis
|
|
||||||
orama inspect --env devnet --ai
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
```
|
|
||||||
orama inspect [flags]
|
|
||||||
```
|
|
||||||
|
|
||||||
| Flag | Default | Description |
|
|
||||||
|------|---------|-------------|
|
|
||||||
| `--config` | `scripts/remote-nodes.conf` | Path to node configuration file |
|
|
||||||
| `--env` | *(required)* | Environment to inspect (`devnet`, `testnet`) |
|
|
||||||
| `--subsystem` | `all` | Comma-separated subsystems to inspect |
|
|
||||||
| `--format` | `table` | Output format: `table` or `json` |
|
|
||||||
| `--timeout` | `30s` | SSH command timeout per node |
|
|
||||||
| `--verbose` | `false` | Print collection progress |
|
|
||||||
| `--ai` | `false` | Enable AI analysis of failures |
|
|
||||||
| `--model` | `moonshotai/kimi-k2.5` | OpenRouter model for AI analysis |
|
|
||||||
| `--api-key` | `$OPENROUTER_API_KEY` | OpenRouter API key |
|
|
||||||
|
|
||||||
### Subsystem Names
|
|
||||||
|
|
||||||
`rqlite`, `olric`, `ipfs`, `dns`, `wireguard` (alias: `wg`), `system`, `network`, `namespace`
|
|
||||||
|
|
||||||
Multiple subsystems can be combined: `--subsystem rqlite,olric,dns`
|
|
||||||
|
|
||||||
## Subsystems
|
|
||||||
|
|
||||||
| Subsystem | What It Checks |
|
|
||||||
|-----------|---------------|
|
|
||||||
| **rqlite** | Raft state, leader election, readyz, commit/applied gap, FSM pending, strong reads, debug vars (query errors, leader_not_found, snapshots), cross-node leader agreement, term consistency, applied index convergence, quorum, version match |
|
|
||||||
| **olric** | Service active, memberlist up, restart count, memory usage, log analysis (suspects, flapping, errors), cross-node memberlist consistency |
|
|
||||||
| **ipfs** | Daemon active, cluster active, swarm peer count, cluster peer count, cluster errors, repo usage %, swarm key present, bootstrap list empty, cross-node version consistency |
|
|
||||||
| **dns** | CoreDNS active, Caddy active, ports (53/80/443), memory, restart count, log errors, Corefile exists, SOA/NS/wildcard/base-A resolution, TLS cert expiry, cross-node nameserver availability |
|
|
||||||
| **wireguard** | Interface up, service active, correct 10.0.0.x IP, listen port 51820, peer count vs expected, MTU 1420, config exists + permissions 600, peer handshakes (fresh/stale/never), peer traffic, catch-all route detection, cross-node peer count + MTU consistency |
|
|
||||||
| **system** | Core services (orama-node, rqlite, olric, ipfs, ipfs-cluster, wg-quick), nameserver services (coredns, caddy), failed systemd units, memory/disk/inode usage, load average, OOM kills, swap, UFW active, process user (orama), panic count, expected ports |
|
|
||||||
| **network** | Internet reachability, default route, WireGuard route, TCP connection count, TIME_WAIT count, TCP retransmission rate, WireGuard mesh ping (all peers) |
|
|
||||||
| **namespace** | Per-namespace: RQLite up + raft state + readyz, Olric memberlist, Gateway HTTP health. Cross-namespace: all-healthy check, RQLite quorum per namespace |
|
|
||||||
|
|
||||||
## Severity Levels
|
|
||||||
|
|
||||||
| Level | When Used |
|
|
||||||
|-------|-----------|
|
|
||||||
| **CRITICAL** | Service completely down. Raft quorum lost, RQLite unresponsive, no leader. |
|
|
||||||
| **HIGH** | Service degraded. Olric down, gateway not responding, IPFS swarm key missing. |
|
|
||||||
| **MEDIUM** | Non-ideal but functional. Stale handshakes, elevated memory, log suspects. |
|
|
||||||
| **LOW** | Informational. Non-standard MTU, port mismatch, version skew. |
|
|
||||||
|
|
||||||
## Check Statuses
|
|
||||||
|
|
||||||
| Status | Meaning |
|
|
||||||
|--------|---------|
|
|
||||||
| **pass** | Check passed. |
|
|
||||||
| **fail** | Check failed — action needed. |
|
|
||||||
| **warn** | Degraded — monitor or investigate. |
|
|
||||||
| **skip** | Check could not run (insufficient data). |
|
|
||||||
|
|
||||||
## Output Formats
|
|
||||||
|
|
||||||
### Table (default)
|
|
||||||
|
|
||||||
```
|
|
||||||
Inspecting 14 devnet nodes...
|
|
||||||
|
|
||||||
## RQLITE
|
|
||||||
----------------------------------------------------------------------
|
|
||||||
OK [CRITICAL] RQLite responding (ubuntu@10.0.0.1)
|
|
||||||
responsive=true version=v8.36.16
|
|
||||||
FAIL [CRITICAL] Cluster has exactly one leader
|
|
||||||
leaders=0 (NO LEADER)
|
|
||||||
...
|
|
||||||
|
|
||||||
======================================================================
|
|
||||||
Summary: 800 passed, 12 failed, 31 warnings, 0 skipped (4.2s)
|
|
||||||
```
|
|
||||||
|
|
||||||
Failures sort first, then warnings, then passes. Within each group, higher severity checks appear first.
|
|
||||||
|
|
||||||
### JSON (`--format json`)
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"summary": {
|
|
||||||
"passed": 800,
|
|
||||||
"failed": 12,
|
|
||||||
"warned": 31,
|
|
||||||
"skipped": 0,
|
|
||||||
"total": 843,
|
|
||||||
"duration_seconds": 4.2
|
|
||||||
},
|
|
||||||
"checks": [
|
|
||||||
{
|
|
||||||
"id": "rqlite.responsive",
|
|
||||||
"name": "RQLite responding",
|
|
||||||
"subsystem": "rqlite",
|
|
||||||
"severity": 3,
|
|
||||||
"status": "pass",
|
|
||||||
"message": "responsive=true version=v8.36.16",
|
|
||||||
"node": "ubuntu@10.0.0.1"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## AI Analysis
|
|
||||||
|
|
||||||
When `--ai` is enabled, failures and warnings are sent to an LLM via OpenRouter for root-cause analysis.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Use default model (kimi-k2.5)
|
|
||||||
orama inspect --env devnet --ai
|
|
||||||
|
|
||||||
# Use a different model
|
|
||||||
orama inspect --env devnet --ai --model openai/gpt-4o
|
|
||||||
|
|
||||||
# Pass API key directly
|
|
||||||
orama inspect --env devnet --ai --api-key sk-or-...
|
|
||||||
```
|
|
||||||
|
|
||||||
The API key can be set via:
|
|
||||||
1. `--api-key` flag
|
|
||||||
2. `OPENROUTER_API_KEY` environment variable
|
|
||||||
3. `.env` file in the current directory
|
|
||||||
|
|
||||||
The AI receives the full check results plus cluster metadata and returns a structured analysis with likely root causes and suggested fixes.
|
|
||||||
|
|
||||||
## Exit Codes
|
|
||||||
|
|
||||||
| Code | Meaning |
|
|
||||||
|------|---------|
|
|
||||||
| `0` | All checks passed (or only warnings). |
|
|
||||||
| `1` | At least one check failed. |
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
The inspector reads node definitions from a pipe-delimited config file (default: `scripts/remote-nodes.conf`).
|
|
||||||
|
|
||||||
### Format
|
|
||||||
|
|
||||||
```
|
|
||||||
# environment|user@host|role
|
|
||||||
devnet|ubuntu@1.2.3.4|node
|
|
||||||
devnet|ubuntu@5.6.7.8|nameserver-ns1
|
|
||||||
```
|
|
||||||
|
|
||||||
| Field | Description |
|
|
||||||
|-------|-------------|
|
|
||||||
| `environment` | Cluster name (`devnet`, `testnet`) |
|
|
||||||
| `user@host` | SSH credentials |
|
|
||||||
| `role` | `node` or `nameserver-ns1`, `nameserver-ns2`, etc. |
|
|
||||||
|
|
||||||
SSH keys are resolved from rootwallet (`rw vault ssh get <host>/<user> --priv`).
|
|
||||||
|
|
||||||
Blank lines and lines starting with `#` are ignored.
|
|
||||||
|
|
||||||
### Node Roles
|
|
||||||
|
|
||||||
- **`node`** — Regular cluster node. Runs RQLite, Olric, IPFS, WireGuard, namespaces.
|
|
||||||
- **`nameserver-*`** — DNS nameserver. Runs CoreDNS + Caddy in addition to base services. System checks verify nameserver-specific services.
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Full cluster inspection
|
|
||||||
orama inspect --env devnet
|
|
||||||
|
|
||||||
# Check only networking
|
|
||||||
orama inspect --env devnet --subsystem wireguard,network
|
|
||||||
|
|
||||||
# Quick RQLite health check
|
|
||||||
orama inspect --env devnet --subsystem rqlite
|
|
||||||
|
|
||||||
# Verbose mode (shows collection progress)
|
|
||||||
orama inspect --env devnet --verbose
|
|
||||||
|
|
||||||
# JSON for scripting / piping
|
|
||||||
orama inspect --env devnet --format json | jq '.checks[] | select(.status == "fail")'
|
|
||||||
|
|
||||||
# AI-assisted debugging
|
|
||||||
orama inspect --env devnet --ai --model anthropic/claude-sonnet-4
|
|
||||||
|
|
||||||
# Custom config file
|
|
||||||
orama inspect --config /path/to/nodes.conf --env testnet
|
|
||||||
```
|
|
||||||
@ -1,278 +0,0 @@
|
|||||||
# Monitoring
|
|
||||||
|
|
||||||
Real-time cluster health monitoring via SSH. The system has two parts:
|
|
||||||
|
|
||||||
1. **`orama node report`** — Runs on each VPS node, collects all local health data, outputs JSON
|
|
||||||
2. **`orama monitor`** — Runs on your local machine, SSHes into nodes, aggregates results, displays via TUI or tables
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
Developer Machine VPS Nodes (via SSH)
|
|
||||||
┌──────────────────┐ ┌────────────────────┐
|
|
||||||
│ orama monitor │ ──SSH──────────>│ orama node report │
|
|
||||||
│ (TUI / tables) │ <──JSON─────── │ (local collector) │
|
|
||||||
│ │ └────────────────────┘
|
|
||||||
│ CollectOnce() │ ──SSH──────────>│ orama node report │
|
|
||||||
│ DeriveAlerts() │ <──JSON─────── │ (local collector) │
|
|
||||||
│ Render() │ └────────────────────┘
|
|
||||||
└──────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
Each node runs `orama node report --json` locally (no SSH to other nodes), collecting data via `os/exec` and `net/http` to localhost services. The monitor SSHes into all nodes in parallel, collects reports, then runs cross-node analysis to detect cluster-wide issues.
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Interactive TUI (auto-refreshes every 30s)
|
|
||||||
orama monitor --env testnet
|
|
||||||
|
|
||||||
# Cluster overview table
|
|
||||||
orama monitor cluster --env testnet
|
|
||||||
|
|
||||||
# Alerts only
|
|
||||||
orama monitor alerts --env testnet
|
|
||||||
|
|
||||||
# Full JSON report (pipe to jq or feed to LLM)
|
|
||||||
orama monitor report --env testnet
|
|
||||||
```
|
|
||||||
|
|
||||||
## `orama monitor` — Local Orchestrator
|
|
||||||
|
|
||||||
### Usage
|
|
||||||
|
|
||||||
```
|
|
||||||
orama monitor [subcommand] --env <environment> [flags]
|
|
||||||
```
|
|
||||||
|
|
||||||
Without a subcommand, launches the interactive TUI.
|
|
||||||
|
|
||||||
### Global Flags
|
|
||||||
|
|
||||||
| Flag | Default | Description |
|
|
||||||
|------|---------|-------------|
|
|
||||||
| `--env` | *(required)* | Environment: `devnet`, `testnet`, `mainnet` |
|
|
||||||
| `--json` | `false` | Machine-readable JSON output (for one-shot subcommands) |
|
|
||||||
| `--node` | | Filter to a specific node host/IP |
|
|
||||||
| `--config` | `scripts/remote-nodes.conf` | Path to node configuration file |
|
|
||||||
|
|
||||||
### Subcommands
|
|
||||||
|
|
||||||
| Subcommand | Description |
|
|
||||||
|------------|-------------|
|
|
||||||
| `live` | Interactive TUI monitor (default when no subcommand) |
|
|
||||||
| `cluster` | Cluster overview: all nodes, roles, RQLite state, WG peers |
|
|
||||||
| `node` | Per-node health details (system, services, WG, DNS) |
|
|
||||||
| `service` | Service status matrix across all nodes |
|
|
||||||
| `mesh` | WireGuard mesh connectivity and peer details |
|
|
||||||
| `dns` | DNS health: CoreDNS, Caddy, TLS cert expiry, resolution |
|
|
||||||
| `namespaces` | Namespace health across nodes |
|
|
||||||
| `alerts` | Active alerts and warnings sorted by severity |
|
|
||||||
| `report` | Full JSON dump optimized for LLM consumption |
|
|
||||||
|
|
||||||
### Examples
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Cluster overview
|
|
||||||
orama monitor cluster --env testnet
|
|
||||||
|
|
||||||
# Cluster overview as JSON
|
|
||||||
orama monitor cluster --env testnet --json
|
|
||||||
|
|
||||||
# Alerts for all nodes
|
|
||||||
orama monitor alerts --env testnet
|
|
||||||
|
|
||||||
# Single-node deep dive
|
|
||||||
orama monitor node --env testnet --node 51.195.109.238
|
|
||||||
|
|
||||||
# Services for one node
|
|
||||||
orama monitor service --env testnet --node 51.195.109.238
|
|
||||||
|
|
||||||
# WireGuard mesh details
|
|
||||||
orama monitor mesh --env testnet
|
|
||||||
|
|
||||||
# DNS health
|
|
||||||
orama monitor dns --env testnet
|
|
||||||
|
|
||||||
# Namespace health
|
|
||||||
orama monitor namespaces --env testnet
|
|
||||||
|
|
||||||
# Full report for LLM analysis
|
|
||||||
orama monitor report --env testnet | jq .
|
|
||||||
|
|
||||||
# Single-node report
|
|
||||||
orama monitor report --env testnet --node 51.195.109.238
|
|
||||||
|
|
||||||
# Custom config file
|
|
||||||
orama monitor cluster --config /path/to/nodes.conf --env devnet
|
|
||||||
```
|
|
||||||
|
|
||||||
### Interactive TUI
|
|
||||||
|
|
||||||
The `live` subcommand (default) launches a full-screen terminal UI:
|
|
||||||
|
|
||||||
**Tabs:** Overview | Nodes | Services | WG Mesh | DNS | Namespaces | Alerts
|
|
||||||
|
|
||||||
**Key Bindings:**
|
|
||||||
|
|
||||||
| Key | Action |
|
|
||||||
|-----|--------|
|
|
||||||
| `Tab` / `Shift+Tab` | Switch tabs |
|
|
||||||
| `j` / `k` or `↑` / `↓` | Scroll content |
|
|
||||||
| `r` | Force refresh |
|
|
||||||
| `q` / `Ctrl+C` | Quit |
|
|
||||||
|
|
||||||
The TUI auto-refreshes every 30 seconds. A spinner shows during data collection. Colors indicate health: green = healthy, red = critical, yellow = warning.
|
|
||||||
|
|
||||||
### LLM Report Format
|
|
||||||
|
|
||||||
`orama monitor report` outputs structured JSON designed for AI consumption:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"meta": {
|
|
||||||
"environment": "testnet",
|
|
||||||
"collected_at": "2026-02-16T12:00:00Z",
|
|
||||||
"duration_seconds": 3.2,
|
|
||||||
"node_count": 3,
|
|
||||||
"healthy_count": 3
|
|
||||||
},
|
|
||||||
"summary": {
|
|
||||||
"rqlite_leader": "10.0.0.1",
|
|
||||||
"rqlite_voters": "3/3",
|
|
||||||
"rqlite_raft_term": 42,
|
|
||||||
"wg_mesh_status": "all connected",
|
|
||||||
"service_health": "all nominal",
|
|
||||||
"critical_alerts": 0,
|
|
||||||
"warning_alerts": 1,
|
|
||||||
"info_alerts": 0
|
|
||||||
},
|
|
||||||
"alerts": [...],
|
|
||||||
"nodes": [
|
|
||||||
{
|
|
||||||
"host": "51.195.109.238",
|
|
||||||
"status": "healthy",
|
|
||||||
"collection_ms": 526,
|
|
||||||
"report": { ... }
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## `orama node report` — VPS-Side Collector
|
|
||||||
|
|
||||||
Runs locally on a VPS node. Collects all system and service data in parallel and outputs a single JSON blob. Requires root privileges.
|
|
||||||
|
|
||||||
### Usage
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# On a VPS node
|
|
||||||
sudo orama node report --json
|
|
||||||
```
|
|
||||||
|
|
||||||
### What It Collects
|
|
||||||
|
|
||||||
| Section | Data |
|
|
||||||
|---------|------|
|
|
||||||
| **system** | CPU count, load average, memory/disk/swap usage, OOM kills, kernel version, uptime, clock time |
|
|
||||||
| **services** | Systemd service states (active, restarts, memory, CPU, restart loop detection) for 10 core services |
|
|
||||||
| **rqlite** | Raft state, leader, term, applied/commit index, peers, strong read test, readyz, debug vars |
|
|
||||||
| **olric** | Service state, memberlist, member count, restarts, memory, log analysis |
|
|
||||||
| **ipfs** | Daemon/cluster state, swarm/cluster peers, repo size, versions, swarm key |
|
|
||||||
| **gateway** | HTTP health check, subsystem status |
|
|
||||||
| **wireguard** | Interface state, WG IP, peers, handshake ages, MTU, config permissions |
|
|
||||||
| **dns** | CoreDNS/Caddy state, port bindings, resolution tests, TLS cert expiry |
|
|
||||||
| **anyone** | Relay/client state, bootstrap progress, fingerprint |
|
|
||||||
| **network** | Internet reachability, TCP stats, retransmission rate, listening ports, UFW rules |
|
|
||||||
| **processes** | Zombie count, orphan orama processes, panic/fatal count in logs |
|
|
||||||
| **namespaces** | Per-namespace service probes (RQLite, Olric, Gateway) |
|
|
||||||
|
|
||||||
### Performance
|
|
||||||
|
|
||||||
All 12 collectors run in parallel with goroutines. Typical collection time is **< 1 second** per node. HTTP timeouts are 3 seconds, command timeouts are 4 seconds.
|
|
||||||
|
|
||||||
### Output Schema
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"timestamp": "2026-02-16T12:00:00Z",
|
|
||||||
"hostname": "ns1",
|
|
||||||
"version": "0.107.0",
|
|
||||||
"collect_ms": 526,
|
|
||||||
"errors": [],
|
|
||||||
"system": { "cpu_count": 4, "load_avg_1": 0.1, "mem_total_mb": 7937, ... },
|
|
||||||
"services": { "services": [...], "failed_units": [] },
|
|
||||||
"rqlite": { "responsive": true, "raft_state": "Leader", "term": 42, ... },
|
|
||||||
"olric": { "service_active": true, "memberlist_up": true, ... },
|
|
||||||
"ipfs": { "daemon_active": true, "swarm_peers": 2, ... },
|
|
||||||
"gateway": { "responsive": true, "http_status": 200, ... },
|
|
||||||
"wireguard": { "interface_up": true, "wg_ip": "10.0.0.1", "peers": [...], ... },
|
|
||||||
"dns": { "coredns_active": true, "caddy_active": true, "base_tls_days_left": 88, ... },
|
|
||||||
"anyone": { "relay_active": true, "bootstrapped": true, ... },
|
|
||||||
"network": { "internet_reachable": true, "ufw_active": true, ... },
|
|
||||||
"processes": { "zombie_count": 0, "orphan_count": 0, "panic_count": 0, ... },
|
|
||||||
"namespaces": []
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Alert Detection
|
|
||||||
|
|
||||||
Alerts are derived from cross-node analysis of all collected reports. Each alert has a severity level and identifies the affected subsystem and node.
|
|
||||||
|
|
||||||
### Alert Severities
|
|
||||||
|
|
||||||
| Severity | Examples |
|
|
||||||
|----------|----------|
|
|
||||||
| **critical** | SSH collection failed (node unreachable), no RQLite leader, split brain, RQLite unresponsive, WireGuard interface down, WG peer never handshaked, OOM kills, service failed, UFW inactive |
|
|
||||||
| **warning** | Strong read failed, memory > 90%, disk > 85%, stale WG handshake (> 3min), Raft term inconsistency, applied index lag > 100, restart loop detected, TLS cert < 14 days, DNS down, namespace gateway down, Anyone not bootstrapped, clock skew > 5s, binary version mismatch, internet unreachable, high TCP retransmission |
|
|
||||||
| **info** | Zombie processes, orphan orama processes, swap usage > 30% |
|
|
||||||
|
|
||||||
### Cross-Node Checks
|
|
||||||
|
|
||||||
These checks compare data across all nodes:
|
|
||||||
|
|
||||||
- **RQLite Leader**: Exactly one leader exists (no split brain)
|
|
||||||
- **Leader Agreement**: All nodes agree on the same leader address
|
|
||||||
- **Raft Term Consistency**: Term values within 1 of each other
|
|
||||||
- **Applied Index Lag**: Followers within 100 entries of the leader
|
|
||||||
- **WireGuard Peer Symmetry**: Each node has N-1 peers
|
|
||||||
- **Clock Skew**: Node clocks within 5 seconds of each other
|
|
||||||
- **Binary Version**: All nodes running the same version
|
|
||||||
- **WebRTC SFU Coverage**: SFU running on expected nodes (3/3) per namespace
|
|
||||||
- **WebRTC TURN Redundancy**: TURN running on expected nodes (2/3) per namespace
|
|
||||||
|
|
||||||
### Per-Node Checks
|
|
||||||
|
|
||||||
- **RQLite**: Responsive, ready, strong read
|
|
||||||
- **WireGuard**: Interface up, handshake freshness
|
|
||||||
- **System**: Memory, disk, load, OOM kills, swap
|
|
||||||
- **Services**: Systemd state, restart loops
|
|
||||||
- **DNS**: CoreDNS/Caddy up, TLS cert expiry, SOA resolution
|
|
||||||
- **Anyone**: Bootstrap progress
|
|
||||||
- **Processes**: Zombies, orphans, panics in logs
|
|
||||||
- **Namespaces**: Gateway and RQLite per namespace
|
|
||||||
- **WebRTC**: SFU and TURN service health (when provisioned)
|
|
||||||
- **Network**: UFW, internet reachability, TCP retransmission
|
|
||||||
|
|
||||||
## Monitor vs Inspector
|
|
||||||
|
|
||||||
Both tools check cluster health, but they serve different purposes:
|
|
||||||
|
|
||||||
| | `orama monitor` | `orama inspect` |
|
|
||||||
|---|---|---|
|
|
||||||
| **Data source** | `orama node report --json` (single SSH call per node) | 15+ SSH commands per node per subsystem |
|
|
||||||
| **Speed** | ~3-5s for full cluster | ~4-10s for full cluster |
|
|
||||||
| **Output** | TUI, tables, JSON | Tables, JSON |
|
|
||||||
| **Focus** | Real-time monitoring, alert detection | Deep diagnostic checks with pass/fail/warn |
|
|
||||||
| **AI support** | `report` subcommand for LLM input | `--ai` flag for inline analysis |
|
|
||||||
| **Use case** | "Is anything wrong right now?" | "What exactly is wrong and why?" |
|
|
||||||
|
|
||||||
Use `monitor` for day-to-day health checks and the interactive TUI. Use `inspect` for deep diagnostics when something is already known to be broken.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
Uses the same `scripts/remote-nodes.conf` as the inspector. See [INSPECTOR.md](INSPECTOR.md#configuration) for format details.
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
Nodes must have the `orama` CLI installed (via `orama node install` or `upload-source.sh`). The monitor runs `sudo orama node report --json` over SSH, so the binary must be at `/usr/local/bin/orama` on each node.
|
|
||||||
@ -1,248 +0,0 @@
|
|||||||
# Nameserver Setup Guide
|
|
||||||
|
|
||||||
This guide explains how to configure your domain registrar to use Orama Network nodes as authoritative nameservers.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
When you install Orama with the `--nameserver` flag, the node runs CoreDNS to serve DNS records for your domain. This enables:
|
|
||||||
|
|
||||||
- Dynamic DNS for deployments (e.g., `myapp.node-abc123.dbrs.space`)
|
|
||||||
- Wildcard DNS support for all subdomains
|
|
||||||
- ACME DNS-01 challenges for automatic SSL certificates
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
Before setting up nameservers, you need:
|
|
||||||
|
|
||||||
1. **Domain ownership** - A domain you control (e.g., `dbrs.space`)
|
|
||||||
2. **3+ VPS nodes** - Recommended for redundancy
|
|
||||||
3. **Static IP addresses** - Each VPS must have a static public IP
|
|
||||||
4. **Access to registrar DNS settings** - Admin access to your domain registrar
|
|
||||||
|
|
||||||
## Understanding DNS Records
|
|
||||||
|
|
||||||
### NS Records (Nameserver Records)
|
|
||||||
NS records tell the internet which servers are authoritative for your domain:
|
|
||||||
```
|
|
||||||
dbrs.space. IN NS ns1.dbrs.space.
|
|
||||||
dbrs.space. IN NS ns2.dbrs.space.
|
|
||||||
dbrs.space. IN NS ns3.dbrs.space.
|
|
||||||
```
|
|
||||||
|
|
||||||
### Glue Records
|
|
||||||
Glue records are A records that provide IP addresses for nameservers that are under the same domain. They're required because:
|
|
||||||
- `ns1.dbrs.space` is under `dbrs.space`
|
|
||||||
- To resolve `ns1.dbrs.space`, you need to query `dbrs.space` nameservers
|
|
||||||
- But those nameservers ARE `ns1.dbrs.space` - circular dependency!
|
|
||||||
- Glue records break this cycle by providing IPs at the registry level
|
|
||||||
|
|
||||||
```
|
|
||||||
ns1.dbrs.space. IN A 141.227.165.168
|
|
||||||
ns2.dbrs.space. IN A 141.227.165.154
|
|
||||||
ns3.dbrs.space. IN A 141.227.156.51
|
|
||||||
```
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
### Step 1: Install Orama on Each VPS
|
|
||||||
|
|
||||||
Install Orama with the `--nameserver` flag on each VPS that will serve as a nameserver:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# On VPS 1 (ns1)
|
|
||||||
sudo orama install \
|
|
||||||
--nameserver \
|
|
||||||
--domain dbrs.space \
|
|
||||||
--vps-ip 141.227.165.168
|
|
||||||
|
|
||||||
# On VPS 2 (ns2)
|
|
||||||
sudo orama install \
|
|
||||||
--nameserver \
|
|
||||||
--domain dbrs.space \
|
|
||||||
--vps-ip 141.227.165.154
|
|
||||||
|
|
||||||
# On VPS 3 (ns3)
|
|
||||||
sudo orama install \
|
|
||||||
--nameserver \
|
|
||||||
--domain dbrs.space \
|
|
||||||
--vps-ip 141.227.156.51
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2: Configure Your Registrar
|
|
||||||
|
|
||||||
#### For Namecheap
|
|
||||||
|
|
||||||
1. **Log into Namecheap Dashboard**
|
|
||||||
- Go to https://www.namecheap.com
|
|
||||||
- Navigate to **Domain List** → **Manage** (next to your domain)
|
|
||||||
|
|
||||||
2. **Add Glue Records (Personal DNS Servers)**
|
|
||||||
- Go to **Advanced DNS** tab
|
|
||||||
- Scroll down to **Personal DNS Servers** section
|
|
||||||
- Click **Add Nameserver**
|
|
||||||
- Add each nameserver with its IP:
|
|
||||||
| Nameserver | IP Address |
|
|
||||||
|------------|------------|
|
|
||||||
| ns1.yourdomain.com | 141.227.165.168 |
|
|
||||||
| ns2.yourdomain.com | 141.227.165.154 |
|
|
||||||
| ns3.yourdomain.com | 141.227.156.51 |
|
|
||||||
|
|
||||||
3. **Set Custom Nameservers**
|
|
||||||
- Go back to the **Domain** tab
|
|
||||||
- Under **Nameservers**, select **Custom DNS**
|
|
||||||
- Add your nameserver hostnames:
|
|
||||||
- ns1.yourdomain.com
|
|
||||||
- ns2.yourdomain.com
|
|
||||||
- ns3.yourdomain.com
|
|
||||||
- Click the green checkmark to save
|
|
||||||
|
|
||||||
4. **Wait for Propagation**
|
|
||||||
- DNS changes can take 24-48 hours to propagate globally
|
|
||||||
- Most changes are visible within 1-4 hours
|
|
||||||
|
|
||||||
#### For GoDaddy
|
|
||||||
|
|
||||||
1. Log into GoDaddy account
|
|
||||||
2. Go to **My Products** → **DNS** for your domain
|
|
||||||
3. Under **Nameservers**, click **Change**
|
|
||||||
4. Select **Enter my own nameservers**
|
|
||||||
5. Add your nameserver hostnames
|
|
||||||
6. For glue records, go to **DNS Management** → **Host Names**
|
|
||||||
7. Add A records for ns1, ns2, ns3
|
|
||||||
|
|
||||||
#### For Cloudflare (as Registrar)
|
|
||||||
|
|
||||||
1. Log into Cloudflare Dashboard
|
|
||||||
2. Go to **Domain Registration** → your domain
|
|
||||||
3. Under **Nameservers**, change to custom
|
|
||||||
4. Note: Cloudflare Registrar may require contacting support for glue records
|
|
||||||
|
|
||||||
#### For Google Domains
|
|
||||||
|
|
||||||
1. Log into Google Domains
|
|
||||||
2. Select your domain → **DNS**
|
|
||||||
3. Under **Name servers**, select **Use custom name servers**
|
|
||||||
4. Add your nameserver hostnames
|
|
||||||
5. For glue records, click **Add** under **Glue records**
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
### Step 1: Verify NS Records
|
|
||||||
|
|
||||||
After propagation, check that NS records are visible:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check NS records from Google DNS
|
|
||||||
dig NS yourdomain.com @8.8.8.8
|
|
||||||
|
|
||||||
# Expected output should show:
|
|
||||||
# yourdomain.com. IN NS ns1.yourdomain.com.
|
|
||||||
# yourdomain.com. IN NS ns2.yourdomain.com.
|
|
||||||
# yourdomain.com. IN NS ns3.yourdomain.com.
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2: Verify Glue Records
|
|
||||||
|
|
||||||
Check that glue records resolve:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check glue records
|
|
||||||
dig A ns1.yourdomain.com @8.8.8.8
|
|
||||||
dig A ns2.yourdomain.com @8.8.8.8
|
|
||||||
dig A ns3.yourdomain.com @8.8.8.8
|
|
||||||
|
|
||||||
# Each should return the correct IP address
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 3: Test CoreDNS
|
|
||||||
|
|
||||||
Query your nameservers directly:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Test a query against ns1
|
|
||||||
dig @ns1.yourdomain.com test.yourdomain.com
|
|
||||||
|
|
||||||
# Test wildcard resolution
|
|
||||||
dig @ns1.yourdomain.com myapp.node-abc123.yourdomain.com
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 4: Verify from Multiple Locations
|
|
||||||
|
|
||||||
Use online tools to verify global propagation:
|
|
||||||
- https://dnschecker.org
|
|
||||||
- https://www.whatsmydns.net
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### DNS Not Resolving
|
|
||||||
|
|
||||||
1. **Check CoreDNS is running:**
|
|
||||||
```bash
|
|
||||||
sudo systemctl status coredns
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Check CoreDNS logs:**
|
|
||||||
```bash
|
|
||||||
sudo journalctl -u coredns -f
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Verify port 53 is open:**
|
|
||||||
```bash
|
|
||||||
sudo ufw status
|
|
||||||
# Port 53 (TCP/UDP) should be allowed
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Test locally:**
|
|
||||||
```bash
|
|
||||||
dig @localhost yourdomain.com
|
|
||||||
```
|
|
||||||
|
|
||||||
### Glue Records Not Propagating
|
|
||||||
|
|
||||||
- Glue records are stored at the registry level, not DNS level
|
|
||||||
- They can take longer to propagate (up to 48 hours)
|
|
||||||
- Verify at your registrar that they were saved correctly
|
|
||||||
- Some registrars require the domain to be using their nameservers first
|
|
||||||
|
|
||||||
### SERVFAIL Errors
|
|
||||||
|
|
||||||
Usually indicates CoreDNS configuration issues:
|
|
||||||
|
|
||||||
1. Check Corefile syntax
|
|
||||||
2. Verify RQLite connectivity
|
|
||||||
3. Check firewall rules
|
|
||||||
|
|
||||||
## Security Considerations
|
|
||||||
|
|
||||||
### Firewall Rules
|
|
||||||
|
|
||||||
Only expose necessary ports:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Allow DNS from anywhere
|
|
||||||
sudo ufw allow 53/tcp
|
|
||||||
sudo ufw allow 53/udp
|
|
||||||
|
|
||||||
# Restrict admin ports to internal network
|
|
||||||
sudo ufw allow from 10.0.0.0/8 to any port 8080 # Health
|
|
||||||
sudo ufw allow from 10.0.0.0/8 to any port 9153 # Metrics
|
|
||||||
```
|
|
||||||
|
|
||||||
### Rate Limiting
|
|
||||||
|
|
||||||
Consider adding rate limiting to prevent DNS amplification attacks.
|
|
||||||
This can be configured in the CoreDNS Corefile.
|
|
||||||
|
|
||||||
## Multi-Node Coordination
|
|
||||||
|
|
||||||
When running multiple nameservers:
|
|
||||||
|
|
||||||
1. **All nodes share the same RQLite cluster** - DNS records are automatically synchronized
|
|
||||||
2. **Install in order** - First node bootstraps, others join
|
|
||||||
3. **Same domain configuration** - All nodes must use the same `--domain` value
|
|
||||||
|
|
||||||
## Related Documentation
|
|
||||||
|
|
||||||
- [CoreDNS RQLite Plugin](../pkg/coredns/README.md) - Technical details
|
|
||||||
- [Deployment Guide](./DEPLOYMENT_GUIDE.md) - Full deployment instructions
|
|
||||||
- [Architecture](./ARCHITECTURE.md) - System architecture overview
|
|
||||||
@ -1,233 +0,0 @@
|
|||||||
# OramaOS Deployment Guide
|
|
||||||
|
|
||||||
OramaOS is a custom minimal Linux image built with Buildroot. It replaces the standard Ubuntu-based node deployment for mainnet, devnet, and testnet environments. Sandbox clusters remain on Ubuntu for development convenience.
|
|
||||||
|
|
||||||
## What is OramaOS?
|
|
||||||
|
|
||||||
OramaOS is a locked-down operating system designed specifically for Orama node operators. Key properties:
|
|
||||||
|
|
||||||
- **No SSH, no shell** — operators cannot access the filesystem or run commands on the machine
|
|
||||||
- **LUKS full-disk encryption** — the data partition is encrypted; the key is split via Shamir's Secret Sharing across peer nodes
|
|
||||||
- **Read-only rootfs** — the OS image uses SquashFS with dm-verity integrity verification
|
|
||||||
- **A/B partition updates** — signed OS images are applied atomically with automatic rollback on failure
|
|
||||||
- **Service sandboxing** — each service runs in its own Linux namespace with seccomp syscall filtering
|
|
||||||
- **Signed binaries** — all updates are cryptographically signed with the Orama rootwallet
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
Partition Layout:
|
|
||||||
/dev/sda1 — ESP (EFI System Partition, systemd-boot)
|
|
||||||
/dev/sda2 — rootfs-A (SquashFS, read-only, dm-verity)
|
|
||||||
/dev/sda3 — rootfs-B (standby, for A/B updates)
|
|
||||||
/dev/sda4 — data (LUKS2 encrypted, ext4)
|
|
||||||
|
|
||||||
Boot Flow:
|
|
||||||
systemd-boot → dm-verity rootfs → orama-agent → WireGuard → services
|
|
||||||
```
|
|
||||||
|
|
||||||
The **orama-agent** is the only root process. It manages:
|
|
||||||
- Boot sequence and LUKS key reconstruction
|
|
||||||
- WireGuard tunnel setup
|
|
||||||
- Service lifecycle (start, stop, restart in sandboxed namespaces)
|
|
||||||
- Command reception from the Gateway over WireGuard
|
|
||||||
- OS updates (download, verify signature, A/B swap, reboot)
|
|
||||||
|
|
||||||
## Enrollment Flow
|
|
||||||
|
|
||||||
OramaOS nodes join the cluster through an enrollment process (different from the Ubuntu `orama node install` flow):
|
|
||||||
|
|
||||||
### Step 1: Flash OramaOS to VPS
|
|
||||||
|
|
||||||
Download the OramaOS image and flash it to your VPS:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Download image (URL provided upon acceptance)
|
|
||||||
wget https://releases.orama.network/oramaos-v1.0.0-amd64.qcow2
|
|
||||||
|
|
||||||
# Flash to VPS (provider-specific — Hetzner, Vultr, etc.)
|
|
||||||
# Most providers support uploading custom images via their dashboard
|
|
||||||
```
|
|
||||||
|
|
||||||
### Step 2: First Boot — Enrollment Mode
|
|
||||||
|
|
||||||
On first boot, the agent:
|
|
||||||
1. Generates a random 8-character registration code
|
|
||||||
2. Starts a temporary HTTP server on port 9999
|
|
||||||
3. Opens an outbound WebSocket to the Gateway
|
|
||||||
4. Waits for enrollment to complete
|
|
||||||
|
|
||||||
The registration code is displayed on the VPS console (if available) and served at `http://<vps-ip>:9999/`.
|
|
||||||
|
|
||||||
### Step 3: Run Enrollment from CLI
|
|
||||||
|
|
||||||
On your local machine (where you have the `orama` CLI and rootwallet):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Generate an invite token on any existing cluster node
|
|
||||||
orama node invite --expiry 24h
|
|
||||||
|
|
||||||
# Enroll the OramaOS node
|
|
||||||
orama node enroll --node-ip <vps-public-ip> --token <invite-token> --gateway <gateway-url>
|
|
||||||
```
|
|
||||||
|
|
||||||
The enrollment command:
|
|
||||||
1. Fetches the registration code from the node (port 9999)
|
|
||||||
2. Sends the code + invite token to the Gateway
|
|
||||||
3. Gateway validates everything, assigns a WireGuard IP, and pushes config to the node
|
|
||||||
4. Node configures WireGuard, formats the LUKS-encrypted data partition
|
|
||||||
5. LUKS key is split via Shamir and distributed to peer vault-guardians
|
|
||||||
6. Services start in sandboxed namespaces
|
|
||||||
7. Port 9999 closes permanently
|
|
||||||
|
|
||||||
### Step 4: Verify
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check the node is online and healthy
|
|
||||||
orama monitor report --env <env>
|
|
||||||
```
|
|
||||||
|
|
||||||
## Genesis Node
|
|
||||||
|
|
||||||
The first OramaOS node in a cluster is the **genesis node**. It has a special boot path because there are no peers yet for Shamir key distribution:
|
|
||||||
|
|
||||||
1. Genesis generates a LUKS key and encrypts the data partition
|
|
||||||
2. The LUKS key is encrypted with a rootwallet-derived key and stored on the unencrypted rootfs
|
|
||||||
3. On reboot (before enough peers exist), the operator must manually unlock:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
orama node unlock --genesis --node-ip <wg-ip>
|
|
||||||
```
|
|
||||||
|
|
||||||
This command:
|
|
||||||
1. Fetches the encrypted genesis key from the node
|
|
||||||
2. Decrypts it using the rootwallet (`rw decrypt`)
|
|
||||||
3. Sends the decrypted LUKS key to the agent over WireGuard
|
|
||||||
|
|
||||||
Once 5+ peers have joined, the genesis node distributes Shamir shares to peers, deletes the local encrypted key, and transitions to normal Shamir-based unlock. After this transition, `orama node unlock` is no longer needed.
|
|
||||||
|
|
||||||
## Normal Reboot (Shamir Unlock)
|
|
||||||
|
|
||||||
When an enrolled OramaOS node reboots:
|
|
||||||
|
|
||||||
1. Agent starts, brings up WireGuard
|
|
||||||
2. Contacts peer vault-guardians over WireGuard
|
|
||||||
3. Fetches K Shamir shares (K = threshold, typically `max(3, N/3)`)
|
|
||||||
4. Reconstructs LUKS key via Lagrange interpolation over GF(256)
|
|
||||||
5. Decrypts and mounts data partition
|
|
||||||
6. Starts all services
|
|
||||||
7. Zeros key from memory
|
|
||||||
|
|
||||||
If not enough peers are available, the agent enters a degraded "waiting for peers" state and retries with exponential backoff (1s, 2s, 4s, 8s, 16s, max 5 retries per cycle).
|
|
||||||
|
|
||||||
## Node Management
|
|
||||||
|
|
||||||
Since OramaOS has no SSH, all management happens through the Gateway API:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check node status
|
|
||||||
curl "https://gateway.example.com/v1/node/status?node_id=<id>"
|
|
||||||
|
|
||||||
# Send a command (e.g., restart a service)
|
|
||||||
curl -X POST "https://gateway.example.com/v1/node/command?node_id=<id>" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{"action":"restart","service":"rqlite"}'
|
|
||||||
|
|
||||||
# View logs
|
|
||||||
curl "https://gateway.example.com/v1/node/logs?node_id=<id>&service=gateway&lines=100"
|
|
||||||
|
|
||||||
# Graceful node departure
|
|
||||||
curl -X POST "https://gateway.example.com/v1/node/leave" \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{"node_id":"<id>"}'
|
|
||||||
```
|
|
||||||
|
|
||||||
The Gateway proxies these requests to the agent over WireGuard (port 9998). The agent is never directly accessible from the public internet.
|
|
||||||
|
|
||||||
## OS Updates
|
|
||||||
|
|
||||||
OramaOS uses an A/B partition scheme for atomic, rollback-safe updates:
|
|
||||||
|
|
||||||
1. Agent periodically checks for new versions
|
|
||||||
2. Downloads the signed image (P2P over WireGuard between nodes)
|
|
||||||
3. Verifies the rootwallet EVM signature against the embedded public key
|
|
||||||
4. Writes to the standby partition (if running from A, writes to B)
|
|
||||||
5. Sets systemd-boot to boot from B with `tries_left=3`
|
|
||||||
6. Reboots
|
|
||||||
7. If B boots successfully (agent starts, WG connects, services healthy): marks B as "good"
|
|
||||||
8. If B fails 3 times: systemd-boot automatically falls back to A
|
|
||||||
|
|
||||||
No operator intervention is needed for updates. Failed updates are automatically rolled back.
|
|
||||||
|
|
||||||
## Service Sandboxing
|
|
||||||
|
|
||||||
Each service on OramaOS runs in an isolated environment:
|
|
||||||
|
|
||||||
- **Mount namespace** — each service only sees its own data directory as writable; everything else is read-only
|
|
||||||
- **UTS namespace** — isolated hostname
|
|
||||||
- **Dedicated UID/GID** — each service runs as a different user (not root)
|
|
||||||
- **Seccomp filtering** — per-service syscall allowlist (initially in audit mode, then enforce mode)
|
|
||||||
|
|
||||||
Services and their sandbox profiles:
|
|
||||||
| Service | Writable Path | Extra Syscalls |
|
|
||||||
|---------|--------------|----------------|
|
|
||||||
| RQLite | `/opt/orama/.orama/data/rqlite` | fsync, fdatasync (Raft + SQLite WAL) |
|
|
||||||
| Olric | `/opt/orama/.orama/data/olric` | sendmmsg, recvmmsg (gossip) |
|
|
||||||
| IPFS | `/opt/orama/.orama/data/ipfs` | sendfile, splice (data transfer) |
|
|
||||||
| Gateway | `/opt/orama/.orama/data/gateway` | sendfile, splice (HTTP) |
|
|
||||||
| CoreDNS | `/opt/orama/.orama/data/coredns` | sendmmsg, recvmmsg (DNS) |
|
|
||||||
|
|
||||||
## OramaOS vs Ubuntu Deployment
|
|
||||||
|
|
||||||
| Feature | Ubuntu | OramaOS |
|
|
||||||
|---------|--------|---------|
|
|
||||||
| SSH access | Yes | No |
|
|
||||||
| Shell access | Yes | No |
|
|
||||||
| Disk encryption | No | LUKS2 (Shamir) |
|
|
||||||
| OS updates | Manual (`orama node upgrade`) | Automatic (signed, A/B) |
|
|
||||||
| Service isolation | systemd only | Namespaces + seccomp |
|
|
||||||
| Rootfs integrity | None | dm-verity |
|
|
||||||
| Binary signing | Optional | Required |
|
|
||||||
| Operator data access | Full | None |
|
|
||||||
| Environments | All (including sandbox) | Mainnet, devnet, testnet |
|
|
||||||
|
|
||||||
## Cleaning / Factory Reset
|
|
||||||
|
|
||||||
OramaOS nodes cannot be cleaned with the standard `orama node clean` command (no SSH access). Instead:
|
|
||||||
|
|
||||||
- **Graceful departure:** `orama node leave` via the Gateway API — stops services, redistributes Shamir shares, removes WG peer
|
|
||||||
- **Factory reset:** Reflash the OramaOS image on the VPS via the hosting provider's dashboard
|
|
||||||
- **Data is unrecoverable:** Since the LUKS key is distributed across peers, reflashing destroys all data permanently
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Node stuck in enrollment mode
|
|
||||||
The node boots but enrollment never completes.
|
|
||||||
|
|
||||||
**Check:** Can you reach `http://<vps-ip>:9999/` from your machine? If not, the VPS firewall may be blocking port 9999.
|
|
||||||
|
|
||||||
**Fix:** Ensure port 9999 is open in the VPS provider's firewall. OramaOS opens it automatically via its internal firewall, but external provider firewalls (Hetzner, AWS security groups) must be configured separately.
|
|
||||||
|
|
||||||
### LUKS unlock fails (not enough peers)
|
|
||||||
After reboot, the node can't reconstruct its LUKS key.
|
|
||||||
|
|
||||||
**Check:** How many peer nodes are online? The node needs at least K peers (threshold) to be reachable over WireGuard.
|
|
||||||
|
|
||||||
**Fix:** Ensure enough cluster nodes are online. If this is the genesis node and fewer than 5 peers exist, use:
|
|
||||||
```bash
|
|
||||||
orama node unlock --genesis --node-ip <wg-ip>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Update failed, node rolled back
|
|
||||||
The node applied an update but reverted to the previous version.
|
|
||||||
|
|
||||||
**Check:** The agent logs will show why the new partition failed to boot (accessible via `GET /v1/node/logs?service=agent`).
|
|
||||||
|
|
||||||
**Common causes:** Corrupted download (signature verification should catch this), hardware issue, or incompatible configuration.
|
|
||||||
|
|
||||||
### Services not starting after reboot
|
|
||||||
The node rebooted and LUKS unlocked, but services are unhealthy.
|
|
||||||
|
|
||||||
**Check:** `GET /v1/node/status` — which services are down?
|
|
||||||
|
|
||||||
**Fix:** Try restarting the specific service via `POST /v1/node/command` with `{"action":"restart","service":"<name>"}`. If the issue persists, check service logs.
|
|
||||||
@ -1,208 +0,0 @@
|
|||||||
# Sandbox: Ephemeral Hetzner Cloud Clusters
|
|
||||||
|
|
||||||
Spin up temporary 5-node Orama clusters on Hetzner Cloud for development and testing. Total cost: ~€0.04/hour.
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# One-time setup (API key, domain, floating IPs, SSH key)
|
|
||||||
orama sandbox setup
|
|
||||||
|
|
||||||
# Create a cluster (~5 minutes)
|
|
||||||
orama sandbox create --name my-feature
|
|
||||||
|
|
||||||
# Check health
|
|
||||||
orama sandbox status
|
|
||||||
|
|
||||||
# SSH into a node
|
|
||||||
orama sandbox ssh 1
|
|
||||||
|
|
||||||
# Deploy code changes
|
|
||||||
orama sandbox rollout
|
|
||||||
|
|
||||||
# Tear it down
|
|
||||||
orama sandbox destroy
|
|
||||||
```
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
### 1. Hetzner Cloud Account
|
|
||||||
|
|
||||||
Create a project at [console.hetzner.cloud](https://console.hetzner.cloud) and generate an API token with read/write permissions under **Security > API Tokens**.
|
|
||||||
|
|
||||||
### 2. Domain with Glue Records
|
|
||||||
|
|
||||||
You need a domain (or subdomain) that points to Hetzner Floating IPs. The `orama sandbox setup` wizard will guide you through this.
|
|
||||||
|
|
||||||
**Example:** Using `sbx.dbrs.space`
|
|
||||||
|
|
||||||
At your domain registrar:
|
|
||||||
1. Create glue records (Personal DNS Servers):
|
|
||||||
- `ns1.sbx.dbrs.space` → `<floating-ip-1>`
|
|
||||||
- `ns2.sbx.dbrs.space` → `<floating-ip-2>`
|
|
||||||
2. Set custom nameservers for `sbx.dbrs.space`:
|
|
||||||
- `ns1.sbx.dbrs.space`
|
|
||||||
- `ns2.sbx.dbrs.space`
|
|
||||||
|
|
||||||
DNS propagation can take up to 48 hours.
|
|
||||||
|
|
||||||
### 3. Binary Archive
|
|
||||||
|
|
||||||
Build the binary archive before creating a cluster:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
orama build
|
|
||||||
```
|
|
||||||
|
|
||||||
This creates `/tmp/orama-<version>-linux-amd64.tar.gz` with all pre-compiled binaries.
|
|
||||||
|
|
||||||
## Setup
|
|
||||||
|
|
||||||
Run the interactive setup wizard:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
orama sandbox setup
|
|
||||||
```
|
|
||||||
|
|
||||||
This will:
|
|
||||||
1. Prompt for your Hetzner API token and validate it
|
|
||||||
2. Ask for your sandbox domain
|
|
||||||
3. Create or reuse 2 Hetzner Floating IPs (~$0.005/hr each)
|
|
||||||
4. Create a firewall with sandbox rules
|
|
||||||
5. Create a rootwallet SSH entry (`sandbox/root`) if it doesn't exist
|
|
||||||
6. Upload the wallet-derived public key to Hetzner
|
|
||||||
7. Display DNS configuration instructions
|
|
||||||
|
|
||||||
Config is saved to `~/.orama/sandbox.yaml`.
|
|
||||||
|
|
||||||
## Commands
|
|
||||||
|
|
||||||
### `orama sandbox create [--name <name>]`
|
|
||||||
|
|
||||||
Creates a new 5-node cluster. If `--name` is omitted, a random name is generated (e.g., "swift-falcon").
|
|
||||||
|
|
||||||
**Cluster layout:**
|
|
||||||
- Nodes 1-2: Nameservers (CoreDNS + Caddy + all services)
|
|
||||||
- Nodes 3-5: Regular nodes (all services except CoreDNS)
|
|
||||||
|
|
||||||
**Phases:**
|
|
||||||
1. Provision 5 CX22 servers on Hetzner (parallel, ~90s)
|
|
||||||
2. Assign floating IPs to nameserver nodes (~10s)
|
|
||||||
3. Upload binary archive to all nodes (parallel, ~60s)
|
|
||||||
4. Install genesis node + generate invite tokens (~120s)
|
|
||||||
5. Join remaining 4 nodes (serial with health checks, ~180s)
|
|
||||||
6. Verify cluster health (~15s)
|
|
||||||
|
|
||||||
**One sandbox at a time.** Since the floating IPs are shared, only one sandbox can own the nameservers. Destroy the active sandbox before creating a new one.
|
|
||||||
|
|
||||||
### `orama sandbox destroy [--name <name>] [--force]`
|
|
||||||
|
|
||||||
Tears down a cluster:
|
|
||||||
1. Unassigns floating IPs
|
|
||||||
2. Deletes all 5 servers (parallel)
|
|
||||||
3. Removes state file
|
|
||||||
|
|
||||||
Use `--force` to skip confirmation.
|
|
||||||
|
|
||||||
### `orama sandbox list`
|
|
||||||
|
|
||||||
Lists all sandboxes with their status. Also checks Hetzner for orphaned servers that don't have a corresponding state file.
|
|
||||||
|
|
||||||
### `orama sandbox status [--name <name>]`
|
|
||||||
|
|
||||||
Shows per-node health including:
|
|
||||||
- Service status (active/inactive)
|
|
||||||
- RQLite role (Leader/Follower)
|
|
||||||
- Cluster summary (commit index, voter count)
|
|
||||||
|
|
||||||
### `orama sandbox rollout [--name <name>]`
|
|
||||||
|
|
||||||
Deploys code changes:
|
|
||||||
1. Uses the latest binary archive from `/tmp/` (run `orama build` first)
|
|
||||||
2. Pushes to all nodes
|
|
||||||
3. Rolling upgrade: followers first, leader last, 15s between nodes
|
|
||||||
|
|
||||||
### `orama sandbox ssh <node-number>`
|
|
||||||
|
|
||||||
Opens an interactive SSH session to a sandbox node (1-5).
|
|
||||||
|
|
||||||
```bash
|
|
||||||
orama sandbox ssh 1 # SSH into node 1 (genesis/ns1)
|
|
||||||
orama sandbox ssh 3 # SSH into node 3 (regular node)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
### Floating IPs
|
|
||||||
|
|
||||||
Hetzner Floating IPs are persistent IPv4 addresses that can be reassigned between servers. They solve the DNS chicken-and-egg problem:
|
|
||||||
|
|
||||||
- Glue records at the registrar point to 2 Floating IPs (configured once)
|
|
||||||
- Each new sandbox assigns the Floating IPs to its nameserver nodes
|
|
||||||
- DNS works instantly — no propagation delay between clusters
|
|
||||||
|
|
||||||
### SSH Authentication
|
|
||||||
|
|
||||||
Sandbox uses a rootwallet-derived SSH key (`sandbox/root` vault entry), the same mechanism as production. The wallet must be unlocked (`rw unlock`) before running sandbox commands that use SSH. The public key is uploaded to Hetzner during setup and injected into every server at creation time.
|
|
||||||
|
|
||||||
### Server Naming
|
|
||||||
|
|
||||||
Servers: `sbx-<name>-<N>` (e.g., `sbx-swift-falcon-1` through `sbx-swift-falcon-5`)
|
|
||||||
|
|
||||||
### State Files
|
|
||||||
|
|
||||||
Sandbox state is stored at `~/.orama/sandboxes/<name>.yaml`. This tracks server IDs, IPs, roles, and cluster status.
|
|
||||||
|
|
||||||
## Cost
|
|
||||||
|
|
||||||
| Resource | Cost | Qty | Total |
|
|
||||||
|----------|------|-----|-------|
|
|
||||||
| CX22 (2 vCPU, 4GB) | €0.006/hr | 5 | €0.03/hr |
|
|
||||||
| Floating IPv4 | €0.005/hr | 2 | €0.01/hr |
|
|
||||||
| **Total** | | | **~€0.04/hr** |
|
|
||||||
|
|
||||||
Servers are billed per hour. Floating IPs are billed as long as they exist (even unassigned). Destroy the sandbox when not in use to save on server costs.
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### "sandbox not configured"
|
|
||||||
|
|
||||||
Run `orama sandbox setup` first.
|
|
||||||
|
|
||||||
### "no binary archive found"
|
|
||||||
|
|
||||||
Run `orama build` to create the binary archive.
|
|
||||||
|
|
||||||
### "sandbox X is already active"
|
|
||||||
|
|
||||||
Only one sandbox can be active at a time. Destroy it first:
|
|
||||||
```bash
|
|
||||||
orama sandbox destroy --name <name>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Server creation fails
|
|
||||||
|
|
||||||
Check:
|
|
||||||
- Hetzner API token is valid and has read/write permissions
|
|
||||||
- You haven't hit Hetzner's server limit (default: 10 per project)
|
|
||||||
- The selected location has CX22 capacity
|
|
||||||
|
|
||||||
### Genesis install fails
|
|
||||||
|
|
||||||
SSH into the node to debug:
|
|
||||||
```bash
|
|
||||||
orama sandbox ssh 1
|
|
||||||
journalctl -u orama-node -f
|
|
||||||
```
|
|
||||||
|
|
||||||
The sandbox will be left in "error" state. You can destroy and recreate it.
|
|
||||||
|
|
||||||
### DNS not resolving
|
|
||||||
|
|
||||||
1. Verify glue records are configured at your registrar
|
|
||||||
2. Check propagation: `dig NS sbx.dbrs.space @8.8.8.8`
|
|
||||||
3. Propagation can take 24-48 hours for new domains
|
|
||||||
|
|
||||||
### Orphaned servers
|
|
||||||
|
|
||||||
If `orama sandbox list` shows orphaned servers, delete them manually at [console.hetzner.cloud](https://console.hetzner.cloud). Sandbox servers are labeled `orama-sandbox=<name>` for easy identification.
|
|
||||||
@ -1,194 +0,0 @@
|
|||||||
# Security Hardening
|
|
||||||
|
|
||||||
This document describes all security measures applied to the Orama Network, covering both Phase 1 (service hardening on existing Ubuntu nodes) and Phase 2 (OramaOS locked-down image).
|
|
||||||
|
|
||||||
## Phase 1: Service Hardening
|
|
||||||
|
|
||||||
These measures apply to all nodes (Ubuntu and OramaOS).
|
|
||||||
|
|
||||||
### Network Isolation
|
|
||||||
|
|
||||||
**CIDR Validation (Step 1.1)**
|
|
||||||
- WireGuard subnet restricted to `10.0.0.0/24` across all components: firewall rules, rate limiter, auth module, and WireGuard PostUp/PostDown iptables rules
|
|
||||||
- Prevents other tenants on shared VPS providers from bypassing the firewall via overlapping `10.x.x.x` ranges
|
|
||||||
|
|
||||||
**IPv6 Disabled (Step 1.2)**
|
|
||||||
- IPv6 disabled system-wide via sysctl: `net.ipv6.conf.all.disable_ipv6=1`
|
|
||||||
- Prevents services bound to `0.0.0.0` from being reachable via IPv6 (which had no firewall rules)
|
|
||||||
|
|
||||||
### Authentication
|
|
||||||
|
|
||||||
**Internal Endpoint Auth (Step 1.3)**
|
|
||||||
- `/v1/internal/wg/peers` and `/v1/internal/wg/peer/remove` now require cluster secret validation
|
|
||||||
- Peer removal additionally validates the request originates from a WireGuard subnet IP
|
|
||||||
|
|
||||||
**RQLite Authentication (Step 1.7)**
|
|
||||||
- RQLite runs with `-auth` flag pointing to a credentials file
|
|
||||||
- All RQLite HTTP requests include `Authorization: Basic <base64>` headers
|
|
||||||
- Credentials generated at cluster genesis, distributed to joining nodes via join response
|
|
||||||
- Both the central RQLite client wrapper and the standalone CoreDNS RQLite client send auth
|
|
||||||
|
|
||||||
**Olric Gossip Encryption (Step 1.8)**
|
|
||||||
- Olric memberlist uses a 32-byte encryption key for all gossip traffic
|
|
||||||
- Key generated at genesis, distributed via join response
|
|
||||||
- Prevents rogue nodes from joining the gossip ring and poisoning caches
|
|
||||||
- Note: encryption is all-or-nothing (coordinated restart required when enabling)
|
|
||||||
|
|
||||||
**IPFS Cluster TrustedPeers (Step 1.9)**
|
|
||||||
- IPFS Cluster `TrustedPeers` populated with actual cluster peer IDs (was `["*"]`)
|
|
||||||
- New peers added to TrustedPeers on all existing nodes during join
|
|
||||||
- Prevents unauthorized peers from controlling IPFS pinning
|
|
||||||
|
|
||||||
**Vault V1 Auth Enforcement (Step 1.14)**
|
|
||||||
- V1 push/pull endpoints require a valid session token when vault-guardian is configured
|
|
||||||
- Previously, auth was optional for backward compatibility — any WG peer could read/overwrite Shamir shares
|
|
||||||
|
|
||||||
### Token & Key Storage
|
|
||||||
|
|
||||||
**Refresh Token Hashing (Step 1.5)**
|
|
||||||
- Refresh tokens stored as SHA-256 hashes in RQLite (never plaintext)
|
|
||||||
- On lookup: hash the incoming token, query by hash
|
|
||||||
- On revocation: hash before revoking (both single-token and by-subject)
|
|
||||||
- Existing tokens invalidated on upgrade (users re-authenticate)
|
|
||||||
|
|
||||||
**API Key Hashing (Step 1.6)**
|
|
||||||
- API keys stored as HMAC-SHA256 hashes using a server-side secret
|
|
||||||
- HMAC secret generated at cluster genesis, stored in `~/.orama/secrets/api-key-hmac-secret`
|
|
||||||
- On lookup: compute HMAC, query by hash — fast enough for every request (unlike bcrypt)
|
|
||||||
- In-memory cache uses raw key as cache key (never persisted)
|
|
||||||
- During rolling upgrade: dual lookup (HMAC first, then raw as fallback) until all nodes upgraded
|
|
||||||
|
|
||||||
**TURN Secret Encryption (Step 1.15)**
|
|
||||||
- TURN shared secrets encrypted at rest in RQLite using AES-256-GCM
|
|
||||||
- Encryption key derived via HKDF from the cluster secret with purpose string `"turn-encryption"`
|
|
||||||
|
|
||||||
### TLS & Transport
|
|
||||||
|
|
||||||
**InsecureSkipVerify Fix (Step 1.10)**
|
|
||||||
- During node join, TLS verification uses TOFU (Trust On First Use)
|
|
||||||
- Invite token output includes the CA certificate fingerprint (SHA-256)
|
|
||||||
- Joining node verifies the server cert fingerprint matches before proceeding
|
|
||||||
- After join: CA cert stored locally for future connections
|
|
||||||
|
|
||||||
**WebSocket Origin Validation (Step 1.4)**
|
|
||||||
- All WebSocket upgraders validate the `Origin` header against the node's configured domain
|
|
||||||
- Non-browser clients (no Origin header) are still allowed
|
|
||||||
- Prevents cross-site WebSocket hijacking attacks
|
|
||||||
|
|
||||||
### Process Isolation
|
|
||||||
|
|
||||||
**Dedicated User (Step 1.11)**
|
|
||||||
- All services run as the `orama` user (not root)
|
|
||||||
- Caddy and CoreDNS get `AmbientCapabilities=CAP_NET_BIND_SERVICE` for ports 80/443 and 53
|
|
||||||
- WireGuard stays as root (kernel netlink requires it)
|
|
||||||
- vault-guardian already had proper hardening
|
|
||||||
|
|
||||||
**systemd Hardening (Step 1.12)**
|
|
||||||
- All service units include:
|
|
||||||
```ini
|
|
||||||
ProtectSystem=strict
|
|
||||||
ProtectHome=yes
|
|
||||||
NoNewPrivileges=yes
|
|
||||||
PrivateDevices=yes
|
|
||||||
ProtectKernelTunables=yes
|
|
||||||
ProtectKernelModules=yes
|
|
||||||
RestrictNamespaces=yes
|
|
||||||
ReadWritePaths=/opt/orama/.orama
|
|
||||||
```
|
|
||||||
- Applied to both template files (`pkg/environments/templates/`) and hardcoded unit generators (`pkg/environments/production/services.go`)
|
|
||||||
|
|
||||||
### Supply Chain
|
|
||||||
|
|
||||||
**Binary Signing (Step 1.13)**
|
|
||||||
- Build archives include `manifest.sig` — a rootwallet EVM signature of the manifest hash
|
|
||||||
- During install, the signature is verified against the embedded Orama public key
|
|
||||||
- Unsigned or tampered archives are rejected
|
|
||||||
|
|
||||||
## Phase 2: OramaOS
|
|
||||||
|
|
||||||
These measures apply only to OramaOS nodes (mainnet, devnet, testnet).
|
|
||||||
|
|
||||||
### Immutable OS
|
|
||||||
|
|
||||||
- **Read-only rootfs** — SquashFS with dm-verity integrity verification
|
|
||||||
- **No shell** — `/bin/sh` symlinked to `/bin/false`, no bash/ash/ssh
|
|
||||||
- **No SSH** — OpenSSH not included in the image
|
|
||||||
- **Minimal packages** — only what's needed for systemd, cryptsetup, and the agent
|
|
||||||
|
|
||||||
### Full-Disk Encryption
|
|
||||||
|
|
||||||
- **LUKS2** with AES-XTS-Plain64 on the data partition
|
|
||||||
- **Shamir's Secret Sharing** over GF(256) — LUKS key split across peer vault-guardians
|
|
||||||
- **Adaptive threshold** — K = max(3, N/3) where N is the number of peers
|
|
||||||
- **Key zeroing** — LUKS key wiped from memory immediately after use
|
|
||||||
- **Malicious share detection** — fetch K+1 shares when possible, verify consistency
|
|
||||||
|
|
||||||
### Service Sandboxing
|
|
||||||
|
|
||||||
Each service runs in isolated Linux namespaces:
|
|
||||||
- **CLONE_NEWNS** — mount namespace (filesystem isolation)
|
|
||||||
- **CLONE_NEWUTS** — hostname namespace
|
|
||||||
- **Dedicated UID/GID** — each service has its own user
|
|
||||||
- **Seccomp filtering** — per-service syscall allowlist
|
|
||||||
|
|
||||||
Note: CLONE_NEWPID is intentionally omitted — it makes services PID 1 in their namespace, which changes signal semantics (SIGTERM ignored by default for PID 1).
|
|
||||||
|
|
||||||
### Signed Updates
|
|
||||||
|
|
||||||
- A/B partition scheme with systemd-boot and boot counting (`tries_left=3`)
|
|
||||||
- All updates signed with rootwallet EVM signature (secp256k1 + keccak256)
|
|
||||||
- Signer address: `0xb5d8a496c8b2412990d7D467E17727fdF5954afC`
|
|
||||||
- P2P distribution over WireGuard between nodes
|
|
||||||
- Automatic rollback on 3 consecutive boot failures
|
|
||||||
|
|
||||||
### Zero Operator Access
|
|
||||||
|
|
||||||
- Operators cannot read data on the machine (LUKS encrypted, no shell)
|
|
||||||
- Management only through Gateway API → agent over WireGuard
|
|
||||||
- All commands are logged and auditable
|
|
||||||
- No root access, no console access, no file system access
|
|
||||||
|
|
||||||
## Rollout Strategy
|
|
||||||
|
|
||||||
### Phase 1 Batches
|
|
||||||
|
|
||||||
```
|
|
||||||
Batch 1 (zero-risk, no restart):
|
|
||||||
- CIDR fix
|
|
||||||
- IPv6 disable
|
|
||||||
- Internal endpoint auth
|
|
||||||
- WebSocket origin check
|
|
||||||
|
|
||||||
Batch 2 (medium-risk, restart needed):
|
|
||||||
- Hash refresh tokens
|
|
||||||
- Hash API keys
|
|
||||||
- Binary signing
|
|
||||||
- Vault V1 auth enforcement
|
|
||||||
- TURN secret encryption
|
|
||||||
|
|
||||||
Batch 3 (high-risk, coordinated rollout):
|
|
||||||
- RQLite auth (followers first, leader last)
|
|
||||||
- Olric encryption (simultaneous restart)
|
|
||||||
- IPFS Cluster TrustedPeers
|
|
||||||
|
|
||||||
Batch 4 (infrastructure changes):
|
|
||||||
- InsecureSkipVerify fix
|
|
||||||
- Dedicated user
|
|
||||||
- systemd hardening
|
|
||||||
```
|
|
||||||
|
|
||||||
### Phase 2
|
|
||||||
|
|
||||||
1. Build and test OramaOS image in QEMU
|
|
||||||
2. Deploy to sandbox cluster alongside Ubuntu nodes
|
|
||||||
3. Verify interop and stability
|
|
||||||
4. Gradual migration: testnet → devnet → mainnet (one node at a time, maintaining Raft quorum)
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
All changes verified on sandbox cluster before production deployment:
|
|
||||||
|
|
||||||
- `make test` — all unit tests pass
|
|
||||||
- `orama monitor report --env sandbox` — full cluster health
|
|
||||||
- Manual endpoint testing (e.g., curl without auth → 401)
|
|
||||||
- Security-specific checks (IPv6 listeners, RQLite auth, binary signatures)
|
|
||||||
@ -1,374 +0,0 @@
|
|||||||
# Serverless Functions
|
|
||||||
|
|
||||||
Orama Network runs serverless functions as sandboxed WebAssembly (WASM) modules. Functions are written in Go, compiled to WASM with TinyGo, and executed in an isolated wazero runtime with configurable memory limits and timeouts.
|
|
||||||
|
|
||||||
Functions receive input via **stdin** (JSON) and return output via **stdout** (JSON). They can also access Orama services — database, cache, storage, secrets, PubSub, and HTTP — through **host functions** injected by the runtime.
|
|
||||||
|
|
||||||
## Quick Start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Scaffold a new function
|
|
||||||
orama function init my-function
|
|
||||||
|
|
||||||
# 2. Edit your handler
|
|
||||||
cd my-function
|
|
||||||
# edit function.go
|
|
||||||
|
|
||||||
# 3. Build to WASM
|
|
||||||
orama function build
|
|
||||||
|
|
||||||
# 4. Deploy
|
|
||||||
orama function deploy
|
|
||||||
|
|
||||||
# 5. Invoke
|
|
||||||
orama function invoke my-function --data '{"name": "World"}'
|
|
||||||
|
|
||||||
# 6. View logs
|
|
||||||
orama function logs my-function
|
|
||||||
```
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
my-function/
|
|
||||||
├── function.go # Handler code
|
|
||||||
└── function.yaml # Configuration
|
|
||||||
```
|
|
||||||
|
|
||||||
### function.yaml
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
name: my-function # Required. Letters, digits, hyphens, underscores.
|
|
||||||
public: false # Allow unauthenticated invocation (default: false)
|
|
||||||
memory: 64 # Memory limit in MB (1-256, default: 64)
|
|
||||||
timeout: 30 # Execution timeout in seconds (1-300, default: 30)
|
|
||||||
retry:
|
|
||||||
count: 0 # Retry attempts on failure (default: 0)
|
|
||||||
delay: 5 # Seconds between retries (default: 5)
|
|
||||||
env: # Environment variables (accessible via get_env)
|
|
||||||
MY_VAR: "value"
|
|
||||||
```
|
|
||||||
|
|
||||||
### function.go (minimal)
|
|
||||||
|
|
||||||
```go
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
)
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
// Read JSON input from stdin
|
|
||||||
var input []byte
|
|
||||||
buf := make([]byte, 4096)
|
|
||||||
for {
|
|
||||||
n, err := os.Stdin.Read(buf)
|
|
||||||
if n > 0 {
|
|
||||||
input = append(input, buf[:n]...)
|
|
||||||
}
|
|
||||||
if err != nil {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
var payload map[string]interface{}
|
|
||||||
json.Unmarshal(input, &payload)
|
|
||||||
|
|
||||||
// Process and return JSON output via stdout
|
|
||||||
response := map[string]interface{}{
|
|
||||||
"result": "Hello!",
|
|
||||||
}
|
|
||||||
output, _ := json.Marshal(response)
|
|
||||||
os.Stdout.Write(output)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Building
|
|
||||||
|
|
||||||
Functions are compiled to WASM using [TinyGo](https://tinygo.org/):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Using the CLI (recommended)
|
|
||||||
orama function build
|
|
||||||
|
|
||||||
# Or manually
|
|
||||||
tinygo build -o function.wasm -target wasi function.go
|
|
||||||
```
|
|
||||||
|
|
||||||
## Host Functions API
|
|
||||||
|
|
||||||
Host functions let your WASM code interact with Orama services. They are imported from the `"env"` or `"host"` module (both work) and use a pointer/length ABI for string parameters.
|
|
||||||
|
|
||||||
All host functions are registered at runtime by the engine. They are available to every function without additional configuration.
|
|
||||||
|
|
||||||
### Context
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `get_caller_wallet()` → string | Wallet address of the caller (from JWT) |
|
|
||||||
| `get_request_id()` → string | Unique invocation ID |
|
|
||||||
| `get_env(key)` → string | Environment variable from function.yaml |
|
|
||||||
| `get_secret(name)` → string | Decrypted secret value (see [Managing Secrets](#managing-secrets)) |
|
|
||||||
|
|
||||||
### Database (RQLite)
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `db_query(sql, argsJSON)` → JSON | Execute SELECT query. Args as JSON array. Returns JSON array of row objects. |
|
|
||||||
| `db_execute(sql, argsJSON)` → int | Execute INSERT/UPDATE/DELETE. Returns affected row count. |
|
|
||||||
|
|
||||||
Example query from WASM:
|
|
||||||
```
|
|
||||||
db_query("SELECT push_token, device_type FROM devices WHERE user_id = ?", '["user123"]')
|
|
||||||
→ [{"push_token": "abc...", "device_type": "ios"}]
|
|
||||||
```
|
|
||||||
|
|
||||||
### Cache (Olric Distributed Cache)
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `cache_get(key)` → bytes | Get cached value by key. Returns empty on miss. |
|
|
||||||
| `cache_set(key, value, ttl)` | Store value with TTL in seconds. |
|
|
||||||
| `cache_incr(key)` → int64 | Atomically increment by 1 (init to 0 if missing). |
|
|
||||||
| `cache_incr_by(key, delta)` → int64 | Atomically increment by delta. |
|
|
||||||
|
|
||||||
### HTTP
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `http_fetch(method, url, headersJSON, body)` → JSON | Make outbound HTTP request. Headers as JSON object. Returns `{"status": 200, "headers": {...}, "body": "..."}`. Timeout: 30s. |
|
|
||||||
|
|
||||||
### PubSub
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `pubsub_publish(topic, dataJSON)` → bool | Publish message to a PubSub topic. Returns true on success. |
|
|
||||||
|
|
||||||
### Logging
|
|
||||||
|
|
||||||
| Function | Description |
|
|
||||||
|----------|-------------|
|
|
||||||
| `log_info(message)` | Log info-level message (captured in invocation logs). |
|
|
||||||
| `log_error(message)` | Log error-level message. |
|
|
||||||
|
|
||||||
## Managing Secrets
|
|
||||||
|
|
||||||
Secrets are encrypted at rest (AES-256-GCM) and scoped to your namespace. Functions read them via `get_secret("name")` at runtime.
|
|
||||||
|
|
||||||
### CLI Commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Set a secret (inline value)
|
|
||||||
orama function secrets set APNS_KEY_ID "ABC123DEF"
|
|
||||||
|
|
||||||
# Set a secret from a file (useful for PEM keys, certificates)
|
|
||||||
orama function secrets set APNS_AUTH_KEY --from-file ./AuthKey_ABC123.p8
|
|
||||||
|
|
||||||
# List all secret names (values are never shown)
|
|
||||||
orama function secrets list
|
|
||||||
|
|
||||||
# Delete a secret
|
|
||||||
orama function secrets delete APNS_KEY_ID
|
|
||||||
|
|
||||||
# Delete without confirmation
|
|
||||||
orama function secrets delete APNS_KEY_ID --force
|
|
||||||
```
|
|
||||||
|
|
||||||
### How It Works
|
|
||||||
|
|
||||||
1. **You set secrets** via the CLI → encrypted and stored in the database
|
|
||||||
2. **Functions read secrets** at runtime via `get_secret("name")` → decrypted on demand
|
|
||||||
3. **Namespace isolation** → each namespace has its own secret store; functions in namespace A cannot read secrets from namespace B
|
|
||||||
|
|
||||||
## PubSub Triggers
|
|
||||||
|
|
||||||
Triggers let functions react to events automatically. When a message is published to a PubSub topic, all functions with a trigger on that topic are invoked asynchronously.
|
|
||||||
|
|
||||||
### CLI Commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Add a trigger: invoke "call-push-handler" when messages hit "calls:invite"
|
|
||||||
orama function triggers add call-push-handler --topic calls:invite
|
|
||||||
|
|
||||||
# List triggers for a function
|
|
||||||
orama function triggers list call-push-handler
|
|
||||||
|
|
||||||
# Delete a trigger
|
|
||||||
orama function triggers delete call-push-handler <trigger-id>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Trigger Event Payload
|
|
||||||
|
|
||||||
When triggered via PubSub, the function receives this JSON via stdin:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"topic": "calls:invite",
|
|
||||||
"data": { ... },
|
|
||||||
"namespace": "my-namespace",
|
|
||||||
"trigger_depth": 1,
|
|
||||||
"timestamp": 1708972800
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Depth Limiting
|
|
||||||
|
|
||||||
To prevent infinite loops (function A publishes to topic → triggers function A again), trigger depth is tracked. Maximum depth is **5**. If a function's output triggers another function, `trigger_depth` increments. At depth 5, no further triggers fire.
|
|
||||||
|
|
||||||
## Function Lifecycle
|
|
||||||
|
|
||||||
### Versioning
|
|
||||||
|
|
||||||
Each deploy creates a new version. The WASM binary is stored in **IPFS** (content-addressed) and metadata is stored in **RQLite**.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# List versions
|
|
||||||
orama function versions my-function
|
|
||||||
|
|
||||||
# Invoke a specific version
|
|
||||||
curl -X POST .../v1/functions/my-function@2/invoke
|
|
||||||
```
|
|
||||||
|
|
||||||
### Invocation Logging
|
|
||||||
|
|
||||||
Every invocation is logged with: request ID, duration, status (success/error/timeout), input/output size, and any `log_info`/`log_error` messages.
|
|
||||||
|
|
||||||
```bash
|
|
||||||
orama function logs my-function
|
|
||||||
```
|
|
||||||
|
|
||||||
## CLI Reference
|
|
||||||
|
|
||||||
| Command | Description |
|
|
||||||
|---------|-------------|
|
|
||||||
| `orama function init <name>` | Scaffold a new function project |
|
|
||||||
| `orama function build [dir]` | Compile Go to WASM |
|
|
||||||
| `orama function deploy [dir]` | Deploy WASM to the network |
|
|
||||||
| `orama function invoke <name> --data <json>` | Invoke a function |
|
|
||||||
| `orama function list` | List deployed functions |
|
|
||||||
| `orama function get <name>` | Get function details |
|
|
||||||
| `orama function delete <name>` | Delete a function |
|
|
||||||
| `orama function logs <name>` | View invocation logs |
|
|
||||||
| `orama function versions <name>` | List function versions |
|
|
||||||
| `orama function secrets set <name> <value>` | Set an encrypted secret |
|
|
||||||
| `orama function secrets list` | List secret names |
|
|
||||||
| `orama function secrets delete <name>` | Delete a secret |
|
|
||||||
| `orama function triggers add <fn> --topic <t>` | Add PubSub trigger |
|
|
||||||
| `orama function triggers list <fn>` | List triggers |
|
|
||||||
| `orama function triggers delete <fn> <id>` | Delete a trigger |
|
|
||||||
|
|
||||||
## HTTP API Reference
|
|
||||||
|
|
||||||
| Method | Endpoint | Description |
|
|
||||||
|--------|----------|-------------|
|
|
||||||
| POST | `/v1/functions` | Deploy function (multipart/form-data) |
|
|
||||||
| GET | `/v1/functions` | List functions |
|
|
||||||
| GET | `/v1/functions/{name}` | Get function info |
|
|
||||||
| DELETE | `/v1/functions/{name}` | Delete function |
|
|
||||||
| POST | `/v1/functions/{name}/invoke` | Invoke function |
|
|
||||||
| GET | `/v1/functions/{name}/versions` | List versions |
|
|
||||||
| GET | `/v1/functions/{name}/logs` | Get logs |
|
|
||||||
| WS | `/v1/functions/{name}/ws` | WebSocket invoke (streaming) |
|
|
||||||
| PUT | `/v1/functions/secrets` | Set a secret |
|
|
||||||
| GET | `/v1/functions/secrets` | List secret names |
|
|
||||||
| DELETE | `/v1/functions/secrets/{name}` | Delete a secret |
|
|
||||||
| POST | `/v1/functions/{name}/triggers` | Add PubSub trigger |
|
|
||||||
| GET | `/v1/functions/{name}/triggers` | List triggers |
|
|
||||||
| DELETE | `/v1/functions/{name}/triggers/{id}` | Delete trigger |
|
|
||||||
| POST | `/v1/invoke/{namespace}/{name}` | Direct invoke (alt endpoint) |
|
|
||||||
|
|
||||||
## Example: Call Push Handler
|
|
||||||
|
|
||||||
A real-world function that sends VoIP push notifications when a call invite is published to PubSub:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# function.yaml
|
|
||||||
name: call-push-handler
|
|
||||||
memory: 128
|
|
||||||
timeout: 30
|
|
||||||
```
|
|
||||||
|
|
||||||
```go
|
|
||||||
// function.go — triggered by PubSub on "calls:invite"
|
|
||||||
package main
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"os"
|
|
||||||
)
|
|
||||||
|
|
||||||
// This function:
|
|
||||||
// 1. Receives a call invite event from PubSub trigger
|
|
||||||
// 2. Queries the database for the callee's device info
|
|
||||||
// 3. Reads push notification credentials from secrets
|
|
||||||
// 4. Sends a push notification via http_fetch
|
|
||||||
|
|
||||||
func main() {
|
|
||||||
// Read PubSub trigger event from stdin
|
|
||||||
var input []byte
|
|
||||||
buf := make([]byte, 4096)
|
|
||||||
for {
|
|
||||||
n, err := os.Stdin.Read(buf)
|
|
||||||
if n > 0 {
|
|
||||||
input = append(input, buf[:n]...)
|
|
||||||
}
|
|
||||||
if err != nil {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Parse the trigger event wrapper
|
|
||||||
var event struct {
|
|
||||||
Topic string `json:"topic"`
|
|
||||||
Data json.RawMessage `json:"data"`
|
|
||||||
}
|
|
||||||
json.Unmarshal(input, &event)
|
|
||||||
|
|
||||||
// Parse the actual call invite data
|
|
||||||
var invite struct {
|
|
||||||
CalleeID string `json:"calleeId"`
|
|
||||||
CallerName string `json:"callerName"`
|
|
||||||
CallType string `json:"callType"`
|
|
||||||
}
|
|
||||||
json.Unmarshal(event.Data, &invite)
|
|
||||||
|
|
||||||
// At this point, the function would use host functions:
|
|
||||||
//
|
|
||||||
// 1. db_query("SELECT push_token, device_type FROM devices WHERE user_id = ?",
|
|
||||||
// json.Marshal([]string{invite.CalleeID}))
|
|
||||||
//
|
|
||||||
// 2. get_secret("FCM_SERVER_KEY") for Android push
|
|
||||||
// get_secret("APNS_KEY_PEM") for iOS push
|
|
||||||
//
|
|
||||||
// 3. http_fetch("POST", "https://fcm.googleapis.com/v1/...", headers, body)
|
|
||||||
//
|
|
||||||
// 4. log_info("Push sent to " + invite.CalleeID)
|
|
||||||
//
|
|
||||||
// Note: Host functions use the WASM ABI (pointer/length).
|
|
||||||
// A Go SDK for ergonomic access is planned.
|
|
||||||
|
|
||||||
response := map[string]interface{}{
|
|
||||||
"status": "sent",
|
|
||||||
"callee": invite.CalleeID,
|
|
||||||
}
|
|
||||||
output, _ := json.Marshal(response)
|
|
||||||
os.Stdout.Write(output)
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Deploy and wire the trigger:
|
|
||||||
```bash
|
|
||||||
orama function build
|
|
||||||
orama function deploy
|
|
||||||
|
|
||||||
# Set push notification secrets
|
|
||||||
orama function secrets set FCM_SERVER_KEY "your-fcm-key"
|
|
||||||
orama function secrets set APNS_KEY_PEM --from-file ./AuthKey.p8
|
|
||||||
orama function secrets set APNS_KEY_ID "ABC123"
|
|
||||||
orama function secrets set APNS_TEAM_ID "TEAM456"
|
|
||||||
|
|
||||||
# Wire the PubSub trigger
|
|
||||||
orama function triggers add call-push-handler --topic calls:invite
|
|
||||||
```
|
|
||||||
@ -1,291 +0,0 @@
|
|||||||
# WebRTC Integration
|
|
||||||
|
|
||||||
Real-time voice, video, and data channels for Orama Network namespaces.
|
|
||||||
|
|
||||||
## Architecture
|
|
||||||
|
|
||||||
```
|
|
||||||
Client A Client B
|
|
||||||
│ │
|
|
||||||
│ 1. Get TURN credentials (REST) │
|
|
||||||
│ 2. Connect WebSocket (signaling) │
|
|
||||||
│ 3. Exchange SDP/ICE via SFU │
|
|
||||||
│ │
|
|
||||||
▼ ▼
|
|
||||||
┌──────────┐ UDP relay ┌──────────┐
|
|
||||||
│ TURN │◄──────────────────►│ TURN │
|
|
||||||
│ Server │ (public IPs) │ Server │
|
|
||||||
│ Node 1 │ │ Node 2 │
|
|
||||||
└────┬─────┘ └────┬─────┘
|
|
||||||
│ WireGuard │ WireGuard
|
|
||||||
▼ ▼
|
|
||||||
┌──────────────────────────────────────────┐
|
|
||||||
│ SFU Servers (3 nodes) │
|
|
||||||
│ - WebSocket signaling (WireGuard only) │
|
|
||||||
│ - Pion WebRTC (RTP forwarding) │
|
|
||||||
│ - Room management │
|
|
||||||
│ - Track publish/subscribe │
|
|
||||||
└──────────────────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
**Key design decisions:**
|
|
||||||
- **TURN-shielded**: SFU binds only to WireGuard IPs. All client media flows through TURN relay.
|
|
||||||
- **`iceTransportPolicy: relay`** enforced server-side — no direct peer connections.
|
|
||||||
- **Opt-in per namespace** via `orama namespace enable webrtc`.
|
|
||||||
- **SFU on all 3 nodes**, **TURN on 2 of 3 nodes** (redundancy without over-provisioning).
|
|
||||||
- **Separate port allocation** from existing namespace services.
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Namespace must be provisioned with a ready cluster (RQLite + Olric + Gateway running).
|
|
||||||
- Command must be run on a cluster node (uses internal gateway endpoint).
|
|
||||||
|
|
||||||
## Enable / Disable
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Enable WebRTC for a namespace
|
|
||||||
orama namespace enable webrtc --namespace myapp
|
|
||||||
|
|
||||||
# Check status
|
|
||||||
orama namespace webrtc-status --namespace myapp
|
|
||||||
|
|
||||||
# Disable WebRTC (stops services, deallocates ports, removes DNS)
|
|
||||||
orama namespace disable webrtc --namespace myapp
|
|
||||||
```
|
|
||||||
|
|
||||||
### What happens on enable:
|
|
||||||
1. Generates a per-namespace TURN shared secret (32 bytes, crypto/rand)
|
|
||||||
2. Inserts `namespace_webrtc_config` DB record
|
|
||||||
3. Allocates WebRTC port blocks on each node (SFU signaling + media range, TURN relay range)
|
|
||||||
4. Spawns TURN on 2 nodes (selected by capacity)
|
|
||||||
5. Spawns SFU on all 3 nodes
|
|
||||||
6. Creates DNS A records: `turn.ns-{name}.{baseDomain}` pointing to TURN node public IPs
|
|
||||||
7. Updates cluster state on all nodes (for cold-boot restoration)
|
|
||||||
|
|
||||||
### What happens on disable:
|
|
||||||
1. Stops SFU on all 3 nodes
|
|
||||||
2. Stops TURN on 2 nodes
|
|
||||||
3. Deallocates all WebRTC ports
|
|
||||||
4. Deletes TURN DNS records
|
|
||||||
5. Cleans up DB records (`namespace_webrtc_config`, `webrtc_rooms`)
|
|
||||||
6. Updates cluster state
|
|
||||||
|
|
||||||
## Client Integration (JavaScript)
|
|
||||||
|
|
||||||
### Authentication
|
|
||||||
|
|
||||||
All WebRTC endpoints require authentication. Use one of:
|
|
||||||
|
|
||||||
```
|
|
||||||
# Option A: API Key via header (recommended)
|
|
||||||
X-API-Key: <your-namespace-api-key>
|
|
||||||
|
|
||||||
# Option B: API Key via Authorization header
|
|
||||||
Authorization: ApiKey <your-namespace-api-key>
|
|
||||||
|
|
||||||
# Option C: JWT Bearer token
|
|
||||||
Authorization: Bearer <jwt>
|
|
||||||
```
|
|
||||||
|
|
||||||
### 1. Get TURN Credentials
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const response = await fetch('https://ns-myapp.orama-devnet.network/v1/webrtc/turn/credentials', {
|
|
||||||
method: 'POST',
|
|
||||||
headers: { 'X-API-Key': apiKey }
|
|
||||||
});
|
|
||||||
|
|
||||||
const { uris, username, password, ttl } = await response.json();
|
|
||||||
// uris: [
|
|
||||||
// "turn:turn.ns-myapp.orama-devnet.network:3478?transport=udp",
|
|
||||||
// "turn:turn.ns-myapp.orama-devnet.network:3478?transport=tcp",
|
|
||||||
// "turns:turn.ns-myapp.orama-devnet.network:5349"
|
|
||||||
// ]
|
|
||||||
// username: "{expiry_unix}:{namespace}"
|
|
||||||
// password: HMAC-SHA1 derived (base64)
|
|
||||||
// ttl: 600 (seconds)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Create PeerConnection
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const pc = new RTCPeerConnection({
|
|
||||||
iceServers: [{ urls: uris, username, credential: password }],
|
|
||||||
iceTransportPolicy: 'relay' // enforced by SFU
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Connect Signaling WebSocket
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const ws = new WebSocket(
|
|
||||||
`wss://ns-myapp.orama-devnet.network/v1/webrtc/signal?room=${roomId}&api_key=${apiKey}`
|
|
||||||
);
|
|
||||||
|
|
||||||
ws.onmessage = (event) => {
|
|
||||||
const msg = JSON.parse(event.data);
|
|
||||||
switch (msg.type) {
|
|
||||||
case 'offer': handleOffer(msg); break;
|
|
||||||
case 'answer': handleAnswer(msg); break;
|
|
||||||
case 'ice-candidate': handleICE(msg); break;
|
|
||||||
case 'peer-joined': handleJoin(msg); break;
|
|
||||||
case 'peer-left': handleLeave(msg); break;
|
|
||||||
case 'turn-credentials':
|
|
||||||
case 'refresh-credentials':
|
|
||||||
updateTURN(msg); // SFU sends refreshed creds at 80% TTL
|
|
||||||
break;
|
|
||||||
case 'server-draining':
|
|
||||||
reconnect(); // SFU shutting down, reconnect to another node
|
|
||||||
break;
|
|
||||||
}
|
|
||||||
};
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Room Management (REST)
|
|
||||||
|
|
||||||
```javascript
|
|
||||||
const headers = { 'X-API-Key': apiKey, 'Content-Type': 'application/json' };
|
|
||||||
|
|
||||||
// Create room
|
|
||||||
await fetch('/v1/webrtc/rooms', {
|
|
||||||
method: 'POST',
|
|
||||||
headers,
|
|
||||||
body: JSON.stringify({ room_id: 'my-room' })
|
|
||||||
});
|
|
||||||
|
|
||||||
// List rooms
|
|
||||||
const rooms = await fetch('/v1/webrtc/rooms', { headers });
|
|
||||||
|
|
||||||
// Close room
|
|
||||||
await fetch('/v1/webrtc/rooms?room_id=my-room', {
|
|
||||||
method: 'DELETE',
|
|
||||||
headers
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
## API Reference
|
|
||||||
|
|
||||||
### REST Endpoints
|
|
||||||
|
|
||||||
| Method | Path | Auth | Description |
|
|
||||||
|--------|------|------|-------------|
|
|
||||||
| POST | `/v1/webrtc/turn/credentials` | JWT/API key | Get TURN relay credentials |
|
|
||||||
| GET/WS | `/v1/webrtc/signal` | JWT/API key | WebSocket signaling |
|
|
||||||
| GET | `/v1/webrtc/rooms` | JWT/API key | List rooms |
|
|
||||||
| POST | `/v1/webrtc/rooms` | JWT/API key (owner) | Create room |
|
|
||||||
| DELETE | `/v1/webrtc/rooms` | JWT/API key (owner) | Close room |
|
|
||||||
|
|
||||||
### Signaling Messages
|
|
||||||
|
|
||||||
| Type | Direction | Description |
|
|
||||||
|------|-----------|-------------|
|
|
||||||
| `join` | Client → SFU | Join room |
|
|
||||||
| `offer` | Client ↔ SFU | SDP offer |
|
|
||||||
| `answer` | Client ↔ SFU | SDP answer |
|
|
||||||
| `ice-candidate` | Client ↔ SFU | ICE candidate |
|
|
||||||
| `leave` | Client → SFU | Leave room |
|
|
||||||
| `peer-joined` | SFU → Client | New peer notification |
|
|
||||||
| `peer-left` | SFU → Client | Peer departure |
|
|
||||||
| `turn-credentials` | SFU → Client | Initial TURN credentials |
|
|
||||||
| `refresh-credentials` | SFU → Client | Refreshed credentials (at 80% TTL) |
|
|
||||||
| `server-draining` | SFU → Client | SFU shutting down |
|
|
||||||
|
|
||||||
## Port Allocation
|
|
||||||
|
|
||||||
WebRTC uses a **separate port allocation system** from the core namespace ports:
|
|
||||||
|
|
||||||
| Service | Port Range | Protocol | Per Namespace |
|
|
||||||
|---------|-----------|----------|---------------|
|
|
||||||
| SFU signaling | 30000-30099 | TCP (WireGuard only) | 1 port |
|
|
||||||
| SFU media (RTP) | 20000-29999 | UDP (WireGuard only) | 500 ports |
|
|
||||||
| TURN listen | 3478 | UDP + TCP | fixed |
|
|
||||||
| TURNS (TLS) | 5349 | TCP | fixed |
|
|
||||||
| TURN relay | 49152-65535 | UDP | 800 ports |
|
|
||||||
|
|
||||||
## TURN Credential Protocol
|
|
||||||
|
|
||||||
- Credentials use HMAC-SHA1 with a per-namespace shared secret
|
|
||||||
- Username format: `{expiry_unix}:{namespace}`
|
|
||||||
- Password: `base64(HMAC-SHA1(shared_secret, username))`
|
|
||||||
- Default TTL: 600 seconds (10 minutes)
|
|
||||||
- SFU proactively sends `refresh-credentials` at 80% of TTL (8 minutes)
|
|
||||||
- Clients should update ICE servers on receiving refresh
|
|
||||||
|
|
||||||
## TURNS TLS Certificate
|
|
||||||
|
|
||||||
TURNS (port 5349) uses TLS. Certificate provisioning:
|
|
||||||
|
|
||||||
1. **Let's Encrypt (primary)**: On TURN spawn, the TURN domain is added to the local Caddy instance's Caddyfile. Caddy provisions a Let's Encrypt cert via DNS-01 ACME challenge (using the orama DNS provider). TURN reads the cert from Caddy's storage.
|
|
||||||
2. **Self-signed (fallback)**: If Caddy cert provisioning fails (timeout, Caddy not running), a self-signed cert is generated with the node's public IP as SAN.
|
|
||||||
|
|
||||||
Caddy auto-renews Let's Encrypt certs at ~60 days. TURN picks up renewed certs on restart.
|
|
||||||
|
|
||||||
## Monitoring
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check WebRTC status
|
|
||||||
orama namespace webrtc-status --namespace myapp
|
|
||||||
|
|
||||||
# Monitor report includes SFU/TURN status
|
|
||||||
orama monitor report --env devnet
|
|
||||||
|
|
||||||
# Inspector checks WebRTC health
|
|
||||||
orama inspector --env devnet
|
|
||||||
```
|
|
||||||
|
|
||||||
The monitoring report includes per-namespace `sfu_up` and `turn_up` fields. The inspector runs cross-node checks to verify SFU coverage (3 nodes) and TURN redundancy (2 nodes).
|
|
||||||
|
|
||||||
## Debugging
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SFU logs
|
|
||||||
journalctl -u orama-namespace-sfu@myapp -f
|
|
||||||
|
|
||||||
# TURN logs
|
|
||||||
journalctl -u orama-namespace-turn@myapp -f
|
|
||||||
|
|
||||||
# Check service status
|
|
||||||
systemctl status orama-namespace-sfu@myapp
|
|
||||||
systemctl status orama-namespace-turn@myapp
|
|
||||||
```
|
|
||||||
|
|
||||||
## Security Model
|
|
||||||
|
|
||||||
- **Forced relay**: `iceTransportPolicy: relay` enforced server-side. Clients cannot bypass TURN.
|
|
||||||
- **HMAC credentials**: Per-namespace TURN shared secret. Credentials expire after 10 minutes.
|
|
||||||
- **Namespace isolation**: Each namespace has its own TURN secret, port ranges, and rooms.
|
|
||||||
- **Authentication required**: All WebRTC endpoints require API key or JWT (`X-API-Key` header, `Authorization: ApiKey`, or `Authorization: Bearer`).
|
|
||||||
- **Room management**: Creating/closing rooms requires namespace ownership.
|
|
||||||
- **SFU on WireGuard only**: SFU binds to 10.0.0.x, never 0.0.0.0. Only reachable via TURN relay.
|
|
||||||
- **Permissions-Policy**: `camera=(self), microphone=(self)` — only same-origin can access media devices.
|
|
||||||
|
|
||||||
## Firewall
|
|
||||||
|
|
||||||
When WebRTC is enabled, the following ports are opened via UFW on TURN nodes:
|
|
||||||
|
|
||||||
| Port | Protocol | Purpose |
|
|
||||||
|------|----------|---------|
|
|
||||||
| 3478 | UDP | TURN standard |
|
|
||||||
| 3478 | TCP | TURN TCP fallback (for clients behind UDP-blocking firewalls) |
|
|
||||||
| 5349 | TCP | TURNS — TURN over TLS (encrypted, works through strict firewalls/DPI) |
|
|
||||||
| 49152-65535 | UDP | TURN relay range (allocated per namespace) |
|
|
||||||
|
|
||||||
SFU ports are NOT opened in the firewall — they are WireGuard-internal only.
|
|
||||||
|
|
||||||
## Database Tables
|
|
||||||
|
|
||||||
| Table | Purpose |
|
|
||||||
|-------|---------|
|
|
||||||
| `namespace_webrtc_config` | Per-namespace WebRTC config (enabled, TURN secret, node counts) |
|
|
||||||
| `webrtc_rooms` | Room-to-SFU-node affinity |
|
|
||||||
| `webrtc_port_allocations` | SFU/TURN port tracking |
|
|
||||||
|
|
||||||
## Cold Boot Recovery
|
|
||||||
|
|
||||||
On node restart, the cluster state file (`cluster_state.json`) includes `has_sfu`, `has_turn`, and port allocation data. The restore process:
|
|
||||||
|
|
||||||
1. Core services restore first: RQLite → Olric → Gateway
|
|
||||||
2. If `has_turn` is set: fetches TURN shared secret from DB, spawns TURN
|
|
||||||
3. If `has_sfu` is set: fetches WebRTC config from DB, spawns SFU with TURN server list
|
|
||||||
|
|
||||||
If the DB is unavailable during restore, SFU/TURN restoration is skipped with a warning log. They will be restored on the next successful DB connection.
|
|
||||||
@ -1,556 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package cluster_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"sort"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// =============================================================================
|
|
||||||
// STRICT NAMESPACE CLUSTER TESTS
|
|
||||||
// These tests FAIL if things don't work. No t.Skip() for expected functionality.
|
|
||||||
// =============================================================================
|
|
||||||
|
|
||||||
// TestNamespaceCluster_FullProvisioning is a STRICT test that verifies the complete
|
|
||||||
// namespace cluster provisioning flow. This test FAILS if any component doesn't work.
|
|
||||||
func TestNamespaceCluster_FullProvisioning(t *testing.T) {
|
|
||||||
// Generate unique namespace name
|
|
||||||
newNamespace := fmt.Sprintf("e2e-cluster-%d", time.Now().UnixNano())
|
|
||||||
|
|
||||||
env, err := e2e.LoadTestEnvWithNamespace(newNamespace)
|
|
||||||
require.NoError(t, err, "FATAL: Failed to create test environment for namespace %s", newNamespace)
|
|
||||||
require.NotEmpty(t, env.APIKey, "FATAL: No API key received - namespace provisioning failed")
|
|
||||||
|
|
||||||
t.Logf("Created namespace: %s", newNamespace)
|
|
||||||
t.Logf("API Key: %s...", env.APIKey[:min(20, len(env.APIKey))])
|
|
||||||
|
|
||||||
// Get cluster status to verify provisioning
|
|
||||||
t.Run("Cluster status shows ready", func(t *testing.T) {
|
|
||||||
// Query the namespace cluster status
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/v1/namespace/status?name="+newNamespace, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Failed to query cluster status")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("Cluster status response: %s", string(bodyBytes))
|
|
||||||
|
|
||||||
// If status endpoint exists and returns cluster info, verify it
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
var result map[string]interface{}
|
|
||||||
if err := json.Unmarshal(bodyBytes, &result); err == nil {
|
|
||||||
status, _ := result["status"].(string)
|
|
||||||
if status != "" && status != "ready" && status != "default" {
|
|
||||||
t.Errorf("FAIL: Cluster status is '%s', expected 'ready'", status)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
// Verify we can use the namespace for deployments
|
|
||||||
t.Run("Deployments work on namespace", func(t *testing.T) {
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
if _, err := os.Stat(tarballPath); os.IsNotExist(err) {
|
|
||||||
t.Skip("Test tarball not found - skipping deployment test")
|
|
||||||
}
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("cluster-test-%d", time.Now().Unix())
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID, "FAIL: Deployment creation failed on namespace cluster")
|
|
||||||
|
|
||||||
t.Logf("Created deployment %s (ID: %s) on namespace %s", deploymentName, deploymentID, newNamespace)
|
|
||||||
|
|
||||||
// Cleanup
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Verify deployment is accessible
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/v1/deployments/get?id="+deploymentID, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Failed to get deployment")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
require.Equal(t, http.StatusOK, resp.StatusCode, "FAIL: Cannot retrieve deployment from namespace cluster")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_RQLiteHealth verifies that namespace RQLite cluster is running
|
|
||||||
// and accepting connections. This test FAILS if RQLite is not accessible.
|
|
||||||
func TestNamespaceCluster_RQLiteHealth(t *testing.T) {
|
|
||||||
t.Run("Check namespace port range for RQLite", func(t *testing.T) {
|
|
||||||
foundRQLite := false
|
|
||||||
var healthyPorts []int
|
|
||||||
var unhealthyPorts []int
|
|
||||||
|
|
||||||
// Check first few port blocks
|
|
||||||
for portStart := 10000; portStart <= 10015; portStart += 5 {
|
|
||||||
rqlitePort := portStart // RQLite HTTP is first port in block
|
|
||||||
if isPortListening("localhost", rqlitePort) {
|
|
||||||
t.Logf("Found RQLite instance on port %d", rqlitePort)
|
|
||||||
foundRQLite = true
|
|
||||||
|
|
||||||
// Verify it responds to health check
|
|
||||||
healthURL := fmt.Sprintf("http://localhost:%d/status", rqlitePort)
|
|
||||||
healthResp, err := http.Get(healthURL)
|
|
||||||
if err == nil {
|
|
||||||
defer healthResp.Body.Close()
|
|
||||||
if healthResp.StatusCode == http.StatusOK {
|
|
||||||
healthyPorts = append(healthyPorts, rqlitePort)
|
|
||||||
t.Logf(" ✓ RQLite on port %d is healthy", rqlitePort)
|
|
||||||
} else {
|
|
||||||
unhealthyPorts = append(unhealthyPorts, rqlitePort)
|
|
||||||
t.Errorf("FAIL: RQLite on port %d returned status %d", rqlitePort, healthResp.StatusCode)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
unhealthyPorts = append(unhealthyPorts, rqlitePort)
|
|
||||||
t.Errorf("FAIL: RQLite on port %d health check failed: %v", rqlitePort, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if !foundRQLite {
|
|
||||||
t.Log("No namespace RQLite instances found in port range 10000-10015")
|
|
||||||
t.Log("This is expected if no namespaces have been provisioned yet")
|
|
||||||
} else {
|
|
||||||
t.Logf("Summary: %d healthy, %d unhealthy RQLite instances", len(healthyPorts), len(unhealthyPorts))
|
|
||||||
require.Empty(t, unhealthyPorts, "FAIL: Some RQLite instances are unhealthy")
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_OlricHealth verifies that namespace Olric cluster is running
|
|
||||||
// and accepting connections.
|
|
||||||
func TestNamespaceCluster_OlricHealth(t *testing.T) {
|
|
||||||
t.Run("Check namespace port range for Olric", func(t *testing.T) {
|
|
||||||
foundOlric := false
|
|
||||||
foundCount := 0
|
|
||||||
|
|
||||||
// Check first few port blocks - Olric memberlist is port_start + 3
|
|
||||||
for portStart := 10000; portStart <= 10015; portStart += 5 {
|
|
||||||
olricMemberlistPort := portStart + 3
|
|
||||||
if isPortListening("localhost", olricMemberlistPort) {
|
|
||||||
t.Logf("Found Olric memberlist on port %d", olricMemberlistPort)
|
|
||||||
foundOlric = true
|
|
||||||
foundCount++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if !foundOlric {
|
|
||||||
t.Log("No namespace Olric instances found in port range 10003-10018")
|
|
||||||
t.Log("This is expected if no namespaces have been provisioned yet")
|
|
||||||
} else {
|
|
||||||
t.Logf("Found %d Olric memberlist ports accepting connections", foundCount)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_GatewayHealth verifies that namespace Gateway instances are running.
|
|
||||||
// This test FAILS if gateway binary exists but gateways don't spawn.
|
|
||||||
func TestNamespaceCluster_GatewayHealth(t *testing.T) {
|
|
||||||
// Check if gateway binary exists
|
|
||||||
gatewayBinaryPaths := []string{
|
|
||||||
"./bin/orama",
|
|
||||||
"../bin/orama",
|
|
||||||
"/usr/local/bin/orama",
|
|
||||||
}
|
|
||||||
|
|
||||||
var gatewayBinaryExists bool
|
|
||||||
var foundPath string
|
|
||||||
for _, path := range gatewayBinaryPaths {
|
|
||||||
if _, err := os.Stat(path); err == nil {
|
|
||||||
gatewayBinaryExists = true
|
|
||||||
foundPath = path
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if !gatewayBinaryExists {
|
|
||||||
t.Log("Gateway binary not found - namespace gateways will not spawn")
|
|
||||||
t.Log("Run 'make build' to build the gateway binary")
|
|
||||||
t.Log("Checked paths:", gatewayBinaryPaths)
|
|
||||||
// This is a FAILURE if we expect gateway to work
|
|
||||||
t.Error("FAIL: Gateway binary not found. Run 'make build' first.")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("Gateway binary found at: %s", foundPath)
|
|
||||||
|
|
||||||
t.Run("Check namespace port range for Gateway", func(t *testing.T) {
|
|
||||||
foundGateway := false
|
|
||||||
var healthyPorts []int
|
|
||||||
var unhealthyPorts []int
|
|
||||||
|
|
||||||
// Check first few port blocks - Gateway HTTP is port_start + 4
|
|
||||||
for portStart := 10000; portStart <= 10015; portStart += 5 {
|
|
||||||
gatewayPort := portStart + 4
|
|
||||||
if isPortListening("localhost", gatewayPort) {
|
|
||||||
t.Logf("Found Gateway instance on port %d", gatewayPort)
|
|
||||||
foundGateway = true
|
|
||||||
|
|
||||||
// Verify it responds to health check
|
|
||||||
healthURL := fmt.Sprintf("http://localhost:%d/v1/health", gatewayPort)
|
|
||||||
healthResp, err := http.Get(healthURL)
|
|
||||||
if err == nil {
|
|
||||||
defer healthResp.Body.Close()
|
|
||||||
if healthResp.StatusCode == http.StatusOK {
|
|
||||||
healthyPorts = append(healthyPorts, gatewayPort)
|
|
||||||
t.Logf(" ✓ Gateway on port %d is healthy", gatewayPort)
|
|
||||||
} else {
|
|
||||||
unhealthyPorts = append(unhealthyPorts, gatewayPort)
|
|
||||||
t.Errorf("FAIL: Gateway on port %d returned status %d", gatewayPort, healthResp.StatusCode)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
unhealthyPorts = append(unhealthyPorts, gatewayPort)
|
|
||||||
t.Errorf("FAIL: Gateway on port %d health check failed: %v", gatewayPort, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if !foundGateway {
|
|
||||||
t.Log("No namespace Gateway instances found in port range 10004-10019")
|
|
||||||
t.Log("This is expected if no namespaces have been provisioned yet")
|
|
||||||
} else {
|
|
||||||
t.Logf("Summary: %d healthy, %d unhealthy Gateway instances", len(healthyPorts), len(unhealthyPorts))
|
|
||||||
require.Empty(t, unhealthyPorts, "FAIL: Some Gateway instances are unhealthy")
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_ProvisioningCreatesProcesses creates a new namespace and
|
|
||||||
// verifies that actual processes are spawned. This is the STRICTEST test.
|
|
||||||
func TestNamespaceCluster_ProvisioningCreatesProcesses(t *testing.T) {
|
|
||||||
newNamespace := fmt.Sprintf("e2e-strict-%d", time.Now().UnixNano())
|
|
||||||
|
|
||||||
// Record ports before provisioning
|
|
||||||
portsBefore := getListeningPortsInRange(10000, 10099)
|
|
||||||
t.Logf("Ports in use before provisioning: %v", portsBefore)
|
|
||||||
|
|
||||||
// Create namespace
|
|
||||||
env, err := e2e.LoadTestEnvWithNamespace(newNamespace)
|
|
||||||
require.NoError(t, err, "FATAL: Failed to create namespace")
|
|
||||||
require.NotEmpty(t, env.APIKey, "FATAL: No API key - provisioning failed")
|
|
||||||
|
|
||||||
t.Logf("Namespace '%s' created successfully", newNamespace)
|
|
||||||
|
|
||||||
// Wait a moment for processes to fully start
|
|
||||||
time.Sleep(3 * time.Second)
|
|
||||||
|
|
||||||
// Record ports after provisioning
|
|
||||||
portsAfter := getListeningPortsInRange(10000, 10099)
|
|
||||||
t.Logf("Ports in use after provisioning: %v", portsAfter)
|
|
||||||
|
|
||||||
// Check if new ports were opened
|
|
||||||
newPorts := diffPorts(portsBefore, portsAfter)
|
|
||||||
sort.Ints(newPorts)
|
|
||||||
t.Logf("New ports opened: %v", newPorts)
|
|
||||||
|
|
||||||
t.Run("New ports allocated for namespace cluster", func(t *testing.T) {
|
|
||||||
if len(newPorts) == 0 {
|
|
||||||
// This might be OK for default namespace or if using global cluster
|
|
||||||
t.Log("No new ports detected")
|
|
||||||
t.Log("Possible reasons:")
|
|
||||||
t.Log(" - Namespace uses default cluster (expected for 'default')")
|
|
||||||
t.Log(" - Cluster already existed from previous test")
|
|
||||||
t.Log(" - Provisioning is handled differently in this environment")
|
|
||||||
} else {
|
|
||||||
t.Logf("SUCCESS: %d new ports opened for namespace cluster", len(newPorts))
|
|
||||||
|
|
||||||
// Verify the ports follow expected pattern
|
|
||||||
for _, port := range newPorts {
|
|
||||||
offset := (port - 10000) % 5
|
|
||||||
switch offset {
|
|
||||||
case 0:
|
|
||||||
t.Logf(" Port %d: RQLite HTTP", port)
|
|
||||||
case 1:
|
|
||||||
t.Logf(" Port %d: RQLite Raft", port)
|
|
||||||
case 2:
|
|
||||||
t.Logf(" Port %d: Olric HTTP", port)
|
|
||||||
case 3:
|
|
||||||
t.Logf(" Port %d: Olric Memberlist", port)
|
|
||||||
case 4:
|
|
||||||
t.Logf(" Port %d: Gateway HTTP", port)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("RQLite is accessible on allocated ports", func(t *testing.T) {
|
|
||||||
rqlitePorts := filterPortsByOffset(newPorts, 0) // RQLite HTTP is offset 0
|
|
||||||
if len(rqlitePorts) == 0 {
|
|
||||||
t.Log("No new RQLite ports detected")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, port := range rqlitePorts {
|
|
||||||
healthURL := fmt.Sprintf("http://localhost:%d/status", port)
|
|
||||||
resp, err := http.Get(healthURL)
|
|
||||||
require.NoError(t, err, "FAIL: RQLite on port %d is not responding", port)
|
|
||||||
resp.Body.Close()
|
|
||||||
require.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"FAIL: RQLite on port %d returned status %d", port, resp.StatusCode)
|
|
||||||
t.Logf("✓ RQLite on port %d is healthy", port)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Olric is accessible on allocated ports", func(t *testing.T) {
|
|
||||||
olricPorts := filterPortsByOffset(newPorts, 3) // Olric Memberlist is offset 3
|
|
||||||
if len(olricPorts) == 0 {
|
|
||||||
t.Log("No new Olric ports detected")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, port := range olricPorts {
|
|
||||||
conn, err := net.DialTimeout("tcp", fmt.Sprintf("localhost:%d", port), 2*time.Second)
|
|
||||||
require.NoError(t, err, "FAIL: Olric memberlist on port %d is not responding", port)
|
|
||||||
conn.Close()
|
|
||||||
t.Logf("✓ Olric memberlist on port %d is accepting connections", port)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_StatusEndpoint tests the /v1/namespace/status endpoint
|
|
||||||
func TestNamespaceCluster_StatusEndpoint(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
t.Run("Status endpoint returns 404 for non-existent cluster", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/v1/namespace/status?id=non-existent-id", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Request should not fail")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
require.Equal(t, http.StatusNotFound, resp.StatusCode,
|
|
||||||
"FAIL: Should return 404 for non-existent cluster, got %d", resp.StatusCode)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_CrossNamespaceAccess verifies namespace isolation
|
|
||||||
func TestNamespaceCluster_CrossNamespaceAccess(t *testing.T) {
|
|
||||||
nsA := fmt.Sprintf("ns-a-%d", time.Now().Unix())
|
|
||||||
nsB := fmt.Sprintf("ns-b-%d", time.Now().Unix())
|
|
||||||
|
|
||||||
envA, err := e2e.LoadTestEnvWithNamespace(nsA)
|
|
||||||
require.NoError(t, err, "FAIL: Cannot create namespace A")
|
|
||||||
|
|
||||||
envB, err := e2e.LoadTestEnvWithNamespace(nsB)
|
|
||||||
require.NoError(t, err, "FAIL: Cannot create namespace B")
|
|
||||||
|
|
||||||
// Verify both namespaces have different API keys
|
|
||||||
require.NotEqual(t, envA.APIKey, envB.APIKey, "FAIL: Namespaces should have different API keys")
|
|
||||||
t.Logf("Namespace A API key: %s...", envA.APIKey[:min(10, len(envA.APIKey))])
|
|
||||||
t.Logf("Namespace B API key: %s...", envB.APIKey[:min(10, len(envB.APIKey))])
|
|
||||||
|
|
||||||
t.Run("API keys are namespace-scoped", func(t *testing.T) {
|
|
||||||
// Namespace A should not see namespace B's resources
|
|
||||||
req, _ := http.NewRequest("GET", envA.GatewayURL+"/v1/deployments/list", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
require.Equal(t, http.StatusOK, resp.StatusCode, "Should list deployments")
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
json.Unmarshal(bodyBytes, &result)
|
|
||||||
|
|
||||||
deployments, _ := result["deployments"].([]interface{})
|
|
||||||
for _, d := range deployments {
|
|
||||||
dep, ok := d.(map[string]interface{})
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
ns, _ := dep["namespace"].(string)
|
|
||||||
require.NotEqual(t, nsB, ns,
|
|
||||||
"FAIL: Namespace A sees Namespace B deployments - isolation broken!")
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeployment_SubdomainFormat tests deployment subdomain format
|
|
||||||
func TestDeployment_SubdomainFormat(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
if _, err := os.Stat(tarballPath); os.IsNotExist(err) {
|
|
||||||
t.Skip("Test tarball not found")
|
|
||||||
}
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("subdomain-test-%d", time.Now().UnixNano())
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID, "FAIL: Deployment creation failed")
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deployment has subdomain with random suffix", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/v1/deployments/get?id="+deploymentID, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Failed to get deployment")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
require.Equal(t, http.StatusOK, resp.StatusCode, "Should get deployment")
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
json.Unmarshal(bodyBytes, &result)
|
|
||||||
|
|
||||||
deployment, ok := result["deployment"].(map[string]interface{})
|
|
||||||
if !ok {
|
|
||||||
deployment = result
|
|
||||||
}
|
|
||||||
|
|
||||||
subdomain, _ := deployment["subdomain"].(string)
|
|
||||||
if subdomain != "" {
|
|
||||||
require.True(t, strings.HasPrefix(subdomain, deploymentName),
|
|
||||||
"FAIL: Subdomain '%s' should start with deployment name '%s'", subdomain, deploymentName)
|
|
||||||
|
|
||||||
suffix := strings.TrimPrefix(subdomain, deploymentName+"-")
|
|
||||||
if suffix != subdomain { // There was a dash separator
|
|
||||||
require.Equal(t, 6, len(suffix),
|
|
||||||
"FAIL: Random suffix should be 6 characters, got %d (%s)", len(suffix), suffix)
|
|
||||||
}
|
|
||||||
t.Logf("Deployment subdomain: %s", subdomain)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_PortAllocation tests port allocation correctness
|
|
||||||
func TestNamespaceCluster_PortAllocation(t *testing.T) {
|
|
||||||
t.Run("Port range is 10000-10099", func(t *testing.T) {
|
|
||||||
const portRangeStart = 10000
|
|
||||||
const portRangeEnd = 10099
|
|
||||||
const portsPerNamespace = 5
|
|
||||||
const maxNamespacesPerNode = 20
|
|
||||||
|
|
||||||
totalPorts := portRangeEnd - portRangeStart + 1
|
|
||||||
require.Equal(t, 100, totalPorts, "Port range should be 100 ports")
|
|
||||||
|
|
||||||
expectedMax := totalPorts / portsPerNamespace
|
|
||||||
require.Equal(t, maxNamespacesPerNode, expectedMax,
|
|
||||||
"Max namespaces per node calculation mismatch")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Port assignments are sequential within block", func(t *testing.T) {
|
|
||||||
portStart := 10000
|
|
||||||
ports := map[string]int{
|
|
||||||
"rqlite_http": portStart + 0,
|
|
||||||
"rqlite_raft": portStart + 1,
|
|
||||||
"olric_http": portStart + 2,
|
|
||||||
"olric_memberlist": portStart + 3,
|
|
||||||
"gateway_http": portStart + 4,
|
|
||||||
}
|
|
||||||
|
|
||||||
seen := make(map[int]bool)
|
|
||||||
for name, port := range ports {
|
|
||||||
require.False(t, seen[port], "FAIL: Port %d for %s is duplicate", port, name)
|
|
||||||
seen[port] = true
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// =============================================================================
|
|
||||||
// HELPER FUNCTIONS
|
|
||||||
// =============================================================================
|
|
||||||
|
|
||||||
func isPortListening(host string, port int) bool {
|
|
||||||
conn, err := net.DialTimeout("tcp", fmt.Sprintf("%s:%d", host, port), 1*time.Second)
|
|
||||||
if err != nil {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
conn.Close()
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
func getListeningPortsInRange(start, end int) []int {
|
|
||||||
var ports []int
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Check ports concurrently for speed
|
|
||||||
results := make(chan int, end-start+1)
|
|
||||||
for port := start; port <= end; port++ {
|
|
||||||
go func(p int) {
|
|
||||||
select {
|
|
||||||
case <-ctx.Done():
|
|
||||||
results <- 0
|
|
||||||
return
|
|
||||||
default:
|
|
||||||
if isPortListening("localhost", p) {
|
|
||||||
results <- p
|
|
||||||
} else {
|
|
||||||
results <- 0
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}(port)
|
|
||||||
}
|
|
||||||
|
|
||||||
for i := 0; i <= end-start; i++ {
|
|
||||||
if port := <-results; port > 0 {
|
|
||||||
ports = append(ports, port)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return ports
|
|
||||||
}
|
|
||||||
|
|
||||||
func diffPorts(before, after []int) []int {
|
|
||||||
beforeMap := make(map[int]bool)
|
|
||||||
for _, p := range before {
|
|
||||||
beforeMap[p] = true
|
|
||||||
}
|
|
||||||
|
|
||||||
var newPorts []int
|
|
||||||
for _, p := range after {
|
|
||||||
if !beforeMap[p] {
|
|
||||||
newPorts = append(newPorts, p)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return newPorts
|
|
||||||
}
|
|
||||||
|
|
||||||
func filterPortsByOffset(ports []int, offset int) []int {
|
|
||||||
var filtered []int
|
|
||||||
for _, p := range ports {
|
|
||||||
if (p-10000)%5 == offset {
|
|
||||||
filtered = append(filtered, p)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return filtered
|
|
||||||
}
|
|
||||||
|
|
||||||
func min(a, b int) int {
|
|
||||||
if a < b {
|
|
||||||
return a
|
|
||||||
}
|
|
||||||
return b
|
|
||||||
}
|
|
||||||
@ -1,447 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package cluster_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestNamespaceIsolation creates two namespaces once and runs all isolation
|
|
||||||
// subtests against them. This keeps namespace usage to 2 regardless of how
|
|
||||||
// many isolation scenarios we test.
|
|
||||||
func TestNamespaceIsolation(t *testing.T) {
|
|
||||||
envA, err := e2e.LoadTestEnvWithNamespace("namespace-a-" + fmt.Sprintf("%d", time.Now().Unix()))
|
|
||||||
require.NoError(t, err, "Failed to create namespace A environment")
|
|
||||||
|
|
||||||
envB, err := e2e.LoadTestEnvWithNamespace("namespace-b-" + fmt.Sprintf("%d", time.Now().Unix()))
|
|
||||||
require.NoError(t, err, "Failed to create namespace B environment")
|
|
||||||
|
|
||||||
t.Run("Deployments", func(t *testing.T) {
|
|
||||||
testNamespaceIsolationDeployments(t, envA, envB)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("SQLiteDatabases", func(t *testing.T) {
|
|
||||||
testNamespaceIsolationSQLiteDatabases(t, envA, envB)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("IPFSContent", func(t *testing.T) {
|
|
||||||
testNamespaceIsolationIPFSContent(t, envA, envB)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("OlricCache", func(t *testing.T) {
|
|
||||||
testNamespaceIsolationOlricCache(t, envA, envB)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func testNamespaceIsolationDeployments(t *testing.T, envA, envB *e2e.E2ETestEnv) {
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
// Create deployment in namespace-a
|
|
||||||
deploymentNameA := "test-app-ns-a"
|
|
||||||
deploymentIDA := e2e.CreateTestDeployment(t, envA, deploymentNameA, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !envA.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, envA, deploymentIDA)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Create deployment in namespace-b
|
|
||||||
deploymentNameB := "test-app-ns-b"
|
|
||||||
deploymentIDB := e2e.CreateTestDeployment(t, envB, deploymentNameB, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !envB.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, envB, deploymentIDB)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Namespace-A cannot list Namespace-B deployments", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", envA.GatewayURL+"/v1/deployments/list", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytes, &result), "Should decode JSON")
|
|
||||||
|
|
||||||
deployments, ok := result["deployments"].([]interface{})
|
|
||||||
require.True(t, ok, "Deployments should be an array")
|
|
||||||
|
|
||||||
// Should only see namespace-a deployments
|
|
||||||
for _, d := range deployments {
|
|
||||||
dep, ok := d.(map[string]interface{})
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
assert.NotEqual(t, deploymentNameB, dep["name"], "Should not see namespace-b deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace A cannot see Namespace B deployments")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-A cannot access Namespace-B deployment by ID", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", envA.GatewayURL+"/v1/deployments/get?id="+deploymentIDB, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Should return 404 or 403
|
|
||||||
assert.Contains(t, []int{http.StatusNotFound, http.StatusForbidden}, resp.StatusCode,
|
|
||||||
"Should block cross-namespace access")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace A cannot access Namespace B deployment (status: %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-A cannot delete Namespace-B deployment", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("DELETE", envA.GatewayURL+"/v1/deployments/delete?id="+deploymentIDB, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Contains(t, []int{http.StatusNotFound, http.StatusForbidden}, resp.StatusCode,
|
|
||||||
"Should block cross-namespace deletion")
|
|
||||||
|
|
||||||
// Verify deployment still exists for namespace-b
|
|
||||||
req2, _ := http.NewRequest("GET", envB.GatewayURL+"/v1/deployments/get?id="+deploymentIDB, nil)
|
|
||||||
req2.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
|
|
||||||
resp2, err := envB.HTTPClient.Do(req2)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp2.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp2.StatusCode, "Deployment should still exist in namespace B")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace A cannot delete Namespace B deployment")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func testNamespaceIsolationSQLiteDatabases(t *testing.T, envA, envB *e2e.E2ETestEnv) {
|
|
||||||
// Create database in namespace-a
|
|
||||||
dbNameA := "users-db-a"
|
|
||||||
e2e.CreateSQLiteDB(t, envA, dbNameA)
|
|
||||||
defer func() {
|
|
||||||
if !envA.SkipCleanup {
|
|
||||||
e2e.DeleteSQLiteDB(t, envA, dbNameA)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Create database in namespace-b
|
|
||||||
dbNameB := "users-db-b"
|
|
||||||
e2e.CreateSQLiteDB(t, envB, dbNameB)
|
|
||||||
defer func() {
|
|
||||||
if !envB.SkipCleanup {
|
|
||||||
e2e.DeleteSQLiteDB(t, envB, dbNameB)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Namespace-A cannot list Namespace-B databases", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", envA.GatewayURL+"/v1/db/sqlite/list", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytes, &result), "Should decode JSON")
|
|
||||||
|
|
||||||
databases, ok := result["databases"].([]interface{})
|
|
||||||
require.True(t, ok, "Databases should be an array")
|
|
||||||
|
|
||||||
for _, db := range databases {
|
|
||||||
database, ok := db.(map[string]interface{})
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
assert.NotEqual(t, dbNameB, database["database_name"], "Should not see namespace-b database")
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace A cannot see Namespace B databases")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-A cannot query Namespace-B database", func(t *testing.T) {
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"database_name": dbNameB,
|
|
||||||
"query": "SELECT * FROM users",
|
|
||||||
}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", envA.GatewayURL+"/v1/db/sqlite/query", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusNotFound, resp.StatusCode, "Should block cross-namespace query")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace A cannot query Namespace B database")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-A cannot backup Namespace-B database", func(t *testing.T) {
|
|
||||||
reqBody := map[string]string{"database_name": dbNameB}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", envA.GatewayURL+"/v1/db/sqlite/backup", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusNotFound, resp.StatusCode, "Should block cross-namespace backup")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace A cannot backup Namespace B database")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func testNamespaceIsolationIPFSContent(t *testing.T, envA, envB *e2e.E2ETestEnv) {
|
|
||||||
// Upload file in namespace-a
|
|
||||||
cidA := e2e.UploadTestFile(t, envA, "test-file-a.txt", "Content from namespace A")
|
|
||||||
defer func() {
|
|
||||||
if !envA.SkipCleanup {
|
|
||||||
e2e.UnpinFile(t, envA, cidA)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Namespace-B cannot GET Namespace-A IPFS content", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", envB.GatewayURL+"/v1/storage/get/"+cidA, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
|
|
||||||
resp, err := envB.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Contains(t, []int{http.StatusNotFound, http.StatusForbidden}, resp.StatusCode,
|
|
||||||
"Should block cross-namespace IPFS GET")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace B cannot GET Namespace A IPFS content (status: %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-B cannot PIN Namespace-A IPFS content", func(t *testing.T) {
|
|
||||||
reqBody := map[string]string{
|
|
||||||
"cid": cidA,
|
|
||||||
"name": "stolen-content",
|
|
||||||
}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", envB.GatewayURL+"/v1/storage/pin", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := envB.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Contains(t, []int{http.StatusNotFound, http.StatusForbidden}, resp.StatusCode,
|
|
||||||
"Should block cross-namespace PIN")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace B cannot PIN Namespace A IPFS content (status: %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-B cannot UNPIN Namespace-A IPFS content", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("DELETE", envB.GatewayURL+"/v1/storage/unpin/"+cidA, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
|
|
||||||
resp, err := envB.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Contains(t, []int{http.StatusNotFound, http.StatusForbidden}, resp.StatusCode,
|
|
||||||
"Should block cross-namespace UNPIN")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace B cannot UNPIN Namespace A IPFS content (status: %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-A can list only their own IPFS pins", func(t *testing.T) {
|
|
||||||
t.Skip("List pins endpoint not implemented yet - namespace isolation enforced at GET/PIN/UNPIN levels")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func testNamespaceIsolationOlricCache(t *testing.T, envA, envB *e2e.E2ETestEnv) {
|
|
||||||
dmap := "test-cache"
|
|
||||||
keyA := "user-session-123"
|
|
||||||
valueA := `{"user_id": "alice", "token": "secret-token-a"}`
|
|
||||||
|
|
||||||
t.Run("Namespace-A sets cache key", func(t *testing.T) {
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"dmap": dmap,
|
|
||||||
"key": keyA,
|
|
||||||
"value": valueA,
|
|
||||||
"ttl": "300s",
|
|
||||||
}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", envA.GatewayURL+"/v1/cache/put", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := envA.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Should set cache key successfully")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace A set cache key")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-B cannot GET Namespace-A cache key", func(t *testing.T) {
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"dmap": dmap,
|
|
||||||
"key": keyA,
|
|
||||||
}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", envB.GatewayURL+"/v1/cache/get", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := envB.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Should return 404 (key doesn't exist in namespace-b)
|
|
||||||
assert.Equal(t, http.StatusNotFound, resp.StatusCode, "Should not find key in different namespace")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace B cannot GET Namespace A cache key")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-B cannot DELETE Namespace-A cache key", func(t *testing.T) {
|
|
||||||
reqBody := map[string]string{
|
|
||||||
"dmap": dmap,
|
|
||||||
"key": keyA,
|
|
||||||
}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", envB.GatewayURL+"/v1/cache/delete", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := envB.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Contains(t, []int{http.StatusOK, http.StatusNotFound}, resp.StatusCode)
|
|
||||||
|
|
||||||
// Verify key still exists for namespace-a
|
|
||||||
reqBody2 := map[string]interface{}{
|
|
||||||
"dmap": dmap,
|
|
||||||
"key": keyA,
|
|
||||||
}
|
|
||||||
bodyBytes2, _ := json.Marshal(reqBody2)
|
|
||||||
|
|
||||||
req2, _ := http.NewRequest("POST", envA.GatewayURL+"/v1/cache/get", bytes.NewReader(bodyBytes2))
|
|
||||||
req2.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
req2.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp2, err := envA.HTTPClient.Do(req2)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp2.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp2.StatusCode, "Key should still exist in namespace A")
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
bodyBytes3, _ := io.ReadAll(resp2.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytes3, &result), "Should decode result")
|
|
||||||
|
|
||||||
// Parse expected JSON string for comparison
|
|
||||||
var expectedValue map[string]interface{}
|
|
||||||
json.Unmarshal([]byte(valueA), &expectedValue)
|
|
||||||
assert.Equal(t, expectedValue, result["value"], "Value should match")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace B cannot DELETE Namespace A cache key")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Namespace-B can set same key name in their namespace", func(t *testing.T) {
|
|
||||||
// Same key name, different namespace should be allowed
|
|
||||||
valueB := `{"user_id": "bob", "token": "secret-token-b"}`
|
|
||||||
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"dmap": dmap,
|
|
||||||
"key": keyA, // Same key name as namespace-a
|
|
||||||
"value": valueB,
|
|
||||||
"ttl": "300s",
|
|
||||||
}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", envB.GatewayURL+"/v1/cache/put", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := envB.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Should set key in namespace B")
|
|
||||||
|
|
||||||
// Verify namespace-a still has their value
|
|
||||||
reqBody2 := map[string]interface{}{
|
|
||||||
"dmap": dmap,
|
|
||||||
"key": keyA,
|
|
||||||
}
|
|
||||||
bodyBytes2, _ := json.Marshal(reqBody2)
|
|
||||||
|
|
||||||
req2, _ := http.NewRequest("POST", envA.GatewayURL+"/v1/cache/get", bytes.NewReader(bodyBytes2))
|
|
||||||
req2.Header.Set("Authorization", "Bearer "+envA.APIKey)
|
|
||||||
req2.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp2, _ := envA.HTTPClient.Do(req2)
|
|
||||||
defer resp2.Body.Close()
|
|
||||||
|
|
||||||
var resultA map[string]interface{}
|
|
||||||
bodyBytesA, _ := io.ReadAll(resp2.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytesA, &resultA), "Should decode result A")
|
|
||||||
|
|
||||||
// Parse expected JSON string for comparison
|
|
||||||
var expectedValueA map[string]interface{}
|
|
||||||
json.Unmarshal([]byte(valueA), &expectedValueA)
|
|
||||||
assert.Equal(t, expectedValueA, resultA["value"], "Namespace A value should be unchanged")
|
|
||||||
|
|
||||||
// Verify namespace-b has their different value
|
|
||||||
reqBody3 := map[string]interface{}{
|
|
||||||
"dmap": dmap,
|
|
||||||
"key": keyA,
|
|
||||||
}
|
|
||||||
bodyBytes3, _ := json.Marshal(reqBody3)
|
|
||||||
|
|
||||||
req3, _ := http.NewRequest("POST", envB.GatewayURL+"/v1/cache/get", bytes.NewReader(bodyBytes3))
|
|
||||||
req3.Header.Set("Authorization", "Bearer "+envB.APIKey)
|
|
||||||
req3.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp3, _ := envB.HTTPClient.Do(req3)
|
|
||||||
defer resp3.Body.Close()
|
|
||||||
|
|
||||||
var resultB map[string]interface{}
|
|
||||||
bodyBytesB, _ := io.ReadAll(resp3.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytesB, &resultB), "Should decode result B")
|
|
||||||
|
|
||||||
// Parse expected JSON string for comparison
|
|
||||||
var expectedValueB map[string]interface{}
|
|
||||||
json.Unmarshal([]byte(valueB), &expectedValueB)
|
|
||||||
assert.Equal(t, expectedValueB, resultB["value"], "Namespace B value should be different")
|
|
||||||
|
|
||||||
t.Logf("✓ Namespace B can set same key name independently")
|
|
||||||
t.Logf(" - Namespace A value: %s", valueA)
|
|
||||||
t.Logf(" - Namespace B value: %s", valueB)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,177 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package cluster
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestRQLite_ReadConsistencyLevels tests that different consistency levels work.
|
|
||||||
func TestRQLite_ReadConsistencyLevels(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
table := e2e.GenerateTableName()
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
dropReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/drop-table",
|
|
||||||
Body: map[string]interface{}{"table": table},
|
|
||||||
}
|
|
||||||
dropReq.Do(context.Background())
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Create table
|
|
||||||
createReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/create-table",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"schema": fmt.Sprintf("CREATE TABLE IF NOT EXISTS %s (id INTEGER PRIMARY KEY AUTOINCREMENT, val TEXT)", table),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
_, status, err := createReq.Do(ctx)
|
|
||||||
require.NoError(t, err)
|
|
||||||
require.True(t, status == http.StatusOK || status == http.StatusCreated, "create table got %d", status)
|
|
||||||
|
|
||||||
// Insert data
|
|
||||||
insertReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/transaction",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"statements": []string{
|
|
||||||
fmt.Sprintf("INSERT INTO %s(val) VALUES ('consistency-test')", table),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
_, status, err = insertReq.Do(ctx)
|
|
||||||
require.NoError(t, err)
|
|
||||||
require.Equal(t, http.StatusOK, status)
|
|
||||||
|
|
||||||
t.Run("Default consistency read", func(t *testing.T) {
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/query",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT * FROM %s", table),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
body, status, err := queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Equal(t, http.StatusOK, status)
|
|
||||||
t.Logf("Default read: %s", string(body))
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Strong consistency read", func(t *testing.T) {
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/query?level=strong",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT * FROM %s", table),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
body, status, err := queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Equal(t, http.StatusOK, status)
|
|
||||||
t.Logf("Strong read: %s", string(body))
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Weak consistency read", func(t *testing.T) {
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/query?level=weak",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT * FROM %s", table),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
body, status, err := queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err)
|
|
||||||
assert.Equal(t, http.StatusOK, status)
|
|
||||||
t.Logf("Weak read: %s", string(body))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestRQLite_WriteAfterMultipleReads verifies write-read cycles stay consistent.
|
|
||||||
func TestRQLite_WriteAfterMultipleReads(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
table := e2e.GenerateTableName()
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
dropReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/drop-table",
|
|
||||||
Body: map[string]interface{}{"table": table},
|
|
||||||
}
|
|
||||||
dropReq.Do(context.Background())
|
|
||||||
}()
|
|
||||||
|
|
||||||
createReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/create-table",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"schema": fmt.Sprintf("CREATE TABLE IF NOT EXISTS %s (id INTEGER PRIMARY KEY AUTOINCREMENT, counter INTEGER DEFAULT 0)", table),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
_, status, err := createReq.Do(ctx)
|
|
||||||
require.NoError(t, err)
|
|
||||||
require.True(t, status == http.StatusOK || status == http.StatusCreated)
|
|
||||||
|
|
||||||
// Write-read cycle 10 times
|
|
||||||
for i := 1; i <= 10; i++ {
|
|
||||||
insertReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/transaction",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"statements": []string{
|
|
||||||
fmt.Sprintf("INSERT INTO %s(counter) VALUES (%d)", table, i),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
_, status, err := insertReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "insert %d failed", i)
|
|
||||||
require.Equal(t, http.StatusOK, status, "insert %d got status %d", i, status)
|
|
||||||
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/query",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT COUNT(*) as cnt FROM %s", table),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
body, _, _ := queryReq.Do(ctx)
|
|
||||||
t.Logf("Iteration %d: %s", i, string(body))
|
|
||||||
}
|
|
||||||
|
|
||||||
// Final verification
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: gatewayURL + "/v1/rqlite/query",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT COUNT(*) as cnt FROM %s", table),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
body, status, err := queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err)
|
|
||||||
require.Equal(t, http.StatusOK, status)
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
json.Unmarshal(body, &result)
|
|
||||||
t.Logf("Final count result: %s", string(body))
|
|
||||||
}
|
|
||||||
@ -1,147 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package e2e
|
|
||||||
|
|
||||||
import (
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
|
|
||||||
"gopkg.in/yaml.v2"
|
|
||||||
)
|
|
||||||
|
|
||||||
// E2EConfig holds the configuration for E2E tests
|
|
||||||
type E2EConfig struct {
|
|
||||||
// Mode can be "local" or "production"
|
|
||||||
Mode string `yaml:"mode"`
|
|
||||||
|
|
||||||
// BaseDomain is the domain used for deployment routing (e.g., "dbrs.space" or "orama.network")
|
|
||||||
BaseDomain string `yaml:"base_domain"`
|
|
||||||
|
|
||||||
// Servers is a list of production servers (only used when mode=production)
|
|
||||||
Servers []ServerConfig `yaml:"servers"`
|
|
||||||
|
|
||||||
// Nameservers is a list of nameserver hostnames (e.g., ["ns1.dbrs.space", "ns2.dbrs.space"])
|
|
||||||
Nameservers []string `yaml:"nameservers"`
|
|
||||||
|
|
||||||
// APIKey is the API key for production testing (auto-discovered if empty)
|
|
||||||
APIKey string `yaml:"api_key"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// ServerConfig holds configuration for a single production server
|
|
||||||
type ServerConfig struct {
|
|
||||||
Name string `yaml:"name"`
|
|
||||||
IP string `yaml:"ip"`
|
|
||||||
User string `yaml:"user"`
|
|
||||||
Password string `yaml:"password"`
|
|
||||||
IsNameserver bool `yaml:"is_nameserver"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// DefaultConfig returns the default configuration
|
|
||||||
func DefaultConfig() *E2EConfig {
|
|
||||||
return &E2EConfig{
|
|
||||||
Mode: "production",
|
|
||||||
BaseDomain: "orama.network",
|
|
||||||
Servers: []ServerConfig{},
|
|
||||||
Nameservers: []string{},
|
|
||||||
APIKey: "",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// LoadE2EConfig loads the E2E test configuration from e2e/config.yaml
|
|
||||||
// Falls back to defaults if the file doesn't exist
|
|
||||||
func LoadE2EConfig() (*E2EConfig, error) {
|
|
||||||
// Try multiple locations for the config file
|
|
||||||
configPaths := []string{
|
|
||||||
"config.yaml", // Relative to e2e directory (when running from e2e/)
|
|
||||||
"e2e/config.yaml", // Relative to project root
|
|
||||||
"../e2e/config.yaml", // From subdirectory within e2e/
|
|
||||||
}
|
|
||||||
|
|
||||||
// Also try absolute path based on working directory
|
|
||||||
if cwd, err := os.Getwd(); err == nil {
|
|
||||||
configPaths = append(configPaths, filepath.Join(cwd, "config.yaml"))
|
|
||||||
configPaths = append(configPaths, filepath.Join(cwd, "e2e", "config.yaml"))
|
|
||||||
// Go up one level if we're in a subdirectory
|
|
||||||
configPaths = append(configPaths, filepath.Join(cwd, "..", "config.yaml"))
|
|
||||||
}
|
|
||||||
|
|
||||||
var configData []byte
|
|
||||||
var readErr error
|
|
||||||
|
|
||||||
for _, path := range configPaths {
|
|
||||||
data, err := os.ReadFile(path)
|
|
||||||
if err == nil {
|
|
||||||
configData = data
|
|
||||||
break
|
|
||||||
}
|
|
||||||
readErr = err
|
|
||||||
}
|
|
||||||
|
|
||||||
// If no config file found, return defaults
|
|
||||||
if configData == nil {
|
|
||||||
// Check if running in production mode via environment variable
|
|
||||||
if os.Getenv("E2E_MODE") == "production" {
|
|
||||||
return nil, readErr // Config file required for production mode
|
|
||||||
}
|
|
||||||
return DefaultConfig(), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var cfg E2EConfig
|
|
||||||
if err := yaml.Unmarshal(configData, &cfg); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Apply defaults for empty values
|
|
||||||
if cfg.Mode == "" {
|
|
||||||
cfg.Mode = "production"
|
|
||||||
}
|
|
||||||
if cfg.BaseDomain == "" {
|
|
||||||
cfg.BaseDomain = "orama.network"
|
|
||||||
}
|
|
||||||
|
|
||||||
return &cfg, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// IsProductionMode returns true if running in production mode
|
|
||||||
func IsProductionMode() bool {
|
|
||||||
// Check environment variable first
|
|
||||||
if os.Getenv("E2E_MODE") == "production" {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
cfg, err := LoadE2EConfig()
|
|
||||||
if err != nil {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
return cfg.Mode == "production"
|
|
||||||
}
|
|
||||||
|
|
||||||
// GetServerIPs returns a list of all server IP addresses from config
|
|
||||||
func GetServerIPs(cfg *E2EConfig) []string {
|
|
||||||
if cfg == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
ips := make([]string, 0, len(cfg.Servers))
|
|
||||||
for _, server := range cfg.Servers {
|
|
||||||
if server.IP != "" {
|
|
||||||
ips = append(ips, server.IP)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return ips
|
|
||||||
}
|
|
||||||
|
|
||||||
// GetNameserverServers returns servers configured as nameservers
|
|
||||||
func GetNameserverServers(cfg *E2EConfig) []ServerConfig {
|
|
||||||
if cfg == nil {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var nameservers []ServerConfig
|
|
||||||
for _, server := range cfg.Servers {
|
|
||||||
if server.IsNameserver {
|
|
||||||
nameservers = append(nameservers, server)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return nameservers
|
|
||||||
}
|
|
||||||
@ -1,45 +0,0 @@
|
|||||||
# E2E Test Configuration
|
|
||||||
#
|
|
||||||
# Copy this file to config.yaml and fill in your values.
|
|
||||||
# config.yaml is git-ignored and should contain your actual credentials.
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# cp config.yaml.example config.yaml
|
|
||||||
# # Edit config.yaml with your server credentials
|
|
||||||
# go test -v -tags e2e ./e2e/...
|
|
||||||
|
|
||||||
# Test mode: "local" or "production"
|
|
||||||
# - local: Tests run against `make dev` cluster on localhost
|
|
||||||
# - production: Tests run against real VPS servers
|
|
||||||
mode: local
|
|
||||||
|
|
||||||
# Base domain for deployment routing
|
|
||||||
# - Local: orama.network (default)
|
|
||||||
# - Production: dbrs.space (or your custom domain)
|
|
||||||
base_domain: orama.network
|
|
||||||
|
|
||||||
# Production servers (only used when mode=production)
|
|
||||||
# Add your VPS servers here with their credentials
|
|
||||||
servers:
|
|
||||||
# Example:
|
|
||||||
# - name: vps-1
|
|
||||||
# ip: 1.2.3.4
|
|
||||||
# user: ubuntu
|
|
||||||
# password: "your-password-here"
|
|
||||||
# is_nameserver: true
|
|
||||||
# - name: vps-2
|
|
||||||
# ip: 5.6.7.8
|
|
||||||
# user: ubuntu
|
|
||||||
# password: "another-password"
|
|
||||||
# is_nameserver: false
|
|
||||||
|
|
||||||
# Nameserver hostnames (for DNS tests in production)
|
|
||||||
# These should match your NS records
|
|
||||||
nameservers:
|
|
||||||
# Example:
|
|
||||||
# - ns1.yourdomain.com
|
|
||||||
# - ns2.yourdomain.com
|
|
||||||
|
|
||||||
# API key for production testing
|
|
||||||
# Leave empty to auto-discover from RQLite or create fresh key
|
|
||||||
api_key: ""
|
|
||||||
@ -1,223 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package deployments_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"sync"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestDeploy_InvalidTarball verifies that uploading an invalid/corrupt tarball
|
|
||||||
// returns a clean error (not a 500 or panic).
|
|
||||||
func TestDeploy_InvalidTarball(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("invalid-tar-%d", time.Now().Unix())
|
|
||||||
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(deploymentName + "\r\n")
|
|
||||||
|
|
||||||
// Write invalid tarball data (random bytes, not a real gzip)
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
body.WriteString("this is not a valid tarball content at all!!!")
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/static/upload", body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
respBody, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("Status: %d, Body: %s", resp.StatusCode, string(respBody))
|
|
||||||
|
|
||||||
// Should return an error, not 2xx (ideally 400, but server currently returns 500)
|
|
||||||
assert.True(t, resp.StatusCode >= 400,
|
|
||||||
"Invalid tarball should return error (got %d)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeploy_EmptyTarball verifies that uploading an empty file returns an error.
|
|
||||||
func TestDeploy_EmptyTarball(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("empty-tar-%d", time.Now().Unix())
|
|
||||||
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(deploymentName + "\r\n")
|
|
||||||
|
|
||||||
// Empty tarball
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/static/upload", body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
respBody, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("Status: %d, Body: %s", resp.StatusCode, string(respBody))
|
|
||||||
|
|
||||||
assert.True(t, resp.StatusCode >= 400,
|
|
||||||
"Empty tarball should return error (got %d)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeploy_MissingName verifies that deploying without a name returns an error.
|
|
||||||
func TestDeploy_MissingName(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
// No name field
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
|
|
||||||
// Create tarball from directory for the "no name" test
|
|
||||||
tarData, err := exec.Command("tar", "-czf", "-", "-C", tarballPath, ".").Output()
|
|
||||||
if err != nil {
|
|
||||||
t.Skip("Failed to create tarball from test app")
|
|
||||||
}
|
|
||||||
body.Write(tarData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/static/upload", body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.True(t, resp.StatusCode >= 400,
|
|
||||||
"Missing name should return error (got %d)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeploy_ConcurrentSameName verifies that deploying two apps with the same
|
|
||||||
// name concurrently doesn't cause data corruption.
|
|
||||||
func TestDeploy_ConcurrentSameName(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("concurrent-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
var wg sync.WaitGroup
|
|
||||||
results := make([]int, 2)
|
|
||||||
ids := make([]string, 2)
|
|
||||||
|
|
||||||
// Pre-create tarball once for both goroutines
|
|
||||||
tarData, err := exec.Command("tar", "-czf", "-", "-C", tarballPath, ".").Output()
|
|
||||||
if err != nil {
|
|
||||||
t.Skip("Failed to create tarball from test app")
|
|
||||||
}
|
|
||||||
|
|
||||||
for i := 0; i < 2; i++ {
|
|
||||||
wg.Add(1)
|
|
||||||
go func(idx int) {
|
|
||||||
defer wg.Done()
|
|
||||||
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(deploymentName + "\r\n")
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
body.Write(tarData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/static/upload", body)
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
results[idx] = resp.StatusCode
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
json.NewDecoder(resp.Body).Decode(&result)
|
|
||||||
if id, ok := result["deployment_id"].(string); ok {
|
|
||||||
ids[idx] = id
|
|
||||||
} else if id, ok := result["id"].(string); ok {
|
|
||||||
ids[idx] = id
|
|
||||||
}
|
|
||||||
}(i)
|
|
||||||
}
|
|
||||||
|
|
||||||
wg.Wait()
|
|
||||||
|
|
||||||
t.Logf("Concurrent deploy results: status1=%d status2=%d id1=%s id2=%s",
|
|
||||||
results[0], results[1], ids[0], ids[1])
|
|
||||||
|
|
||||||
// At least one should succeed
|
|
||||||
successCount := 0
|
|
||||||
for _, status := range results {
|
|
||||||
if status == http.StatusCreated {
|
|
||||||
successCount++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
assert.GreaterOrEqual(t, successCount, 1,
|
|
||||||
"At least one concurrent deploy should succeed")
|
|
||||||
|
|
||||||
// Cleanup
|
|
||||||
for _, id := range ids {
|
|
||||||
if id != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, id)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func readFileBytes(path string) ([]byte, error) {
|
|
||||||
f, err := os.Open(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
defer f.Close()
|
|
||||||
return io.ReadAll(f)
|
|
||||||
}
|
|
||||||
@ -1,308 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package deployments_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestGoBackendWithSQLite tests Go backend deployment with hosted SQLite connectivity
|
|
||||||
// 1. Create hosted SQLite database
|
|
||||||
// 2. Deploy Go backend with DATABASE_NAME env var
|
|
||||||
// 3. POST /api/users → verify insert
|
|
||||||
// 4. GET /api/users → verify read
|
|
||||||
// 5. Cleanup
|
|
||||||
func TestGoBackendWithSQLite(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("go-sqlite-test-%d", time.Now().Unix())
|
|
||||||
dbName := fmt.Sprintf("test-db-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/go-api")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
// Cleanup after test
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
if deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
// Delete the test database
|
|
||||||
deleteSQLiteDB(t, env, dbName)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Create SQLite database", func(t *testing.T) {
|
|
||||||
e2e.CreateSQLiteDB(t, env, dbName)
|
|
||||||
t.Logf("Created database: %s", dbName)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Deploy Go backend with DATABASE_NAME", func(t *testing.T) {
|
|
||||||
deploymentID = createGoDeployment(t, env, deploymentName, tarballPath, map[string]string{
|
|
||||||
"DATABASE_NAME": dbName,
|
|
||||||
"GATEWAY_URL": env.GatewayURL,
|
|
||||||
"API_KEY": env.APIKey,
|
|
||||||
})
|
|
||||||
require.NotEmpty(t, deploymentID, "Deployment ID should not be empty")
|
|
||||||
t.Logf("Created Go deployment: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wait for deployment to become healthy", func(t *testing.T) {
|
|
||||||
healthy := e2e.WaitForHealthy(t, env, deploymentID, 90*time.Second)
|
|
||||||
require.True(t, healthy, "Deployment should become healthy")
|
|
||||||
t.Logf("Deployment is healthy")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test health endpoint", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/health")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Health check should return 200")
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
var health map[string]interface{}
|
|
||||||
require.NoError(t, json.Unmarshal(body, &health))
|
|
||||||
|
|
||||||
assert.Contains(t, []string{"healthy", "ok"}, health["status"])
|
|
||||||
t.Logf("Health response: %+v", health)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("POST /api/notes - create note", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
noteData := map[string]string{
|
|
||||||
"title": "Test Note",
|
|
||||||
"content": "This is a test note",
|
|
||||||
}
|
|
||||||
body, _ := json.Marshal(noteData)
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/api/notes", bytes.NewBuffer(body))
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
req.Host = domain
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusCreated, resp.StatusCode, "Should create note successfully")
|
|
||||||
|
|
||||||
var note map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(¬e))
|
|
||||||
|
|
||||||
assert.Equal(t, "Test Note", note["title"])
|
|
||||||
assert.Equal(t, "This is a test note", note["content"])
|
|
||||||
t.Logf("Created note: %+v", note)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("GET /api/notes - list notes", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/api/notes")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
|
|
||||||
var notes []map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(¬es))
|
|
||||||
|
|
||||||
assert.GreaterOrEqual(t, len(notes), 1, "Should have at least one note")
|
|
||||||
|
|
||||||
found := false
|
|
||||||
for _, note := range notes {
|
|
||||||
if note["title"] == "Test Note" {
|
|
||||||
found = true
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
assert.True(t, found, "Test note should be in the list")
|
|
||||||
t.Logf("Notes count: %d", len(notes))
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("DELETE /api/notes - delete note", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
// First get the note ID
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/api/notes")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
var notes []map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(¬es))
|
|
||||||
|
|
||||||
var noteID int
|
|
||||||
for _, note := range notes {
|
|
||||||
if note["title"] == "Test Note" {
|
|
||||||
noteID = int(note["id"].(float64))
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
require.NotZero(t, noteID, "Should find test note ID")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("DELETE", fmt.Sprintf("%s/api/notes/%d", env.GatewayURL, noteID), nil)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Host = domain
|
|
||||||
|
|
||||||
deleteResp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer deleteResp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, deleteResp.StatusCode, "Should delete note successfully")
|
|
||||||
t.Logf("Deleted note ID: %d", noteID)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// createGoDeployment creates a Go backend deployment with environment variables
|
|
||||||
func createGoDeployment(t *testing.T, env *e2e.E2ETestEnv, name, tarballPath string, envVars map[string]string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var fileData []byte
|
|
||||||
info, err := os.Stat(tarballPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to stat tarball path: %v", err)
|
|
||||||
}
|
|
||||||
if info.IsDir() {
|
|
||||||
// Build Go binary for linux/amd64, then tar it
|
|
||||||
tmpDir, err := os.MkdirTemp("", "go-deploy-*")
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create temp dir: %v", err)
|
|
||||||
}
|
|
||||||
defer os.RemoveAll(tmpDir)
|
|
||||||
|
|
||||||
binaryPath := filepath.Join(tmpDir, "app")
|
|
||||||
buildCmd := exec.Command("go", "build", "-o", binaryPath, ".")
|
|
||||||
buildCmd.Dir = tarballPath
|
|
||||||
buildCmd.Env = append(os.Environ(), "GOOS=linux", "GOARCH=amd64", "CGO_ENABLED=0")
|
|
||||||
if out, err := buildCmd.CombinedOutput(); err != nil {
|
|
||||||
t.Fatalf("failed to build Go app: %v\n%s", err, string(out))
|
|
||||||
}
|
|
||||||
|
|
||||||
fileData, err = exec.Command("tar", "-czf", "-", "-C", tmpDir, ".").Output()
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create tarball: %v", err)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
file, err := os.Open(tarballPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to open tarball: %v", err)
|
|
||||||
}
|
|
||||||
defer file.Close()
|
|
||||||
fileData, _ = io.ReadAll(file)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create multipart form
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
// Write name field
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(name + "\r\n")
|
|
||||||
|
|
||||||
// Write environment variables
|
|
||||||
for key, value := range envVars {
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString(fmt.Sprintf("Content-Disposition: form-data; name=\"env_%s\"\r\n\r\n", key))
|
|
||||||
body.WriteString(value + "\r\n")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Write tarball file
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
|
|
||||||
body.Write(fileData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/go/upload", body)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to execute request: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusCreated {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Fatalf("Deployment upload failed with status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
|
||||||
t.Fatalf("failed to decode response: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if id, ok := result["deployment_id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
if id, ok := result["id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
t.Fatalf("Deployment response missing id field: %+v", result)
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
// deleteSQLiteDB deletes a SQLite database
|
|
||||||
func deleteSQLiteDB(t *testing.T, env *e2e.E2ETestEnv, dbName string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
req, err := http.NewRequest("DELETE", env.GatewayURL+"/v1/db/"+dbName, nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("warning: failed to create delete request: %v", err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("warning: failed to delete database: %v", err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
t.Logf("warning: delete database returned status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,264 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package deployments_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestNextJSDeployment_SSR tests Next.js deployment with SSR and API routes
|
|
||||||
// 1. Deploy Next.js app
|
|
||||||
// 2. Test SSR page (verify server-rendered HTML)
|
|
||||||
// 3. Test API routes (/api/hello, /api/data)
|
|
||||||
// 4. Test static assets
|
|
||||||
// 5. Cleanup
|
|
||||||
func TestNextJSDeployment_SSR(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("nextjs-ssr-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/nextjs-ssr.tar.gz")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
// Check if tarball exists
|
|
||||||
if _, err := os.Stat(tarballPath); os.IsNotExist(err) {
|
|
||||||
t.Skip("Next.js SSR tarball not found at " + tarballPath)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cleanup after test
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy Next.js SSR app", func(t *testing.T) {
|
|
||||||
deploymentID = createNextJSDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID, "Deployment ID should not be empty")
|
|
||||||
t.Logf("Created Next.js deployment: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wait for deployment to become healthy", func(t *testing.T) {
|
|
||||||
healthy := e2e.WaitForHealthy(t, env, deploymentID, 120*time.Second)
|
|
||||||
require.True(t, healthy, "Deployment should become healthy")
|
|
||||||
t.Logf("Deployment is healthy")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify deployment in database", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
assert.Equal(t, deploymentName, deployment["name"], "Deployment name should match")
|
|
||||||
|
|
||||||
deploymentType, ok := deployment["type"].(string)
|
|
||||||
require.True(t, ok, "Type should be a string")
|
|
||||||
assert.Contains(t, deploymentType, "nextjs", "Type should be nextjs")
|
|
||||||
|
|
||||||
t.Logf("Deployment type: %s", deploymentType)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test SSR page - verify server-rendered HTML", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "SSR page should return 200")
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err, "Should read response body")
|
|
||||||
bodyStr := string(body)
|
|
||||||
|
|
||||||
// Verify HTML is server-rendered (contains actual content, not just loading state)
|
|
||||||
assert.Contains(t, bodyStr, "Orama Network Next.js Test", "Should contain app title")
|
|
||||||
assert.Contains(t, bodyStr, "Server-Side Rendering Test", "Should contain SSR test marker")
|
|
||||||
assert.Contains(t, resp.Header.Get("Content-Type"), "text/html", "Should be HTML content")
|
|
||||||
|
|
||||||
t.Logf("SSR page loaded successfully")
|
|
||||||
t.Logf("Content-Type: %s", resp.Header.Get("Content-Type"))
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test API route - /api/hello", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/api/hello")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "API route should return 200")
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&result), "Should decode JSON response")
|
|
||||||
|
|
||||||
assert.Contains(t, result["message"], "Hello", "Should contain hello message")
|
|
||||||
assert.NotEmpty(t, result["timestamp"], "Should have timestamp")
|
|
||||||
|
|
||||||
t.Logf("API /hello response: %+v", result)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test API route - /api/data", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/api/data")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "API data route should return 200")
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&result), "Should decode JSON response")
|
|
||||||
|
|
||||||
// Just verify it returns valid JSON
|
|
||||||
t.Logf("API /data response: %+v", result)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test static asset - _next directory", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
// First, get the main page to find the actual static asset path
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
bodyStr := string(body)
|
|
||||||
|
|
||||||
// Look for _next/static references in the HTML
|
|
||||||
if strings.Contains(bodyStr, "_next/static") {
|
|
||||||
t.Logf("Found _next/static references in HTML")
|
|
||||||
|
|
||||||
// Try to fetch a common static chunk
|
|
||||||
// The exact path depends on Next.js build output
|
|
||||||
// We'll just verify the _next directory structure is accessible
|
|
||||||
chunkResp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/_next/static/chunks/main.js")
|
|
||||||
defer chunkResp.Body.Close()
|
|
||||||
|
|
||||||
// It's OK if specific files don't exist (they have hashed names)
|
|
||||||
// Just verify we don't get a 500 error
|
|
||||||
assert.NotEqual(t, http.StatusInternalServerError, chunkResp.StatusCode,
|
|
||||||
"Static asset request should not cause server error")
|
|
||||||
|
|
||||||
t.Logf("Static asset request status: %d", chunkResp.StatusCode)
|
|
||||||
} else {
|
|
||||||
t.Logf("No _next/static references found (may be using different bundling)")
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test 404 handling", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/nonexistent-page-xyz")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Next.js should handle 404 gracefully
|
|
||||||
// Could be 404 or 200 depending on catch-all routes
|
|
||||||
assert.Contains(t, []int{200, 404}, resp.StatusCode,
|
|
||||||
"Should return either 200 (catch-all) or 404")
|
|
||||||
|
|
||||||
t.Logf("404 handling: status=%d", resp.StatusCode)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// createNextJSDeployment creates a Next.js deployment
|
|
||||||
func createNextJSDeployment(t *testing.T, env *e2e.E2ETestEnv, name, tarballPath string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
file, err := os.Open(tarballPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to open tarball: %v", err)
|
|
||||||
}
|
|
||||||
defer file.Close()
|
|
||||||
|
|
||||||
// Create multipart form
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
// Write name field
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(name + "\r\n")
|
|
||||||
|
|
||||||
// Write ssr field (enable SSR mode)
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"ssr\"\r\n\r\n")
|
|
||||||
body.WriteString("true\r\n")
|
|
||||||
|
|
||||||
// Write tarball file
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
|
|
||||||
fileData, _ := io.ReadAll(file)
|
|
||||||
body.Write(fileData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/nextjs/upload", body)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
// Use a longer timeout for large Next.js uploads (can be 50MB+)
|
|
||||||
uploadClient := e2e.NewHTTPClient(5 * time.Minute)
|
|
||||||
resp, err := uploadClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to execute request: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusCreated {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Fatalf("Deployment upload failed with status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
|
||||||
t.Fatalf("failed to decode response: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if id, ok := result["deployment_id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
if id, ok := result["id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
t.Fatalf("Deployment response missing id field: %+v", result)
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
@ -1,203 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package deployments_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestNodeJSDeployment_FullFlow(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("test-nodejs-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/node-api")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
// Cleanup after test
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Upload Node.js backend", func(t *testing.T) {
|
|
||||||
deploymentID = createNodeJSDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
|
|
||||||
assert.NotEmpty(t, deploymentID, "Deployment ID should not be empty")
|
|
||||||
t.Logf("Created deployment: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wait for deployment to become healthy", func(t *testing.T) {
|
|
||||||
healthy := e2e.WaitForHealthy(t, env, deploymentID, 90*time.Second)
|
|
||||||
assert.True(t, healthy, "Deployment should become healthy within timeout")
|
|
||||||
t.Logf("Deployment is healthy")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test health endpoint", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
// Get the deployment URLs (can be array of strings or map)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Test via Host header (localhost testing)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, extractDomain(nodeURL), "/health")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Health check should return 200")
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
var health map[string]interface{}
|
|
||||||
require.NoError(t, json.Unmarshal(body, &health))
|
|
||||||
|
|
||||||
assert.Contains(t, []string{"healthy", "ok"}, health["status"],
|
|
||||||
"Health status should be 'healthy' or 'ok'")
|
|
||||||
t.Logf("Health check passed: %v", health)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test API endpoint", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
// Test health endpoint (node-api app serves /health)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/health")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.Unmarshal(body, &result))
|
|
||||||
|
|
||||||
assert.NotEmpty(t, result["service"])
|
|
||||||
t.Logf("API endpoint response: %v", result)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func createNodeJSDeployment(t *testing.T, env *e2e.E2ETestEnv, name, tarballPath string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var fileData []byte
|
|
||||||
|
|
||||||
info, err := os.Stat(tarballPath)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Failed to stat tarball path: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if info.IsDir() {
|
|
||||||
// Create tarball from directory
|
|
||||||
tarData, err := exec.Command("tar", "-czf", "-", "-C", tarballPath, ".").Output()
|
|
||||||
require.NoError(t, err, "Failed to create tarball from %s", tarballPath)
|
|
||||||
fileData = tarData
|
|
||||||
} else {
|
|
||||||
file, err := os.Open(tarballPath)
|
|
||||||
require.NoError(t, err, "Failed to open tarball: %s", tarballPath)
|
|
||||||
defer file.Close()
|
|
||||||
fileData, _ = io.ReadAll(file)
|
|
||||||
}
|
|
||||||
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(name + "\r\n")
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
|
|
||||||
body.Write(fileData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/nodejs/upload", body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusCreated {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Fatalf("Deployment upload failed with status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&result))
|
|
||||||
|
|
||||||
if id, ok := result["deployment_id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
if id, ok := result["id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
t.Fatalf("Deployment response missing id field: %+v", result)
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
// extractNodeURL gets the node URL from deployment response
|
|
||||||
// Handles both array of strings and map formats
|
|
||||||
func extractNodeURL(t *testing.T, deployment map[string]interface{}) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
// Try as array of strings first (new format)
|
|
||||||
if urls, ok := deployment["urls"].([]interface{}); ok && len(urls) > 0 {
|
|
||||||
if url, ok := urls[0].(string); ok {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Try as map (legacy format)
|
|
||||||
if urls, ok := deployment["urls"].(map[string]interface{}); ok {
|
|
||||||
if url, ok := urls["node"].(string); ok {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractDomain(url string) string {
|
|
||||||
// Extract domain from URL like "https://myapp.node-xyz.dbrs.space"
|
|
||||||
// Remove protocol
|
|
||||||
domain := url
|
|
||||||
if len(url) > 8 && url[:8] == "https://" {
|
|
||||||
domain = url[8:]
|
|
||||||
} else if len(url) > 7 && url[:7] == "http://" {
|
|
||||||
domain = url[7:]
|
|
||||||
}
|
|
||||||
// Remove trailing slash
|
|
||||||
if len(domain) > 0 && domain[len(domain)-1] == '/' {
|
|
||||||
domain = domain[:len(domain)-1]
|
|
||||||
}
|
|
||||||
return domain
|
|
||||||
}
|
|
||||||
@ -1,352 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package deployments_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestStaticReplica_CreatedOnDeploy verifies that deploying a static app
|
|
||||||
// creates replica records on a second node.
|
|
||||||
func TestStaticReplica_CreatedOnDeploy(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("replica-static-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy static app", func(t *testing.T) {
|
|
||||||
deploymentID = e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
t.Logf("Created deployment: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wait for replica setup", func(t *testing.T) {
|
|
||||||
// Static replicas should set up quickly (IPFS content)
|
|
||||||
time.Sleep(10 * time.Second)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Deployment has replica records", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
// Check that replicas field exists and has entries
|
|
||||||
replicas, ok := deployment["replicas"].([]interface{})
|
|
||||||
if !ok {
|
|
||||||
// Replicas might be in a nested structure or separate endpoint
|
|
||||||
t.Logf("Deployment response: %+v", deployment)
|
|
||||||
// Try querying replicas via the deployment details
|
|
||||||
homeNodeID, _ := deployment["home_node_id"].(string)
|
|
||||||
require.NotEmpty(t, homeNodeID, "Deployment should have a home_node_id")
|
|
||||||
t.Logf("Home node: %s", homeNodeID)
|
|
||||||
// If replicas aren't in the response, that's still okay — we verify
|
|
||||||
// via DNS and cross-node serving below
|
|
||||||
t.Log("Replica records not in deployment response; will verify via DNS/serving")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
assert.GreaterOrEqual(t, len(replicas), 1, "Should have at least 1 replica")
|
|
||||||
t.Logf("Found %d replica records", len(replicas))
|
|
||||||
for i, r := range replicas {
|
|
||||||
if replica, ok := r.(map[string]interface{}); ok {
|
|
||||||
t.Logf(" Replica %d: node=%s status=%s", i, replica["node_id"], replica["status"])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Static content served via gateway", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Static content should be served (got %d: %s)", resp.StatusCode, string(body))
|
|
||||||
t.Logf("Served via gateway: status=%d", resp.StatusCode)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDynamicReplica_CreatedOnDeploy verifies that deploying a dynamic (Node.js) app
|
|
||||||
// creates a replica process on a second node.
|
|
||||||
func TestDynamicReplica_CreatedOnDeploy(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("replica-nodejs-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/node-api")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy Node.js backend", func(t *testing.T) {
|
|
||||||
deploymentID = createNodeJSDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
t.Logf("Created deployment: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wait for deployment and replica", func(t *testing.T) {
|
|
||||||
healthy := e2e.WaitForHealthy(t, env, deploymentID, 90*time.Second)
|
|
||||||
assert.True(t, healthy, "Deployment should become healthy")
|
|
||||||
// Extra wait for async replica setup
|
|
||||||
time.Sleep(15 * time.Second)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Dynamic app served from both nodes", func(t *testing.T) {
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL in deployment")
|
|
||||||
}
|
|
||||||
domain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/health")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Dynamic app should be served via gateway (got %d: %s)", resp.StatusCode, string(body))
|
|
||||||
t.Logf("Served via gateway: status=%d body=%s", resp.StatusCode, string(body))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestReplica_UpdatePropagation verifies that updating a deployment propagates to replicas.
|
|
||||||
func TestReplica_UpdatePropagation(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("replica-update-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy v1", func(t *testing.T) {
|
|
||||||
deploymentID = e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
time.Sleep(10 * time.Second) // Wait for replica
|
|
||||||
})
|
|
||||||
|
|
||||||
var v1CID string
|
|
||||||
t.Run("Record v1 CID", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
v1CID, _ = deployment["content_cid"].(string)
|
|
||||||
require.NotEmpty(t, v1CID)
|
|
||||||
t.Logf("v1 CID: %s", v1CID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Update to v2", func(t *testing.T) {
|
|
||||||
updateStaticDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
time.Sleep(10 * time.Second) // Wait for update + replica propagation
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("All nodes serve updated version", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
v2CID, _ := deployment["content_cid"].(string)
|
|
||||||
|
|
||||||
// v2 CID might be same (same tarball) but version should increment
|
|
||||||
version, _ := deployment["version"].(float64)
|
|
||||||
assert.Equal(t, float64(2), version, "Should be version 2")
|
|
||||||
t.Logf("v2 CID: %s, version: %v", v2CID, version)
|
|
||||||
|
|
||||||
// Verify via gateway
|
|
||||||
dep := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
depCID, _ := dep["content_cid"].(string)
|
|
||||||
assert.Equal(t, v2CID, depCID, "CID should match after update")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestReplica_RollbackPropagation verifies rollback propagates to replica nodes.
|
|
||||||
func TestReplica_RollbackPropagation(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("replica-rollback-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy v1 and update to v2", func(t *testing.T) {
|
|
||||||
deploymentID = e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
time.Sleep(10 * time.Second)
|
|
||||||
|
|
||||||
updateStaticDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
time.Sleep(10 * time.Second)
|
|
||||||
})
|
|
||||||
|
|
||||||
var v1CID string
|
|
||||||
t.Run("Get v1 CID from versions", func(t *testing.T) {
|
|
||||||
versions := listVersions(t, env, deploymentName)
|
|
||||||
if len(versions) > 0 {
|
|
||||||
v1CID, _ = versions[0]["content_cid"].(string)
|
|
||||||
}
|
|
||||||
if v1CID == "" {
|
|
||||||
// Fall back: v1 CID from current deployment
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
v1CID, _ = deployment["content_cid"].(string)
|
|
||||||
}
|
|
||||||
t.Logf("v1 CID for rollback comparison: %s", v1CID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Rollback to v1", func(t *testing.T) {
|
|
||||||
rollbackDeployment(t, env, deploymentName, 1)
|
|
||||||
time.Sleep(10 * time.Second) // Wait for rollback + replica propagation
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("All nodes have rolled-back CID", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
currentCID, _ := deployment["content_cid"].(string)
|
|
||||||
t.Logf("Post-rollback CID: %s", currentCID)
|
|
||||||
|
|
||||||
assert.Equal(t, v1CID, currentCID, "CID should match v1 after rollback")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestReplica_TeardownOnDelete verifies that deleting a deployment removes replicas.
|
|
||||||
func TestReplica_TeardownOnDelete(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("replica-delete-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
time.Sleep(10 * time.Second) // Wait for replica
|
|
||||||
|
|
||||||
// Get the domain before deletion
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
domain := ""
|
|
||||||
if nodeURL != "" {
|
|
||||||
domain = extractDomain(nodeURL)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Run("Delete deployment", func(t *testing.T) {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
time.Sleep(10 * time.Second) // Wait for teardown propagation
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Deployment no longer served on any node", func(t *testing.T) {
|
|
||||||
if domain == "" {
|
|
||||||
t.Skip("No domain to test")
|
|
||||||
}
|
|
||||||
|
|
||||||
req, err := http.NewRequest("GET", env.GatewayURL+"/", nil)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Host = domain
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("Connection failed (expected after deletion)")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
assert.NotContains(t, string(body), "<div id=\"root\">",
|
|
||||||
"Deleted deployment should not be served")
|
|
||||||
}
|
|
||||||
t.Logf("status=%d (expected non-200)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// updateStaticDeployment updates an existing static deployment.
|
|
||||||
func updateStaticDeployment(t *testing.T, env *e2e.E2ETestEnv, name, tarballPath string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var fileData []byte
|
|
||||||
info, err := os.Stat(tarballPath)
|
|
||||||
require.NoError(t, err)
|
|
||||||
if info.IsDir() {
|
|
||||||
fileData, err = exec.Command("tar", "-czf", "-", "-C", tarballPath, ".").Output()
|
|
||||||
require.NoError(t, err)
|
|
||||||
} else {
|
|
||||||
file, err := os.Open(tarballPath)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer file.Close()
|
|
||||||
fileData, _ = io.ReadAll(file)
|
|
||||||
}
|
|
||||||
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(name + "\r\n")
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
|
|
||||||
body.Write(fileData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/static/update", body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Fatalf("Update failed with status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,232 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package deployments_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestDeploymentRollback_FullFlow tests the complete rollback workflow:
|
|
||||||
// 1. Deploy v1
|
|
||||||
// 2. Update to v2
|
|
||||||
// 3. Verify v2 content
|
|
||||||
// 4. Rollback to v1
|
|
||||||
// 5. Verify v1 content is restored
|
|
||||||
func TestDeploymentRollback_FullFlow(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("rollback-test-%d", time.Now().Unix())
|
|
||||||
tarballPathV1 := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
// Cleanup after test
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy v1", func(t *testing.T) {
|
|
||||||
deploymentID = e2e.CreateTestDeployment(t, env, deploymentName, tarballPathV1)
|
|
||||||
require.NotEmpty(t, deploymentID, "Deployment ID should not be empty")
|
|
||||||
t.Logf("Created deployment v1: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify v1 deployment", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
version, ok := deployment["version"].(float64)
|
|
||||||
require.True(t, ok, "Version should be a number")
|
|
||||||
assert.Equal(t, float64(1), version, "Initial version should be 1")
|
|
||||||
|
|
||||||
contentCID, ok := deployment["content_cid"].(string)
|
|
||||||
require.True(t, ok, "Content CID should be a string")
|
|
||||||
assert.NotEmpty(t, contentCID, "Content CID should not be empty")
|
|
||||||
|
|
||||||
t.Logf("v1 version: %v, CID: %s", version, contentCID)
|
|
||||||
})
|
|
||||||
|
|
||||||
var v1CID string
|
|
||||||
t.Run("Save v1 CID", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
v1CID = deployment["content_cid"].(string)
|
|
||||||
t.Logf("Saved v1 CID: %s", v1CID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Update to v2", func(t *testing.T) {
|
|
||||||
// Update the deployment with the same tarball (simulates a new version)
|
|
||||||
updateDeployment(t, env, deploymentName, tarballPathV1)
|
|
||||||
|
|
||||||
// Wait for update to complete
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify v2 deployment", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
version, ok := deployment["version"].(float64)
|
|
||||||
require.True(t, ok, "Version should be a number")
|
|
||||||
assert.Equal(t, float64(2), version, "Version should be 2 after update")
|
|
||||||
|
|
||||||
t.Logf("v2 version: %v", version)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("List deployment versions", func(t *testing.T) {
|
|
||||||
versions := listVersions(t, env, deploymentName)
|
|
||||||
t.Logf("Available versions: %+v", versions)
|
|
||||||
|
|
||||||
// Should have at least 2 versions in history
|
|
||||||
assert.GreaterOrEqual(t, len(versions), 1, "Should have version history")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Rollback to v1", func(t *testing.T) {
|
|
||||||
rollbackDeployment(t, env, deploymentName, 1)
|
|
||||||
|
|
||||||
// Wait for rollback to complete
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify rollback succeeded", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
version, ok := deployment["version"].(float64)
|
|
||||||
require.True(t, ok, "Version should be a number")
|
|
||||||
// Note: Version number increases even on rollback (it's a new deployment version)
|
|
||||||
// But the content_cid should be the same as v1
|
|
||||||
t.Logf("Post-rollback version: %v", version)
|
|
||||||
|
|
||||||
contentCID, ok := deployment["content_cid"].(string)
|
|
||||||
require.True(t, ok, "Content CID should be a string")
|
|
||||||
assert.Equal(t, v1CID, contentCID, "Content CID should match v1 after rollback")
|
|
||||||
|
|
||||||
t.Logf("Rollback verified - content CID matches v1: %s", contentCID)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// updateDeployment updates an existing static deployment
|
|
||||||
func updateDeployment(t *testing.T, env *e2e.E2ETestEnv, name, tarballPath string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var fileData []byte
|
|
||||||
info, err := os.Stat(tarballPath)
|
|
||||||
require.NoError(t, err)
|
|
||||||
if info.IsDir() {
|
|
||||||
fileData, err = exec.Command("tar", "-czf", "-", "-C", tarballPath, ".").Output()
|
|
||||||
require.NoError(t, err)
|
|
||||||
} else {
|
|
||||||
file, err := os.Open(tarballPath)
|
|
||||||
require.NoError(t, err, "Failed to open tarball")
|
|
||||||
defer file.Close()
|
|
||||||
fileData, _ = io.ReadAll(file)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create multipart form
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
// Write name field
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(name + "\r\n")
|
|
||||||
|
|
||||||
// Write tarball file
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
|
|
||||||
body.Write(fileData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/static/update", body)
|
|
||||||
require.NoError(t, err, "Failed to create request")
|
|
||||||
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Failed to execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Fatalf("Update failed with status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&result), "Failed to decode response")
|
|
||||||
t.Logf("Update response: %+v", result)
|
|
||||||
}
|
|
||||||
|
|
||||||
// listVersions lists available versions for a deployment
|
|
||||||
func listVersions(t *testing.T, env *e2e.E2ETestEnv, name string) []map[string]interface{} {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
req, err := http.NewRequest("GET", env.GatewayURL+"/v1/deployments/versions?name="+name, nil)
|
|
||||||
require.NoError(t, err, "Failed to create request")
|
|
||||||
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Failed to execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("List versions returned status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
var result struct {
|
|
||||||
Versions []map[string]interface{} `json:"versions"`
|
|
||||||
}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
|
||||||
t.Logf("Failed to decode versions: %v", err)
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
return result.Versions
|
|
||||||
}
|
|
||||||
|
|
||||||
// rollbackDeployment triggers a rollback to a specific version
|
|
||||||
func rollbackDeployment(t *testing.T, env *e2e.E2ETestEnv, name string, targetVersion int) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
reqBody := map[string]interface{}{
|
|
||||||
"name": name,
|
|
||||||
"version": targetVersion,
|
|
||||||
}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/rollback", bytes.NewBuffer(bodyBytes))
|
|
||||||
require.NoError(t, err, "Failed to create request")
|
|
||||||
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Failed to execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Fatalf("Rollback failed with status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&result), "Failed to decode response")
|
|
||||||
t.Logf("Rollback response: %+v", result)
|
|
||||||
}
|
|
||||||
@ -1,210 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package deployments_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestStaticDeployment_FullFlow(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("test-static-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
// Cleanup after test
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Upload static tarball", func(t *testing.T) {
|
|
||||||
deploymentID = e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
|
|
||||||
assert.NotEmpty(t, deploymentID, "Deployment ID should not be empty")
|
|
||||||
t.Logf("✓ Created deployment: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify deployment in database", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
assert.Equal(t, deploymentName, deployment["name"], "Deployment name should match")
|
|
||||||
assert.NotEmpty(t, deployment["content_cid"], "Content CID should not be empty")
|
|
||||||
|
|
||||||
// Status might be "deploying" or "active" depending on timing
|
|
||||||
status, ok := deployment["status"].(string)
|
|
||||||
require.True(t, ok, "Status should be a string")
|
|
||||||
assert.Contains(t, []string{"deploying", "active"}, status, "Status should be deploying or active")
|
|
||||||
|
|
||||||
t.Logf("✓ Deployment verified in database")
|
|
||||||
t.Logf(" - Name: %s", deployment["name"])
|
|
||||||
t.Logf(" - Status: %s", status)
|
|
||||||
t.Logf(" - CID: %s", deployment["content_cid"])
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify DNS record creation", func(t *testing.T) {
|
|
||||||
// Wait for deployment to become active
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
|
|
||||||
// Get the actual domain from deployment response
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
require.NotEmpty(t, nodeURL, "Deployment should have a URL")
|
|
||||||
expectedDomain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
// Make request with Host header (localhost testing)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, expectedDomain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Should return 200 with React app HTML
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Should return 200 OK")
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err, "Should read response body")
|
|
||||||
|
|
||||||
bodyStr := string(body)
|
|
||||||
|
|
||||||
// Verify React app content
|
|
||||||
assert.Contains(t, bodyStr, "<div id=\"root\">", "Should contain React root div")
|
|
||||||
assert.Contains(t, resp.Header.Get("Content-Type"), "text/html", "Content-Type should be text/html")
|
|
||||||
|
|
||||||
t.Logf("✓ Domain routing works")
|
|
||||||
t.Logf(" - Domain: %s", expectedDomain)
|
|
||||||
t.Logf(" - Status: %d", resp.StatusCode)
|
|
||||||
t.Logf(" - Content-Type: %s", resp.Header.Get("Content-Type"))
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify static assets serve correctly", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
require.NotEmpty(t, nodeURL, "Deployment should have a URL")
|
|
||||||
expectedDomain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
// Test CSS file (exact path depends on Vite build output)
|
|
||||||
// We'll just test a few common asset paths
|
|
||||||
assetPaths := []struct {
|
|
||||||
path string
|
|
||||||
contentType string
|
|
||||||
}{
|
|
||||||
{"/index.html", "text/html"},
|
|
||||||
// Note: Asset paths with hashes change on each build
|
|
||||||
// We'll test what we can
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, asset := range assetPaths {
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, expectedDomain, asset.path)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
assert.Contains(t, resp.Header.Get("Content-Type"), asset.contentType,
|
|
||||||
"Content-Type should be %s for %s", asset.contentType, asset.path)
|
|
||||||
|
|
||||||
t.Logf("✓ Asset served correctly: %s (%s)", asset.path, asset.contentType)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify SPA fallback routing", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
require.NotEmpty(t, nodeURL, "Deployment should have a URL")
|
|
||||||
expectedDomain := extractDomain(nodeURL)
|
|
||||||
|
|
||||||
// Request unknown route (should return index.html for SPA)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, expectedDomain, "/about/team")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "SPA fallback should return 200")
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err, "Should read response body")
|
|
||||||
|
|
||||||
assert.Contains(t, string(body), "<div id=\"root\">", "Should return index.html for unknown paths")
|
|
||||||
|
|
||||||
t.Logf("✓ SPA fallback routing works")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("List deployments", func(t *testing.T) {
|
|
||||||
req, err := http.NewRequest("GET", env.GatewayURL+"/v1/deployments/list", nil)
|
|
||||||
require.NoError(t, err, "Should create request")
|
|
||||||
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "List deployments should return 200")
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, e2e.DecodeJSON(mustReadAll(t, resp.Body), &result), "Should decode JSON")
|
|
||||||
|
|
||||||
deployments, ok := result["deployments"].([]interface{})
|
|
||||||
require.True(t, ok, "Deployments should be an array")
|
|
||||||
|
|
||||||
assert.GreaterOrEqual(t, len(deployments), 1, "Should have at least one deployment")
|
|
||||||
|
|
||||||
// Find our deployment
|
|
||||||
found := false
|
|
||||||
for _, d := range deployments {
|
|
||||||
dep, ok := d.(map[string]interface{})
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if dep["name"] == deploymentName {
|
|
||||||
found = true
|
|
||||||
t.Logf("✓ Found deployment in list: %s", deploymentName)
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
assert.True(t, found, "Deployment should be in list")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Delete deployment", func(t *testing.T) {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
// Verify deletion - allow time for replication
|
|
||||||
time.Sleep(3 * time.Second)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/v1/deployments/get?id="+deploymentID, nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("Delete verification response: status=%d body=%s", resp.StatusCode, string(body))
|
|
||||||
|
|
||||||
// After deletion, either 404 (not found) or 200 with empty/error response is acceptable
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
// If 200, check if the deployment is actually gone
|
|
||||||
t.Logf("Got 200 - this may indicate soft delete or eventual consistency")
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("✓ Deployment deleted successfully")
|
|
||||||
|
|
||||||
// Clear deploymentID so cleanup doesn't try to delete again
|
|
||||||
deploymentID = ""
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func mustReadAll(t *testing.T, r io.Reader) []byte {
|
|
||||||
t.Helper()
|
|
||||||
data, err := io.ReadAll(r)
|
|
||||||
require.NoError(t, err, "Should read all data")
|
|
||||||
return data
|
|
||||||
}
|
|
||||||
1731
core/e2e/env.go
1731
core/e2e/env.go
File diff suppressed because it is too large
Load Diff
@ -1,462 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package integration_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// =============================================================================
|
|
||||||
// STRICT DATA PERSISTENCE TESTS
|
|
||||||
// These tests verify that data is properly persisted and survives operations.
|
|
||||||
// Tests FAIL if data is lost or corrupted.
|
|
||||||
// =============================================================================
|
|
||||||
|
|
||||||
// TestRQLite_DataPersistence verifies that RQLite data is persisted through the gateway.
|
|
||||||
func TestRQLite_DataPersistence(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
tableName := fmt.Sprintf("persist_test_%d", time.Now().UnixNano())
|
|
||||||
|
|
||||||
// Cleanup
|
|
||||||
defer func() {
|
|
||||||
dropReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/drop-table",
|
|
||||||
Body: map[string]interface{}{"table": tableName},
|
|
||||||
}
|
|
||||||
dropReq.Do(context.Background())
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Create table
|
|
||||||
createReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/create-table",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"schema": fmt.Sprintf(
|
|
||||||
"CREATE TABLE IF NOT EXISTS %s (id INTEGER PRIMARY KEY, value TEXT, version INTEGER)",
|
|
||||||
tableName,
|
|
||||||
),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := createReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create table")
|
|
||||||
require.True(t, status == http.StatusCreated || status == http.StatusOK,
|
|
||||||
"FAIL: Create table returned status %d", status)
|
|
||||||
|
|
||||||
t.Run("Data_survives_multiple_writes", func(t *testing.T) {
|
|
||||||
// Insert initial data
|
|
||||||
var statements []string
|
|
||||||
for i := 1; i <= 10; i++ {
|
|
||||||
statements = append(statements,
|
|
||||||
fmt.Sprintf("INSERT INTO %s (value, version) VALUES ('item_%d', %d)", tableName, i, i))
|
|
||||||
}
|
|
||||||
|
|
||||||
insertReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/transaction",
|
|
||||||
Body: map[string]interface{}{"statements": statements},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := insertReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not insert rows")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Insert returned status %d", status)
|
|
||||||
|
|
||||||
// Verify all data exists
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/query",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT COUNT(*) FROM %s", tableName),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, status, err := queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not count rows")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Count query returned status %d", status)
|
|
||||||
|
|
||||||
var queryResp map[string]interface{}
|
|
||||||
e2e.DecodeJSON(body, &queryResp)
|
|
||||||
|
|
||||||
if rows, ok := queryResp["rows"].([]interface{}); ok && len(rows) > 0 {
|
|
||||||
row := rows[0].([]interface{})
|
|
||||||
count := int(row[0].(float64))
|
|
||||||
require.Equal(t, 10, count, "FAIL: Expected 10 rows, got %d", count)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Update data
|
|
||||||
updateReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/transaction",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"statements": []string{
|
|
||||||
fmt.Sprintf("UPDATE %s SET version = version + 100 WHERE version <= 5", tableName),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err = updateReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not update rows")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Update returned status %d", status)
|
|
||||||
|
|
||||||
// Verify updates persisted
|
|
||||||
queryUpdatedReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/query",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT COUNT(*) FROM %s WHERE version > 100", tableName),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, status, err = queryUpdatedReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not count updated rows")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Count updated query returned status %d", status)
|
|
||||||
|
|
||||||
e2e.DecodeJSON(body, &queryResp)
|
|
||||||
if rows, ok := queryResp["rows"].([]interface{}); ok && len(rows) > 0 {
|
|
||||||
row := rows[0].([]interface{})
|
|
||||||
count := int(row[0].(float64))
|
|
||||||
require.Equal(t, 5, count, "FAIL: Expected 5 updated rows, got %d", count)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf(" ✓ Data persists through multiple write operations")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Deletes_are_persisted", func(t *testing.T) {
|
|
||||||
// Delete some rows
|
|
||||||
deleteReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/transaction",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"statements": []string{
|
|
||||||
fmt.Sprintf("DELETE FROM %s WHERE version > 100", tableName),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := deleteReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not delete rows")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Delete returned status %d", status)
|
|
||||||
|
|
||||||
// Verify deletes persisted
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/query",
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT COUNT(*) FROM %s", tableName),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, status, err := queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not count remaining rows")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Count query returned status %d", status)
|
|
||||||
|
|
||||||
var queryResp map[string]interface{}
|
|
||||||
e2e.DecodeJSON(body, &queryResp)
|
|
||||||
|
|
||||||
if rows, ok := queryResp["rows"].([]interface{}); ok && len(rows) > 0 {
|
|
||||||
row := rows[0].([]interface{})
|
|
||||||
count := int(row[0].(float64))
|
|
||||||
require.Equal(t, 5, count, "FAIL: Expected 5 rows after delete, got %d", count)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf(" ✓ Deletes are properly persisted")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestRQLite_DataFilesExist verifies RQLite data files are created on disk.
|
|
||||||
func TestRQLite_DataFilesExist(t *testing.T) {
|
|
||||||
homeDir, err := os.UserHomeDir()
|
|
||||||
require.NoError(t, err, "FAIL: Could not get home directory")
|
|
||||||
|
|
||||||
// Check for RQLite data directories
|
|
||||||
dataLocations := []string{
|
|
||||||
filepath.Join(homeDir, ".orama", "node-1", "rqlite"),
|
|
||||||
filepath.Join(homeDir, ".orama", "node-2", "rqlite"),
|
|
||||||
filepath.Join(homeDir, ".orama", "node-3", "rqlite"),
|
|
||||||
filepath.Join(homeDir, ".orama", "node-4", "rqlite"),
|
|
||||||
filepath.Join(homeDir, ".orama", "node-5", "rqlite"),
|
|
||||||
}
|
|
||||||
|
|
||||||
foundDataDirs := 0
|
|
||||||
for _, dataDir := range dataLocations {
|
|
||||||
if _, err := os.Stat(dataDir); err == nil {
|
|
||||||
foundDataDirs++
|
|
||||||
t.Logf(" ✓ Found RQLite data directory: %s", dataDir)
|
|
||||||
|
|
||||||
// Check for Raft log files
|
|
||||||
entries, _ := os.ReadDir(dataDir)
|
|
||||||
for _, entry := range entries {
|
|
||||||
t.Logf(" - %s", entry.Name())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
require.Greater(t, foundDataDirs, 0,
|
|
||||||
"FAIL: No RQLite data directories found - data may not be persisted")
|
|
||||||
t.Logf(" Found %d RQLite data directories", foundDataDirs)
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestOlric_DataPersistence verifies Olric cache data persistence.
|
|
||||||
// Note: Olric is an in-memory cache, so this tests data survival during runtime.
|
|
||||||
func TestOlric_DataPersistence(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "FAIL: Could not load test environment")
|
|
||||||
|
|
||||||
dmap := fmt.Sprintf("persist_cache_%d", time.Now().UnixNano())
|
|
||||||
|
|
||||||
t.Run("Cache_data_survives_multiple_operations", func(t *testing.T) {
|
|
||||||
// Put multiple keys
|
|
||||||
keys := make(map[string]string)
|
|
||||||
for i := 0; i < 10; i++ {
|
|
||||||
key := fmt.Sprintf("persist_key_%d", i)
|
|
||||||
value := fmt.Sprintf("persist_value_%d", i)
|
|
||||||
keys[key] = value
|
|
||||||
|
|
||||||
err := e2e.PutToOlric(env.GatewayURL, env.APIKey, dmap, key, value)
|
|
||||||
require.NoError(t, err, "FAIL: Could not put key %s", key)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Perform other operations
|
|
||||||
err := e2e.PutToOlric(env.GatewayURL, env.APIKey, dmap, "other_key", "other_value")
|
|
||||||
require.NoError(t, err, "FAIL: Could not put other key")
|
|
||||||
|
|
||||||
// Verify original keys still exist
|
|
||||||
for key, expectedValue := range keys {
|
|
||||||
retrieved, err := e2e.GetFromOlric(env.GatewayURL, env.APIKey, dmap, key)
|
|
||||||
require.NoError(t, err, "FAIL: Key %s not found after other operations", key)
|
|
||||||
require.Equal(t, expectedValue, retrieved, "FAIL: Value mismatch for key %s", key)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf(" ✓ Cache data survives multiple operations")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestNamespaceCluster_DataPersistence verifies namespace-specific data is isolated and persisted.
|
|
||||||
func TestNamespaceCluster_DataPersistence(t *testing.T) {
|
|
||||||
// Create namespace
|
|
||||||
namespace := fmt.Sprintf("persist-ns-%d", time.Now().UnixNano())
|
|
||||||
env, err := e2e.LoadTestEnvWithNamespace(namespace)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create namespace")
|
|
||||||
|
|
||||||
t.Logf("Created namespace: %s", namespace)
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
t.Run("Namespace_data_is_isolated", func(t *testing.T) {
|
|
||||||
// Create data via gateway API
|
|
||||||
tableName := fmt.Sprintf("ns_data_%d", time.Now().UnixNano())
|
|
||||||
|
|
||||||
req := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: env.GatewayURL + "/v1/rqlite/create-table",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"schema": fmt.Sprintf("CREATE TABLE IF NOT EXISTS %s (id INTEGER PRIMARY KEY, value TEXT)", tableName),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := req.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create table in namespace")
|
|
||||||
require.True(t, status == http.StatusOK || status == http.StatusCreated,
|
|
||||||
"FAIL: Create table returned status %d", status)
|
|
||||||
|
|
||||||
// Insert data
|
|
||||||
insertReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: env.GatewayURL + "/v1/rqlite/transaction",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"statements": []string{
|
|
||||||
fmt.Sprintf("INSERT INTO %s (value) VALUES ('ns_test_value')", tableName),
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err = insertReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not insert into namespace table")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Insert returned status %d", status)
|
|
||||||
|
|
||||||
// Verify data exists
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: env.GatewayURL + "/v1/rqlite/query",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"sql": fmt.Sprintf("SELECT value FROM %s", tableName),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, status, err := queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Could not query namespace table")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Query returned status %d", status)
|
|
||||||
|
|
||||||
var queryResp map[string]interface{}
|
|
||||||
json.Unmarshal(body, &queryResp)
|
|
||||||
count, _ := queryResp["count"].(float64)
|
|
||||||
require.Equal(t, float64(1), count, "FAIL: Expected 1 row in namespace table")
|
|
||||||
|
|
||||||
t.Logf(" ✓ Namespace data is isolated and persisted")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestIPFS_DataPersistence verifies IPFS content is persisted and pinned.
|
|
||||||
// Note: Detailed IPFS tests are in storage_http_test.go. This test uses the helper from env.go.
|
|
||||||
func TestIPFS_DataPersistence(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "FAIL: Could not load test environment")
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
t.Run("Uploaded_content_persists", func(t *testing.T) {
|
|
||||||
// Use helper function to upload content via multipart form
|
|
||||||
content := fmt.Sprintf("persistent content %d", time.Now().UnixNano())
|
|
||||||
cid := e2e.UploadTestFile(t, env, "persist_test.txt", content)
|
|
||||||
require.NotEmpty(t, cid, "FAIL: No CID returned from upload")
|
|
||||||
t.Logf(" Uploaded content with CID: %s", cid)
|
|
||||||
|
|
||||||
// Verify content can be retrieved
|
|
||||||
getReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: env.GatewayURL + "/v1/storage/get/" + cid,
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
respBody, status, err := getReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Get content failed")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Get returned status %d", status)
|
|
||||||
require.Contains(t, string(respBody), "persistent content",
|
|
||||||
"FAIL: Retrieved content doesn't match uploaded content")
|
|
||||||
|
|
||||||
t.Logf(" ✓ IPFS content persists and is retrievable")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestSQLite_DataPersistence verifies per-deployment SQLite databases persist.
|
|
||||||
func TestSQLite_DataPersistence(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "FAIL: Could not load test environment")
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
dbName := fmt.Sprintf("persist_db_%d", time.Now().UnixNano())
|
|
||||||
|
|
||||||
t.Run("SQLite_database_persists", func(t *testing.T) {
|
|
||||||
// Create database
|
|
||||||
createReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: env.GatewayURL + "/v1/db/sqlite/create",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"database_name": dbName,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := createReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Create database failed")
|
|
||||||
require.True(t, status == http.StatusOK || status == http.StatusCreated,
|
|
||||||
"FAIL: Create returned status %d", status)
|
|
||||||
t.Logf(" Created SQLite database: %s", dbName)
|
|
||||||
|
|
||||||
// Create table and insert data
|
|
||||||
queryReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: env.GatewayURL + "/v1/db/sqlite/query",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"database_name": dbName,
|
|
||||||
"query": "CREATE TABLE IF NOT EXISTS test_table (id INTEGER PRIMARY KEY, data TEXT)",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err = queryReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Create table failed")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Create table returned status %d", status)
|
|
||||||
|
|
||||||
// Insert data
|
|
||||||
insertReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: env.GatewayURL + "/v1/db/sqlite/query",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"database_name": dbName,
|
|
||||||
"query": "INSERT INTO test_table (data) VALUES ('persistent_data')",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err = insertReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Insert failed")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Insert returned status %d", status)
|
|
||||||
|
|
||||||
// Verify data persists
|
|
||||||
selectReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodPost,
|
|
||||||
URL: env.GatewayURL + "/v1/db/sqlite/query",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
Body: map[string]interface{}{
|
|
||||||
"database_name": dbName,
|
|
||||||
"query": "SELECT data FROM test_table",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, status, err := selectReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Select failed")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: Select returned status %d", status)
|
|
||||||
require.Contains(t, string(body), "persistent_data",
|
|
||||||
"FAIL: Data not found in SQLite database")
|
|
||||||
|
|
||||||
t.Logf(" ✓ SQLite database data persists")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("SQLite_database_listed", func(t *testing.T) {
|
|
||||||
// List databases to verify it was persisted
|
|
||||||
listReq := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: env.GatewayURL + "/v1/db/sqlite/list",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + env.APIKey,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
body, status, err := listReq.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: List databases failed")
|
|
||||||
require.Equal(t, http.StatusOK, status, "FAIL: List returned status %d", status)
|
|
||||||
require.Contains(t, string(body), dbName,
|
|
||||||
"FAIL: Created database not found in list")
|
|
||||||
|
|
||||||
t.Logf(" ✓ SQLite database appears in list")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,356 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package integration_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestDomainRouting_BasicRouting(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("test-routing-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for deployment to be active
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
|
|
||||||
// Get deployment details for debugging
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
t.Logf("Deployment created: ID=%s, CID=%s, Name=%s, Status=%s",
|
|
||||||
deploymentID, deployment["content_cid"], deployment["name"], deployment["status"])
|
|
||||||
|
|
||||||
t.Run("Standard domain resolves", func(t *testing.T) {
|
|
||||||
// Domain format: {deploymentName}.{baseDomain}
|
|
||||||
domain := env.BuildDeploymentDomain(deploymentName)
|
|
||||||
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Should return 200 OK")
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err, "Should read response body")
|
|
||||||
|
|
||||||
assert.Contains(t, string(body), "<div id=\"root\">", "Should serve React app")
|
|
||||||
assert.Contains(t, resp.Header.Get("Content-Type"), "text/html", "Content-Type should be HTML")
|
|
||||||
|
|
||||||
t.Logf("✓ Standard domain routing works: %s", domain)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Non-orama domain passes through", func(t *testing.T) {
|
|
||||||
// Request with non-orama domain should not route to deployment
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, "example.com", "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Should either return 404 or pass to default handler
|
|
||||||
assert.NotEqual(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Non-orama domain should not route to deployment")
|
|
||||||
|
|
||||||
t.Logf("✓ Non-orama domains correctly pass through (status: %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("API paths bypass domain routing", func(t *testing.T) {
|
|
||||||
// /v1/* paths should bypass domain routing and use API key auth
|
|
||||||
domain := env.BuildDeploymentDomain(deploymentName)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/v1/deployments/list", nil)
|
|
||||||
req.Host = domain
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Should return API response, not deployment content
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "API endpoint should work")
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
err = json.Unmarshal(bodyBytes, &result)
|
|
||||||
|
|
||||||
// Should be JSON API response
|
|
||||||
assert.NoError(t, err, "Should decode JSON (API response)")
|
|
||||||
assert.NotNil(t, result["deployments"], "Should have deployments field")
|
|
||||||
|
|
||||||
t.Logf("✓ API paths correctly bypass domain routing")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Well-known paths bypass domain routing", func(t *testing.T) {
|
|
||||||
domain := env.BuildDeploymentDomain(deploymentName)
|
|
||||||
|
|
||||||
// /.well-known/ paths should bypass (used for ACME challenges, etc.)
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/.well-known/acme-challenge/test")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Should not serve deployment content
|
|
||||||
// Exact status depends on implementation, but shouldn't be deployment content
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
bodyStr := string(body)
|
|
||||||
|
|
||||||
// Shouldn't contain React app content
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
assert.NotContains(t, bodyStr, "<div id=\"root\">",
|
|
||||||
"Well-known paths should not serve deployment content")
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("✓ Well-known paths bypass routing (status: %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestDomainRouting_MultipleDeployments(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
// Create multiple deployments
|
|
||||||
deployment1Name := fmt.Sprintf("test-multi-1-%d", time.Now().Unix())
|
|
||||||
deployment2Name := fmt.Sprintf("test-multi-2-%d", time.Now().Unix())
|
|
||||||
|
|
||||||
deployment1ID := e2e.CreateTestDeployment(t, env, deployment1Name, tarballPath)
|
|
||||||
time.Sleep(1 * time.Second)
|
|
||||||
deployment2ID := e2e.CreateTestDeployment(t, env, deployment2Name, tarballPath)
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deployment1ID)
|
|
||||||
e2e.DeleteDeployment(t, env, deployment2ID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
|
|
||||||
t.Run("Each deployment routes independently", func(t *testing.T) {
|
|
||||||
domain1 := env.BuildDeploymentDomain(deployment1Name)
|
|
||||||
domain2 := env.BuildDeploymentDomain(deployment2Name)
|
|
||||||
|
|
||||||
// Test deployment 1
|
|
||||||
resp1 := e2e.TestDeploymentWithHostHeader(t, env, domain1, "/")
|
|
||||||
defer resp1.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp1.StatusCode, "Deployment 1 should serve")
|
|
||||||
|
|
||||||
// Test deployment 2
|
|
||||||
resp2 := e2e.TestDeploymentWithHostHeader(t, env, domain2, "/")
|
|
||||||
defer resp2.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp2.StatusCode, "Deployment 2 should serve")
|
|
||||||
|
|
||||||
t.Logf("✓ Multiple deployments route independently")
|
|
||||||
t.Logf(" - Domain 1: %s", domain1)
|
|
||||||
t.Logf(" - Domain 2: %s", domain2)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wrong domain returns 404", func(t *testing.T) {
|
|
||||||
// Request with non-existent deployment subdomain
|
|
||||||
fakeDeploymentDomain := env.BuildDeploymentDomain(fmt.Sprintf("nonexistent-deployment-%d", time.Now().Unix()))
|
|
||||||
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, fakeDeploymentDomain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusNotFound, resp.StatusCode,
|
|
||||||
"Non-existent deployment should return 404")
|
|
||||||
|
|
||||||
t.Logf("✓ Non-existent deployment returns 404")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestDomainRouting_ContentTypes(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("test-content-types-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
|
|
||||||
domain := env.BuildDeploymentDomain(deploymentName)
|
|
||||||
|
|
||||||
contentTypeTests := []struct {
|
|
||||||
path string
|
|
||||||
shouldHave string
|
|
||||||
description string
|
|
||||||
}{
|
|
||||||
{"/", "text/html", "HTML root"},
|
|
||||||
{"/index.html", "text/html", "HTML file"},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, test := range contentTypeTests {
|
|
||||||
t.Run(test.description, func(t *testing.T) {
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, test.path)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
contentType := resp.Header.Get("Content-Type")
|
|
||||||
assert.Contains(t, contentType, test.shouldHave,
|
|
||||||
"Content-Type for %s should contain %s", test.path, test.shouldHave)
|
|
||||||
|
|
||||||
t.Logf("✓ %s: %s", test.description, contentType)
|
|
||||||
} else {
|
|
||||||
t.Logf("⚠ %s returned status %d", test.path, resp.StatusCode)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestDomainRouting_SPAFallback(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("test-spa-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
|
|
||||||
domain := env.BuildDeploymentDomain(deploymentName)
|
|
||||||
|
|
||||||
t.Run("Unknown paths fall back to index.html", func(t *testing.T) {
|
|
||||||
unknownPaths := []string{
|
|
||||||
"/about",
|
|
||||||
"/users/123",
|
|
||||||
"/settings/profile",
|
|
||||||
"/some/deep/nested/path",
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, path := range unknownPaths {
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, path)
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
resp.Body.Close()
|
|
||||||
|
|
||||||
// Should return index.html for SPA routing
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"SPA fallback should return 200 for %s", path)
|
|
||||||
|
|
||||||
assert.Contains(t, string(body), "<div id=\"root\">",
|
|
||||||
"SPA fallback should return index.html for %s", path)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("✓ SPA fallback routing verified for %d paths", len(unknownPaths))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestDeployment_DomainFormat verifies that deployment URLs use the correct format:
|
|
||||||
// - CORRECT: {name}-{random}.{baseDomain} (e.g., "myapp-f3o4if.dbrs.space")
|
|
||||||
// - WRONG: {name}.node-{shortID}.{baseDomain} (should NOT exist)
|
|
||||||
func TestDeployment_DomainFormat(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("format-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for deployment
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
|
|
||||||
t.Run("Deployment URL has correct format", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
// Get the deployment URLs
|
|
||||||
urls, ok := deployment["urls"].([]interface{})
|
|
||||||
if !ok || len(urls) == 0 {
|
|
||||||
// Fall back to single url field
|
|
||||||
if url, ok := deployment["url"].(string); ok && url != "" {
|
|
||||||
urls = []interface{}{url}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get the subdomain from deployment response
|
|
||||||
subdomain, _ := deployment["subdomain"].(string)
|
|
||||||
t.Logf("Deployment subdomain: %s", subdomain)
|
|
||||||
t.Logf("Deployment URLs: %v", urls)
|
|
||||||
|
|
||||||
foundCorrectFormat := false
|
|
||||||
for _, u := range urls {
|
|
||||||
urlStr, ok := u.(string)
|
|
||||||
if !ok {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
// URL should start with https://{name}-
|
|
||||||
expectedPrefix := fmt.Sprintf("https://%s-", deploymentName)
|
|
||||||
if strings.HasPrefix(urlStr, expectedPrefix) {
|
|
||||||
foundCorrectFormat = true
|
|
||||||
}
|
|
||||||
|
|
||||||
// URL should contain base domain
|
|
||||||
assert.Contains(t, urlStr, env.BaseDomain,
|
|
||||||
"URL should contain base domain %s", env.BaseDomain)
|
|
||||||
|
|
||||||
// URL should NOT contain node identifier pattern
|
|
||||||
assert.NotContains(t, urlStr, ".node-",
|
|
||||||
"URL should NOT have node identifier (got: %s)", urlStr)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(urls) > 0 {
|
|
||||||
assert.True(t, foundCorrectFormat, "Should find URL with correct domain format (https://{name}-{random}.{baseDomain})")
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("✓ Domain format verification passed")
|
|
||||||
t.Logf(" - Format: {name}-{random}.{baseDomain}")
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Domain resolves via Host header", func(t *testing.T) {
|
|
||||||
// Get the actual subdomain from the deployment
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
subdomain, _ := deployment["subdomain"].(string)
|
|
||||||
if subdomain == "" {
|
|
||||||
t.Skip("No subdomain set, skipping host header test")
|
|
||||||
}
|
|
||||||
domain := subdomain + "." + env.BaseDomain
|
|
||||||
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Domain %s should resolve successfully", domain)
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
assert.Contains(t, string(body), "<div id=\"root\">",
|
|
||||||
"Should serve deployment content")
|
|
||||||
|
|
||||||
t.Logf("✓ Domain %s resolves correctly", domain)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,278 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package integration_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
func TestFullStack_GoAPI_SQLite(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
appName := fmt.Sprintf("fullstack-app-%d", time.Now().Unix())
|
|
||||||
backendName := appName + "-backend"
|
|
||||||
dbName := appName + "-db"
|
|
||||||
|
|
||||||
var backendID string
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
if backendID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, backendID)
|
|
||||||
}
|
|
||||||
e2e.DeleteSQLiteDB(t, env, dbName)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Step 1: Create SQLite database
|
|
||||||
t.Run("Create SQLite database", func(t *testing.T) {
|
|
||||||
e2e.CreateSQLiteDB(t, env, dbName)
|
|
||||||
|
|
||||||
// Create users table
|
|
||||||
query := `CREATE TABLE users (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
name TEXT NOT NULL,
|
|
||||||
email TEXT UNIQUE NOT NULL,
|
|
||||||
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
|
|
||||||
)`
|
|
||||||
e2e.ExecuteSQLQuery(t, env, dbName, query)
|
|
||||||
|
|
||||||
// Insert test data
|
|
||||||
insertQuery := `INSERT INTO users (name, email) VALUES ('Alice', 'alice@example.com')`
|
|
||||||
result := e2e.ExecuteSQLQuery(t, env, dbName, insertQuery)
|
|
||||||
|
|
||||||
assert.NotNil(t, result, "Should execute INSERT successfully")
|
|
||||||
t.Logf("✓ Database created with users table")
|
|
||||||
})
|
|
||||||
|
|
||||||
// Step 2: Deploy Go backend (this would normally connect to SQLite)
|
|
||||||
// Note: For now we test the Go backend deployment without actual DB connection
|
|
||||||
// as that requires environment variable injection during deployment
|
|
||||||
t.Run("Deploy Go backend", func(t *testing.T) {
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/go-api")
|
|
||||||
|
|
||||||
// Note: In a real implementation, we would pass DATABASE_NAME env var
|
|
||||||
// For now, we just test the deployment mechanism
|
|
||||||
backendID = e2e.CreateTestDeployment(t, env, backendName, tarballPath)
|
|
||||||
|
|
||||||
assert.NotEmpty(t, backendID, "Backend deployment ID should not be empty")
|
|
||||||
t.Logf("✓ Go backend deployed: %s", backendName)
|
|
||||||
|
|
||||||
// Wait for deployment to become active
|
|
||||||
time.Sleep(3 * time.Second)
|
|
||||||
})
|
|
||||||
|
|
||||||
// Step 3: Test database operations
|
|
||||||
t.Run("Test database CRUD operations", func(t *testing.T) {
|
|
||||||
// INSERT
|
|
||||||
insertQuery := `INSERT INTO users (name, email) VALUES ('Bob', 'bob@example.com')`
|
|
||||||
e2e.ExecuteSQLQuery(t, env, dbName, insertQuery)
|
|
||||||
|
|
||||||
// SELECT
|
|
||||||
users := e2e.QuerySQLite(t, env, dbName, "SELECT * FROM users ORDER BY id")
|
|
||||||
require.GreaterOrEqual(t, len(users), 2, "Should have at least 2 users")
|
|
||||||
|
|
||||||
assert.Equal(t, "Alice", users[0]["name"], "First user should be Alice")
|
|
||||||
assert.Equal(t, "Bob", users[1]["name"], "Second user should be Bob")
|
|
||||||
|
|
||||||
t.Logf("✓ Database CRUD operations work")
|
|
||||||
t.Logf(" - Found %d users", len(users))
|
|
||||||
|
|
||||||
// UPDATE
|
|
||||||
updateQuery := `UPDATE users SET email = 'alice.new@example.com' WHERE name = 'Alice'`
|
|
||||||
result := e2e.ExecuteSQLQuery(t, env, dbName, updateQuery)
|
|
||||||
|
|
||||||
rowsAffected, ok := result["rows_affected"].(float64)
|
|
||||||
require.True(t, ok, "Should have rows_affected")
|
|
||||||
assert.Equal(t, float64(1), rowsAffected, "Should update 1 row")
|
|
||||||
|
|
||||||
// Verify update
|
|
||||||
updated := e2e.QuerySQLite(t, env, dbName, "SELECT email FROM users WHERE name = 'Alice'")
|
|
||||||
require.Len(t, updated, 1, "Should find Alice")
|
|
||||||
assert.Equal(t, "alice.new@example.com", updated[0]["email"], "Email should be updated")
|
|
||||||
|
|
||||||
t.Logf("✓ UPDATE operation verified")
|
|
||||||
|
|
||||||
// DELETE
|
|
||||||
deleteQuery := `DELETE FROM users WHERE name = 'Bob'`
|
|
||||||
result = e2e.ExecuteSQLQuery(t, env, dbName, deleteQuery)
|
|
||||||
|
|
||||||
rowsAffected, ok = result["rows_affected"].(float64)
|
|
||||||
require.True(t, ok, "Should have rows_affected")
|
|
||||||
assert.Equal(t, float64(1), rowsAffected, "Should delete 1 row")
|
|
||||||
|
|
||||||
// Verify deletion
|
|
||||||
remaining := e2e.QuerySQLite(t, env, dbName, "SELECT * FROM users")
|
|
||||||
assert.Equal(t, 1, len(remaining), "Should have 1 user remaining")
|
|
||||||
|
|
||||||
t.Logf("✓ DELETE operation verified")
|
|
||||||
})
|
|
||||||
|
|
||||||
// Step 4: Test backend API endpoints (if deployment is active)
|
|
||||||
t.Run("Test backend API endpoints", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, backendID)
|
|
||||||
|
|
||||||
status, ok := deployment["status"].(string)
|
|
||||||
if !ok || status != "active" {
|
|
||||||
t.Skip("Backend deployment not active, skipping API tests")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
backendDomain := env.BuildDeploymentDomain(backendName)
|
|
||||||
|
|
||||||
// Test health endpoint
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, backendDomain, "/health")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
var health map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytes, &health), "Should decode health response")
|
|
||||||
|
|
||||||
assert.Equal(t, "healthy", health["status"], "Status should be healthy")
|
|
||||||
assert.Equal(t, "go-backend-test", health["service"], "Service name should match")
|
|
||||||
|
|
||||||
t.Logf("✓ Backend health check passed")
|
|
||||||
} else {
|
|
||||||
t.Logf("⚠ Health check returned status %d (deployment may still be starting)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Test users API endpoint
|
|
||||||
resp2 := e2e.TestDeploymentWithHostHeader(t, env, backendDomain, "/api/users")
|
|
||||||
defer resp2.Body.Close()
|
|
||||||
|
|
||||||
if resp2.StatusCode == http.StatusOK {
|
|
||||||
var usersResp map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp2.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytes, &usersResp), "Should decode users response")
|
|
||||||
|
|
||||||
users, ok := usersResp["users"].([]interface{})
|
|
||||||
require.True(t, ok, "Should have users array")
|
|
||||||
assert.GreaterOrEqual(t, len(users), 3, "Should have test users")
|
|
||||||
|
|
||||||
t.Logf("✓ Backend API endpoint works")
|
|
||||||
t.Logf(" - Users endpoint returned %d users", len(users))
|
|
||||||
} else {
|
|
||||||
t.Logf("⚠ Users API returned status %d (deployment may still be starting)", resp2.StatusCode)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
// Step 5: Test database backup
|
|
||||||
t.Run("Test database backup", func(t *testing.T) {
|
|
||||||
reqBody := map[string]string{"database_name": dbName}
|
|
||||||
bodyBytes, _ := json.Marshal(reqBody)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("POST", env.GatewayURL+"/v1/db/sqlite/backup", bytes.NewReader(bodyBytes))
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err, "Should execute backup request")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
var result map[string]interface{}
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, json.Unmarshal(bodyBytes, &result), "Should decode backup response")
|
|
||||||
|
|
||||||
backupCID, ok := result["backup_cid"].(string)
|
|
||||||
require.True(t, ok, "Should have backup CID")
|
|
||||||
assert.NotEmpty(t, backupCID, "Backup CID should not be empty")
|
|
||||||
|
|
||||||
t.Logf("✓ Database backup created")
|
|
||||||
t.Logf(" - CID: %s", backupCID)
|
|
||||||
} else {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("⚠ Backup returned status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
// Step 6: Test concurrent database queries
|
|
||||||
t.Run("Test concurrent database reads", func(t *testing.T) {
|
|
||||||
// WAL mode should allow concurrent reads — run sequentially to avoid t.Fatal in goroutines
|
|
||||||
for i := 0; i < 5; i++ {
|
|
||||||
users := e2e.QuerySQLite(t, env, dbName, "SELECT * FROM users")
|
|
||||||
assert.GreaterOrEqual(t, len(users), 0, "Should query successfully")
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Logf("✓ Sequential reads successful")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFullStack_StaticSite_SQLite(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
appName := fmt.Sprintf("fullstack-static-%d", time.Now().Unix())
|
|
||||||
frontendName := appName + "-frontend"
|
|
||||||
dbName := appName + "-db"
|
|
||||||
|
|
||||||
var frontendID string
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
if frontendID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, frontendID)
|
|
||||||
}
|
|
||||||
e2e.DeleteSQLiteDB(t, env, dbName)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy static site and create database", func(t *testing.T) {
|
|
||||||
// Create database
|
|
||||||
e2e.CreateSQLiteDB(t, env, dbName)
|
|
||||||
e2e.ExecuteSQLQuery(t, env, dbName, "CREATE TABLE page_views (id INTEGER PRIMARY KEY, page TEXT, count INTEGER)")
|
|
||||||
e2e.ExecuteSQLQuery(t, env, dbName, "INSERT INTO page_views (page, count) VALUES ('home', 0)")
|
|
||||||
|
|
||||||
// Deploy frontend
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
frontendID = e2e.CreateTestDeployment(t, env, frontendName, tarballPath)
|
|
||||||
|
|
||||||
assert.NotEmpty(t, frontendID, "Frontend deployment should succeed")
|
|
||||||
t.Logf("✓ Static site deployed with SQLite backend")
|
|
||||||
|
|
||||||
// Wait for deployment
|
|
||||||
time.Sleep(2 * time.Second)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test frontend serving and database interaction", func(t *testing.T) {
|
|
||||||
frontendDomain := env.BuildDeploymentDomain(frontendName)
|
|
||||||
|
|
||||||
// Test frontend
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, frontendDomain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Frontend should serve")
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
assert.Contains(t, string(body), "<div id=\"root\">", "Should contain React app")
|
|
||||||
|
|
||||||
// Simulate page view tracking
|
|
||||||
e2e.ExecuteSQLQuery(t, env, dbName, "UPDATE page_views SET count = count + 1 WHERE page = 'home'")
|
|
||||||
|
|
||||||
// Verify count
|
|
||||||
views := e2e.QuerySQLite(t, env, dbName, "SELECT count FROM page_views WHERE page = 'home'")
|
|
||||||
require.Len(t, views, 1, "Should have page view record")
|
|
||||||
|
|
||||||
count, ok := views[0]["count"].(float64)
|
|
||||||
require.True(t, ok, "Count should be a number")
|
|
||||||
assert.Equal(t, float64(1), count, "Page view count should be incremented")
|
|
||||||
|
|
||||||
t.Logf("✓ Full stack integration verified")
|
|
||||||
t.Logf(" - Frontend: %s", frontendDomain)
|
|
||||||
t.Logf(" - Database: %s", dbName)
|
|
||||||
t.Logf(" - Page views tracked: %.0f", count)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,125 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package integration
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestIPFS_ContentPinnedOnMultipleNodes verifies that deploying a static app
|
|
||||||
// makes the IPFS content available across multiple nodes.
|
|
||||||
func TestIPFS_ContentPinnedOnMultipleNodes(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("ipfs-pin-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
time.Sleep(15 * time.Second) // Wait for IPFS content replication
|
|
||||||
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
contentCID, _ := deployment["content_cid"].(string)
|
|
||||||
require.NotEmpty(t, contentCID, "Deployment should have a content CID")
|
|
||||||
|
|
||||||
t.Run("Content served via gateway", func(t *testing.T) {
|
|
||||||
// Extract domain from deployment URLs
|
|
||||||
urls, _ := deployment["urls"].([]interface{})
|
|
||||||
require.NotEmpty(t, urls, "Deployment should have URLs")
|
|
||||||
urlStr, _ := urls[0].(string)
|
|
||||||
domain := urlStr
|
|
||||||
if len(urlStr) > 8 && urlStr[:8] == "https://" {
|
|
||||||
domain = urlStr[8:]
|
|
||||||
} else if len(urlStr) > 7 && urlStr[:7] == "http://" {
|
|
||||||
domain = urlStr[7:]
|
|
||||||
}
|
|
||||||
if len(domain) > 0 && domain[len(domain)-1] == '/' {
|
|
||||||
domain = domain[:len(domain)-1]
|
|
||||||
}
|
|
||||||
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("status=%d, body=%d bytes", resp.StatusCode, len(body))
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"IPFS content should be served via gateway (CID: %s)", contentCID)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestIPFS_LargeFileDeployment verifies that deploying an app with larger
|
|
||||||
// static assets works correctly.
|
|
||||||
func TestIPFS_LargeFileDeployment(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("ipfs-large-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
// The react-vite tarball is our largest test asset
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
time.Sleep(5 * time.Second)
|
|
||||||
|
|
||||||
t.Run("Deployment has valid CID", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
contentCID, _ := deployment["content_cid"].(string)
|
|
||||||
assert.NotEmpty(t, contentCID, "Should have a content CID")
|
|
||||||
assert.True(t, len(contentCID) > 10, "CID should be a valid IPFS hash")
|
|
||||||
t.Logf("Content CID: %s", contentCID)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Static content serves correctly", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
urls, ok := deployment["urls"].([]interface{})
|
|
||||||
if !ok || len(urls) == 0 {
|
|
||||||
t.Skip("No URLs in deployment")
|
|
||||||
}
|
|
||||||
|
|
||||||
nodeURL, _ := urls[0].(string)
|
|
||||||
domain := nodeURL
|
|
||||||
if len(nodeURL) > 8 && nodeURL[:8] == "https://" {
|
|
||||||
domain = nodeURL[8:]
|
|
||||||
} else if len(nodeURL) > 7 && nodeURL[:7] == "http://" {
|
|
||||||
domain = nodeURL[7:]
|
|
||||||
}
|
|
||||||
if len(domain) > 0 && domain[len(domain)-1] == '/' {
|
|
||||||
domain = domain[:len(domain)-1]
|
|
||||||
}
|
|
||||||
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
assert.Greater(t, len(body), 100, "Response should have substantial content")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,136 +0,0 @@
|
|||||||
//go:build e2e && production
|
|
||||||
|
|
||||||
package production
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestCrossNode_ProxyRouting tests that requests routed through the gateway
|
|
||||||
// are served correctly for a deployment.
|
|
||||||
func TestCrossNode_ProxyRouting(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Cross-node testing requires at least 2 servers in config")
|
|
||||||
}
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("proxy-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for deployment to be active
|
|
||||||
time.Sleep(3 * time.Second)
|
|
||||||
|
|
||||||
domain := env.BuildDeploymentDomain(deploymentName)
|
|
||||||
t.Logf("Testing routing for: %s", domain)
|
|
||||||
|
|
||||||
t.Run("Request via gateway succeeds", func(t *testing.T) {
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Request should return 200 (got %d: %s)", resp.StatusCode, string(body))
|
|
||||||
|
|
||||||
assert.Contains(t, string(body), "<div id=\"root\">",
|
|
||||||
"Should serve deployment content")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestCrossNode_APIConsistency tests that API responses are consistent
|
|
||||||
func TestCrossNode_APIConsistency(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("consistency-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for replication
|
|
||||||
time.Sleep(5 * time.Second)
|
|
||||||
|
|
||||||
t.Run("Deployment list contains our deployment", func(t *testing.T) {
|
|
||||||
req, err := http.NewRequest("GET", env.GatewayURL+"/v1/deployments/list", nil)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode)
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&result))
|
|
||||||
|
|
||||||
deployments, ok := result["deployments"].([]interface{})
|
|
||||||
require.True(t, ok, "Response should have deployments array")
|
|
||||||
t.Logf("Gateway reports %d deployments", len(deployments))
|
|
||||||
|
|
||||||
found := false
|
|
||||||
for _, d := range deployments {
|
|
||||||
dep, _ := d.(map[string]interface{})
|
|
||||||
if dep["name"] == deploymentName {
|
|
||||||
found = true
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
assert.True(t, found, "Our deployment should be in the list")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestCrossNode_DeploymentGetConsistency tests that deployment details are correct
|
|
||||||
func TestCrossNode_DeploymentGetConsistency(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("get-consistency-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for replication
|
|
||||||
time.Sleep(5 * time.Second)
|
|
||||||
|
|
||||||
t.Run("Deployment details are correct", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
cid, _ := deployment["content_cid"].(string)
|
|
||||||
assert.NotEmpty(t, cid, "Should have a content CID")
|
|
||||||
|
|
||||||
name, _ := deployment["name"].(string)
|
|
||||||
assert.Equal(t, deploymentName, name, "Name should match")
|
|
||||||
|
|
||||||
t.Logf("Deployment: name=%s, cid=%s, status=%s", name, cid, deployment["status"])
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,228 +0,0 @@
|
|||||||
//go:build e2e && production
|
|
||||||
|
|
||||||
package production
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestFailover_HomeNodeDown verifies that when the home node's deployment process
|
|
||||||
// is down, requests still succeed via the replica node.
|
|
||||||
func TestFailover_HomeNodeDown(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Failover testing requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Deploy a Node.js backend so we have a process to stop
|
|
||||||
deploymentName := fmt.Sprintf("failover-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/node-api")
|
|
||||||
|
|
||||||
deploymentID := createNodeJSDeploymentProd(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for deployment and replica
|
|
||||||
healthy := e2e.WaitForHealthy(t, env, deploymentID, 90*time.Second)
|
|
||||||
require.True(t, healthy, "Deployment should become healthy")
|
|
||||||
time.Sleep(20 * time.Second) // Wait for async replica setup
|
|
||||||
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURLProd(t, deployment)
|
|
||||||
require.NotEmpty(t, nodeURL)
|
|
||||||
domain := extractDomainProd(nodeURL)
|
|
||||||
|
|
||||||
t.Run("Deployment serves via gateway", func(t *testing.T) {
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/health")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Deployment should be served via gateway (got %d: %s)", resp.StatusCode, string(body))
|
|
||||||
t.Logf("Gateway response: status=%d body=%s", resp.StatusCode, string(body))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFailover_5xxRetry verifies that if one node returns a gateway error,
|
|
||||||
// the middleware retries on the next replica.
|
|
||||||
func TestFailover_5xxRetry(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Deploy a static app (always works via IPFS, no process to crash)
|
|
||||||
deploymentName := fmt.Sprintf("retry-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
time.Sleep(10 * time.Second)
|
|
||||||
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
nodeURL := extractNodeURLProd(t, deployment)
|
|
||||||
if nodeURL == "" {
|
|
||||||
t.Skip("No node URL")
|
|
||||||
}
|
|
||||||
domain := extractDomainProd(nodeURL)
|
|
||||||
|
|
||||||
t.Run("Deployment serves successfully", func(t *testing.T) {
|
|
||||||
resp := e2e.TestDeploymentWithHostHeader(t, env, domain, "/")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Static content should be served (got %d: %s)", resp.StatusCode, string(body))
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestFailover_CrossNodeProxyTimeout verifies that cross-node proxy fails fast
|
|
||||||
// (within a reasonable timeout) rather than hanging.
|
|
||||||
func TestFailover_CrossNodeProxyTimeout(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
if len(env.Config.Servers) < 2 {
|
|
||||||
t.Skip("Requires at least 2 servers")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Make a request to a non-existent deployment — should fail fast
|
|
||||||
domain := fmt.Sprintf("nonexistent-%d.%s", time.Now().Unix(), env.BaseDomain)
|
|
||||||
|
|
||||||
start := time.Now()
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/", nil)
|
|
||||||
req.Host = domain
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
elapsed := time.Since(start)
|
|
||||||
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("Request failed in %v: %v", elapsed, err)
|
|
||||||
} else {
|
|
||||||
resp.Body.Close()
|
|
||||||
t.Logf("Got status %d in %v", resp.StatusCode, elapsed)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Should respond within 15 seconds (our proxy timeout is 5s)
|
|
||||||
assert.Less(t, elapsed.Seconds(), 15.0,
|
|
||||||
"Request to non-existent deployment should fail fast, took %v", elapsed)
|
|
||||||
}
|
|
||||||
|
|
||||||
func createNodeJSDeploymentProd(t *testing.T, env *e2e.E2ETestEnv, name, tarballPath string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
var fileData []byte
|
|
||||||
|
|
||||||
info, err := os.Stat(tarballPath)
|
|
||||||
require.NoError(t, err, "Failed to stat: %s", tarballPath)
|
|
||||||
|
|
||||||
if info.IsDir() {
|
|
||||||
tarData, err := exec.Command("tar", "-czf", "-", "-C", tarballPath, ".").Output()
|
|
||||||
require.NoError(t, err, "Failed to create tarball from %s", tarballPath)
|
|
||||||
fileData = tarData
|
|
||||||
} else {
|
|
||||||
file, err := os.Open(tarballPath)
|
|
||||||
require.NoError(t, err, "Failed to open tarball: %s", tarballPath)
|
|
||||||
defer file.Close()
|
|
||||||
fileData, _ = io.ReadAll(file)
|
|
||||||
}
|
|
||||||
|
|
||||||
body := &bytes.Buffer{}
|
|
||||||
boundary := "----WebKitFormBoundary7MA4YWxkTrZu0gW"
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"name\"\r\n\r\n")
|
|
||||||
body.WriteString(name + "\r\n")
|
|
||||||
|
|
||||||
body.WriteString("--" + boundary + "\r\n")
|
|
||||||
body.WriteString("Content-Disposition: form-data; name=\"tarball\"; filename=\"app.tar.gz\"\r\n")
|
|
||||||
body.WriteString("Content-Type: application/gzip\r\n\r\n")
|
|
||||||
|
|
||||||
body.Write(fileData)
|
|
||||||
body.WriteString("\r\n--" + boundary + "--\r\n")
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", env.GatewayURL+"/v1/deployments/nodejs/upload", body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
req.Header.Set("Content-Type", "multipart/form-data; boundary="+boundary)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusCreated {
|
|
||||||
bodyBytes, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Fatalf("Deployment upload failed with status %d: %s", resp.StatusCode, string(bodyBytes))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result map[string]interface{}
|
|
||||||
require.NoError(t, json.NewDecoder(resp.Body).Decode(&result))
|
|
||||||
|
|
||||||
if id, ok := result["deployment_id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
if id, ok := result["id"].(string); ok {
|
|
||||||
return id
|
|
||||||
}
|
|
||||||
t.Fatalf("Deployment response missing id: %+v", result)
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractNodeURLProd(t *testing.T, deployment map[string]interface{}) string {
|
|
||||||
t.Helper()
|
|
||||||
if urls, ok := deployment["urls"].([]interface{}); ok && len(urls) > 0 {
|
|
||||||
if url, ok := urls[0].(string); ok {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if urls, ok := deployment["urls"].(map[string]interface{}); ok {
|
|
||||||
if url, ok := urls["node"].(string); ok {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractDomainProd(url string) string {
|
|
||||||
domain := url
|
|
||||||
if len(url) > 8 && url[:8] == "https://" {
|
|
||||||
domain = url[8:]
|
|
||||||
} else if len(url) > 7 && url[:7] == "http://" {
|
|
||||||
domain = url[7:]
|
|
||||||
}
|
|
||||||
if len(domain) > 0 && domain[len(domain)-1] == '/' {
|
|
||||||
domain = domain[:len(domain)-1]
|
|
||||||
}
|
|
||||||
return domain
|
|
||||||
}
|
|
||||||
@ -1,185 +0,0 @@
|
|||||||
//go:build e2e && production
|
|
||||||
|
|
||||||
package production
|
|
||||||
|
|
||||||
import (
|
|
||||||
"crypto/tls"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestHTTPS_CertificateValid tests that HTTPS works with a valid certificate
|
|
||||||
func TestHTTPS_CertificateValid(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("https-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
|
|
||||||
deploymentID := e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
// Wait for deployment and certificate provisioning
|
|
||||||
time.Sleep(5 * time.Second)
|
|
||||||
|
|
||||||
domain := env.BuildDeploymentDomain(deploymentName)
|
|
||||||
httpsURL := fmt.Sprintf("https://%s", domain)
|
|
||||||
|
|
||||||
t.Run("HTTPS connection with certificate verification", func(t *testing.T) {
|
|
||||||
// Create client that DOES verify certificates
|
|
||||||
client := &http.Client{
|
|
||||||
Timeout: 30 * time.Second,
|
|
||||||
Transport: &http.Transport{
|
|
||||||
TLSClientConfig: &tls.Config{
|
|
||||||
// Do NOT skip verification - we want to test real certs
|
|
||||||
InsecureSkipVerify: false,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
req, err := http.NewRequest("GET", httpsURL+"/", nil)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
// Certificate might not be ready yet, or domain might not resolve
|
|
||||||
t.Logf("⚠ HTTPS request failed (this may be expected if certs are still provisioning): %v", err)
|
|
||||||
t.Skip("HTTPS not available or certificate not ready")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
t.Logf("HTTPS returned %d (deployment may not be routed yet): %s", resp.StatusCode, string(body))
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check TLS connection state
|
|
||||||
if resp.TLS != nil {
|
|
||||||
t.Logf("✓ HTTPS works with valid certificate")
|
|
||||||
t.Logf(" - Domain: %s", domain)
|
|
||||||
t.Logf(" - TLS Version: %x", resp.TLS.Version)
|
|
||||||
t.Logf(" - Cipher Suite: %x", resp.TLS.CipherSuite)
|
|
||||||
if len(resp.TLS.PeerCertificates) > 0 {
|
|
||||||
cert := resp.TLS.PeerCertificates[0]
|
|
||||||
t.Logf(" - Certificate Subject: %s", cert.Subject)
|
|
||||||
t.Logf(" - Certificate Issuer: %s", cert.Issuer)
|
|
||||||
t.Logf(" - Valid Until: %s", cert.NotAfter)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHTTPS_CertificateDetails tests certificate properties
|
|
||||||
func TestHTTPS_CertificateDetails(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
t.Run("Base domain certificate", func(t *testing.T) {
|
|
||||||
httpsURL := fmt.Sprintf("https://%s", env.BaseDomain)
|
|
||||||
|
|
||||||
// Connect and get certificate info
|
|
||||||
conn, err := tls.Dial("tcp", env.BaseDomain+":443", &tls.Config{
|
|
||||||
InsecureSkipVerify: true, // We just want to inspect the cert
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("⚠ Could not connect to %s:443: %v", env.BaseDomain, err)
|
|
||||||
t.Skip("HTTPS not available on base domain")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer conn.Close()
|
|
||||||
|
|
||||||
certs := conn.ConnectionState().PeerCertificates
|
|
||||||
require.NotEmpty(t, certs, "Should have certificates")
|
|
||||||
|
|
||||||
cert := certs[0]
|
|
||||||
t.Logf("Certificate for %s:", env.BaseDomain)
|
|
||||||
t.Logf(" - Subject: %s", cert.Subject)
|
|
||||||
t.Logf(" - DNS Names: %v", cert.DNSNames)
|
|
||||||
t.Logf(" - Valid From: %s", cert.NotBefore)
|
|
||||||
t.Logf(" - Valid Until: %s", cert.NotAfter)
|
|
||||||
t.Logf(" - Issuer: %s", cert.Issuer)
|
|
||||||
|
|
||||||
// Check that certificate covers our domain
|
|
||||||
coversDomain := false
|
|
||||||
for _, name := range cert.DNSNames {
|
|
||||||
if name == env.BaseDomain || name == "*."+env.BaseDomain {
|
|
||||||
coversDomain = true
|
|
||||||
break
|
|
||||||
}
|
|
||||||
}
|
|
||||||
assert.True(t, coversDomain, "Certificate should cover %s", env.BaseDomain)
|
|
||||||
|
|
||||||
// Check certificate is not expired
|
|
||||||
assert.True(t, time.Now().Before(cert.NotAfter), "Certificate should not be expired")
|
|
||||||
assert.True(t, time.Now().After(cert.NotBefore), "Certificate should be valid now")
|
|
||||||
|
|
||||||
// Make actual HTTPS request to verify it works
|
|
||||||
client := &http.Client{
|
|
||||||
Timeout: 30 * time.Second,
|
|
||||||
Transport: &http.Transport{
|
|
||||||
TLSClientConfig: &tls.Config{
|
|
||||||
InsecureSkipVerify: false,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Get(httpsURL)
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("⚠ HTTPS request failed: %v", err)
|
|
||||||
} else {
|
|
||||||
resp.Body.Close()
|
|
||||||
t.Logf("✓ HTTPS request succeeded with status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHTTPS_HTTPRedirect tests that HTTP requests are redirected to HTTPS
|
|
||||||
func TestHTTPS_HTTPRedirect(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
t.Run("HTTP redirects to HTTPS", func(t *testing.T) {
|
|
||||||
// Create client that doesn't follow redirects
|
|
||||||
client := &http.Client{
|
|
||||||
Timeout: 30 * time.Second,
|
|
||||||
CheckRedirect: func(req *http.Request, via []*http.Request) error {
|
|
||||||
return http.ErrUseLastResponse
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
httpURL := fmt.Sprintf("http://%s", env.BaseDomain)
|
|
||||||
|
|
||||||
resp, err := client.Get(httpURL)
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("⚠ HTTP request failed: %v", err)
|
|
||||||
t.Skip("HTTP not available or redirects not configured")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Check for redirect
|
|
||||||
if resp.StatusCode >= 300 && resp.StatusCode < 400 {
|
|
||||||
location := resp.Header.Get("Location")
|
|
||||||
t.Logf("✓ HTTP redirects to: %s (status %d)", location, resp.StatusCode)
|
|
||||||
assert.Contains(t, location, "https://", "Should redirect to HTTPS")
|
|
||||||
} else if resp.StatusCode == http.StatusOK {
|
|
||||||
// HTTP might just serve content directly in some configurations
|
|
||||||
t.Logf("⚠ HTTP returned 200 instead of redirect (HTTPS redirect may not be configured)")
|
|
||||||
} else {
|
|
||||||
t.Logf("HTTP returned status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,204 +0,0 @@
|
|||||||
//go:build e2e && production
|
|
||||||
|
|
||||||
package production
|
|
||||||
|
|
||||||
import (
|
|
||||||
"crypto/tls"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestHTTPS_ExternalAccess tests that deployed apps are accessible via HTTPS
|
|
||||||
// from the public internet with valid SSL certificates.
|
|
||||||
//
|
|
||||||
// This test requires:
|
|
||||||
// - Orama deployed on a VPS with a real domain
|
|
||||||
// - DNS properly configured
|
|
||||||
// - Run with: go test -v -tags "e2e production" -run TestHTTPS ./e2e/production/...
|
|
||||||
func TestHTTPS_ExternalAccess(t *testing.T) {
|
|
||||||
// Skip if not configured for external testing
|
|
||||||
externalURL := os.Getenv("ORAMA_EXTERNAL_URL")
|
|
||||||
if externalURL == "" {
|
|
||||||
t.Skip("ORAMA_EXTERNAL_URL not set - skipping external HTTPS test")
|
|
||||||
}
|
|
||||||
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("https-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
// Cleanup after test
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy static app", func(t *testing.T) {
|
|
||||||
deploymentID = e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
t.Logf("Created deployment: %s (ID: %s)", deploymentName, deploymentID)
|
|
||||||
})
|
|
||||||
|
|
||||||
var deploymentDomain string
|
|
||||||
|
|
||||||
t.Run("Get deployment domain", func(t *testing.T) {
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
nodeURL := extractNodeURL(t, deployment)
|
|
||||||
require.NotEmpty(t, nodeURL, "Deployment should have node URL")
|
|
||||||
|
|
||||||
deploymentDomain = extractDomain(nodeURL)
|
|
||||||
t.Logf("Deployment domain: %s", deploymentDomain)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wait for DNS propagation", func(t *testing.T) {
|
|
||||||
// Poll DNS until the domain resolves
|
|
||||||
deadline := time.Now().Add(2 * time.Minute)
|
|
||||||
|
|
||||||
for time.Now().Before(deadline) {
|
|
||||||
ips, err := net.LookupHost(deploymentDomain)
|
|
||||||
if err == nil && len(ips) > 0 {
|
|
||||||
t.Logf("DNS resolved: %s -> %v", deploymentDomain, ips)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
t.Logf("DNS not yet resolved, waiting...")
|
|
||||||
time.Sleep(5 * time.Second)
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Fatalf("DNS did not resolve within timeout for %s", deploymentDomain)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Test HTTPS access with valid certificate", func(t *testing.T) {
|
|
||||||
// Create HTTP client that DOES verify certificates
|
|
||||||
// (no InsecureSkipVerify - we want to test real SSL)
|
|
||||||
client := &http.Client{
|
|
||||||
Timeout: 30 * time.Second,
|
|
||||||
Transport: &http.Transport{
|
|
||||||
TLSClientConfig: &tls.Config{
|
|
||||||
// Use default verification (validates certificate)
|
|
||||||
InsecureSkipVerify: false,
|
|
||||||
},
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
url := fmt.Sprintf("https://%s/", deploymentDomain)
|
|
||||||
t.Logf("Testing HTTPS: %s", url)
|
|
||||||
|
|
||||||
resp, err := client.Get(url)
|
|
||||||
require.NoError(t, err, "HTTPS request should succeed with valid certificate")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode, "Should return 200 OK")
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
// Verify it's our React app
|
|
||||||
assert.Contains(t, string(body), "<div id=\"root\">", "Should serve React app")
|
|
||||||
|
|
||||||
t.Logf("HTTPS test passed: %s returned %d", url, resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Verify SSL certificate details", func(t *testing.T) {
|
|
||||||
conn, err := tls.Dial("tcp", deploymentDomain+":443", nil)
|
|
||||||
require.NoError(t, err, "TLS dial should succeed")
|
|
||||||
defer conn.Close()
|
|
||||||
|
|
||||||
state := conn.ConnectionState()
|
|
||||||
require.NotEmpty(t, state.PeerCertificates, "Should have peer certificates")
|
|
||||||
|
|
||||||
cert := state.PeerCertificates[0]
|
|
||||||
t.Logf("Certificate subject: %s", cert.Subject)
|
|
||||||
t.Logf("Certificate issuer: %s", cert.Issuer)
|
|
||||||
t.Logf("Certificate valid from: %s to %s", cert.NotBefore, cert.NotAfter)
|
|
||||||
|
|
||||||
// Verify certificate is not expired
|
|
||||||
assert.True(t, time.Now().After(cert.NotBefore), "Certificate should be valid (not before)")
|
|
||||||
assert.True(t, time.Now().Before(cert.NotAfter), "Certificate should be valid (not expired)")
|
|
||||||
|
|
||||||
// Verify domain matches
|
|
||||||
err = cert.VerifyHostname(deploymentDomain)
|
|
||||||
assert.NoError(t, err, "Certificate should be valid for domain %s", deploymentDomain)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestHTTPS_DomainFormat verifies deployment URL format
|
|
||||||
func TestHTTPS_DomainFormat(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err, "Failed to load test environment")
|
|
||||||
|
|
||||||
deploymentName := fmt.Sprintf("domain-test-%d", time.Now().Unix())
|
|
||||||
tarballPath := filepath.Join("../../testdata/apps/react-app")
|
|
||||||
var deploymentID string
|
|
||||||
|
|
||||||
// Cleanup after test
|
|
||||||
defer func() {
|
|
||||||
if !env.SkipCleanup && deploymentID != "" {
|
|
||||||
e2e.DeleteDeployment(t, env, deploymentID)
|
|
||||||
}
|
|
||||||
}()
|
|
||||||
|
|
||||||
t.Run("Deploy app and verify domain format", func(t *testing.T) {
|
|
||||||
deploymentID = e2e.CreateTestDeployment(t, env, deploymentName, tarballPath)
|
|
||||||
require.NotEmpty(t, deploymentID)
|
|
||||||
|
|
||||||
deployment := e2e.GetDeployment(t, env, deploymentID)
|
|
||||||
|
|
||||||
t.Logf("Deployment URLs: %+v", deployment["urls"])
|
|
||||||
|
|
||||||
// Get deployment URL (handles both array and map formats)
|
|
||||||
deploymentURL := extractNodeURL(t, deployment)
|
|
||||||
assert.NotEmpty(t, deploymentURL, "Should have deployment URL")
|
|
||||||
|
|
||||||
// URL should be simple format: {name}.{baseDomain} (NOT {name}.node-{shortID}.{baseDomain})
|
|
||||||
if deploymentURL != "" {
|
|
||||||
assert.NotContains(t, deploymentURL, ".node-", "URL should NOT contain node identifier (simplified format)")
|
|
||||||
assert.Contains(t, deploymentURL, deploymentName, "URL should contain deployment name")
|
|
||||||
t.Logf("Deployment URL: %s", deploymentURL)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractNodeURL(t *testing.T, deployment map[string]interface{}) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
if urls, ok := deployment["urls"].([]interface{}); ok && len(urls) > 0 {
|
|
||||||
if url, ok := urls[0].(string); ok {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if urls, ok := deployment["urls"].(map[string]interface{}); ok {
|
|
||||||
if url, ok := urls["node"].(string); ok {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
func extractDomain(url string) string {
|
|
||||||
domain := url
|
|
||||||
if len(url) > 8 && url[:8] == "https://" {
|
|
||||||
domain = url[8:]
|
|
||||||
} else if len(url) > 7 && url[:7] == "http://" {
|
|
||||||
domain = url[7:]
|
|
||||||
}
|
|
||||||
if len(domain) > 0 && domain[len(domain)-1] == '/' {
|
|
||||||
domain = domain[:len(domain)-1]
|
|
||||||
}
|
|
||||||
return domain
|
|
||||||
}
|
|
||||||
@ -1,95 +0,0 @@
|
|||||||
//go:build e2e && production
|
|
||||||
|
|
||||||
package production
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestMiddleware_NonExistentDeployment verifies that requests to a non-existent
|
|
||||||
// deployment return 404 (not 502 or hang).
|
|
||||||
func TestMiddleware_NonExistentDeployment(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
domain := fmt.Sprintf("does-not-exist-%d.%s", time.Now().Unix(), env.BaseDomain)
|
|
||||||
|
|
||||||
req, _ := http.NewRequest("GET", env.GatewayURL+"/", nil)
|
|
||||||
req.Host = domain
|
|
||||||
|
|
||||||
start := time.Now()
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
elapsed := time.Since(start)
|
|
||||||
|
|
||||||
if err != nil {
|
|
||||||
t.Logf("Request failed in %v: %v", elapsed, err)
|
|
||||||
// Connection refused or timeout is acceptable
|
|
||||||
assert.Less(t, elapsed.Seconds(), 15.0, "Should fail fast")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
t.Logf("Status: %d, elapsed: %v, body: %s", resp.StatusCode, elapsed, string(body))
|
|
||||||
|
|
||||||
// Should be 404 or 502, not 200
|
|
||||||
assert.NotEqual(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Non-existent deployment should not return 200")
|
|
||||||
assert.Less(t, elapsed.Seconds(), 15.0, "Should respond fast")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestMiddleware_InternalAPIAuthRejection verifies that internal replica API
|
|
||||||
// endpoints reject requests without the proper internal auth header.
|
|
||||||
func TestMiddleware_InternalAPIAuthRejection(t *testing.T) {
|
|
||||||
env, err := e2e.LoadTestEnv()
|
|
||||||
require.NoError(t, err)
|
|
||||||
|
|
||||||
t.Run("No auth header rejected", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("POST",
|
|
||||||
env.GatewayURL+"/v1/internal/deployments/replica/setup", nil)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Should be rejected (401 or 403)
|
|
||||||
assert.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden,
|
|
||||||
"Internal API without auth should be rejected (got %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Wrong auth header rejected", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("POST",
|
|
||||||
env.GatewayURL+"/v1/internal/deployments/replica/setup", nil)
|
|
||||||
req.Header.Set("X-Orama-Internal-Auth", "wrong-token")
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden || resp.StatusCode == http.StatusBadRequest,
|
|
||||||
"Internal API with wrong auth should be rejected (got %d)", resp.StatusCode)
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("Regular API key does not grant internal access", func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("POST",
|
|
||||||
env.GatewayURL+"/v1/internal/deployments/replica/setup", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env.APIKey)
|
|
||||||
|
|
||||||
resp, err := env.HTTPClient.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// The request may pass auth but fail on bad body — 400 is acceptable
|
|
||||||
// But it should NOT succeed with 200
|
|
||||||
assert.NotEqual(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"Regular API key should not fully authenticate internal endpoints")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,148 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package shared
|
|
||||||
|
|
||||||
import (
|
|
||||||
"net/http"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/e2e"
|
|
||||||
"github.com/stretchr/testify/assert"
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// TestAuth_ExpiredOrInvalidJWT verifies that an expired/invalid JWT token is rejected.
|
|
||||||
func TestAuth_ExpiredOrInvalidJWT(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
|
|
||||||
// Craft an obviously invalid JWT
|
|
||||||
invalidJWT := "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwiZXhwIjoxfQ.invalid"
|
|
||||||
|
|
||||||
req, err := http.NewRequest("GET", gatewayURL+"/v1/deployments/list", nil)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+invalidJWT)
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode,
|
|
||||||
"Invalid JWT should return 401")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestAuth_EmptyAPIKey verifies that an empty API key is rejected.
|
|
||||||
func TestAuth_EmptyAPIKey(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
|
|
||||||
req, err := http.NewRequest("GET", gatewayURL+"/v1/deployments/list", nil)
|
|
||||||
require.NoError(t, err)
|
|
||||||
req.Header.Set("Authorization", "Bearer ")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode,
|
|
||||||
"Empty API key should return 401")
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestAuth_SQLInjectionInAPIKey verifies that SQL injection in the API key
|
|
||||||
// does not bypass authentication.
|
|
||||||
func TestAuth_SQLInjectionInAPIKey(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
|
|
||||||
injectionAttempts := []string{
|
|
||||||
"' OR '1'='1",
|
|
||||||
"'; DROP TABLE api_keys; --",
|
|
||||||
"\" OR \"1\"=\"1",
|
|
||||||
"admin'--",
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, attempt := range injectionAttempts {
|
|
||||||
t.Run(attempt, func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", gatewayURL+"/v1/deployments/list", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+attempt)
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusUnauthorized, resp.StatusCode,
|
|
||||||
"SQL injection attempt should be rejected")
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestAuth_NamespaceScopedAccess verifies that an API key for one namespace
|
|
||||||
// cannot access another namespace's deployments.
|
|
||||||
func TestAuth_NamespaceScopedAccess(t *testing.T) {
|
|
||||||
// Create two environments with different namespaces
|
|
||||||
env1, err := e2e.LoadTestEnvWithNamespace("auth-test-ns1")
|
|
||||||
if err != nil {
|
|
||||||
t.Skip("Could not create namespace env1: " + err.Error())
|
|
||||||
}
|
|
||||||
|
|
||||||
env2, err := e2e.LoadTestEnvWithNamespace("auth-test-ns2")
|
|
||||||
if err != nil {
|
|
||||||
t.Skip("Could not create namespace env2: " + err.Error())
|
|
||||||
}
|
|
||||||
|
|
||||||
t.Run("Namespace 1 key cannot list namespace 2 deployments", func(t *testing.T) {
|
|
||||||
// Use env1's API key to query env2's gateway
|
|
||||||
// The namespace should be scoped to the API key
|
|
||||||
req, _ := http.NewRequest("GET", env2.GatewayURL+"/v1/deployments/list", nil)
|
|
||||||
req.Header.Set("Authorization", "Bearer "+env1.APIKey)
|
|
||||||
req.Header.Set("X-Namespace", "auth-test-ns2")
|
|
||||||
|
|
||||||
resp, err := env1.HTTPClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Skip("Gateway unreachable")
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// The API should either reject (403) or return only ns1's deployments
|
|
||||||
t.Logf("Cross-namespace access returned: %d", resp.StatusCode)
|
|
||||||
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
t.Log("API returned 200 — namespace isolation may be enforced at data level")
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestAuth_PublicEndpointsNoAuth verifies that health/status endpoints
|
|
||||||
// don't require authentication.
|
|
||||||
func TestAuth_PublicEndpointsNoAuth(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
|
|
||||||
publicPaths := []string{
|
|
||||||
"/v1/health",
|
|
||||||
"/v1/status",
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, path := range publicPaths {
|
|
||||||
t.Run(path, func(t *testing.T) {
|
|
||||||
req, _ := http.NewRequest("GET", gatewayURL+path, nil)
|
|
||||||
// No auth header
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err)
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
assert.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"%s should be accessible without auth", path)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,333 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package shared_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"net/http"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
"unicode"
|
|
||||||
|
|
||||||
e2e "github.com/DeBrosOfficial/network/e2e"
|
|
||||||
|
|
||||||
"github.com/stretchr/testify/require"
|
|
||||||
)
|
|
||||||
|
|
||||||
// =============================================================================
|
|
||||||
// STRICT AUTHENTICATION NEGATIVE TESTS
|
|
||||||
// These tests verify that authentication is properly enforced.
|
|
||||||
// Tests FAIL if unauthenticated/invalid requests are allowed through.
|
|
||||||
// =============================================================================
|
|
||||||
|
|
||||||
func TestAuth_MissingAPIKey(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request protected endpoint without auth headers
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/cache/health", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// STRICT: Must reject requests without authentication
|
|
||||||
require.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden,
|
|
||||||
"FAIL: Protected endpoint allowed request without auth - expected 401/403, got %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Missing API key correctly rejected with status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_InvalidAPIKey(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request with invalid API key
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/cache/health", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
req.Header.Set("Authorization", "Bearer invalid-key-xyz-123456789")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// STRICT: Must reject invalid API keys
|
|
||||||
require.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden,
|
|
||||||
"FAIL: Invalid API key was accepted - expected 401/403, got %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Invalid API key correctly rejected with status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_CacheWithoutAuth(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request cache endpoint without auth
|
|
||||||
req := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/cache/health",
|
|
||||||
SkipAuth: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := req.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
|
|
||||||
// STRICT: Cache endpoint must require authentication
|
|
||||||
require.True(t, status == http.StatusUnauthorized || status == http.StatusForbidden,
|
|
||||||
"FAIL: Cache endpoint accessible without auth - expected 401/403, got %d", status)
|
|
||||||
t.Logf(" ✓ Cache endpoint correctly requires auth (status %d)", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_StorageWithoutAuth(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request storage endpoint without auth
|
|
||||||
req := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/storage/status/QmTest",
|
|
||||||
SkipAuth: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := req.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
|
|
||||||
// STRICT: Storage endpoint must require authentication
|
|
||||||
require.True(t, status == http.StatusUnauthorized || status == http.StatusForbidden,
|
|
||||||
"FAIL: Storage endpoint accessible without auth - expected 401/403, got %d", status)
|
|
||||||
t.Logf(" ✓ Storage endpoint correctly requires auth (status %d)", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_RQLiteWithoutAuth(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request rqlite endpoint without auth
|
|
||||||
req := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/rqlite/schema",
|
|
||||||
SkipAuth: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := req.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
|
|
||||||
// STRICT: RQLite endpoint must require authentication
|
|
||||||
require.True(t, status == http.StatusUnauthorized || status == http.StatusForbidden,
|
|
||||||
"FAIL: RQLite endpoint accessible without auth - expected 401/403, got %d", status)
|
|
||||||
t.Logf(" ✓ RQLite endpoint correctly requires auth (status %d)", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_MalformedBearerToken(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request with malformed bearer token (missing "Bearer " prefix)
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/cache/health", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
req.Header.Set("Authorization", "invalid-token-format-no-bearer")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// STRICT: Must reject malformed authorization headers
|
|
||||||
require.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden,
|
|
||||||
"FAIL: Malformed auth header accepted - expected 401/403, got %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Malformed bearer token correctly rejected (status %d)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_ExpiredJWT(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Test with a clearly invalid JWT structure
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/cache/health", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
req.Header.Set("Authorization", "Bearer expired.jwt.token.invalid")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// STRICT: Must reject invalid/expired JWT tokens
|
|
||||||
require.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden,
|
|
||||||
"FAIL: Invalid JWT accepted - expected 401/403, got %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Invalid JWT correctly rejected (status %d)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_EmptyBearerToken(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request with empty bearer token
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/cache/health", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
req.Header.Set("Authorization", "Bearer ")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// STRICT: Must reject empty bearer tokens
|
|
||||||
require.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden,
|
|
||||||
"FAIL: Empty bearer token accepted - expected 401/403, got %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Empty bearer token correctly rejected (status %d)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_DuplicateAuthHeaders(t *testing.T) {
|
|
||||||
if e2e.GetAPIKey() == "" {
|
|
||||||
t.Skip("No API key configured")
|
|
||||||
}
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request with both valid API key in Authorization header
|
|
||||||
req := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/cache/health",
|
|
||||||
Headers: map[string]string{
|
|
||||||
"Authorization": "Bearer " + e2e.GetAPIKey(),
|
|
||||||
"X-API-Key": e2e.GetAPIKey(),
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := req.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
|
|
||||||
// Should succeed since we have a valid API key
|
|
||||||
require.Equal(t, http.StatusOK, status,
|
|
||||||
"FAIL: Valid API key rejected when multiple auth headers present - got %d", status)
|
|
||||||
t.Logf(" ✓ Duplicate auth headers with valid key succeeds (status %d)", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_CaseSensitiveAPIKey(t *testing.T) {
|
|
||||||
apiKey := e2e.GetAPIKey()
|
|
||||||
if apiKey == "" {
|
|
||||||
t.Skip("No API key configured")
|
|
||||||
}
|
|
||||||
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Create incorrectly cased API key
|
|
||||||
incorrectKey := ""
|
|
||||||
for i, ch := range apiKey {
|
|
||||||
if i%2 == 0 && unicode.IsLetter(ch) {
|
|
||||||
if unicode.IsLower(ch) {
|
|
||||||
incorrectKey += string(unicode.ToUpper(ch))
|
|
||||||
} else {
|
|
||||||
incorrectKey += string(unicode.ToLower(ch))
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
incorrectKey += string(ch)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Skip if the key didn't change (no letters)
|
|
||||||
if incorrectKey == apiKey {
|
|
||||||
t.Skip("API key has no letters to change case")
|
|
||||||
}
|
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/cache/health", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
req.Header.Set("Authorization", "Bearer "+incorrectKey)
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// STRICT: API keys MUST be case-sensitive
|
|
||||||
require.True(t, resp.StatusCode == http.StatusUnauthorized || resp.StatusCode == http.StatusForbidden,
|
|
||||||
"FAIL: API key check is not case-sensitive - modified key accepted with status %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Case-modified API key correctly rejected (status %d)", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_HealthEndpointNoAuth(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Health endpoint at /v1/health should NOT require auth
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/health", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Health endpoint should be publicly accessible
|
|
||||||
require.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"FAIL: Health endpoint should not require auth - expected 200, got %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Health endpoint correctly accessible without auth")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_StatusEndpointNoAuth(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Status endpoint at /v1/status should NOT require auth
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, e2e.GetGatewayURL()+"/v1/status", nil)
|
|
||||||
require.NoError(t, err, "FAIL: Could not create request")
|
|
||||||
|
|
||||||
client := e2e.NewHTTPClient(30 * time.Second)
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Status endpoint should be publicly accessible
|
|
||||||
require.Equal(t, http.StatusOK, resp.StatusCode,
|
|
||||||
"FAIL: Status endpoint should not require auth - expected 200, got %d", resp.StatusCode)
|
|
||||||
t.Logf(" ✓ Status endpoint correctly accessible without auth")
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_DeploymentsWithoutAuth(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request deployments endpoint without auth
|
|
||||||
req := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/deployments/list",
|
|
||||||
SkipAuth: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := req.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
|
|
||||||
// STRICT: Deployments endpoint must require authentication
|
|
||||||
require.True(t, status == http.StatusUnauthorized || status == http.StatusForbidden,
|
|
||||||
"FAIL: Deployments endpoint accessible without auth - expected 401/403, got %d", status)
|
|
||||||
t.Logf(" ✓ Deployments endpoint correctly requires auth (status %d)", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAuth_SQLiteWithoutAuth(t *testing.T) {
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
|
||||||
defer cancel()
|
|
||||||
|
|
||||||
// Request SQLite endpoint without auth
|
|
||||||
req := &e2e.HTTPRequest{
|
|
||||||
Method: http.MethodGet,
|
|
||||||
URL: e2e.GetGatewayURL() + "/v1/db/sqlite/list",
|
|
||||||
SkipAuth: true,
|
|
||||||
}
|
|
||||||
|
|
||||||
_, status, err := req.Do(ctx)
|
|
||||||
require.NoError(t, err, "FAIL: Request failed")
|
|
||||||
|
|
||||||
// STRICT: SQLite endpoint must require authentication
|
|
||||||
require.True(t, status == http.StatusUnauthorized || status == http.StatusForbidden,
|
|
||||||
"FAIL: SQLite endpoint accessible without auth - expected 401/403, got %d", status)
|
|
||||||
t.Logf(" ✓ SQLite endpoint correctly requires auth (status %d)", status)
|
|
||||||
}
|
|
||||||
@ -1,241 +0,0 @@
|
|||||||
//go:build e2e
|
|
||||||
|
|
||||||
package shared_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
e2e "github.com/DeBrosOfficial/network/e2e"
|
|
||||||
)
|
|
||||||
|
|
||||||
// turnCredentialsResponse is the expected response from the TURN credentials endpoint.
|
|
||||||
type turnCredentialsResponse struct {
|
|
||||||
URLs []string `json:"urls"`
|
|
||||||
Username string `json:"username"`
|
|
||||||
Credential string `json:"credential"`
|
|
||||||
TTL int `json:"ttl"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestWebRTC_TURNCredentials_RequiresAuth verifies that the TURN credentials endpoint
|
|
||||||
// rejects unauthenticated requests.
|
|
||||||
func TestWebRTC_TURNCredentials_RequiresAuth(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", gatewayURL+"/v1/webrtc/turn/credentials", nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("request failed: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusUnauthorized {
|
|
||||||
t.Fatalf("expected 401 Unauthorized, got %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestWebRTC_TURNCredentials_ValidResponse verifies that authenticated requests to the
|
|
||||||
// TURN credentials endpoint return a valid credential structure.
|
|
||||||
func TestWebRTC_TURNCredentials_ValidResponse(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
apiKey := e2e.GetAPIKey()
|
|
||||||
if apiKey == "" {
|
|
||||||
t.Skip("no API key configured")
|
|
||||||
}
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
|
|
||||||
req, err := http.NewRequest("POST", gatewayURL+"/v1/webrtc/turn/credentials", nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("request failed: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
t.Fatalf("expected 200 OK, got %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
var creds turnCredentialsResponse
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&creds); err != nil {
|
|
||||||
t.Fatalf("failed to decode response: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(creds.URLs) == 0 {
|
|
||||||
t.Fatal("expected at least one TURN URL")
|
|
||||||
}
|
|
||||||
if creds.Username == "" {
|
|
||||||
t.Fatal("expected non-empty username")
|
|
||||||
}
|
|
||||||
if creds.Credential == "" {
|
|
||||||
t.Fatal("expected non-empty credential")
|
|
||||||
}
|
|
||||||
if creds.TTL <= 0 {
|
|
||||||
t.Fatalf("expected positive TTL, got %d", creds.TTL)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestWebRTC_Rooms_RequiresAuth verifies that the rooms endpoint rejects unauthenticated requests.
|
|
||||||
func TestWebRTC_Rooms_RequiresAuth(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
|
|
||||||
req, err := http.NewRequest("GET", gatewayURL+"/v1/webrtc/rooms", nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("request failed: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusUnauthorized {
|
|
||||||
t.Fatalf("expected 401 Unauthorized, got %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestWebRTC_Signal_RequiresAuth verifies that the signaling WebSocket rejects
|
|
||||||
// unauthenticated connections.
|
|
||||||
func TestWebRTC_Signal_RequiresAuth(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
|
|
||||||
// Use regular HTTP GET to the signal endpoint — without auth it should return 401
|
|
||||||
// before WebSocket upgrade
|
|
||||||
req, err := http.NewRequest("GET", gatewayURL+"/v1/webrtc/signal?room=test-room", nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("request failed: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusUnauthorized {
|
|
||||||
t.Fatalf("expected 401, got %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestWebRTC_Rooms_CreateAndList verifies room creation and listing with proper auth.
|
|
||||||
func TestWebRTC_Rooms_CreateAndList(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
apiKey := e2e.GetAPIKey()
|
|
||||||
if apiKey == "" {
|
|
||||||
t.Skip("no API key configured")
|
|
||||||
}
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
|
|
||||||
roomID := e2e.GenerateUniqueID("e2e-webrtc-room")
|
|
||||||
|
|
||||||
// Create room
|
|
||||||
createBody, _ := json.Marshal(map[string]string{"room_id": roomID})
|
|
||||||
req, err := http.NewRequest("POST", gatewayURL+"/v1/webrtc/rooms", bytes.NewReader(createBody))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("create room failed: %v", err)
|
|
||||||
}
|
|
||||||
resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusCreated {
|
|
||||||
t.Fatalf("expected 200/201, got %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
// List rooms
|
|
||||||
req, err = http.NewRequest("GET", gatewayURL+"/v1/webrtc/rooms", nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
|
||||||
|
|
||||||
resp, err = client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("list rooms failed: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
t.Fatalf("expected 200, got %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Clean up: delete room
|
|
||||||
req, err = http.NewRequest("DELETE", gatewayURL+"/v1/webrtc/rooms?room_id="+roomID, nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
|
||||||
|
|
||||||
resp2, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("delete room failed: %v", err)
|
|
||||||
}
|
|
||||||
resp2.Body.Close()
|
|
||||||
}
|
|
||||||
|
|
||||||
// TestWebRTC_PermissionsPolicy verifies the Permissions-Policy header allows camera and microphone.
|
|
||||||
func TestWebRTC_PermissionsPolicy(t *testing.T) {
|
|
||||||
e2e.SkipIfMissingGateway(t)
|
|
||||||
|
|
||||||
gatewayURL := e2e.GetGatewayURL()
|
|
||||||
apiKey := e2e.GetAPIKey()
|
|
||||||
if apiKey == "" {
|
|
||||||
t.Skip("no API key configured")
|
|
||||||
}
|
|
||||||
client := e2e.NewHTTPClient(10 * time.Second)
|
|
||||||
|
|
||||||
req, err := http.NewRequest("GET", gatewayURL+"/v1/webrtc/rooms", nil)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("failed to create request: %v", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+apiKey)
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("request failed: %v", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
pp := resp.Header.Get("Permissions-Policy")
|
|
||||||
if pp == "" {
|
|
||||||
t.Skip("Permissions-Policy header not set")
|
|
||||||
}
|
|
||||||
|
|
||||||
if !strings.Contains(pp, "camera=(self)") {
|
|
||||||
t.Errorf("Permissions-Policy missing camera=(self), got: %s", pp)
|
|
||||||
}
|
|
||||||
if !strings.Contains(pp, "microphone=(self)") {
|
|
||||||
t.Errorf("Permissions-Policy missing microphone=(self), got: %s", pp)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,77 +0,0 @@
|
|||||||
-- Migration 005: DNS Records for CoreDNS Integration
|
|
||||||
-- This migration creates tables for managing DNS records with RQLite backend for CoreDNS
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
-- DNS records table for dynamic DNS management
|
|
||||||
CREATE TABLE IF NOT EXISTS dns_records (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
fqdn TEXT NOT NULL UNIQUE, -- Fully qualified domain name (e.g., myapp.node-7prvNa.orama.network)
|
|
||||||
record_type TEXT NOT NULL DEFAULT 'A', -- DNS record type: A, AAAA, CNAME, TXT
|
|
||||||
value TEXT NOT NULL, -- IP address or target value
|
|
||||||
ttl INTEGER NOT NULL DEFAULT 300, -- Time to live in seconds
|
|
||||||
namespace TEXT NOT NULL, -- Namespace that owns this record
|
|
||||||
deployment_id TEXT, -- Optional: deployment that created this record
|
|
||||||
node_id TEXT, -- Optional: specific node ID for node-specific routing
|
|
||||||
is_active BOOLEAN NOT NULL DEFAULT TRUE,-- Enable/disable without deleting
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
created_by TEXT NOT NULL -- Wallet address or 'system' for auto-created records
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for fast DNS lookups
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_fqdn ON dns_records(fqdn);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_namespace ON dns_records(namespace);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_deployment ON dns_records(deployment_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_node_id ON dns_records(node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_active ON dns_records(is_active);
|
|
||||||
|
|
||||||
-- DNS nodes registry for tracking active nodes
|
|
||||||
CREATE TABLE IF NOT EXISTS dns_nodes (
|
|
||||||
id TEXT PRIMARY KEY, -- Node ID (e.g., node-7prvNa)
|
|
||||||
ip_address TEXT NOT NULL, -- Public IP address
|
|
||||||
internal_ip TEXT, -- Private IP for cluster communication
|
|
||||||
region TEXT, -- Geographic region
|
|
||||||
status TEXT NOT NULL DEFAULT 'active', -- active, draining, offline
|
|
||||||
last_seen TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
capabilities TEXT, -- JSON: ["wasm", "ipfs", "cache"]
|
|
||||||
metadata TEXT, -- JSON: additional node info
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for node health monitoring
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_nodes_status ON dns_nodes(status);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_nodes_last_seen ON dns_nodes(last_seen);
|
|
||||||
|
|
||||||
-- Reserved domains table to prevent subdomain collisions
|
|
||||||
CREATE TABLE IF NOT EXISTS reserved_domains (
|
|
||||||
domain TEXT PRIMARY KEY,
|
|
||||||
reason TEXT NOT NULL,
|
|
||||||
reserved_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Seed reserved domains
|
|
||||||
INSERT INTO reserved_domains (domain, reason) VALUES
|
|
||||||
('api.orama.network', 'API gateway endpoint'),
|
|
||||||
('www.orama.network', 'Marketing website'),
|
|
||||||
('admin.orama.network', 'Admin panel'),
|
|
||||||
('ns1.orama.network', 'Nameserver 1'),
|
|
||||||
('ns2.orama.network', 'Nameserver 2'),
|
|
||||||
('ns3.orama.network', 'Nameserver 3'),
|
|
||||||
('ns4.orama.network', 'Nameserver 4'),
|
|
||||||
('mail.orama.network', 'Email service'),
|
|
||||||
('cdn.orama.network', 'Content delivery'),
|
|
||||||
('docs.orama.network', 'Documentation'),
|
|
||||||
('status.orama.network', 'Status page')
|
|
||||||
ON CONFLICT(domain) DO NOTHING;
|
|
||||||
|
|
||||||
-- Mark migration as applied
|
|
||||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
|
||||||
version INTEGER PRIMARY KEY,
|
|
||||||
applied_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (5);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,74 +0,0 @@
|
|||||||
-- Migration 006: Per-Namespace SQLite Databases
|
|
||||||
-- This migration creates infrastructure for isolated SQLite databases per namespace
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
-- Namespace SQLite databases registry
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_sqlite_databases (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
namespace TEXT NOT NULL, -- Namespace that owns this database
|
|
||||||
database_name TEXT NOT NULL, -- Database name (unique per namespace)
|
|
||||||
home_node_id TEXT NOT NULL, -- Node ID where database file resides
|
|
||||||
file_path TEXT NOT NULL, -- Absolute path on home node
|
|
||||||
size_bytes BIGINT DEFAULT 0, -- Current database size
|
|
||||||
backup_cid TEXT, -- Latest backup CID in IPFS
|
|
||||||
last_backup_at TIMESTAMP, -- Last backup timestamp
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
created_by TEXT NOT NULL, -- Wallet address that created the database
|
|
||||||
|
|
||||||
UNIQUE(namespace, database_name)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for database lookups
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_sqlite_databases_namespace ON namespace_sqlite_databases(namespace);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_sqlite_databases_home_node ON namespace_sqlite_databases(home_node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_sqlite_databases_name ON namespace_sqlite_databases(namespace, database_name);
|
|
||||||
|
|
||||||
-- SQLite database backups history
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_sqlite_backups (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
database_id TEXT NOT NULL, -- References namespace_sqlite_databases.id
|
|
||||||
backup_cid TEXT NOT NULL, -- IPFS CID of backup file
|
|
||||||
size_bytes BIGINT NOT NULL, -- Backup file size
|
|
||||||
backup_type TEXT NOT NULL, -- 'manual', 'scheduled', 'migration'
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
created_by TEXT NOT NULL,
|
|
||||||
|
|
||||||
FOREIGN KEY (database_id) REFERENCES namespace_sqlite_databases(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Index for backup history queries
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_sqlite_backups_database ON namespace_sqlite_backups(database_id, created_at DESC);
|
|
||||||
|
|
||||||
-- Namespace quotas for resource management (future use)
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_quotas (
|
|
||||||
namespace TEXT PRIMARY KEY,
|
|
||||||
|
|
||||||
-- Storage quotas
|
|
||||||
max_sqlite_databases INTEGER DEFAULT 10, -- Max SQLite databases per namespace
|
|
||||||
max_storage_bytes BIGINT DEFAULT 5368709120, -- 5GB default
|
|
||||||
max_ipfs_pins INTEGER DEFAULT 1000, -- Max pinned IPFS objects
|
|
||||||
|
|
||||||
-- Compute quotas
|
|
||||||
max_deployments INTEGER DEFAULT 20, -- Max concurrent deployments
|
|
||||||
max_cpu_percent INTEGER DEFAULT 200, -- Total CPU quota (2 cores)
|
|
||||||
max_memory_mb INTEGER DEFAULT 2048, -- Total memory quota
|
|
||||||
|
|
||||||
-- Rate limits
|
|
||||||
max_rqlite_queries_per_minute INTEGER DEFAULT 1000,
|
|
||||||
max_olric_ops_per_minute INTEGER DEFAULT 10000,
|
|
||||||
|
|
||||||
-- Current usage (updated periodically)
|
|
||||||
current_storage_bytes BIGINT DEFAULT 0,
|
|
||||||
current_deployments INTEGER DEFAULT 0,
|
|
||||||
current_sqlite_databases INTEGER DEFAULT 0,
|
|
||||||
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Mark migration as applied
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (6);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,178 +0,0 @@
|
|||||||
-- Migration 007: Deployments System
|
|
||||||
-- This migration creates the complete schema for managing custom deployments
|
|
||||||
-- (Static sites, Next.js, Go backends, Node.js backends)
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
-- Main deployments table
|
|
||||||
CREATE TABLE IF NOT EXISTS deployments (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
namespace TEXT NOT NULL, -- Owner namespace
|
|
||||||
name TEXT NOT NULL, -- Deployment name (unique per namespace)
|
|
||||||
type TEXT NOT NULL, -- 'static', 'nextjs', 'nextjs-static', 'go-backend', 'go-wasm', 'nodejs-backend'
|
|
||||||
version INTEGER NOT NULL DEFAULT 1, -- Monotonic version counter
|
|
||||||
status TEXT NOT NULL DEFAULT 'deploying', -- 'deploying', 'active', 'failed', 'stopped', 'updating'
|
|
||||||
|
|
||||||
-- Content storage
|
|
||||||
content_cid TEXT, -- IPFS CID for static content or built assets
|
|
||||||
build_cid TEXT, -- IPFS CID for build artifacts (Next.js SSR, binaries)
|
|
||||||
|
|
||||||
-- Runtime configuration
|
|
||||||
home_node_id TEXT, -- Node ID hosting stateful data/processes
|
|
||||||
port INTEGER, -- Allocated port (NULL for static/WASM)
|
|
||||||
subdomain TEXT, -- Custom subdomain (e.g., myapp)
|
|
||||||
environment TEXT, -- JSON: {"KEY": "value", ...}
|
|
||||||
|
|
||||||
-- Resource limits
|
|
||||||
memory_limit_mb INTEGER DEFAULT 256,
|
|
||||||
cpu_limit_percent INTEGER DEFAULT 50,
|
|
||||||
disk_limit_mb INTEGER DEFAULT 1024,
|
|
||||||
|
|
||||||
-- Health & monitoring
|
|
||||||
health_check_path TEXT DEFAULT '/health', -- HTTP path for health checks
|
|
||||||
health_check_interval INTEGER DEFAULT 30, -- Seconds between health checks
|
|
||||||
restart_policy TEXT DEFAULT 'always', -- 'always', 'on-failure', 'never'
|
|
||||||
max_restart_count INTEGER DEFAULT 10, -- Max restarts before marking as failed
|
|
||||||
|
|
||||||
-- Metadata
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
deployed_by TEXT NOT NULL, -- Wallet address or API key
|
|
||||||
|
|
||||||
UNIQUE(namespace, name)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for deployment lookups
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployments_namespace ON deployments(namespace);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployments_status ON deployments(status);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployments_home_node ON deployments(home_node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployments_type ON deployments(type);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployments_subdomain ON deployments(subdomain);
|
|
||||||
|
|
||||||
-- Port allocations table (prevents port conflicts)
|
|
||||||
CREATE TABLE IF NOT EXISTS port_allocations (
|
|
||||||
node_id TEXT NOT NULL,
|
|
||||||
port INTEGER NOT NULL,
|
|
||||||
deployment_id TEXT NOT NULL,
|
|
||||||
allocated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
PRIMARY KEY (node_id, port),
|
|
||||||
FOREIGN KEY (deployment_id) REFERENCES deployments(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Index for finding allocated ports by node
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_port_allocations_node ON port_allocations(node_id, port);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_port_allocations_deployment ON port_allocations(deployment_id);
|
|
||||||
|
|
||||||
-- Home node assignments (namespace → node mapping)
|
|
||||||
CREATE TABLE IF NOT EXISTS home_node_assignments (
|
|
||||||
namespace TEXT PRIMARY KEY,
|
|
||||||
home_node_id TEXT NOT NULL,
|
|
||||||
assigned_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
last_heartbeat TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
deployment_count INTEGER DEFAULT 0, -- Cached count for capacity planning
|
|
||||||
total_memory_mb INTEGER DEFAULT 0, -- Cached total memory usage
|
|
||||||
total_cpu_percent INTEGER DEFAULT 0 -- Cached total CPU usage
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Index for querying by node
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_home_node_by_node ON home_node_assignments(home_node_id);
|
|
||||||
|
|
||||||
-- Deployment domains (custom domain mapping)
|
|
||||||
CREATE TABLE IF NOT EXISTS deployment_domains (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
deployment_id TEXT NOT NULL,
|
|
||||||
namespace TEXT NOT NULL,
|
|
||||||
domain TEXT NOT NULL UNIQUE, -- Full domain (e.g., myapp.orama.network or custom)
|
|
||||||
routing_type TEXT NOT NULL DEFAULT 'balanced', -- 'balanced' or 'node_specific'
|
|
||||||
node_id TEXT, -- For node_specific routing
|
|
||||||
is_custom BOOLEAN DEFAULT FALSE, -- True for user's own domain
|
|
||||||
tls_cert_cid TEXT, -- IPFS CID for custom TLS certificate
|
|
||||||
verified_at TIMESTAMP, -- When custom domain was verified
|
|
||||||
verification_token TEXT, -- TXT record token for domain verification
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
FOREIGN KEY (deployment_id) REFERENCES deployments(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for domain lookups
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_domains_deployment ON deployment_domains(deployment_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_domains_domain ON deployment_domains(domain);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_domains_namespace ON deployment_domains(namespace);
|
|
||||||
|
|
||||||
-- Deployment history (version tracking and rollback)
|
|
||||||
CREATE TABLE IF NOT EXISTS deployment_history (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
deployment_id TEXT NOT NULL,
|
|
||||||
version INTEGER NOT NULL,
|
|
||||||
content_cid TEXT,
|
|
||||||
build_cid TEXT,
|
|
||||||
deployed_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
deployed_by TEXT NOT NULL,
|
|
||||||
status TEXT NOT NULL DEFAULT 'success', -- 'success', 'failed', 'rolled_back'
|
|
||||||
error_message TEXT,
|
|
||||||
rollback_from_version INTEGER, -- If this is a rollback, original version
|
|
||||||
|
|
||||||
FOREIGN KEY (deployment_id) REFERENCES deployments(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Indexes for history queries
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_history_deployment ON deployment_history(deployment_id, version DESC);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_history_status ON deployment_history(status);
|
|
||||||
|
|
||||||
-- Deployment environment variables (separate for security)
|
|
||||||
CREATE TABLE IF NOT EXISTS deployment_env_vars (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
deployment_id TEXT NOT NULL,
|
|
||||||
key TEXT NOT NULL,
|
|
||||||
value TEXT NOT NULL, -- Encrypted in production
|
|
||||||
is_secret BOOLEAN DEFAULT FALSE, -- True for sensitive values
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
UNIQUE(deployment_id, key),
|
|
||||||
FOREIGN KEY (deployment_id) REFERENCES deployments(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Index for env var lookups
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_env_vars_deployment ON deployment_env_vars(deployment_id);
|
|
||||||
|
|
||||||
-- Deployment events log (audit trail)
|
|
||||||
CREATE TABLE IF NOT EXISTS deployment_events (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
deployment_id TEXT NOT NULL,
|
|
||||||
event_type TEXT NOT NULL, -- 'created', 'started', 'stopped', 'restarted', 'updated', 'deleted', 'health_check_failed'
|
|
||||||
message TEXT,
|
|
||||||
metadata TEXT, -- JSON: additional context
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
created_by TEXT, -- Wallet address or 'system'
|
|
||||||
|
|
||||||
FOREIGN KEY (deployment_id) REFERENCES deployments(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Index for event queries
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_events_deployment ON deployment_events(deployment_id, created_at DESC);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_events_type ON deployment_events(event_type);
|
|
||||||
|
|
||||||
-- Process health checks (for dynamic deployments)
|
|
||||||
CREATE TABLE IF NOT EXISTS deployment_health_checks (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
deployment_id TEXT NOT NULL,
|
|
||||||
node_id TEXT NOT NULL,
|
|
||||||
status TEXT NOT NULL, -- 'healthy', 'unhealthy', 'unknown'
|
|
||||||
response_time_ms INTEGER,
|
|
||||||
status_code INTEGER,
|
|
||||||
error_message TEXT,
|
|
||||||
checked_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
FOREIGN KEY (deployment_id) REFERENCES deployments(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Index for health check queries (keep only recent checks)
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_health_checks_deployment ON deployment_health_checks(deployment_id, checked_at DESC);
|
|
||||||
|
|
||||||
-- Mark migration as applied
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (7);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,31 +0,0 @@
|
|||||||
-- Migration 008: IPFS Namespace Tracking
|
|
||||||
-- This migration adds namespace isolation for IPFS content by tracking CID ownership.
|
|
||||||
|
|
||||||
-- Table: ipfs_content_ownership
|
|
||||||
-- Tracks which namespace owns each CID uploaded to IPFS.
|
|
||||||
-- This enables namespace isolation so that:
|
|
||||||
-- - Namespace-A cannot GET/PIN/UNPIN Namespace-B's content
|
|
||||||
-- - Same CID can be uploaded by different namespaces (shared content)
|
|
||||||
CREATE TABLE IF NOT EXISTS ipfs_content_ownership (
|
|
||||||
id TEXT PRIMARY KEY,
|
|
||||||
cid TEXT NOT NULL,
|
|
||||||
namespace TEXT NOT NULL,
|
|
||||||
name TEXT,
|
|
||||||
size_bytes BIGINT DEFAULT 0,
|
|
||||||
is_pinned BOOLEAN DEFAULT FALSE,
|
|
||||||
uploaded_at TIMESTAMP NOT NULL,
|
|
||||||
uploaded_by TEXT NOT NULL,
|
|
||||||
UNIQUE(cid, namespace)
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Index for fast namespace + CID lookup
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_ipfs_ownership_namespace_cid
|
|
||||||
ON ipfs_content_ownership(namespace, cid);
|
|
||||||
|
|
||||||
-- Index for fast CID lookup across all namespaces
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_ipfs_ownership_cid
|
|
||||||
ON ipfs_content_ownership(cid);
|
|
||||||
|
|
||||||
-- Index for namespace-only queries (list all content for a namespace)
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_ipfs_ownership_namespace
|
|
||||||
ON ipfs_content_ownership(namespace);
|
|
||||||
@ -1,45 +0,0 @@
|
|||||||
-- Migration 009: Update DNS Records to Support Multiple Records per FQDN
|
|
||||||
-- This allows round-robin A records and multiple NS records for the same domain
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
-- SQLite doesn't support DROP CONSTRAINT, so we recreate the table
|
|
||||||
-- First, create the new table structure
|
|
||||||
CREATE TABLE IF NOT EXISTS dns_records_new (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
fqdn TEXT NOT NULL, -- Fully qualified domain name (e.g., myapp.node-7prvNa.orama.network)
|
|
||||||
record_type TEXT NOT NULL DEFAULT 'A',-- DNS record type: A, AAAA, CNAME, TXT, NS, SOA
|
|
||||||
value TEXT NOT NULL, -- IP address or target value
|
|
||||||
ttl INTEGER NOT NULL DEFAULT 300, -- Time to live in seconds
|
|
||||||
priority INTEGER DEFAULT 0, -- Priority for MX/SRV records, or weight for round-robin
|
|
||||||
namespace TEXT NOT NULL DEFAULT 'system', -- Namespace that owns this record
|
|
||||||
deployment_id TEXT, -- Optional: deployment that created this record
|
|
||||||
node_id TEXT, -- Optional: specific node ID for node-specific routing
|
|
||||||
is_active BOOLEAN NOT NULL DEFAULT TRUE,-- Enable/disable without deleting
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
created_by TEXT NOT NULL DEFAULT 'system', -- Wallet address or 'system' for auto-created records
|
|
||||||
UNIQUE(fqdn, record_type, value) -- Allow multiple records of same type for same FQDN, but not duplicates
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Copy existing data if the old table exists
|
|
||||||
INSERT OR IGNORE INTO dns_records_new (id, fqdn, record_type, value, ttl, namespace, deployment_id, node_id, is_active, created_at, updated_at, created_by)
|
|
||||||
SELECT id, fqdn, record_type, value, ttl, namespace, deployment_id, node_id, is_active, created_at, updated_at, created_by
|
|
||||||
FROM dns_records WHERE 1=1;
|
|
||||||
|
|
||||||
-- Drop old table and rename new one
|
|
||||||
DROP TABLE IF EXISTS dns_records;
|
|
||||||
ALTER TABLE dns_records_new RENAME TO dns_records;
|
|
||||||
|
|
||||||
-- Recreate indexes
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_fqdn ON dns_records(fqdn);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_fqdn_type ON dns_records(fqdn, record_type);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_namespace ON dns_records(namespace);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_deployment ON dns_records(deployment_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_node_id ON dns_records(node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_dns_records_active ON dns_records(is_active);
|
|
||||||
|
|
||||||
-- Mark migration as applied
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (9);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,190 +0,0 @@
|
|||||||
-- Migration 010: Namespace Clusters for Physical Isolation
|
|
||||||
-- Creates tables to manage per-namespace RQLite and Olric clusters
|
|
||||||
-- Each namespace gets its own 3-node cluster for complete isolation
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
-- Extend namespaces table with cluster status tracking
|
|
||||||
-- Note: SQLite doesn't support ADD COLUMN IF NOT EXISTS, so we handle this carefully
|
|
||||||
-- These columns track the provisioning state of the namespace's dedicated cluster
|
|
||||||
|
|
||||||
-- First check if columns exist, if not add them
|
|
||||||
-- cluster_status: 'none', 'provisioning', 'ready', 'degraded', 'failed', 'deprovisioning'
|
|
||||||
|
|
||||||
-- Create a new namespaces table with additional columns if needed
|
|
||||||
CREATE TABLE IF NOT EXISTS namespaces_new (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
name TEXT NOT NULL UNIQUE,
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
cluster_status TEXT DEFAULT 'none',
|
|
||||||
cluster_created_at TIMESTAMP,
|
|
||||||
cluster_ready_at TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Copy data from old table if it exists and new columns don't
|
|
||||||
INSERT OR IGNORE INTO namespaces_new (id, name, created_at, cluster_status)
|
|
||||||
SELECT id, name, created_at, 'none' FROM namespaces WHERE NOT EXISTS (
|
|
||||||
SELECT 1 FROM pragma_table_info('namespaces') WHERE name = 'cluster_status'
|
|
||||||
);
|
|
||||||
|
|
||||||
-- If the column already exists, this migration was partially applied - skip the table swap
|
|
||||||
-- We'll use a different approach: just ensure the new tables exist
|
|
||||||
|
|
||||||
-- Namespace clusters registry
|
|
||||||
-- One record per namespace that has a dedicated cluster
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_clusters (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
namespace_id INTEGER NOT NULL UNIQUE, -- FK to namespaces
|
|
||||||
namespace_name TEXT NOT NULL, -- Cached for easier lookups
|
|
||||||
status TEXT NOT NULL DEFAULT 'provisioning', -- provisioning, ready, degraded, failed, deprovisioning
|
|
||||||
|
|
||||||
-- Cluster configuration
|
|
||||||
rqlite_node_count INTEGER NOT NULL DEFAULT 3,
|
|
||||||
olric_node_count INTEGER NOT NULL DEFAULT 3,
|
|
||||||
gateway_node_count INTEGER NOT NULL DEFAULT 3,
|
|
||||||
|
|
||||||
-- Provisioning metadata
|
|
||||||
provisioned_by TEXT NOT NULL, -- Wallet address that triggered provisioning
|
|
||||||
provisioned_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
ready_at TIMESTAMP,
|
|
||||||
last_health_check TIMESTAMP,
|
|
||||||
|
|
||||||
-- Error tracking
|
|
||||||
error_message TEXT,
|
|
||||||
retry_count INTEGER DEFAULT 0,
|
|
||||||
|
|
||||||
FOREIGN KEY (namespace_id) REFERENCES namespaces(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_namespace_clusters_status ON namespace_clusters(status);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_namespace_clusters_namespace ON namespace_clusters(namespace_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_namespace_clusters_name ON namespace_clusters(namespace_name);
|
|
||||||
|
|
||||||
-- Namespace cluster nodes
|
|
||||||
-- Tracks which physical nodes host services for each namespace cluster
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_cluster_nodes (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
namespace_cluster_id TEXT NOT NULL, -- FK to namespace_clusters
|
|
||||||
node_id TEXT NOT NULL, -- FK to dns_nodes (physical node)
|
|
||||||
|
|
||||||
-- Role in the cluster
|
|
||||||
-- Each node can have multiple roles (rqlite + olric + gateway)
|
|
||||||
role TEXT NOT NULL, -- 'rqlite_leader', 'rqlite_follower', 'olric', 'gateway'
|
|
||||||
|
|
||||||
-- Service ports (allocated from reserved range 10000-10099)
|
|
||||||
rqlite_http_port INTEGER, -- Port for RQLite HTTP API
|
|
||||||
rqlite_raft_port INTEGER, -- Port for RQLite Raft consensus
|
|
||||||
olric_http_port INTEGER, -- Port for Olric HTTP API
|
|
||||||
olric_memberlist_port INTEGER, -- Port for Olric memberlist gossip
|
|
||||||
gateway_http_port INTEGER, -- Port for Gateway HTTP
|
|
||||||
|
|
||||||
-- Service status
|
|
||||||
status TEXT NOT NULL DEFAULT 'pending', -- pending, starting, running, stopped, failed
|
|
||||||
process_pid INTEGER, -- PID of running process (for local management)
|
|
||||||
last_heartbeat TIMESTAMP,
|
|
||||||
error_message TEXT,
|
|
||||||
|
|
||||||
-- Join addresses for cluster formation
|
|
||||||
rqlite_join_address TEXT, -- Address to join RQLite cluster
|
|
||||||
olric_peers TEXT, -- JSON array of Olric peer addresses
|
|
||||||
|
|
||||||
-- Metadata
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
UNIQUE(namespace_cluster_id, node_id, role),
|
|
||||||
FOREIGN KEY (namespace_cluster_id) REFERENCES namespace_clusters(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_cluster_nodes_cluster ON namespace_cluster_nodes(namespace_cluster_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_cluster_nodes_node ON namespace_cluster_nodes(node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_cluster_nodes_status ON namespace_cluster_nodes(status);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_cluster_nodes_role ON namespace_cluster_nodes(role);
|
|
||||||
|
|
||||||
-- Namespace port allocations
|
|
||||||
-- Manages the reserved port range (10000-10099) for namespace services
|
|
||||||
-- Each namespace instance on a node gets a block of 5 consecutive ports
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_port_allocations (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
node_id TEXT NOT NULL, -- Physical node ID
|
|
||||||
namespace_cluster_id TEXT NOT NULL, -- Namespace cluster this allocation belongs to
|
|
||||||
|
|
||||||
-- Port block (5 consecutive ports)
|
|
||||||
port_start INTEGER NOT NULL, -- Start of port block (e.g., 10000)
|
|
||||||
port_end INTEGER NOT NULL, -- End of port block (e.g., 10004)
|
|
||||||
|
|
||||||
-- Individual port assignments within the block
|
|
||||||
rqlite_http_port INTEGER NOT NULL, -- port_start + 0
|
|
||||||
rqlite_raft_port INTEGER NOT NULL, -- port_start + 1
|
|
||||||
olric_http_port INTEGER NOT NULL, -- port_start + 2
|
|
||||||
olric_memberlist_port INTEGER NOT NULL, -- port_start + 3
|
|
||||||
gateway_http_port INTEGER NOT NULL, -- port_start + 4
|
|
||||||
|
|
||||||
allocated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
-- Prevent overlapping allocations on same node
|
|
||||||
UNIQUE(node_id, port_start),
|
|
||||||
-- One allocation per namespace per node
|
|
||||||
UNIQUE(namespace_cluster_id, node_id),
|
|
||||||
FOREIGN KEY (namespace_cluster_id) REFERENCES namespace_clusters(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_ns_port_alloc_node ON namespace_port_allocations(node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_ns_port_alloc_cluster ON namespace_port_allocations(namespace_cluster_id);
|
|
||||||
|
|
||||||
-- Namespace cluster events
|
|
||||||
-- Audit log for cluster provisioning and lifecycle events
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_cluster_events (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
namespace_cluster_id TEXT NOT NULL,
|
|
||||||
event_type TEXT NOT NULL, -- Event types listed below
|
|
||||||
node_id TEXT, -- Optional: specific node this event relates to
|
|
||||||
message TEXT,
|
|
||||||
metadata TEXT, -- JSON for additional event data
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
FOREIGN KEY (namespace_cluster_id) REFERENCES namespace_clusters(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
-- Event types:
|
|
||||||
-- 'provisioning_started' - Cluster provisioning began
|
|
||||||
-- 'nodes_selected' - 3 nodes were selected for the cluster
|
|
||||||
-- 'ports_allocated' - Ports allocated on a node
|
|
||||||
-- 'rqlite_started' - RQLite instance started on a node
|
|
||||||
-- 'rqlite_joined' - RQLite instance joined the cluster
|
|
||||||
-- 'rqlite_leader_elected' - RQLite leader election completed
|
|
||||||
-- 'olric_started' - Olric instance started on a node
|
|
||||||
-- 'olric_joined' - Olric instance joined memberlist
|
|
||||||
-- 'gateway_started' - Gateway instance started on a node
|
|
||||||
-- 'dns_created' - DNS records created for namespace
|
|
||||||
-- 'cluster_ready' - All services ready, cluster is operational
|
|
||||||
-- 'cluster_degraded' - One or more nodes are unhealthy
|
|
||||||
-- 'cluster_failed' - Cluster failed to provision or operate
|
|
||||||
-- 'node_failed' - Specific node became unhealthy
|
|
||||||
-- 'node_recovered' - Node recovered from failure
|
|
||||||
-- 'deprovisioning_started' - Cluster deprovisioning began
|
|
||||||
-- 'deprovisioned' - Cluster fully deprovisioned
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_cluster_events_cluster ON namespace_cluster_events(namespace_cluster_id, created_at DESC);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_cluster_events_type ON namespace_cluster_events(event_type);
|
|
||||||
|
|
||||||
-- Global deployment registry
|
|
||||||
-- Prevents duplicate deployment subdomains across all namespaces
|
|
||||||
-- Since deployments now use {name}-{random}.{domain}, we track used subdomains globally
|
|
||||||
CREATE TABLE IF NOT EXISTS global_deployment_subdomains (
|
|
||||||
subdomain TEXT PRIMARY KEY, -- Full subdomain (e.g., 'myapp-f3o4if')
|
|
||||||
namespace TEXT NOT NULL, -- Owner namespace
|
|
||||||
deployment_id TEXT NOT NULL, -- FK to deployments (in namespace cluster)
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
-- No FK to deployments since deployments are in namespace-specific clusters
|
|
||||||
UNIQUE(subdomain)
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_global_subdomains_namespace ON global_deployment_subdomains(namespace);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_global_subdomains_deployment ON global_deployment_subdomains(deployment_id);
|
|
||||||
|
|
||||||
-- Mark migration as applied
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (10);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,19 +0,0 @@
|
|||||||
-- Migration 011: DNS Nameservers Table
|
|
||||||
-- Maps NS hostnames (ns1, ns2, ns3) to specific node IDs and IPs
|
|
||||||
-- Provides stable NS assignment that survives restarts and re-seeding
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
CREATE TABLE IF NOT EXISTS dns_nameservers (
|
|
||||||
hostname TEXT PRIMARY KEY, -- e.g., "ns1", "ns2", "ns3"
|
|
||||||
node_id TEXT NOT NULL, -- Peer ID of the assigned node
|
|
||||||
ip_address TEXT NOT NULL, -- IP address of the assigned node
|
|
||||||
domain TEXT NOT NULL, -- Base domain (e.g., "dbrs.space")
|
|
||||||
assigned_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
UNIQUE(node_id, domain) -- A node can only hold one NS slot per domain
|
|
||||||
);
|
|
||||||
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (11);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,15 +0,0 @@
|
|||||||
-- Deployment replicas: tracks which nodes host replicas of each deployment
|
|
||||||
CREATE TABLE IF NOT EXISTS deployment_replicas (
|
|
||||||
deployment_id TEXT NOT NULL,
|
|
||||||
node_id TEXT NOT NULL,
|
|
||||||
port INTEGER DEFAULT 0,
|
|
||||||
status TEXT NOT NULL DEFAULT 'pending',
|
|
||||||
is_primary BOOLEAN NOT NULL DEFAULT FALSE,
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
PRIMARY KEY (deployment_id, node_id),
|
|
||||||
FOREIGN KEY (deployment_id) REFERENCES deployments(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_replicas_node ON deployment_replicas(node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_deployment_replicas_status ON deployment_replicas(deployment_id, status);
|
|
||||||
@ -1,9 +0,0 @@
|
|||||||
-- WireGuard mesh peer tracking
|
|
||||||
CREATE TABLE IF NOT EXISTS wireguard_peers (
|
|
||||||
node_id TEXT PRIMARY KEY,
|
|
||||||
wg_ip TEXT NOT NULL UNIQUE,
|
|
||||||
public_key TEXT NOT NULL UNIQUE,
|
|
||||||
public_ip TEXT NOT NULL,
|
|
||||||
wg_port INTEGER DEFAULT 51820,
|
|
||||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
@ -1,8 +0,0 @@
|
|||||||
CREATE TABLE IF NOT EXISTS invite_tokens (
|
|
||||||
token TEXT PRIMARY KEY,
|
|
||||||
created_by TEXT NOT NULL,
|
|
||||||
created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
expires_at DATETIME NOT NULL,
|
|
||||||
used_at DATETIME,
|
|
||||||
used_by_ip TEXT
|
|
||||||
);
|
|
||||||
@ -1,3 +0,0 @@
|
|||||||
-- Store IPFS peer IDs alongside WireGuard peers for automatic swarm discovery
|
|
||||||
-- Each node registers its IPFS peer ID so other nodes can connect via ipfs swarm connect
|
|
||||||
ALTER TABLE wireguard_peers ADD COLUMN ipfs_peer_id TEXT DEFAULT '';
|
|
||||||
@ -1,19 +0,0 @@
|
|||||||
-- Migration 016: Node health events for failure detection
|
|
||||||
-- Tracks peer-to-peer health observations for quorum-based dead node detection
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
CREATE TABLE IF NOT EXISTS node_health_events (
|
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
||||||
observer_id TEXT NOT NULL, -- node that detected the failure
|
|
||||||
target_id TEXT NOT NULL, -- node that is suspect/dead
|
|
||||||
status TEXT NOT NULL, -- 'suspect', 'dead', 'recovered'
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_nhe_target_status ON node_health_events(target_id, status);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_nhe_created_at ON node_health_events(created_at);
|
|
||||||
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (16);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,21 +0,0 @@
|
|||||||
-- Migration 017: Phantom auth sessions for QR code + deep link authentication
|
|
||||||
-- Stores session state for the CLI-to-phone relay pattern via the gateway
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
CREATE TABLE IF NOT EXISTS phantom_auth_sessions (
|
|
||||||
id TEXT PRIMARY KEY,
|
|
||||||
namespace TEXT NOT NULL,
|
|
||||||
status TEXT NOT NULL DEFAULT 'pending',
|
|
||||||
wallet TEXT,
|
|
||||||
api_key TEXT,
|
|
||||||
error_message TEXT,
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
expires_at TIMESTAMP NOT NULL
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_phantom_sessions_status ON phantom_auth_sessions(status);
|
|
||||||
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (17);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,96 +0,0 @@
|
|||||||
-- Migration 018: WebRTC Services (SFU + TURN) for Namespace Clusters
|
|
||||||
-- Adds per-namespace WebRTC configuration, room tracking, and port allocation
|
|
||||||
-- WebRTC is opt-in: enabled via `orama namespace enable webrtc`
|
|
||||||
|
|
||||||
BEGIN;
|
|
||||||
|
|
||||||
-- Per-namespace WebRTC configuration
|
|
||||||
-- One row per namespace that has WebRTC enabled
|
|
||||||
CREATE TABLE IF NOT EXISTS namespace_webrtc_config (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
namespace_cluster_id TEXT NOT NULL UNIQUE, -- FK to namespace_clusters
|
|
||||||
namespace_name TEXT NOT NULL, -- Cached for easier lookups
|
|
||||||
enabled INTEGER NOT NULL DEFAULT 1, -- 1 = enabled, 0 = disabled
|
|
||||||
|
|
||||||
-- TURN authentication
|
|
||||||
turn_shared_secret TEXT NOT NULL, -- HMAC-SHA1 shared secret (base64, 32 bytes)
|
|
||||||
turn_credential_ttl INTEGER NOT NULL DEFAULT 600, -- Credential TTL in seconds (default: 10 min)
|
|
||||||
|
|
||||||
-- Service topology
|
|
||||||
sfu_node_count INTEGER NOT NULL DEFAULT 3, -- SFU instances (all 3 nodes)
|
|
||||||
turn_node_count INTEGER NOT NULL DEFAULT 2, -- TURN instances (2 of 3 nodes for HA)
|
|
||||||
|
|
||||||
-- Metadata
|
|
||||||
enabled_by TEXT NOT NULL, -- Wallet address that enabled WebRTC
|
|
||||||
enabled_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
disabled_at TIMESTAMP,
|
|
||||||
|
|
||||||
FOREIGN KEY (namespace_cluster_id) REFERENCES namespace_clusters(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_config_namespace ON namespace_webrtc_config(namespace_name);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_config_cluster ON namespace_webrtc_config(namespace_cluster_id);
|
|
||||||
|
|
||||||
-- WebRTC room tracking
|
|
||||||
-- Tracks active rooms and their SFU node affinity
|
|
||||||
CREATE TABLE IF NOT EXISTS webrtc_rooms (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
namespace_cluster_id TEXT NOT NULL, -- FK to namespace_clusters
|
|
||||||
namespace_name TEXT NOT NULL, -- Cached for easier lookups
|
|
||||||
room_id TEXT NOT NULL, -- Application-defined room identifier
|
|
||||||
|
|
||||||
-- SFU affinity
|
|
||||||
sfu_node_id TEXT NOT NULL, -- Node hosting this room's SFU
|
|
||||||
sfu_internal_ip TEXT NOT NULL, -- WireGuard IP of SFU node
|
|
||||||
sfu_signaling_port INTEGER NOT NULL, -- SFU WebSocket signaling port
|
|
||||||
|
|
||||||
-- Room state
|
|
||||||
participant_count INTEGER NOT NULL DEFAULT 0,
|
|
||||||
max_participants INTEGER NOT NULL DEFAULT 100,
|
|
||||||
created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
last_activity TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
-- Prevent duplicate rooms within a namespace
|
|
||||||
UNIQUE(namespace_cluster_id, room_id),
|
|
||||||
FOREIGN KEY (namespace_cluster_id) REFERENCES namespace_clusters(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_rooms_namespace ON webrtc_rooms(namespace_name);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_rooms_node ON webrtc_rooms(sfu_node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_rooms_activity ON webrtc_rooms(last_activity);
|
|
||||||
|
|
||||||
-- WebRTC port allocations
|
|
||||||
-- Separate from namespace_port_allocations to avoid breaking existing port blocks
|
|
||||||
-- Each namespace gets SFU + TURN ports on each node where those services run
|
|
||||||
CREATE TABLE IF NOT EXISTS webrtc_port_allocations (
|
|
||||||
id TEXT PRIMARY KEY, -- UUID
|
|
||||||
node_id TEXT NOT NULL, -- Physical node ID
|
|
||||||
namespace_cluster_id TEXT NOT NULL, -- FK to namespace_clusters
|
|
||||||
service_type TEXT NOT NULL, -- 'sfu' or 'turn'
|
|
||||||
|
|
||||||
-- SFU ports (when service_type = 'sfu')
|
|
||||||
sfu_signaling_port INTEGER, -- WebSocket signaling port
|
|
||||||
sfu_media_port_start INTEGER, -- Start of RTP media port range
|
|
||||||
sfu_media_port_end INTEGER, -- End of RTP media port range
|
|
||||||
|
|
||||||
-- TURN ports (when service_type = 'turn')
|
|
||||||
turn_listen_port INTEGER, -- TURN listener port (3478)
|
|
||||||
turn_tls_port INTEGER, -- TURN TLS port (443/UDP)
|
|
||||||
turn_relay_port_start INTEGER, -- Start of relay port range
|
|
||||||
turn_relay_port_end INTEGER, -- End of relay port range
|
|
||||||
|
|
||||||
allocated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
|
|
||||||
|
|
||||||
-- Prevent overlapping allocations
|
|
||||||
UNIQUE(node_id, namespace_cluster_id, service_type),
|
|
||||||
FOREIGN KEY (namespace_cluster_id) REFERENCES namespace_clusters(id) ON DELETE CASCADE
|
|
||||||
);
|
|
||||||
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_ports_node ON webrtc_port_allocations(node_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_ports_cluster ON webrtc_port_allocations(namespace_cluster_id);
|
|
||||||
CREATE INDEX IF NOT EXISTS idx_webrtc_ports_type ON webrtc_port_allocations(service_type);
|
|
||||||
|
|
||||||
-- Mark migration as applied
|
|
||||||
INSERT OR IGNORE INTO schema_migrations(version) VALUES (18);
|
|
||||||
|
|
||||||
COMMIT;
|
|
||||||
@ -1,4 +0,0 @@
|
|||||||
-- Invalidate all existing refresh tokens.
|
|
||||||
-- Tokens were stored in plaintext; the application now stores SHA-256 hashes.
|
|
||||||
-- Users will need to re-authenticate (tokens have 30-day expiry anyway).
|
|
||||||
UPDATE refresh_tokens SET revoked_at = datetime('now') WHERE revoked_at IS NULL;
|
|
||||||
@ -1,6 +0,0 @@
|
|||||||
package migrations
|
|
||||||
|
|
||||||
import "embed"
|
|
||||||
|
|
||||||
//go:embed *.sql
|
|
||||||
var FS embed.FS
|
|
||||||
@ -1,350 +0,0 @@
|
|||||||
package auth
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/hex"
|
|
||||||
"os"
|
|
||||||
"strings"
|
|
||||||
"testing"
|
|
||||||
)
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// extractDomainFromURL
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
func TestExtractDomainFromURL(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
input string
|
|
||||||
want string
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "https with domain only",
|
|
||||||
input: "https://example.com",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "http with port and path",
|
|
||||||
input: "http://example.com:8080/path",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "https with subdomain and path",
|
|
||||||
input: "https://sub.domain.com/api/v1",
|
|
||||||
want: "sub.domain.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "no scheme bare domain",
|
|
||||||
input: "example.com",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "https with IP and port",
|
|
||||||
input: "https://192.168.1.1:443",
|
|
||||||
want: "192.168.1.1",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "empty string",
|
|
||||||
input: "",
|
|
||||||
want: "",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "bare domain no scheme",
|
|
||||||
input: "gateway.orama.network",
|
|
||||||
want: "gateway.orama.network",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "https with query params",
|
|
||||||
input: "https://example.com?foo=bar",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "https with path and query params",
|
|
||||||
input: "https://example.com/page?q=1&r=2",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "bare domain with port",
|
|
||||||
input: "example.com:9090",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "https with fragment",
|
|
||||||
input: "https://example.com/page#section",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "https with user info",
|
|
||||||
input: "https://user:pass@example.com/path",
|
|
||||||
want: "example.com",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
got := extractDomainFromURL(tt.input)
|
|
||||||
if got != tt.want {
|
|
||||||
t.Errorf("extractDomainFromURL(%q) = %q, want %q", tt.input, got, tt.want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// ValidateWalletAddress
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
func TestValidateWalletAddress(t *testing.T) {
|
|
||||||
validHex40 := "aabbccddee1122334455aabbccddee1122334455"
|
|
||||||
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
address string
|
|
||||||
want bool
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "valid 40 char hex with 0x prefix",
|
|
||||||
address: "0x" + validHex40,
|
|
||||||
want: true,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "valid 40 char hex without prefix",
|
|
||||||
address: validHex40,
|
|
||||||
want: true,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "valid uppercase hex with 0x prefix",
|
|
||||||
address: "0x" + strings.ToUpper(validHex40),
|
|
||||||
want: true,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "too short",
|
|
||||||
address: "0xaabbccdd",
|
|
||||||
want: false,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "too long",
|
|
||||||
address: "0x" + validHex40 + "ff",
|
|
||||||
want: false,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "non hex characters",
|
|
||||||
address: "0x" + "zzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzz",
|
|
||||||
want: false,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "empty string",
|
|
||||||
address: "",
|
|
||||||
want: false,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "just 0x prefix",
|
|
||||||
address: "0x",
|
|
||||||
want: false,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "39 hex chars with 0x prefix",
|
|
||||||
address: "0x" + validHex40[:39],
|
|
||||||
want: false,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "41 hex chars with 0x prefix",
|
|
||||||
address: "0x" + validHex40 + "a",
|
|
||||||
want: false,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "mixed case hex is valid",
|
|
||||||
address: "0xAaBbCcDdEe1122334455aAbBcCdDeE1122334455",
|
|
||||||
want: true,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
got := ValidateWalletAddress(tt.address)
|
|
||||||
if got != tt.want {
|
|
||||||
t.Errorf("ValidateWalletAddress(%q) = %v, want %v", tt.address, got, tt.want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// FormatWalletAddress
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
func TestFormatWalletAddress(t *testing.T) {
|
|
||||||
tests := []struct {
|
|
||||||
name string
|
|
||||||
address string
|
|
||||||
want string
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
name: "already lowercase with 0x",
|
|
||||||
address: "0xaabbccddee1122334455aabbccddee1122334455",
|
|
||||||
want: "0xaabbccddee1122334455aabbccddee1122334455",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "uppercase gets lowercased",
|
|
||||||
address: "0xAABBCCDDEE1122334455AABBCCDDEE1122334455",
|
|
||||||
want: "0xaabbccddee1122334455aabbccddee1122334455",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "without 0x prefix gets it added",
|
|
||||||
address: "aabbccddee1122334455aabbccddee1122334455",
|
|
||||||
want: "0xaabbccddee1122334455aabbccddee1122334455",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "0X uppercase prefix gets normalized",
|
|
||||||
address: "0XAABBCCDDEE1122334455AABBCCDDEE1122334455",
|
|
||||||
want: "0xaabbccddee1122334455aabbccddee1122334455",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "mixed case gets normalized",
|
|
||||||
address: "0xAaBbCcDdEe1122334455AaBbCcDdEe1122334455",
|
|
||||||
want: "0xaabbccddee1122334455aabbccddee1122334455",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "empty string gets 0x prefix",
|
|
||||||
address: "",
|
|
||||||
want: "0x",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
name: "just 0x stays as 0x",
|
|
||||||
address: "0x",
|
|
||||||
want: "0x",
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tt := range tests {
|
|
||||||
t.Run(tt.name, func(t *testing.T) {
|
|
||||||
got := FormatWalletAddress(tt.address)
|
|
||||||
if got != tt.want {
|
|
||||||
t.Errorf("FormatWalletAddress(%q) = %q, want %q", tt.address, got, tt.want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// GenerateRandomString
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
func TestGenerateRandomString(t *testing.T) {
|
|
||||||
t.Run("returns correct length", func(t *testing.T) {
|
|
||||||
lengths := []int{8, 16, 32, 64}
|
|
||||||
for _, l := range lengths {
|
|
||||||
s, err := GenerateRandomString(l)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GenerateRandomString(%d) returned error: %v", l, err)
|
|
||||||
}
|
|
||||||
if len(s) != l {
|
|
||||||
t.Errorf("GenerateRandomString(%d) returned string of length %d, want %d", l, len(s), l)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("two calls produce different values", func(t *testing.T) {
|
|
||||||
s1, err := GenerateRandomString(32)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("first call returned error: %v", err)
|
|
||||||
}
|
|
||||||
s2, err := GenerateRandomString(32)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("second call returned error: %v", err)
|
|
||||||
}
|
|
||||||
if s1 == s2 {
|
|
||||||
t.Errorf("two calls to GenerateRandomString(32) produced the same value: %q", s1)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("returns hex characters only", func(t *testing.T) {
|
|
||||||
s, err := GenerateRandomString(32)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GenerateRandomString(32) returned error: %v", err)
|
|
||||||
}
|
|
||||||
// hex.DecodeString requires even-length input; pad if needed
|
|
||||||
toDecode := s
|
|
||||||
if len(toDecode)%2 != 0 {
|
|
||||||
toDecode = toDecode + "0"
|
|
||||||
}
|
|
||||||
if _, err := hex.DecodeString(toDecode); err != nil {
|
|
||||||
t.Errorf("GenerateRandomString(32) returned non-hex string: %q, err: %v", s, err)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("length zero returns empty string", func(t *testing.T) {
|
|
||||||
s, err := GenerateRandomString(0)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GenerateRandomString(0) returned error: %v", err)
|
|
||||||
}
|
|
||||||
if s != "" {
|
|
||||||
t.Errorf("GenerateRandomString(0) = %q, want empty string", s)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("length one returns single hex char", func(t *testing.T) {
|
|
||||||
s, err := GenerateRandomString(1)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("GenerateRandomString(1) returned error: %v", err)
|
|
||||||
}
|
|
||||||
if len(s) != 1 {
|
|
||||||
t.Errorf("GenerateRandomString(1) returned string of length %d, want 1", len(s))
|
|
||||||
}
|
|
||||||
// Must be a valid hex character
|
|
||||||
const hexChars = "0123456789abcdef"
|
|
||||||
if !strings.Contains(hexChars, s) {
|
|
||||||
t.Errorf("GenerateRandomString(1) = %q, not a valid hex character", s)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
// phantomAuthURL
|
|
||||||
// ---------------------------------------------------------------------------
|
|
||||||
|
|
||||||
func TestPhantomAuthURL(t *testing.T) {
|
|
||||||
t.Run("returns default when env var not set", func(t *testing.T) {
|
|
||||||
// Ensure the env var is not set
|
|
||||||
os.Unsetenv("ORAMA_PHANTOM_AUTH_URL")
|
|
||||||
|
|
||||||
got := phantomAuthURL()
|
|
||||||
if got != defaultPhantomAuthURL {
|
|
||||||
t.Errorf("phantomAuthURL() = %q, want default %q", got, defaultPhantomAuthURL)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("returns custom URL when env var is set", func(t *testing.T) {
|
|
||||||
custom := "https://custom-phantom.example.com"
|
|
||||||
os.Setenv("ORAMA_PHANTOM_AUTH_URL", custom)
|
|
||||||
defer os.Unsetenv("ORAMA_PHANTOM_AUTH_URL")
|
|
||||||
|
|
||||||
got := phantomAuthURL()
|
|
||||||
if got != custom {
|
|
||||||
t.Errorf("phantomAuthURL() = %q, want %q", got, custom)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("trailing slash stripped from env var", func(t *testing.T) {
|
|
||||||
custom := "https://custom-phantom.example.com/"
|
|
||||||
os.Setenv("ORAMA_PHANTOM_AUTH_URL", custom)
|
|
||||||
defer os.Unsetenv("ORAMA_PHANTOM_AUTH_URL")
|
|
||||||
|
|
||||||
got := phantomAuthURL()
|
|
||||||
want := "https://custom-phantom.example.com"
|
|
||||||
if got != want {
|
|
||||||
t.Errorf("phantomAuthURL() = %q, want %q (trailing slash should be stripped)", got, want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
|
|
||||||
t.Run("multiple trailing slashes stripped from env var", func(t *testing.T) {
|
|
||||||
custom := "https://custom-phantom.example.com///"
|
|
||||||
os.Setenv("ORAMA_PHANTOM_AUTH_URL", custom)
|
|
||||||
defer os.Unsetenv("ORAMA_PHANTOM_AUTH_URL")
|
|
||||||
|
|
||||||
got := phantomAuthURL()
|
|
||||||
want := "https://custom-phantom.example.com"
|
|
||||||
if got != want {
|
|
||||||
t.Errorf("phantomAuthURL() = %q, want %q (trailing slashes should be stripped)", got, want)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
@ -1,22 +0,0 @@
|
|||||||
package auth
|
|
||||||
|
|
||||||
import "net"
|
|
||||||
|
|
||||||
// WireGuardSubnet is the internal WireGuard mesh CIDR.
|
|
||||||
const WireGuardSubnet = "10.0.0.0/24"
|
|
||||||
|
|
||||||
// IsWireGuardPeer checks whether remoteAddr (host:port format) originates
|
|
||||||
// from the WireGuard mesh subnet. This provides cryptographic peer
|
|
||||||
// authentication since WireGuard validates keys at the tunnel layer.
|
|
||||||
func IsWireGuardPeer(remoteAddr string) bool {
|
|
||||||
host, _, err := net.SplitHostPort(remoteAddr)
|
|
||||||
if err != nil {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
ip := net.ParseIP(host)
|
|
||||||
if ip == nil {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
_, wgNet, _ := net.ParseCIDR(WireGuardSubnet)
|
|
||||||
return wgNet.Contains(ip)
|
|
||||||
}
|
|
||||||
@ -1,214 +0,0 @@
|
|||||||
package auth
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bufio"
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"net/url"
|
|
||||||
"os"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/tlsutil"
|
|
||||||
qrterminal "github.com/mdp/qrterminal/v3"
|
|
||||||
)
|
|
||||||
|
|
||||||
// defaultPhantomAuthURL is the default Phantom auth React app URL (deployed on Orama devnet).
|
|
||||||
// Override with ORAMA_PHANTOM_AUTH_URL environment variable.
|
|
||||||
const defaultPhantomAuthURL = "https://phantom-auth-y0w9aa.orama-devnet.network"
|
|
||||||
|
|
||||||
// phantomAuthURL returns the Phantom auth URL, preferring the environment variable.
|
|
||||||
func phantomAuthURL() string {
|
|
||||||
if u := os.Getenv("ORAMA_PHANTOM_AUTH_URL"); u != "" {
|
|
||||||
return strings.TrimRight(u, "/")
|
|
||||||
}
|
|
||||||
return defaultPhantomAuthURL
|
|
||||||
}
|
|
||||||
|
|
||||||
// PhantomSession represents a phantom auth session from the gateway.
|
|
||||||
type PhantomSession struct {
|
|
||||||
SessionID string `json:"session_id"`
|
|
||||||
ExpiresAt string `json:"expires_at"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// PhantomSessionStatus represents the polled status of a phantom auth session.
|
|
||||||
type PhantomSessionStatus struct {
|
|
||||||
SessionID string `json:"session_id"`
|
|
||||||
Status string `json:"status"`
|
|
||||||
Wallet string `json:"wallet"`
|
|
||||||
APIKey string `json:"api_key"`
|
|
||||||
Namespace string `json:"namespace"`
|
|
||||||
Error string `json:"error"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// PerformPhantomAuthentication runs the Phantom Solana auth flow:
|
|
||||||
// 1. Prompt for namespace
|
|
||||||
// 2. Create session via gateway
|
|
||||||
// 3. Display QR code in terminal
|
|
||||||
// 4. Poll for completion
|
|
||||||
// 5. Return credentials
|
|
||||||
func PerformPhantomAuthentication(gatewayURL, namespace string) (*Credentials, error) {
|
|
||||||
reader := bufio.NewReader(os.Stdin)
|
|
||||||
|
|
||||||
fmt.Println("\n🟣 Phantom Wallet Authentication (Solana)")
|
|
||||||
fmt.Println("==========================================")
|
|
||||||
fmt.Println("Requires an NFT from the authorized collection.")
|
|
||||||
|
|
||||||
// Prompt for namespace if empty
|
|
||||||
if namespace == "" {
|
|
||||||
for {
|
|
||||||
fmt.Print("Enter namespace (required): ")
|
|
||||||
nsInput, err := reader.ReadString('\n')
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to read namespace: %w", err)
|
|
||||||
}
|
|
||||||
namespace = strings.TrimSpace(nsInput)
|
|
||||||
if namespace != "" {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
fmt.Println("Namespace cannot be empty.")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
domain := extractDomainFromURL(gatewayURL)
|
|
||||||
client := tlsutil.NewHTTPClientForDomain(30*time.Second, domain)
|
|
||||||
|
|
||||||
// 1. Create phantom session
|
|
||||||
fmt.Println("\nCreating authentication session...")
|
|
||||||
session, err := createPhantomSession(client, gatewayURL, namespace)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to create session: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// 2. Build auth URL and display QR code
|
|
||||||
authURL := fmt.Sprintf("%s/?session=%s&gateway=%s&namespace=%s",
|
|
||||||
phantomAuthURL(), session.SessionID, url.QueryEscape(gatewayURL), url.QueryEscape(namespace))
|
|
||||||
|
|
||||||
fmt.Println("\nScan this QR code with your phone to authenticate:")
|
|
||||||
fmt.Println()
|
|
||||||
qrterminal.GenerateWithConfig(authURL, qrterminal.Config{
|
|
||||||
Level: qrterminal.M,
|
|
||||||
Writer: os.Stdout,
|
|
||||||
BlackChar: qrterminal.BLACK,
|
|
||||||
WhiteChar: qrterminal.WHITE,
|
|
||||||
QuietZone: 1,
|
|
||||||
})
|
|
||||||
fmt.Println()
|
|
||||||
fmt.Printf("Or open this URL on your phone:\n%s\n\n", authURL)
|
|
||||||
fmt.Println("Waiting for authentication... (timeout: 5 minutes)")
|
|
||||||
|
|
||||||
// 3. Poll for completion
|
|
||||||
creds, err := pollPhantomSession(client, gatewayURL, session.SessionID)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Set namespace and build namespace URL
|
|
||||||
creds.Namespace = namespace
|
|
||||||
if domain := extractDomainFromURL(gatewayURL); domain != "" {
|
|
||||||
if namespace == "default" {
|
|
||||||
creds.NamespaceURL = fmt.Sprintf("https://%s", domain)
|
|
||||||
} else {
|
|
||||||
creds.NamespaceURL = fmt.Sprintf("https://ns-%s.%s", namespace, domain)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\n🎉 Authentication successful!\n")
|
|
||||||
truncatedKey := creds.APIKey
|
|
||||||
if len(truncatedKey) > 8 {
|
|
||||||
truncatedKey = truncatedKey[:8] + "..."
|
|
||||||
}
|
|
||||||
fmt.Printf("📝 API Key: %s\n", truncatedKey)
|
|
||||||
|
|
||||||
return creds, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// createPhantomSession creates a new phantom auth session via the gateway.
|
|
||||||
func createPhantomSession(client *http.Client, gatewayURL, namespace string) (*PhantomSession, error) {
|
|
||||||
reqBody := map[string]string{
|
|
||||||
"namespace": namespace,
|
|
||||||
}
|
|
||||||
payload, err := json.Marshal(reqBody)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Post(gatewayURL+"/v1/auth/phantom/session", "application/json", bytes.NewReader(payload))
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to call gateway: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
return nil, fmt.Errorf("gateway returned status %d: %s", resp.StatusCode, string(body))
|
|
||||||
}
|
|
||||||
|
|
||||||
var session PhantomSession
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&session); err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to decode response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return &session, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// pollPhantomSession polls the gateway for session completion.
|
|
||||||
func pollPhantomSession(client *http.Client, gatewayURL, sessionID string) (*Credentials, error) {
|
|
||||||
pollInterval := 2 * time.Second
|
|
||||||
maxDuration := 5 * time.Minute
|
|
||||||
deadline := time.Now().Add(maxDuration)
|
|
||||||
|
|
||||||
spinnerChars := []string{"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"}
|
|
||||||
spinnerIdx := 0
|
|
||||||
|
|
||||||
for time.Now().Before(deadline) {
|
|
||||||
resp, err := client.Get(gatewayURL + "/v1/auth/phantom/session/" + sessionID)
|
|
||||||
if err != nil {
|
|
||||||
time.Sleep(pollInterval)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
var status PhantomSessionStatus
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&status); err != nil {
|
|
||||||
resp.Body.Close()
|
|
||||||
time.Sleep(pollInterval)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
resp.Body.Close()
|
|
||||||
|
|
||||||
switch status.Status {
|
|
||||||
case "completed":
|
|
||||||
fmt.Printf("\r✅ Authenticated! \n")
|
|
||||||
return &Credentials{
|
|
||||||
APIKey: status.APIKey,
|
|
||||||
Wallet: status.Wallet,
|
|
||||||
UserID: status.Wallet,
|
|
||||||
IssuedAt: time.Now(),
|
|
||||||
}, nil
|
|
||||||
|
|
||||||
case "failed":
|
|
||||||
fmt.Printf("\r❌ Authentication failed \n")
|
|
||||||
errMsg := status.Error
|
|
||||||
if errMsg == "" {
|
|
||||||
errMsg = "unknown error"
|
|
||||||
}
|
|
||||||
return nil, fmt.Errorf("authentication failed: %s", errMsg)
|
|
||||||
|
|
||||||
case "expired":
|
|
||||||
fmt.Printf("\r⏰ Session expired \n")
|
|
||||||
return nil, fmt.Errorf("authentication session expired")
|
|
||||||
|
|
||||||
case "pending":
|
|
||||||
fmt.Printf("\r%s Waiting for phone authentication... ", spinnerChars[spinnerIdx%len(spinnerChars)])
|
|
||||||
spinnerIdx++
|
|
||||||
}
|
|
||||||
|
|
||||||
time.Sleep(pollInterval)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\r⏰ Timeout \n")
|
|
||||||
return nil, fmt.Errorf("authentication timed out after 5 minutes")
|
|
||||||
}
|
|
||||||
@ -1,290 +0,0 @@
|
|||||||
package auth
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bufio"
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/tlsutil"
|
|
||||||
)
|
|
||||||
|
|
||||||
// IsRootWalletInstalled checks if the `rw` CLI is available in PATH
|
|
||||||
func IsRootWalletInstalled() bool {
|
|
||||||
_, err := exec.LookPath("rw")
|
|
||||||
return err == nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// getRootWalletAddress gets the EVM address from the RootWallet keystore
|
|
||||||
func getRootWalletAddress() (string, error) {
|
|
||||||
cmd := exec.Command("rw", "address", "--chain", "evm")
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
out, err := cmd.Output()
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to get address from rw: %w", err)
|
|
||||||
}
|
|
||||||
addr := strings.TrimSpace(string(out))
|
|
||||||
if addr == "" {
|
|
||||||
return "", fmt.Errorf("rw returned empty address — run 'rw init' first")
|
|
||||||
}
|
|
||||||
return addr, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// signWithRootWallet signs a message using RootWallet's EVM key.
|
|
||||||
// Stdin is passed through so the user can enter their password if the session is expired.
|
|
||||||
func signWithRootWallet(message string) (string, error) {
|
|
||||||
cmd := exec.Command("rw", "sign", message, "--chain", "evm")
|
|
||||||
cmd.Stdin = os.Stdin
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
out, err := cmd.Output()
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to sign with rw: %w", err)
|
|
||||||
}
|
|
||||||
sig := strings.TrimSpace(string(out))
|
|
||||||
if sig == "" {
|
|
||||||
return "", fmt.Errorf("rw returned empty signature")
|
|
||||||
}
|
|
||||||
return sig, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// PerformRootWalletAuthentication performs a challenge-response authentication flow
|
|
||||||
// using the RootWallet CLI to sign a gateway-issued nonce
|
|
||||||
func PerformRootWalletAuthentication(gatewayURL, namespace string) (*Credentials, error) {
|
|
||||||
reader := bufio.NewReader(os.Stdin)
|
|
||||||
|
|
||||||
fmt.Println("\n🔐 RootWallet Authentication")
|
|
||||||
fmt.Println("=============================")
|
|
||||||
|
|
||||||
// 1. Get wallet address from RootWallet
|
|
||||||
fmt.Println("⏳ Reading wallet address from RootWallet...")
|
|
||||||
wallet, err := getRootWalletAddress()
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to get wallet address: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !ValidateWalletAddress(wallet) {
|
|
||||||
return nil, fmt.Errorf("invalid wallet address from rw: %s", wallet)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("✅ Wallet: %s\n", wallet)
|
|
||||||
|
|
||||||
// 2. Prompt for namespace if not provided
|
|
||||||
if namespace == "" {
|
|
||||||
for {
|
|
||||||
fmt.Print("Enter namespace (required): ")
|
|
||||||
nsInput, err := reader.ReadString('\n')
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to read namespace: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
namespace = strings.TrimSpace(nsInput)
|
|
||||||
if namespace != "" {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
fmt.Println("⚠️ Namespace cannot be empty. Please enter a namespace.")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
fmt.Printf("✅ Namespace: %s\n", namespace)
|
|
||||||
|
|
||||||
// 3. Request challenge nonce from gateway
|
|
||||||
fmt.Println("⏳ Requesting authentication challenge...")
|
|
||||||
domain := extractDomainFromURL(gatewayURL)
|
|
||||||
client := tlsutil.NewHTTPClientForDomain(30*time.Second, domain)
|
|
||||||
|
|
||||||
nonce, err := requestChallenge(client, gatewayURL, wallet, namespace)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to get challenge: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// 4. Sign the nonce with RootWallet
|
|
||||||
fmt.Println("⏳ Signing challenge with RootWallet...")
|
|
||||||
signature, err := signWithRootWallet(nonce)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to sign challenge: %w", err)
|
|
||||||
}
|
|
||||||
fmt.Println("✅ Challenge signed")
|
|
||||||
|
|
||||||
// 5. Verify signature with gateway
|
|
||||||
fmt.Println("⏳ Verifying signature with gateway...")
|
|
||||||
creds, err := verifySignature(client, gatewayURL, wallet, nonce, signature, namespace)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to verify signature: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// If namespace cluster is being provisioned, poll until ready
|
|
||||||
if creds.ProvisioningPollURL != "" {
|
|
||||||
fmt.Println("⏳ Provisioning namespace cluster...")
|
|
||||||
pollErr := pollNamespaceProvisioning(client, gatewayURL, creds.ProvisioningPollURL)
|
|
||||||
if pollErr != nil {
|
|
||||||
fmt.Printf("⚠️ Provisioning poll failed: %v\n", pollErr)
|
|
||||||
fmt.Println(" Credentials are saved. Cluster may still be provisioning in background.")
|
|
||||||
} else {
|
|
||||||
fmt.Println("✅ Namespace cluster ready!")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\n🎉 Authentication successful!\n")
|
|
||||||
fmt.Printf("🏢 Namespace: %s\n", creds.Namespace)
|
|
||||||
|
|
||||||
return creds, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// requestChallenge sends POST /v1/auth/challenge and returns the nonce
|
|
||||||
func requestChallenge(client *http.Client, gatewayURL, wallet, namespace string) (string, error) {
|
|
||||||
reqBody := map[string]string{
|
|
||||||
"wallet": wallet,
|
|
||||||
"namespace": namespace,
|
|
||||||
}
|
|
||||||
|
|
||||||
payload, err := json.Marshal(reqBody)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to marshal request: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Post(gatewayURL+"/v1/auth/challenge", "application/json", bytes.NewReader(payload))
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to call gateway: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
return "", fmt.Errorf("gateway returned status %d: %s", resp.StatusCode, string(body))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result struct {
|
|
||||||
Nonce string `json:"nonce"`
|
|
||||||
Wallet string `json:"wallet"`
|
|
||||||
Namespace string `json:"namespace"`
|
|
||||||
ExpiresAt string `json:"expires_at"`
|
|
||||||
}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
|
||||||
return "", fmt.Errorf("failed to decode response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if result.Nonce == "" {
|
|
||||||
return "", fmt.Errorf("no nonce in challenge response")
|
|
||||||
}
|
|
||||||
|
|
||||||
return result.Nonce, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// verifySignature sends POST /v1/auth/verify and returns credentials
|
|
||||||
func verifySignature(client *http.Client, gatewayURL, wallet, nonce, signature, namespace string) (*Credentials, error) {
|
|
||||||
reqBody := map[string]string{
|
|
||||||
"wallet": wallet,
|
|
||||||
"nonce": nonce,
|
|
||||||
"signature": signature,
|
|
||||||
"namespace": namespace,
|
|
||||||
"chain_type": "ETH",
|
|
||||||
}
|
|
||||||
|
|
||||||
payload, err := json.Marshal(reqBody)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to marshal request: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Post(gatewayURL+"/v1/auth/verify", "application/json", bytes.NewReader(payload))
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to call gateway: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusAccepted {
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
return nil, fmt.Errorf("gateway returned status %d: %s", resp.StatusCode, string(body))
|
|
||||||
}
|
|
||||||
|
|
||||||
var result struct {
|
|
||||||
AccessToken string `json:"access_token"`
|
|
||||||
RefreshToken string `json:"refresh_token"`
|
|
||||||
ExpiresIn int `json:"expires_in"`
|
|
||||||
Subject string `json:"subject"`
|
|
||||||
Namespace string `json:"namespace"`
|
|
||||||
APIKey string `json:"api_key"`
|
|
||||||
// Provisioning fields (202 Accepted)
|
|
||||||
Status string `json:"status"`
|
|
||||||
PollURL string `json:"poll_url"`
|
|
||||||
}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to decode response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if result.APIKey == "" {
|
|
||||||
return nil, fmt.Errorf("no api_key in verify response")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Build namespace gateway URL
|
|
||||||
namespaceURL := ""
|
|
||||||
if d := extractDomainFromURL(gatewayURL); d != "" {
|
|
||||||
if namespace == "default" {
|
|
||||||
namespaceURL = fmt.Sprintf("https://%s", d)
|
|
||||||
} else {
|
|
||||||
namespaceURL = fmt.Sprintf("https://ns-%s.%s", namespace, d)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
creds := &Credentials{
|
|
||||||
APIKey: result.APIKey,
|
|
||||||
RefreshToken: result.RefreshToken,
|
|
||||||
Namespace: result.Namespace,
|
|
||||||
UserID: result.Subject,
|
|
||||||
Wallet: result.Subject,
|
|
||||||
IssuedAt: time.Now(),
|
|
||||||
NamespaceURL: namespaceURL,
|
|
||||||
}
|
|
||||||
|
|
||||||
// If 202, namespace cluster is being provisioned — set poll URL
|
|
||||||
if resp.StatusCode == http.StatusAccepted && result.PollURL != "" {
|
|
||||||
creds.ProvisioningPollURL = result.PollURL
|
|
||||||
}
|
|
||||||
|
|
||||||
// Note: result.ExpiresIn is the JWT access token lifetime (15min),
|
|
||||||
// NOT the API key lifetime. Don't set ExpiresAt — the API key is permanent.
|
|
||||||
|
|
||||||
return creds, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// pollNamespaceProvisioning polls the namespace status endpoint until the cluster is ready.
|
|
||||||
func pollNamespaceProvisioning(client *http.Client, gatewayURL, pollPath string) error {
|
|
||||||
pollURL := gatewayURL + pollPath
|
|
||||||
timeout := time.After(120 * time.Second)
|
|
||||||
ticker := time.NewTicker(5 * time.Second)
|
|
||||||
defer ticker.Stop()
|
|
||||||
|
|
||||||
for {
|
|
||||||
select {
|
|
||||||
case <-timeout:
|
|
||||||
return fmt.Errorf("timed out after 120s waiting for namespace cluster")
|
|
||||||
case <-ticker.C:
|
|
||||||
resp, err := client.Get(pollURL)
|
|
||||||
if err != nil {
|
|
||||||
continue // Retry on network error
|
|
||||||
}
|
|
||||||
|
|
||||||
var status struct {
|
|
||||||
Status string `json:"status"`
|
|
||||||
}
|
|
||||||
decErr := json.NewDecoder(resp.Body).Decode(&status)
|
|
||||||
resp.Body.Close()
|
|
||||||
if decErr != nil {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
switch status.Status {
|
|
||||||
case "ready":
|
|
||||||
return nil
|
|
||||||
case "failed", "error":
|
|
||||||
return fmt.Errorf("namespace provisioning failed")
|
|
||||||
}
|
|
||||||
// "provisioning" or other — keep polling
|
|
||||||
fmt.Print(".")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,351 +0,0 @@
|
|||||||
package auth
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bufio"
|
|
||||||
"bytes"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"net/url"
|
|
||||||
"os"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/tlsutil"
|
|
||||||
)
|
|
||||||
|
|
||||||
// PerformSimpleAuthentication performs a simple authentication flow where the user
|
|
||||||
// provides a wallet address and receives an API key without signature verification.
|
|
||||||
// Requires an existing valid API key (convenience re-auth only).
|
|
||||||
func PerformSimpleAuthentication(gatewayURL, wallet, namespace, existingAPIKey string) (*Credentials, error) {
|
|
||||||
reader := bufio.NewReader(os.Stdin)
|
|
||||||
|
|
||||||
fmt.Println("\n🔐 Simple Wallet Authentication")
|
|
||||||
fmt.Println("================================")
|
|
||||||
|
|
||||||
// Read wallet address (skip prompt if provided via flag)
|
|
||||||
if wallet == "" {
|
|
||||||
fmt.Print("Enter your wallet address (0x...): ")
|
|
||||||
walletInput, err := reader.ReadString('\n')
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to read wallet address: %w", err)
|
|
||||||
}
|
|
||||||
wallet = strings.TrimSpace(walletInput)
|
|
||||||
}
|
|
||||||
|
|
||||||
if wallet == "" {
|
|
||||||
return nil, fmt.Errorf("wallet address cannot be empty")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Validate wallet format (basic check)
|
|
||||||
if !strings.HasPrefix(wallet, "0x") && !strings.HasPrefix(wallet, "0X") {
|
|
||||||
wallet = "0x" + wallet
|
|
||||||
}
|
|
||||||
|
|
||||||
if !ValidateWalletAddress(wallet) {
|
|
||||||
return nil, fmt.Errorf("invalid wallet address format")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Read namespace (skip prompt if provided via flag)
|
|
||||||
if namespace == "" {
|
|
||||||
for {
|
|
||||||
fmt.Print("Enter namespace (required): ")
|
|
||||||
nsInput, err := reader.ReadString('\n')
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to read namespace: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
namespace = strings.TrimSpace(nsInput)
|
|
||||||
if namespace != "" {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
fmt.Println("⚠️ Namespace cannot be empty. Please enter a namespace.")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\n✅ Wallet: %s\n", wallet)
|
|
||||||
fmt.Printf("✅ Namespace: %s\n", namespace)
|
|
||||||
fmt.Println("⏳ Requesting API key from gateway...")
|
|
||||||
|
|
||||||
// Request API key from gateway
|
|
||||||
apiKey, err := requestAPIKeyFromGateway(gatewayURL, wallet, namespace, existingAPIKey)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to request API key: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Build namespace gateway URL from the gateway URL
|
|
||||||
namespaceURL := ""
|
|
||||||
if domain := extractDomainFromURL(gatewayURL); domain != "" {
|
|
||||||
if namespace == "default" {
|
|
||||||
namespaceURL = fmt.Sprintf("https://%s", domain)
|
|
||||||
} else {
|
|
||||||
namespaceURL = fmt.Sprintf("https://ns-%s.%s", namespace, domain)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create credentials
|
|
||||||
creds := &Credentials{
|
|
||||||
APIKey: apiKey,
|
|
||||||
Namespace: namespace,
|
|
||||||
UserID: wallet,
|
|
||||||
Wallet: wallet,
|
|
||||||
IssuedAt: time.Now(),
|
|
||||||
NamespaceURL: namespaceURL,
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\n🎉 Authentication successful!\n")
|
|
||||||
truncatedKey := creds.APIKey
|
|
||||||
if len(truncatedKey) > 8 {
|
|
||||||
truncatedKey = truncatedKey[:8] + "..."
|
|
||||||
}
|
|
||||||
fmt.Printf("📝 API Key: %s\n", truncatedKey)
|
|
||||||
|
|
||||||
return creds, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// requestAPIKeyFromGateway calls the gateway's simple-key endpoint to generate an API key
|
|
||||||
// For non-default namespaces, this may trigger cluster provisioning and require polling
|
|
||||||
func requestAPIKeyFromGateway(gatewayURL, wallet, namespace, existingAPIKey string) (string, error) {
|
|
||||||
reqBody := map[string]string{
|
|
||||||
"wallet": wallet,
|
|
||||||
"namespace": namespace,
|
|
||||||
}
|
|
||||||
|
|
||||||
payload, err := json.Marshal(reqBody)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to marshal request: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
endpoint := gatewayURL + "/v1/auth/simple-key"
|
|
||||||
|
|
||||||
// Extract domain from URL for TLS configuration
|
|
||||||
// This uses tlsutil which handles Let's Encrypt staging certificates for *.orama.network
|
|
||||||
domain := extractDomainFromURL(gatewayURL)
|
|
||||||
client := tlsutil.NewHTTPClientForDomain(30*time.Second, domain)
|
|
||||||
|
|
||||||
req, err := http.NewRequest(http.MethodPost, endpoint, bytes.NewReader(payload))
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to create request: %w", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
if existingAPIKey != "" {
|
|
||||||
req.Header.Set("X-API-Key", existingAPIKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to call gateway: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
// Handle 202 Accepted - namespace cluster is being provisioned
|
|
||||||
if resp.StatusCode == http.StatusAccepted {
|
|
||||||
return handleProvisioningResponse(gatewayURL, client, resp, wallet, namespace, existingAPIKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
return "", fmt.Errorf("gateway returned status %d: %s", resp.StatusCode, string(body))
|
|
||||||
}
|
|
||||||
|
|
||||||
var respBody map[string]interface{}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&respBody); err != nil {
|
|
||||||
return "", fmt.Errorf("failed to decode response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
apiKey, ok := respBody["api_key"].(string)
|
|
||||||
if !ok || apiKey == "" {
|
|
||||||
return "", fmt.Errorf("no api_key in response")
|
|
||||||
}
|
|
||||||
|
|
||||||
return apiKey, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// handleProvisioningResponse handles 202 Accepted responses when namespace cluster provisioning is needed
|
|
||||||
func handleProvisioningResponse(gatewayURL string, client *http.Client, resp *http.Response, wallet, namespace, existingAPIKey string) (string, error) {
|
|
||||||
var provResp map[string]interface{}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&provResp); err != nil {
|
|
||||||
return "", fmt.Errorf("failed to decode provisioning response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
status, _ := provResp["status"].(string)
|
|
||||||
pollURL, _ := provResp["poll_url"].(string)
|
|
||||||
clusterID, _ := provResp["cluster_id"].(string)
|
|
||||||
message, _ := provResp["message"].(string)
|
|
||||||
|
|
||||||
if status != "provisioning" {
|
|
||||||
return "", fmt.Errorf("unexpected status: %s", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\n🏗️ Provisioning namespace cluster...\n")
|
|
||||||
if message != "" {
|
|
||||||
fmt.Printf(" %s\n", message)
|
|
||||||
}
|
|
||||||
if clusterID != "" {
|
|
||||||
fmt.Printf(" Cluster ID: %s\n", clusterID)
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
|
|
||||||
// Poll until cluster is ready
|
|
||||||
if err := pollProvisioningStatus(gatewayURL, client, pollURL); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cluster is ready, retry the API key request
|
|
||||||
fmt.Println("\n✅ Namespace cluster ready!")
|
|
||||||
fmt.Println("⏳ Retrieving API key...")
|
|
||||||
|
|
||||||
return retryAPIKeyRequest(gatewayURL, client, wallet, namespace, existingAPIKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
// pollProvisioningStatus polls the status endpoint until the cluster is ready
|
|
||||||
func pollProvisioningStatus(gatewayURL string, client *http.Client, pollURL string) error {
|
|
||||||
// Build full poll URL if it's a relative path
|
|
||||||
if strings.HasPrefix(pollURL, "/") {
|
|
||||||
pollURL = gatewayURL + pollURL
|
|
||||||
} else {
|
|
||||||
// Validate that absolute poll URLs point to the same gateway domain
|
|
||||||
gatewayDomain := extractDomainFromURL(gatewayURL)
|
|
||||||
pollDomain := extractDomainFromURL(pollURL)
|
|
||||||
if gatewayDomain != pollDomain {
|
|
||||||
return fmt.Errorf("poll URL domain mismatch: expected %s, got %s", gatewayDomain, pollDomain)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
maxAttempts := 120 // 10 minutes (5 seconds per poll)
|
|
||||||
pollInterval := 5 * time.Second
|
|
||||||
|
|
||||||
spinnerChars := []string{"⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"}
|
|
||||||
spinnerIdx := 0
|
|
||||||
|
|
||||||
for i := 0; i < maxAttempts; i++ {
|
|
||||||
// Show progress spinner
|
|
||||||
fmt.Printf("\r%s Waiting for cluster... ", spinnerChars[spinnerIdx%len(spinnerChars)])
|
|
||||||
spinnerIdx++
|
|
||||||
|
|
||||||
resp, err := client.Get(pollURL)
|
|
||||||
if err != nil {
|
|
||||||
time.Sleep(pollInterval)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
var statusResp map[string]interface{}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&statusResp); err != nil {
|
|
||||||
resp.Body.Close()
|
|
||||||
time.Sleep(pollInterval)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
resp.Body.Close()
|
|
||||||
|
|
||||||
status, _ := statusResp["status"].(string)
|
|
||||||
|
|
||||||
switch status {
|
|
||||||
case "ready":
|
|
||||||
fmt.Printf("\r✅ Cluster ready! \n")
|
|
||||||
return nil
|
|
||||||
|
|
||||||
case "failed":
|
|
||||||
errMsg, _ := statusResp["error"].(string)
|
|
||||||
fmt.Printf("\r❌ Provisioning failed \n")
|
|
||||||
return fmt.Errorf("cluster provisioning failed: %s", errMsg)
|
|
||||||
|
|
||||||
case "provisioning":
|
|
||||||
// Show progress details
|
|
||||||
rqliteReady, _ := statusResp["rqlite_ready"].(bool)
|
|
||||||
olricReady, _ := statusResp["olric_ready"].(bool)
|
|
||||||
gatewayReady, _ := statusResp["gateway_ready"].(bool)
|
|
||||||
dnsReady, _ := statusResp["dns_ready"].(bool)
|
|
||||||
|
|
||||||
progressStr := ""
|
|
||||||
if rqliteReady {
|
|
||||||
progressStr += "RQLite✓ "
|
|
||||||
}
|
|
||||||
if olricReady {
|
|
||||||
progressStr += "Olric✓ "
|
|
||||||
}
|
|
||||||
if gatewayReady {
|
|
||||||
progressStr += "Gateway✓ "
|
|
||||||
}
|
|
||||||
if dnsReady {
|
|
||||||
progressStr += "DNS✓"
|
|
||||||
}
|
|
||||||
if progressStr != "" {
|
|
||||||
fmt.Printf("\r%s Provisioning... [%s]", spinnerChars[spinnerIdx%len(spinnerChars)], progressStr)
|
|
||||||
}
|
|
||||||
|
|
||||||
default:
|
|
||||||
// Unknown status, continue polling
|
|
||||||
}
|
|
||||||
|
|
||||||
time.Sleep(pollInterval)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\r⚠️ Timeout waiting for cluster \n")
|
|
||||||
return fmt.Errorf("timeout waiting for namespace cluster provisioning")
|
|
||||||
}
|
|
||||||
|
|
||||||
// retryAPIKeyRequest retries the API key request after cluster provisioning
|
|
||||||
func retryAPIKeyRequest(gatewayURL string, client *http.Client, wallet, namespace, existingAPIKey string) (string, error) {
|
|
||||||
reqBody := map[string]string{
|
|
||||||
"wallet": wallet,
|
|
||||||
"namespace": namespace,
|
|
||||||
}
|
|
||||||
|
|
||||||
payload, err := json.Marshal(reqBody)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to marshal request: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
endpoint := gatewayURL + "/v1/auth/simple-key"
|
|
||||||
|
|
||||||
req, err := http.NewRequest(http.MethodPost, endpoint, bytes.NewReader(payload))
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to create request: %w", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
if existingAPIKey != "" {
|
|
||||||
req.Header.Set("X-API-Key", existingAPIKey)
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return "", fmt.Errorf("failed to call gateway: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode == http.StatusAccepted {
|
|
||||||
// Still provisioning? This shouldn't happen but handle gracefully
|
|
||||||
return "", fmt.Errorf("cluster still provisioning, please try again")
|
|
||||||
}
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
return "", fmt.Errorf("gateway returned status %d: %s", resp.StatusCode, string(body))
|
|
||||||
}
|
|
||||||
|
|
||||||
var respBody map[string]interface{}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&respBody); err != nil {
|
|
||||||
return "", fmt.Errorf("failed to decode response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
apiKey, ok := respBody["api_key"].(string)
|
|
||||||
if !ok || apiKey == "" {
|
|
||||||
return "", fmt.Errorf("no api_key in response")
|
|
||||||
}
|
|
||||||
|
|
||||||
return apiKey, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// extractDomainFromURL extracts the hostname from a URL, stripping scheme, port, and path.
|
|
||||||
func extractDomainFromURL(rawURL string) string {
|
|
||||||
// Ensure the URL has a scheme so net/url.Parse works correctly
|
|
||||||
if !strings.Contains(rawURL, "://") {
|
|
||||||
rawURL = "https://" + rawURL
|
|
||||||
}
|
|
||||||
u, err := url.Parse(rawURL)
|
|
||||||
if err != nil {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
return u.Hostname()
|
|
||||||
}
|
|
||||||
@ -1,466 +0,0 @@
|
|||||||
package cli
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bufio"
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"strings"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/auth"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleAuthCommand handles authentication commands
|
|
||||||
func HandleAuthCommand(args []string) {
|
|
||||||
if len(args) == 0 {
|
|
||||||
showAuthHelp()
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
subcommand := args[0]
|
|
||||||
switch subcommand {
|
|
||||||
case "login":
|
|
||||||
var wallet, namespace string
|
|
||||||
var simple bool
|
|
||||||
fs := flag.NewFlagSet("auth login", flag.ExitOnError)
|
|
||||||
fs.StringVar(&wallet, "wallet", "", "Wallet address (implies --simple)")
|
|
||||||
fs.StringVar(&namespace, "namespace", "", "Namespace name")
|
|
||||||
fs.BoolVar(&simple, "simple", false, "Use simple auth without signature verification")
|
|
||||||
_ = fs.Parse(args[1:])
|
|
||||||
handleAuthLogin(wallet, namespace, simple)
|
|
||||||
case "logout":
|
|
||||||
handleAuthLogout()
|
|
||||||
case "whoami":
|
|
||||||
handleAuthWhoami()
|
|
||||||
case "status":
|
|
||||||
handleAuthStatus()
|
|
||||||
case "list":
|
|
||||||
handleAuthList()
|
|
||||||
case "switch":
|
|
||||||
handleAuthSwitch()
|
|
||||||
default:
|
|
||||||
fmt.Fprintf(os.Stderr, "Unknown auth command: %s\n", subcommand)
|
|
||||||
showAuthHelp()
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func showAuthHelp() {
|
|
||||||
fmt.Printf("🔐 Authentication Commands\n\n")
|
|
||||||
fmt.Printf("Usage: orama auth <subcommand>\n\n")
|
|
||||||
fmt.Printf("Subcommands:\n")
|
|
||||||
fmt.Printf(" login - Authenticate with RootWallet (default) or simple auth\n")
|
|
||||||
fmt.Printf(" logout - Clear stored credentials\n")
|
|
||||||
fmt.Printf(" whoami - Show current authentication status\n")
|
|
||||||
fmt.Printf(" status - Show detailed authentication info\n")
|
|
||||||
fmt.Printf(" list - List all stored credentials for current environment\n")
|
|
||||||
fmt.Printf(" switch - Switch between stored credentials\n\n")
|
|
||||||
fmt.Printf("Login Flags:\n")
|
|
||||||
fmt.Printf(" --namespace <name> - Target namespace\n")
|
|
||||||
fmt.Printf(" --simple - Use simple auth (no signature, dev only)\n")
|
|
||||||
fmt.Printf(" --wallet <0x...> - Wallet address (implies --simple)\n\n")
|
|
||||||
fmt.Printf("Examples:\n")
|
|
||||||
fmt.Printf(" orama auth login # Sign with RootWallet (default)\n")
|
|
||||||
fmt.Printf(" orama auth login --namespace myns # Sign with RootWallet + namespace\n")
|
|
||||||
fmt.Printf(" orama auth login --simple # Simple auth (no signature)\n")
|
|
||||||
fmt.Printf(" orama auth whoami # Check who you're logged in as\n")
|
|
||||||
fmt.Printf(" orama auth logout # Clear all stored credentials\n\n")
|
|
||||||
fmt.Printf("Environment Variables:\n")
|
|
||||||
fmt.Printf(" ORAMA_GATEWAY_URL - Gateway URL (overrides environment config)\n\n")
|
|
||||||
fmt.Printf("Authentication Flow (RootWallet):\n")
|
|
||||||
fmt.Printf(" 1. Run 'orama auth login'\n")
|
|
||||||
fmt.Printf(" 2. Your wallet address is read from RootWallet automatically\n")
|
|
||||||
fmt.Printf(" 3. Enter your namespace when prompted\n")
|
|
||||||
fmt.Printf(" 4. A challenge nonce is signed with your wallet key\n")
|
|
||||||
fmt.Printf(" 5. Credentials are saved to ~/.orama/credentials.json\n\n")
|
|
||||||
fmt.Printf("Note: Requires RootWallet CLI (rw) in PATH.\n")
|
|
||||||
fmt.Printf(" Install: cd rootwallet/cli && ./install.sh\n")
|
|
||||||
fmt.Printf(" Authentication uses the currently active environment.\n")
|
|
||||||
fmt.Printf(" Use 'orama env current' to see your active environment.\n")
|
|
||||||
}
|
|
||||||
|
|
||||||
func handleAuthLogin(wallet, namespace string, simple bool) {
|
|
||||||
// Get gateway URL from active environment
|
|
||||||
gatewayURL := getGatewayURL()
|
|
||||||
|
|
||||||
// Show active environment
|
|
||||||
env, err := GetActiveEnvironment()
|
|
||||||
if err == nil {
|
|
||||||
fmt.Printf("🌍 Environment: %s\n", env.Name)
|
|
||||||
}
|
|
||||||
fmt.Printf("🔐 Authenticating with gateway at: %s\n\n", gatewayURL)
|
|
||||||
|
|
||||||
// Load enhanced credential store
|
|
||||||
store, err := auth.LoadEnhancedCredentials()
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to load credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check if we already have credentials for this gateway
|
|
||||||
gwCreds := store.Gateways[gatewayURL]
|
|
||||||
if gwCreds != nil && len(gwCreds.Credentials) > 0 {
|
|
||||||
// Show existing credentials and offer choice
|
|
||||||
choice, credIndex, err := store.DisplayCredentialMenu(gatewayURL)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Menu selection failed: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
switch choice {
|
|
||||||
case auth.AuthChoiceUseCredential:
|
|
||||||
selectedCreds := gwCreds.Credentials[credIndex]
|
|
||||||
store.SetDefaultCredential(gatewayURL, credIndex)
|
|
||||||
selectedCreds.UpdateLastUsed()
|
|
||||||
if err := store.Save(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to save credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
fmt.Printf("✅ Switched to wallet: %s\n", selectedCreds.Wallet)
|
|
||||||
fmt.Printf("🏢 Namespace: %s\n", selectedCreds.Namespace)
|
|
||||||
return
|
|
||||||
|
|
||||||
case auth.AuthChoiceLogout:
|
|
||||||
store.ClearAllCredentials()
|
|
||||||
if err := store.Save(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to clear credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
fmt.Println("✅ All credentials cleared")
|
|
||||||
return
|
|
||||||
|
|
||||||
case auth.AuthChoiceExit:
|
|
||||||
fmt.Println("Exiting...")
|
|
||||||
return
|
|
||||||
|
|
||||||
case auth.AuthChoiceAddCredential:
|
|
||||||
// Fall through to add new credential
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Choose authentication method
|
|
||||||
var creds *auth.Credentials
|
|
||||||
reader := bufio.NewReader(os.Stdin)
|
|
||||||
|
|
||||||
if simple || wallet != "" {
|
|
||||||
// Explicit simple auth — requires existing credentials
|
|
||||||
existingCreds := store.GetDefaultCredential(gatewayURL)
|
|
||||||
if existingCreds == nil || !existingCreds.IsValid() {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Simple auth requires existing credentials. Authenticate with RootWallet or Phantom first.\n")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
creds, err = auth.PerformSimpleAuthentication(gatewayURL, wallet, namespace, existingCreds.APIKey)
|
|
||||||
} else {
|
|
||||||
// Show auth method selection
|
|
||||||
fmt.Println("How would you like to authenticate?")
|
|
||||||
fmt.Println(" 1. RootWallet (EVM signature)")
|
|
||||||
fmt.Println(" 2. Phantom (Solana + NFT required)")
|
|
||||||
fmt.Print("\nSelect [1/2]: ")
|
|
||||||
|
|
||||||
choice, _ := reader.ReadString('\n')
|
|
||||||
choice = strings.TrimSpace(choice)
|
|
||||||
|
|
||||||
switch choice {
|
|
||||||
case "2":
|
|
||||||
creds, err = auth.PerformPhantomAuthentication(gatewayURL, namespace)
|
|
||||||
default:
|
|
||||||
// Default to RootWallet
|
|
||||||
if auth.IsRootWalletInstalled() {
|
|
||||||
creds, err = auth.PerformRootWalletAuthentication(gatewayURL, namespace)
|
|
||||||
} else {
|
|
||||||
fmt.Println("\n⚠️ RootWallet CLI (rw) not found in PATH.")
|
|
||||||
fmt.Println(" Install it: cd rootwallet/cli && ./install.sh")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Authentication failed: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add to enhanced store
|
|
||||||
store.AddCredential(gatewayURL, creds)
|
|
||||||
|
|
||||||
// Set as default
|
|
||||||
gwCreds = store.Gateways[gatewayURL]
|
|
||||||
if gwCreds != nil {
|
|
||||||
store.SetDefaultCredential(gatewayURL, len(gwCreds.Credentials)-1)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := store.Save(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to save credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
credsPath, _ := auth.GetCredentialsPath()
|
|
||||||
fmt.Printf("✅ Authentication successful!\n")
|
|
||||||
fmt.Printf("📁 Credentials saved to: %s\n", credsPath)
|
|
||||||
fmt.Printf("🎯 Wallet: %s\n", creds.Wallet)
|
|
||||||
fmt.Printf("🏢 Namespace: %s\n", creds.Namespace)
|
|
||||||
if creds.NamespaceURL != "" {
|
|
||||||
fmt.Printf("🌐 Namespace URL: %s\n", creds.NamespaceURL)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func handleAuthLogout() {
|
|
||||||
if err := auth.ClearAllCredentials(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to clear credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
fmt.Println("✅ Logged out successfully - all credentials have been cleared")
|
|
||||||
}
|
|
||||||
|
|
||||||
func handleAuthWhoami() {
|
|
||||||
store, err := auth.LoadEnhancedCredentials()
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to load credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
gatewayURL := getGatewayURL()
|
|
||||||
creds := store.GetDefaultCredential(gatewayURL)
|
|
||||||
|
|
||||||
if creds == nil || !creds.IsValid() {
|
|
||||||
fmt.Println("❌ Not authenticated - run 'orama auth login' to authenticate")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println("✅ Authenticated")
|
|
||||||
fmt.Printf(" Wallet: %s\n", creds.Wallet)
|
|
||||||
fmt.Printf(" Namespace: %s\n", creds.Namespace)
|
|
||||||
if creds.NamespaceURL != "" {
|
|
||||||
fmt.Printf(" NS Gateway: %s\n", creds.NamespaceURL)
|
|
||||||
}
|
|
||||||
fmt.Printf(" Issued At: %s\n", creds.IssuedAt.Format("2006-01-02 15:04:05"))
|
|
||||||
if !creds.ExpiresAt.IsZero() {
|
|
||||||
fmt.Printf(" Expires At: %s\n", creds.ExpiresAt.Format("2006-01-02 15:04:05"))
|
|
||||||
}
|
|
||||||
if !creds.LastUsedAt.IsZero() {
|
|
||||||
fmt.Printf(" Last Used: %s\n", creds.LastUsedAt.Format("2006-01-02 15:04:05"))
|
|
||||||
}
|
|
||||||
if creds.Plan != "" {
|
|
||||||
fmt.Printf(" Plan: %s\n", creds.Plan)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func handleAuthStatus() {
|
|
||||||
store, err := auth.LoadEnhancedCredentials()
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to load credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
gatewayURL := getGatewayURL()
|
|
||||||
creds := store.GetDefaultCredential(gatewayURL)
|
|
||||||
|
|
||||||
// Show active environment
|
|
||||||
env, err := GetActiveEnvironment()
|
|
||||||
if err == nil {
|
|
||||||
fmt.Printf("🌍 Active Environment: %s\n", env.Name)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println("🔐 Authentication Status")
|
|
||||||
fmt.Printf(" Gateway URL: %s\n", gatewayURL)
|
|
||||||
|
|
||||||
if creds == nil {
|
|
||||||
fmt.Println(" Status: ❌ Not authenticated")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if !creds.IsValid() {
|
|
||||||
fmt.Println(" Status: ⚠️ Credentials expired")
|
|
||||||
if !creds.ExpiresAt.IsZero() {
|
|
||||||
fmt.Printf(" Expired At: %s\n", creds.ExpiresAt.Format("2006-01-02 15:04:05"))
|
|
||||||
}
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println(" Status: ✅ Authenticated")
|
|
||||||
fmt.Printf(" Wallet: %s\n", creds.Wallet)
|
|
||||||
fmt.Printf(" Namespace: %s\n", creds.Namespace)
|
|
||||||
if creds.NamespaceURL != "" {
|
|
||||||
fmt.Printf(" NS Gateway: %s\n", creds.NamespaceURL)
|
|
||||||
}
|
|
||||||
if !creds.ExpiresAt.IsZero() {
|
|
||||||
fmt.Printf(" Expires: %s\n", creds.ExpiresAt.Format("2006-01-02 15:04:05"))
|
|
||||||
}
|
|
||||||
if !creds.LastUsedAt.IsZero() {
|
|
||||||
fmt.Printf(" Last Used: %s\n", creds.LastUsedAt.Format("2006-01-02 15:04:05"))
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// promptForGatewayURL interactively prompts for the gateway URL
|
|
||||||
// Uses the active environment or allows entering a custom domain
|
|
||||||
func promptForGatewayURL() string {
|
|
||||||
// Check environment variable first (allows override without prompting)
|
|
||||||
if url := os.Getenv("ORAMA_GATEWAY_URL"); url != "" {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
|
|
||||||
// Try active environment
|
|
||||||
env, err := GetActiveEnvironment()
|
|
||||||
if err == nil {
|
|
||||||
reader := bufio.NewReader(os.Stdin)
|
|
||||||
|
|
||||||
fmt.Println("\n🌐 Node Connection")
|
|
||||||
fmt.Println("==================")
|
|
||||||
fmt.Printf("1. Use active environment: %s (%s)\n", env.Name, env.GatewayURL)
|
|
||||||
fmt.Println("2. Enter custom domain")
|
|
||||||
fmt.Print("\nSelect option [1/2]: ")
|
|
||||||
|
|
||||||
choice, _ := reader.ReadString('\n')
|
|
||||||
choice = strings.TrimSpace(choice)
|
|
||||||
|
|
||||||
if choice == "1" || choice == "" {
|
|
||||||
return env.GatewayURL
|
|
||||||
}
|
|
||||||
|
|
||||||
if choice == "2" {
|
|
||||||
fmt.Print("Enter node domain (e.g., node-hk19de.orama.network): ")
|
|
||||||
domain, _ := reader.ReadString('\n')
|
|
||||||
domain = strings.TrimSpace(domain)
|
|
||||||
|
|
||||||
if domain == "" {
|
|
||||||
fmt.Printf("⚠️ No domain entered, using %s\n", env.Name)
|
|
||||||
return env.GatewayURL
|
|
||||||
}
|
|
||||||
|
|
||||||
// Remove any protocol prefix if user included it
|
|
||||||
domain = strings.TrimPrefix(domain, "https://")
|
|
||||||
domain = strings.TrimPrefix(domain, "http://")
|
|
||||||
// Remove trailing slash
|
|
||||||
domain = strings.TrimSuffix(domain, "/")
|
|
||||||
|
|
||||||
return fmt.Sprintf("https://%s", domain)
|
|
||||||
}
|
|
||||||
|
|
||||||
return env.GatewayURL
|
|
||||||
}
|
|
||||||
|
|
||||||
return "https://orama-devnet.network"
|
|
||||||
}
|
|
||||||
|
|
||||||
// getGatewayURL returns the gateway URL based on environment or env var
|
|
||||||
// Used by other commands that don't need interactive node selection
|
|
||||||
func getGatewayURL() string {
|
|
||||||
// Check environment variable first (for backwards compatibility)
|
|
||||||
if url := os.Getenv("ORAMA_GATEWAY_URL"); url != "" {
|
|
||||||
return url
|
|
||||||
}
|
|
||||||
|
|
||||||
// Get from active environment
|
|
||||||
env, err := GetActiveEnvironment()
|
|
||||||
if err == nil {
|
|
||||||
return env.GatewayURL
|
|
||||||
}
|
|
||||||
|
|
||||||
// Fallback to devnet
|
|
||||||
return "https://orama-devnet.network"
|
|
||||||
}
|
|
||||||
|
|
||||||
func handleAuthList() {
|
|
||||||
store, err := auth.LoadEnhancedCredentials()
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to load credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
gatewayURL := getGatewayURL()
|
|
||||||
|
|
||||||
// Show active environment
|
|
||||||
env, err := GetActiveEnvironment()
|
|
||||||
if err == nil {
|
|
||||||
fmt.Printf("🌍 Environment: %s\n", env.Name)
|
|
||||||
}
|
|
||||||
fmt.Printf("🔗 Gateway: %s\n\n", gatewayURL)
|
|
||||||
|
|
||||||
gwCreds := store.Gateways[gatewayURL]
|
|
||||||
if gwCreds == nil || len(gwCreds.Credentials) == 0 {
|
|
||||||
fmt.Println("No credentials stored for this environment.")
|
|
||||||
fmt.Println("Run 'orama auth login' to authenticate.")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("🔐 Stored Credentials (%d):\n\n", len(gwCreds.Credentials))
|
|
||||||
for i, creds := range gwCreds.Credentials {
|
|
||||||
defaultMark := ""
|
|
||||||
if i == gwCreds.DefaultIndex {
|
|
||||||
defaultMark = " ← active"
|
|
||||||
}
|
|
||||||
|
|
||||||
statusEmoji := "✅"
|
|
||||||
statusText := "valid"
|
|
||||||
if !creds.IsValid() {
|
|
||||||
statusEmoji = "❌"
|
|
||||||
statusText = "expired"
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf(" %d. %s Wallet: %s%s\n", i+1, statusEmoji, creds.Wallet, defaultMark)
|
|
||||||
fmt.Printf(" Namespace: %s | Status: %s\n", creds.Namespace, statusText)
|
|
||||||
if creds.Plan != "" {
|
|
||||||
fmt.Printf(" Plan: %s\n", creds.Plan)
|
|
||||||
}
|
|
||||||
if !creds.IssuedAt.IsZero() {
|
|
||||||
fmt.Printf(" Issued: %s\n", creds.IssuedAt.Format("2006-01-02 15:04:05"))
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func handleAuthSwitch() {
|
|
||||||
store, err := auth.LoadEnhancedCredentials()
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to load credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
gatewayURL := getGatewayURL()
|
|
||||||
|
|
||||||
gwCreds := store.Gateways[gatewayURL]
|
|
||||||
if gwCreds == nil || len(gwCreds.Credentials) == 0 {
|
|
||||||
fmt.Println("No credentials stored for this environment.")
|
|
||||||
fmt.Println("Run 'orama auth login' to authenticate first.")
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(gwCreds.Credentials) == 1 {
|
|
||||||
fmt.Println("Only one credential stored. Nothing to switch to.")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// Display menu
|
|
||||||
choice, credIndex, err := store.DisplayCredentialMenu(gatewayURL)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Menu selection failed: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
switch choice {
|
|
||||||
case auth.AuthChoiceUseCredential:
|
|
||||||
selectedCreds := gwCreds.Credentials[credIndex]
|
|
||||||
store.SetDefaultCredential(gatewayURL, credIndex)
|
|
||||||
selectedCreds.UpdateLastUsed()
|
|
||||||
if err := store.Save(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to save credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
fmt.Printf("✅ Switched to wallet: %s\n", selectedCreds.Wallet)
|
|
||||||
fmt.Printf("🏢 Namespace: %s\n", selectedCreds.Namespace)
|
|
||||||
|
|
||||||
case auth.AuthChoiceAddCredential:
|
|
||||||
fmt.Println("Use 'orama auth login' to add a new credential.")
|
|
||||||
|
|
||||||
case auth.AuthChoiceLogout:
|
|
||||||
store.ClearAllCredentials()
|
|
||||||
if err := store.Save(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "❌ Failed to clear credentials: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
fmt.Println("✅ All credentials cleared")
|
|
||||||
|
|
||||||
case auth.AuthChoiceExit:
|
|
||||||
fmt.Println("Cancelled.")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@ -1,318 +0,0 @@
|
|||||||
package build
|
|
||||||
|
|
||||||
import (
|
|
||||||
"archive/tar"
|
|
||||||
"compress/gzip"
|
|
||||||
"crypto/sha256"
|
|
||||||
"encoding/hex"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Manifest describes the contents of a binary archive.
|
|
||||||
type Manifest struct {
|
|
||||||
Version string `json:"version"`
|
|
||||||
Commit string `json:"commit"`
|
|
||||||
Date string `json:"date"`
|
|
||||||
Arch string `json:"arch"`
|
|
||||||
Checksums map[string]string `json:"checksums"` // filename -> sha256
|
|
||||||
}
|
|
||||||
|
|
||||||
// generateManifest creates the manifest with SHA256 checksums of all binaries.
|
|
||||||
func (b *Builder) generateManifest() (*Manifest, error) {
|
|
||||||
m := &Manifest{
|
|
||||||
Version: b.version,
|
|
||||||
Commit: b.commit,
|
|
||||||
Date: b.date,
|
|
||||||
Arch: b.flags.Arch,
|
|
||||||
Checksums: make(map[string]string),
|
|
||||||
}
|
|
||||||
|
|
||||||
entries, err := os.ReadDir(b.binDir)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, entry := range entries {
|
|
||||||
if entry.IsDir() {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
path := filepath.Join(b.binDir, entry.Name())
|
|
||||||
hash, err := sha256File(path)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("failed to hash %s: %w", entry.Name(), err)
|
|
||||||
}
|
|
||||||
m.Checksums[entry.Name()] = hash
|
|
||||||
}
|
|
||||||
|
|
||||||
return m, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// createArchive creates the tar.gz archive from the build directory.
|
|
||||||
func (b *Builder) createArchive(outputPath string, manifest *Manifest) error {
|
|
||||||
fmt.Printf("\nCreating archive: %s\n", outputPath)
|
|
||||||
|
|
||||||
// Write manifest.json to tmpDir
|
|
||||||
manifestData, err := json.MarshalIndent(manifest, "", " ")
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(filepath.Join(b.tmpDir, "manifest.json"), manifestData, 0644); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Create output file
|
|
||||||
f, err := os.Create(outputPath)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer f.Close()
|
|
||||||
|
|
||||||
gw := gzip.NewWriter(f)
|
|
||||||
defer gw.Close()
|
|
||||||
|
|
||||||
tw := tar.NewWriter(gw)
|
|
||||||
defer tw.Close()
|
|
||||||
|
|
||||||
// Add bin/ directory
|
|
||||||
if err := addDirToTar(tw, b.binDir, "bin"); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add systemd/ directory
|
|
||||||
systemdDir := filepath.Join(b.tmpDir, "systemd")
|
|
||||||
if _, err := os.Stat(systemdDir); err == nil {
|
|
||||||
if err := addDirToTar(tw, systemdDir, "systemd"); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add packages/ directory if it exists
|
|
||||||
packagesDir := filepath.Join(b.tmpDir, "packages")
|
|
||||||
if _, err := os.Stat(packagesDir); err == nil {
|
|
||||||
if err := addDirToTar(tw, packagesDir, "packages"); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add manifest.json
|
|
||||||
if err := addFileToTar(tw, filepath.Join(b.tmpDir, "manifest.json"), "manifest.json"); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add manifest.sig if it exists (created by --sign)
|
|
||||||
sigPath := filepath.Join(b.tmpDir, "manifest.sig")
|
|
||||||
if _, err := os.Stat(sigPath); err == nil {
|
|
||||||
if err := addFileToTar(tw, sigPath, "manifest.sig"); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Print summary
|
|
||||||
fmt.Printf(" bin/: %d binaries\n", len(manifest.Checksums))
|
|
||||||
fmt.Printf(" systemd/: namespace templates\n")
|
|
||||||
fmt.Printf(" manifest: v%s (%s) linux/%s\n", manifest.Version, manifest.Commit, manifest.Arch)
|
|
||||||
|
|
||||||
info, err := f.Stat()
|
|
||||||
if err == nil {
|
|
||||||
fmt.Printf(" size: %s\n", formatBytes(info.Size()))
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// signManifest signs the manifest hash using rootwallet CLI.
|
|
||||||
// Produces manifest.sig containing the hex-encoded EVM signature.
|
|
||||||
func (b *Builder) signManifest(manifest *Manifest) error {
|
|
||||||
fmt.Printf("\nSigning manifest with rootwallet...\n")
|
|
||||||
|
|
||||||
// Serialize manifest deterministically (compact JSON, sorted keys via json.Marshal)
|
|
||||||
manifestData, err := json.Marshal(manifest)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("failed to marshal manifest: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Hash the manifest JSON
|
|
||||||
hash := sha256.Sum256(manifestData)
|
|
||||||
hashHex := hex.EncodeToString(hash[:])
|
|
||||||
|
|
||||||
// Call rw sign <hash> --chain evm
|
|
||||||
cmd := exec.Command("rw", "sign", hashHex, "--chain", "evm")
|
|
||||||
var stdout, stderr strings.Builder
|
|
||||||
cmd.Stdout = &stdout
|
|
||||||
cmd.Stderr = &stderr
|
|
||||||
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("rw sign failed: %w\n%s", err, stderr.String())
|
|
||||||
}
|
|
||||||
|
|
||||||
signature := strings.TrimSpace(stdout.String())
|
|
||||||
if signature == "" {
|
|
||||||
return fmt.Errorf("rw sign produced empty signature")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Write signature file
|
|
||||||
sigPath := filepath.Join(b.tmpDir, "manifest.sig")
|
|
||||||
if err := os.WriteFile(sigPath, []byte(signature), 0644); err != nil {
|
|
||||||
return fmt.Errorf("failed to write manifest.sig: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf(" Manifest signed (SHA256: %s...)\n", hashHex[:16])
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// addDirToTar adds all files in a directory to the tar archive under the given prefix.
|
|
||||||
func addDirToTar(tw *tar.Writer, srcDir, prefix string) error {
|
|
||||||
return filepath.Walk(srcDir, func(path string, info os.FileInfo, err error) error {
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Calculate relative path
|
|
||||||
relPath, err := filepath.Rel(srcDir, path)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
tarPath := filepath.Join(prefix, relPath)
|
|
||||||
|
|
||||||
if info.IsDir() {
|
|
||||||
header := &tar.Header{
|
|
||||||
Name: tarPath + "/",
|
|
||||||
Mode: 0755,
|
|
||||||
Typeflag: tar.TypeDir,
|
|
||||||
}
|
|
||||||
return tw.WriteHeader(header)
|
|
||||||
}
|
|
||||||
|
|
||||||
return addFileToTar(tw, path, tarPath)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// addFileToTar adds a single file to the tar archive.
|
|
||||||
func addFileToTar(tw *tar.Writer, srcPath, tarPath string) error {
|
|
||||||
f, err := os.Open(srcPath)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer f.Close()
|
|
||||||
|
|
||||||
info, err := f.Stat()
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
header := &tar.Header{
|
|
||||||
Name: tarPath,
|
|
||||||
Size: info.Size(),
|
|
||||||
Mode: int64(info.Mode()),
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := tw.WriteHeader(header); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = io.Copy(tw, f)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// sha256File computes the SHA256 hash of a file.
|
|
||||||
func sha256File(path string) (string, error) {
|
|
||||||
f, err := os.Open(path)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
defer f.Close()
|
|
||||||
|
|
||||||
h := sha256.New()
|
|
||||||
if _, err := io.Copy(h, f); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
return hex.EncodeToString(h.Sum(nil)), nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// downloadFile downloads a URL to a local file path.
|
|
||||||
func downloadFile(url, destPath string) error {
|
|
||||||
client := &http.Client{Timeout: 5 * time.Minute}
|
|
||||||
resp, err := client.Get(url)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("failed to download %s: %w", url, err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
return fmt.Errorf("download %s returned status %d", url, resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
f, err := os.Create(destPath)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer f.Close()
|
|
||||||
|
|
||||||
_, err = io.Copy(f, resp.Body)
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// extractFileFromTarball extracts a single file from a tar.gz archive.
|
|
||||||
func extractFileFromTarball(tarPath, targetFile, destPath string) error {
|
|
||||||
f, err := os.Open(tarPath)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer f.Close()
|
|
||||||
|
|
||||||
gr, err := gzip.NewReader(f)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer gr.Close()
|
|
||||||
|
|
||||||
tr := tar.NewReader(gr)
|
|
||||||
for {
|
|
||||||
header, err := tr.Next()
|
|
||||||
if err == io.EOF {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Match the target file (strip leading ./ if present)
|
|
||||||
name := strings.TrimPrefix(header.Name, "./")
|
|
||||||
if name == targetFile {
|
|
||||||
out, err := os.OpenFile(destPath, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0755)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer out.Close()
|
|
||||||
|
|
||||||
if _, err := io.Copy(out, tr); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return fmt.Errorf("file %s not found in archive %s", targetFile, tarPath)
|
|
||||||
}
|
|
||||||
|
|
||||||
// formatBytes formats bytes into a human-readable string.
|
|
||||||
func formatBytes(b int64) string {
|
|
||||||
const unit = 1024
|
|
||||||
if b < unit {
|
|
||||||
return fmt.Sprintf("%d B", b)
|
|
||||||
}
|
|
||||||
div, exp := int64(unit), 0
|
|
||||||
for n := b / unit; n >= unit; n /= unit {
|
|
||||||
div *= unit
|
|
||||||
exp++
|
|
||||||
}
|
|
||||||
return fmt.Sprintf("%.1f %cB", float64(b)/float64(div), "KMGTPE"[exp])
|
|
||||||
}
|
|
||||||
@ -1,829 +0,0 @@
|
|||||||
package build
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"os/exec"
|
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/DeBrosOfficial/network/pkg/constants"
|
|
||||||
)
|
|
||||||
|
|
||||||
// oramaBinary defines a binary to cross-compile from the project source.
|
|
||||||
type oramaBinary struct {
|
|
||||||
Name string // output binary name
|
|
||||||
Package string // Go package path relative to project root
|
|
||||||
// Extra ldflags beyond the standard ones
|
|
||||||
ExtraLDFlags string
|
|
||||||
}
|
|
||||||
|
|
||||||
// Builder orchestrates the entire build process.
|
|
||||||
type Builder struct {
|
|
||||||
flags *Flags
|
|
||||||
projectDir string
|
|
||||||
tmpDir string
|
|
||||||
binDir string
|
|
||||||
version string
|
|
||||||
commit string
|
|
||||||
date string
|
|
||||||
}
|
|
||||||
|
|
||||||
// NewBuilder creates a new Builder.
|
|
||||||
func NewBuilder(flags *Flags) *Builder {
|
|
||||||
return &Builder{flags: flags}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Build runs the full build pipeline.
|
|
||||||
func (b *Builder) Build() error {
|
|
||||||
start := time.Now()
|
|
||||||
|
|
||||||
// Find project root
|
|
||||||
projectDir, err := findProjectRoot()
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
b.projectDir = projectDir
|
|
||||||
|
|
||||||
// Read version from Makefile or use "dev"
|
|
||||||
b.version = b.readVersion()
|
|
||||||
b.commit = b.readCommit()
|
|
||||||
b.date = time.Now().UTC().Format("2006-01-02T15:04:05Z")
|
|
||||||
|
|
||||||
// Create temp build directory
|
|
||||||
b.tmpDir, err = os.MkdirTemp("", "orama-build-*")
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("failed to create temp dir: %w", err)
|
|
||||||
}
|
|
||||||
defer os.RemoveAll(b.tmpDir)
|
|
||||||
|
|
||||||
b.binDir = filepath.Join(b.tmpDir, "bin")
|
|
||||||
if err := os.MkdirAll(b.binDir, 0755); err != nil {
|
|
||||||
return fmt.Errorf("failed to create bin dir: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("Building orama %s for linux/%s\n", b.version, b.flags.Arch)
|
|
||||||
fmt.Printf("Project: %s\n\n", b.projectDir)
|
|
||||||
|
|
||||||
// Step 1: Cross-compile Orama binaries
|
|
||||||
if err := b.buildOramaBinaries(); err != nil {
|
|
||||||
return fmt.Errorf("failed to build orama binaries: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 2: Cross-compile Vault Guardian (Zig)
|
|
||||||
if err := b.buildVaultGuardian(); err != nil {
|
|
||||||
return fmt.Errorf("failed to build vault-guardian: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 3: Cross-compile Olric
|
|
||||||
if err := b.buildOlric(); err != nil {
|
|
||||||
return fmt.Errorf("failed to build olric: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 4: Cross-compile IPFS Cluster
|
|
||||||
if err := b.buildIPFSCluster(); err != nil {
|
|
||||||
return fmt.Errorf("failed to build ipfs-cluster: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 5: Build CoreDNS with RQLite plugin
|
|
||||||
if err := b.buildCoreDNS(); err != nil {
|
|
||||||
return fmt.Errorf("failed to build coredns: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 6: Build Caddy with Orama DNS module
|
|
||||||
if err := b.buildCaddy(); err != nil {
|
|
||||||
return fmt.Errorf("failed to build caddy: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 7: Download pre-built IPFS Kubo
|
|
||||||
if err := b.downloadIPFS(); err != nil {
|
|
||||||
return fmt.Errorf("failed to download ipfs: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 8: Download pre-built RQLite
|
|
||||||
if err := b.downloadRQLite(); err != nil {
|
|
||||||
return fmt.Errorf("failed to download rqlite: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 9: Copy systemd templates
|
|
||||||
if err := b.copySystemdTemplates(); err != nil {
|
|
||||||
return fmt.Errorf("failed to copy systemd templates: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 10: Generate manifest
|
|
||||||
manifest, err := b.generateManifest()
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("failed to generate manifest: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 11: Sign manifest (optional)
|
|
||||||
if b.flags.Sign {
|
|
||||||
if err := b.signManifest(manifest); err != nil {
|
|
||||||
return fmt.Errorf("failed to sign manifest: %w", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 12: Create archive
|
|
||||||
outputPath := b.flags.Output
|
|
||||||
if outputPath == "" {
|
|
||||||
outputPath = fmt.Sprintf("/tmp/orama-%s-linux-%s.tar.gz", b.version, b.flags.Arch)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := b.createArchive(outputPath, manifest); err != nil {
|
|
||||||
return fmt.Errorf("failed to create archive: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
elapsed := time.Since(start).Round(time.Second)
|
|
||||||
fmt.Printf("\nBuild complete in %s\n", elapsed)
|
|
||||||
fmt.Printf("Archive: %s\n", outputPath)
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) buildOramaBinaries() error {
|
|
||||||
fmt.Println("[1/8] Cross-compiling Orama binaries...")
|
|
||||||
|
|
||||||
ldflags := fmt.Sprintf("-s -w -X 'main.version=%s' -X 'main.commit=%s' -X 'main.date=%s'",
|
|
||||||
b.version, b.commit, b.date)
|
|
||||||
|
|
||||||
gatewayLDFlags := fmt.Sprintf("%s -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildVersion=%s' -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildCommit=%s' -X 'github.com/DeBrosOfficial/network/pkg/gateway.BuildTime=%s'",
|
|
||||||
ldflags, b.version, b.commit, b.date)
|
|
||||||
|
|
||||||
binaries := []oramaBinary{
|
|
||||||
{Name: "orama", Package: "./cmd/cli/"},
|
|
||||||
{Name: "orama-node", Package: "./cmd/node/"},
|
|
||||||
{Name: "gateway", Package: "./cmd/gateway/", ExtraLDFlags: gatewayLDFlags},
|
|
||||||
{Name: "identity", Package: "./cmd/identity/"},
|
|
||||||
{Name: "sfu", Package: "./cmd/sfu/"},
|
|
||||||
{Name: "turn", Package: "./cmd/turn/"},
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, bin := range binaries {
|
|
||||||
flags := ldflags
|
|
||||||
if bin.ExtraLDFlags != "" {
|
|
||||||
flags = bin.ExtraLDFlags
|
|
||||||
}
|
|
||||||
|
|
||||||
output := filepath.Join(b.binDir, bin.Name)
|
|
||||||
cmd := exec.Command("go", "build",
|
|
||||||
"-ldflags", flags,
|
|
||||||
"-trimpath",
|
|
||||||
"-o", output,
|
|
||||||
bin.Package)
|
|
||||||
cmd.Dir = b.projectDir
|
|
||||||
cmd.Env = b.crossEnv()
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
|
|
||||||
if b.flags.Verbose {
|
|
||||||
fmt.Printf(" go build -o %s %s\n", bin.Name, bin.Package)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("failed to build %s: %w", bin.Name, err)
|
|
||||||
}
|
|
||||||
fmt.Printf(" ✓ %s\n", bin.Name)
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) buildVaultGuardian() error {
|
|
||||||
fmt.Println("[2/8] Cross-compiling Vault Guardian (Zig)...")
|
|
||||||
|
|
||||||
// Ensure zig is available
|
|
||||||
if _, err := exec.LookPath("zig"); err != nil {
|
|
||||||
return fmt.Errorf("zig not found in PATH — install from https://ziglang.org/download/")
|
|
||||||
}
|
|
||||||
|
|
||||||
// Vault source is sibling to orama project
|
|
||||||
vaultDir := filepath.Join(b.projectDir, "..", "orama-vault")
|
|
||||||
if _, err := os.Stat(filepath.Join(vaultDir, "build.zig")); err != nil {
|
|
||||||
return fmt.Errorf("vault source not found at %s — expected orama-vault as sibling directory: %w", vaultDir, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Map Go arch to Zig target triple
|
|
||||||
var zigTarget string
|
|
||||||
switch b.flags.Arch {
|
|
||||||
case "amd64":
|
|
||||||
zigTarget = "x86_64-linux-musl"
|
|
||||||
case "arm64":
|
|
||||||
zigTarget = "aarch64-linux-musl"
|
|
||||||
default:
|
|
||||||
return fmt.Errorf("unsupported architecture for vault: %s", b.flags.Arch)
|
|
||||||
}
|
|
||||||
|
|
||||||
if b.flags.Verbose {
|
|
||||||
fmt.Printf(" zig build -Dtarget=%s -Doptimize=ReleaseSafe\n", zigTarget)
|
|
||||||
}
|
|
||||||
|
|
||||||
cmd := exec.Command("zig", "build",
|
|
||||||
fmt.Sprintf("-Dtarget=%s", zigTarget),
|
|
||||||
"-Doptimize=ReleaseSafe")
|
|
||||||
cmd.Dir = vaultDir
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("zig build failed: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Copy output binary to build bin dir
|
|
||||||
src := filepath.Join(vaultDir, "zig-out", "bin", "vault-guardian")
|
|
||||||
dst := filepath.Join(b.binDir, "vault-guardian")
|
|
||||||
if err := copyFile(src, dst); err != nil {
|
|
||||||
return fmt.Errorf("failed to copy vault-guardian binary: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println(" ✓ vault-guardian")
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// copyFile copies a file from src to dst, preserving executable permissions.
|
|
||||||
func copyFile(src, dst string) error {
|
|
||||||
srcFile, err := os.Open(src)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer srcFile.Close()
|
|
||||||
|
|
||||||
dstFile, err := os.OpenFile(dst, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0755)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer dstFile.Close()
|
|
||||||
|
|
||||||
if _, err := srcFile.WriteTo(dstFile); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) buildOlric() error {
|
|
||||||
fmt.Printf("[3/8] Cross-compiling Olric %s...\n", constants.OlricVersion)
|
|
||||||
|
|
||||||
// go install doesn't support cross-compilation with GOBIN set,
|
|
||||||
// so we create a temporary module and use go build -o instead.
|
|
||||||
tmpDir, err := os.MkdirTemp("", "olric-build-*")
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("create temp dir: %w", err)
|
|
||||||
}
|
|
||||||
defer os.RemoveAll(tmpDir)
|
|
||||||
|
|
||||||
modInit := exec.Command("go", "mod", "init", "olric-build")
|
|
||||||
modInit.Dir = tmpDir
|
|
||||||
modInit.Stderr = os.Stderr
|
|
||||||
if err := modInit.Run(); err != nil {
|
|
||||||
return fmt.Errorf("go mod init: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
modGet := exec.Command("go", "get",
|
|
||||||
fmt.Sprintf("github.com/olric-data/olric/cmd/olric-server@%s", constants.OlricVersion))
|
|
||||||
modGet.Dir = tmpDir
|
|
||||||
modGet.Env = append(os.Environ(),
|
|
||||||
"GOPROXY=https://proxy.golang.org|direct",
|
|
||||||
"GONOSUMDB=*")
|
|
||||||
modGet.Stderr = os.Stderr
|
|
||||||
if err := modGet.Run(); err != nil {
|
|
||||||
return fmt.Errorf("go get olric: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
cmd := exec.Command("go", "build",
|
|
||||||
"-ldflags", "-s -w",
|
|
||||||
"-trimpath",
|
|
||||||
"-o", filepath.Join(b.binDir, "olric-server"),
|
|
||||||
fmt.Sprintf("github.com/olric-data/olric/cmd/olric-server"))
|
|
||||||
cmd.Dir = tmpDir
|
|
||||||
cmd.Env = append(b.crossEnv(),
|
|
||||||
"GOPROXY=https://proxy.golang.org|direct",
|
|
||||||
"GONOSUMDB=*")
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
fmt.Println(" ✓ olric-server")
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) buildIPFSCluster() error {
|
|
||||||
fmt.Printf("[4/8] Cross-compiling IPFS Cluster %s...\n", constants.IPFSClusterVersion)
|
|
||||||
|
|
||||||
tmpDir, err := os.MkdirTemp("", "ipfs-cluster-build-*")
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("create temp dir: %w", err)
|
|
||||||
}
|
|
||||||
defer os.RemoveAll(tmpDir)
|
|
||||||
|
|
||||||
modInit := exec.Command("go", "mod", "init", "ipfs-cluster-build")
|
|
||||||
modInit.Dir = tmpDir
|
|
||||||
modInit.Stderr = os.Stderr
|
|
||||||
if err := modInit.Run(); err != nil {
|
|
||||||
return fmt.Errorf("go mod init: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
modGet := exec.Command("go", "get",
|
|
||||||
fmt.Sprintf("github.com/ipfs-cluster/ipfs-cluster/cmd/ipfs-cluster-service@%s", constants.IPFSClusterVersion))
|
|
||||||
modGet.Dir = tmpDir
|
|
||||||
modGet.Env = append(os.Environ(),
|
|
||||||
"GOPROXY=https://proxy.golang.org|direct",
|
|
||||||
"GONOSUMDB=*")
|
|
||||||
modGet.Stderr = os.Stderr
|
|
||||||
if err := modGet.Run(); err != nil {
|
|
||||||
return fmt.Errorf("go get ipfs-cluster: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
cmd := exec.Command("go", "build",
|
|
||||||
"-ldflags", "-s -w",
|
|
||||||
"-trimpath",
|
|
||||||
"-o", filepath.Join(b.binDir, "ipfs-cluster-service"),
|
|
||||||
"github.com/ipfs-cluster/ipfs-cluster/cmd/ipfs-cluster-service")
|
|
||||||
cmd.Dir = tmpDir
|
|
||||||
cmd.Env = append(b.crossEnv(),
|
|
||||||
"GOPROXY=https://proxy.golang.org|direct",
|
|
||||||
"GONOSUMDB=*")
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
fmt.Println(" ✓ ipfs-cluster-service")
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) buildCoreDNS() error {
|
|
||||||
fmt.Printf("[5/8] Building CoreDNS %s with RQLite plugin...\n", constants.CoreDNSVersion)
|
|
||||||
|
|
||||||
buildDir := filepath.Join(b.tmpDir, "coredns-build")
|
|
||||||
|
|
||||||
// Clone CoreDNS
|
|
||||||
fmt.Println(" Cloning CoreDNS...")
|
|
||||||
cmd := exec.Command("git", "clone", "--depth", "1",
|
|
||||||
"--branch", "v"+constants.CoreDNSVersion,
|
|
||||||
"https://github.com/coredns/coredns.git", buildDir)
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("failed to clone coredns: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Copy RQLite plugin from local source
|
|
||||||
pluginSrc := filepath.Join(b.projectDir, "pkg", "coredns", "rqlite")
|
|
||||||
pluginDst := filepath.Join(buildDir, "plugin", "rqlite")
|
|
||||||
if err := os.MkdirAll(pluginDst, 0755); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
entries, err := os.ReadDir(pluginSrc)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("failed to read rqlite plugin source at %s: %w", pluginSrc, err)
|
|
||||||
}
|
|
||||||
for _, entry := range entries {
|
|
||||||
if entry.IsDir() || filepath.Ext(entry.Name()) != ".go" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
data, err := os.ReadFile(filepath.Join(pluginSrc, entry.Name()))
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(filepath.Join(pluginDst, entry.Name()), data, 0644); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Write plugin.cfg (same as build-linux-coredns.sh)
|
|
||||||
pluginCfg := `metadata:metadata
|
|
||||||
cancel:cancel
|
|
||||||
tls:tls
|
|
||||||
reload:reload
|
|
||||||
nsid:nsid
|
|
||||||
bufsize:bufsize
|
|
||||||
root:root
|
|
||||||
bind:bind
|
|
||||||
debug:debug
|
|
||||||
trace:trace
|
|
||||||
ready:ready
|
|
||||||
health:health
|
|
||||||
pprof:pprof
|
|
||||||
prometheus:metrics
|
|
||||||
errors:errors
|
|
||||||
log:log
|
|
||||||
dnstap:dnstap
|
|
||||||
local:local
|
|
||||||
dns64:dns64
|
|
||||||
acl:acl
|
|
||||||
any:any
|
|
||||||
chaos:chaos
|
|
||||||
loadbalance:loadbalance
|
|
||||||
cache:cache
|
|
||||||
rewrite:rewrite
|
|
||||||
header:header
|
|
||||||
dnssec:dnssec
|
|
||||||
autopath:autopath
|
|
||||||
minimal:minimal
|
|
||||||
template:template
|
|
||||||
transfer:transfer
|
|
||||||
hosts:hosts
|
|
||||||
file:file
|
|
||||||
auto:auto
|
|
||||||
secondary:secondary
|
|
||||||
loop:loop
|
|
||||||
forward:forward
|
|
||||||
grpc:grpc
|
|
||||||
erratic:erratic
|
|
||||||
whoami:whoami
|
|
||||||
on:github.com/coredns/caddy/onevent
|
|
||||||
sign:sign
|
|
||||||
view:view
|
|
||||||
rqlite:rqlite
|
|
||||||
`
|
|
||||||
if err := os.WriteFile(filepath.Join(buildDir, "plugin.cfg"), []byte(pluginCfg), 0644); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Add dependencies
|
|
||||||
fmt.Println(" Adding dependencies...")
|
|
||||||
goPath := os.Getenv("PATH")
|
|
||||||
baseEnv := append(os.Environ(),
|
|
||||||
"PATH="+goPath,
|
|
||||||
"GOPROXY=https://proxy.golang.org|direct",
|
|
||||||
"GONOSUMDB=*")
|
|
||||||
|
|
||||||
for _, dep := range []string{"github.com/miekg/dns@latest", "go.uber.org/zap@latest"} {
|
|
||||||
cmd := exec.Command("go", "get", dep)
|
|
||||||
cmd.Dir = buildDir
|
|
||||||
cmd.Env = baseEnv
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("failed to get %s: %w", dep, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
cmd = exec.Command("go", "mod", "tidy")
|
|
||||||
cmd.Dir = buildDir
|
|
||||||
cmd.Env = baseEnv
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("go mod tidy failed: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Generate plugin code
|
|
||||||
fmt.Println(" Generating plugin code...")
|
|
||||||
cmd = exec.Command("go", "generate")
|
|
||||||
cmd.Dir = buildDir
|
|
||||||
cmd.Env = baseEnv
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("go generate failed: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Cross-compile
|
|
||||||
fmt.Println(" Building binary...")
|
|
||||||
cmd = exec.Command("go", "build",
|
|
||||||
"-ldflags", "-s -w",
|
|
||||||
"-trimpath",
|
|
||||||
"-o", filepath.Join(b.binDir, "coredns"))
|
|
||||||
cmd.Dir = buildDir
|
|
||||||
cmd.Env = append(baseEnv,
|
|
||||||
"GOOS=linux",
|
|
||||||
fmt.Sprintf("GOARCH=%s", b.flags.Arch),
|
|
||||||
"CGO_ENABLED=0")
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("build failed: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println(" ✓ coredns")
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) buildCaddy() error {
|
|
||||||
fmt.Printf("[6/8] Building Caddy %s with Orama DNS module...\n", constants.CaddyVersion)
|
|
||||||
|
|
||||||
// Ensure xcaddy is available
|
|
||||||
if _, err := exec.LookPath("xcaddy"); err != nil {
|
|
||||||
return fmt.Errorf("xcaddy not found in PATH — install with: go install github.com/caddyserver/xcaddy/cmd/xcaddy@latest")
|
|
||||||
}
|
|
||||||
|
|
||||||
moduleDir := filepath.Join(b.tmpDir, "caddy-dns-orama")
|
|
||||||
if err := os.MkdirAll(moduleDir, 0755); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Write go.mod
|
|
||||||
goMod := fmt.Sprintf(`module github.com/DeBrosOfficial/caddy-dns-orama
|
|
||||||
|
|
||||||
go 1.22
|
|
||||||
|
|
||||||
require (
|
|
||||||
github.com/caddyserver/caddy/v2 v2.%s
|
|
||||||
github.com/libdns/libdns v1.1.0
|
|
||||||
)
|
|
||||||
`, constants.CaddyVersion[2:])
|
|
||||||
if err := os.WriteFile(filepath.Join(moduleDir, "go.mod"), []byte(goMod), 0644); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Write provider.go — read from the caddy installer's generated code
|
|
||||||
// We inline the same provider code used by the VPS-side caddy installer
|
|
||||||
providerCode := generateCaddyProviderCode()
|
|
||||||
if err := os.WriteFile(filepath.Join(moduleDir, "provider.go"), []byte(providerCode), 0644); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// go mod tidy
|
|
||||||
cmd := exec.Command("go", "mod", "tidy")
|
|
||||||
cmd.Dir = moduleDir
|
|
||||||
cmd.Env = append(os.Environ(),
|
|
||||||
"GOPROXY=https://proxy.golang.org|direct",
|
|
||||||
"GONOSUMDB=*")
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("go mod tidy failed: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Build with xcaddy
|
|
||||||
fmt.Println(" Building binary...")
|
|
||||||
cmd = exec.Command("xcaddy", "build",
|
|
||||||
"v"+constants.CaddyVersion,
|
|
||||||
"--with", "github.com/DeBrosOfficial/caddy-dns-orama="+moduleDir,
|
|
||||||
"--output", filepath.Join(b.binDir, "caddy"))
|
|
||||||
cmd.Env = append(os.Environ(),
|
|
||||||
"GOOS=linux",
|
|
||||||
fmt.Sprintf("GOARCH=%s", b.flags.Arch),
|
|
||||||
"GOPROXY=https://proxy.golang.org|direct",
|
|
||||||
"GONOSUMDB=*")
|
|
||||||
cmd.Stdout = os.Stdout
|
|
||||||
cmd.Stderr = os.Stderr
|
|
||||||
if err := cmd.Run(); err != nil {
|
|
||||||
return fmt.Errorf("xcaddy build failed: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println(" ✓ caddy")
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) downloadIPFS() error {
|
|
||||||
fmt.Printf("[7/8] Downloading IPFS Kubo %s...\n", constants.IPFSKuboVersion)
|
|
||||||
|
|
||||||
arch := b.flags.Arch
|
|
||||||
tarball := fmt.Sprintf("kubo_%s_linux-%s.tar.gz", constants.IPFSKuboVersion, arch)
|
|
||||||
url := fmt.Sprintf("https://dist.ipfs.tech/kubo/%s/%s", constants.IPFSKuboVersion, tarball)
|
|
||||||
tarPath := filepath.Join(b.tmpDir, tarball)
|
|
||||||
|
|
||||||
if err := downloadFile(url, tarPath); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Extract ipfs binary from kubo/ipfs
|
|
||||||
if err := extractFileFromTarball(tarPath, "kubo/ipfs", filepath.Join(b.binDir, "ipfs")); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println(" ✓ ipfs")
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) downloadRQLite() error {
|
|
||||||
fmt.Printf("[8/8] Downloading RQLite %s...\n", constants.RQLiteVersion)
|
|
||||||
|
|
||||||
arch := b.flags.Arch
|
|
||||||
tarball := fmt.Sprintf("rqlite-v%s-linux-%s.tar.gz", constants.RQLiteVersion, arch)
|
|
||||||
url := fmt.Sprintf("https://github.com/rqlite/rqlite/releases/download/v%s/%s", constants.RQLiteVersion, tarball)
|
|
||||||
tarPath := filepath.Join(b.tmpDir, tarball)
|
|
||||||
|
|
||||||
if err := downloadFile(url, tarPath); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
// Extract rqlited binary
|
|
||||||
extractDir := fmt.Sprintf("rqlite-v%s-linux-%s", constants.RQLiteVersion, arch)
|
|
||||||
if err := extractFileFromTarball(tarPath, extractDir+"/rqlited", filepath.Join(b.binDir, "rqlited")); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Println(" ✓ rqlited")
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) copySystemdTemplates() error {
|
|
||||||
systemdSrc := filepath.Join(b.projectDir, "systemd")
|
|
||||||
systemdDst := filepath.Join(b.tmpDir, "systemd")
|
|
||||||
if err := os.MkdirAll(systemdDst, 0755); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
|
|
||||||
entries, err := os.ReadDir(systemdSrc)
|
|
||||||
if err != nil {
|
|
||||||
return fmt.Errorf("failed to read systemd dir: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, entry := range entries {
|
|
||||||
if entry.IsDir() || !strings.HasSuffix(entry.Name(), ".service") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
data, err := os.ReadFile(filepath.Join(systemdSrc, entry.Name()))
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(filepath.Join(systemdDst, entry.Name()), data, 0644); err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// crossEnv returns the environment for cross-compilation.
|
|
||||||
func (b *Builder) crossEnv() []string {
|
|
||||||
return append(os.Environ(),
|
|
||||||
"GOOS=linux",
|
|
||||||
fmt.Sprintf("GOARCH=%s", b.flags.Arch),
|
|
||||||
"CGO_ENABLED=0")
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) readVersion() string {
|
|
||||||
// Try to read from Makefile
|
|
||||||
data, err := os.ReadFile(filepath.Join(b.projectDir, "Makefile"))
|
|
||||||
if err != nil {
|
|
||||||
return "dev"
|
|
||||||
}
|
|
||||||
for _, line := range strings.Split(string(data), "\n") {
|
|
||||||
line = strings.TrimSpace(line)
|
|
||||||
if strings.HasPrefix(line, "VERSION") {
|
|
||||||
parts := strings.SplitN(line, ":=", 2)
|
|
||||||
if len(parts) == 2 {
|
|
||||||
return strings.TrimSpace(parts[1])
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return "dev"
|
|
||||||
}
|
|
||||||
|
|
||||||
func (b *Builder) readCommit() string {
|
|
||||||
cmd := exec.Command("git", "rev-parse", "--short", "HEAD")
|
|
||||||
cmd.Dir = b.projectDir
|
|
||||||
out, err := cmd.Output()
|
|
||||||
if err != nil {
|
|
||||||
return "unknown"
|
|
||||||
}
|
|
||||||
return strings.TrimSpace(string(out))
|
|
||||||
}
|
|
||||||
|
|
||||||
// generateCaddyProviderCode returns the Caddy DNS provider Go source.
|
|
||||||
// This is the same code used by the VPS-side caddy installer.
|
|
||||||
func generateCaddyProviderCode() string {
|
|
||||||
return `// Package orama implements a DNS provider for Caddy that uses the Orama Network
|
|
||||||
// gateway's internal ACME API for DNS-01 challenge validation.
|
|
||||||
package orama
|
|
||||||
|
|
||||||
import (
|
|
||||||
"bytes"
|
|
||||||
"context"
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"net/http"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"github.com/caddyserver/caddy/v2"
|
|
||||||
"github.com/caddyserver/caddy/v2/caddyconfig/caddyfile"
|
|
||||||
"github.com/libdns/libdns"
|
|
||||||
)
|
|
||||||
|
|
||||||
func init() {
|
|
||||||
caddy.RegisterModule(Provider{})
|
|
||||||
}
|
|
||||||
|
|
||||||
// Provider wraps the Orama DNS provider for Caddy.
|
|
||||||
type Provider struct {
|
|
||||||
// Endpoint is the URL of the Orama gateway's ACME API
|
|
||||||
// Default: http://localhost:6001/v1/internal/acme
|
|
||||||
Endpoint string ` + "`json:\"endpoint,omitempty\"`" + `
|
|
||||||
}
|
|
||||||
|
|
||||||
// CaddyModule returns the Caddy module information.
|
|
||||||
func (Provider) CaddyModule() caddy.ModuleInfo {
|
|
||||||
return caddy.ModuleInfo{
|
|
||||||
ID: "dns.providers.orama",
|
|
||||||
New: func() caddy.Module { return new(Provider) },
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Provision sets up the module.
|
|
||||||
func (p *Provider) Provision(ctx caddy.Context) error {
|
|
||||||
if p.Endpoint == "" {
|
|
||||||
p.Endpoint = "http://localhost:6001/v1/internal/acme"
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// UnmarshalCaddyfile parses the Caddyfile configuration.
|
|
||||||
func (p *Provider) UnmarshalCaddyfile(d *caddyfile.Dispenser) error {
|
|
||||||
for d.Next() {
|
|
||||||
for d.NextBlock(0) {
|
|
||||||
switch d.Val() {
|
|
||||||
case "endpoint":
|
|
||||||
if !d.NextArg() {
|
|
||||||
return d.ArgErr()
|
|
||||||
}
|
|
||||||
p.Endpoint = d.Val()
|
|
||||||
default:
|
|
||||||
return d.Errf("unrecognized option: %s", d.Val())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// AppendRecords adds records to the zone.
|
|
||||||
func (p *Provider) AppendRecords(ctx context.Context, zone string, records []libdns.Record) ([]libdns.Record, error) {
|
|
||||||
var added []libdns.Record
|
|
||||||
for _, rec := range records {
|
|
||||||
rr := rec.RR()
|
|
||||||
if rr.Type != "TXT" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
fqdn := rr.Name + "." + zone
|
|
||||||
payload := map[string]string{"fqdn": fqdn, "value": rr.Data}
|
|
||||||
body, err := json.Marshal(payload)
|
|
||||||
if err != nil {
|
|
||||||
return added, fmt.Errorf("failed to marshal request: %w", err)
|
|
||||||
}
|
|
||||||
req, err := http.NewRequestWithContext(ctx, "POST", p.Endpoint+"/present", bytes.NewReader(body))
|
|
||||||
if err != nil {
|
|
||||||
return added, fmt.Errorf("failed to create request: %w", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
client := &http.Client{Timeout: 30 * time.Second}
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return added, fmt.Errorf("failed to present challenge: %w", err)
|
|
||||||
}
|
|
||||||
resp.Body.Close()
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
return added, fmt.Errorf("present failed with status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
added = append(added, rec)
|
|
||||||
}
|
|
||||||
return added, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// DeleteRecords removes records from the zone.
|
|
||||||
func (p *Provider) DeleteRecords(ctx context.Context, zone string, records []libdns.Record) ([]libdns.Record, error) {
|
|
||||||
var deleted []libdns.Record
|
|
||||||
for _, rec := range records {
|
|
||||||
rr := rec.RR()
|
|
||||||
if rr.Type != "TXT" {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
fqdn := rr.Name + "." + zone
|
|
||||||
payload := map[string]string{"fqdn": fqdn, "value": rr.Data}
|
|
||||||
body, err := json.Marshal(payload)
|
|
||||||
if err != nil {
|
|
||||||
return deleted, fmt.Errorf("failed to marshal request: %w", err)
|
|
||||||
}
|
|
||||||
req, err := http.NewRequestWithContext(ctx, "POST", p.Endpoint+"/cleanup", bytes.NewReader(body))
|
|
||||||
if err != nil {
|
|
||||||
return deleted, fmt.Errorf("failed to create request: %w", err)
|
|
||||||
}
|
|
||||||
req.Header.Set("Content-Type", "application/json")
|
|
||||||
client := &http.Client{Timeout: 30 * time.Second}
|
|
||||||
resp, err := client.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return deleted, fmt.Errorf("failed to cleanup challenge: %w", err)
|
|
||||||
}
|
|
||||||
resp.Body.Close()
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
return deleted, fmt.Errorf("cleanup failed with status %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
deleted = append(deleted, rec)
|
|
||||||
}
|
|
||||||
return deleted, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// GetRecords returns the records in the zone. Not used for ACME.
|
|
||||||
func (p *Provider) GetRecords(ctx context.Context, zone string) ([]libdns.Record, error) {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// SetRecords sets the records in the zone. Not used for ACME.
|
|
||||||
func (p *Provider) SetRecords(ctx context.Context, zone string, records []libdns.Record) ([]libdns.Record, error) {
|
|
||||||
return nil, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// Interface guards
|
|
||||||
var (
|
|
||||||
_ caddy.Module = (*Provider)(nil)
|
|
||||||
_ caddy.Provisioner = (*Provider)(nil)
|
|
||||||
_ caddyfile.Unmarshaler = (*Provider)(nil)
|
|
||||||
_ libdns.RecordAppender = (*Provider)(nil)
|
|
||||||
_ libdns.RecordDeleter = (*Provider)(nil)
|
|
||||||
_ libdns.RecordGetter = (*Provider)(nil)
|
|
||||||
_ libdns.RecordSetter = (*Provider)(nil)
|
|
||||||
)
|
|
||||||
`
|
|
||||||
}
|
|
||||||
@ -1,82 +0,0 @@
|
|||||||
package build
|
|
||||||
|
|
||||||
import (
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
"path/filepath"
|
|
||||||
"runtime"
|
|
||||||
)
|
|
||||||
|
|
||||||
// Flags represents build command flags.
|
|
||||||
type Flags struct {
|
|
||||||
Arch string
|
|
||||||
Output string
|
|
||||||
Verbose bool
|
|
||||||
Sign bool // Sign the archive manifest with rootwallet
|
|
||||||
}
|
|
||||||
|
|
||||||
// Handle is the entry point for the build command.
|
|
||||||
func Handle(args []string) {
|
|
||||||
flags, err := parseFlags(args)
|
|
||||||
if err != nil {
|
|
||||||
if err == flag.ErrHelp {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
b := NewBuilder(flags)
|
|
||||||
if err := b.Build(); err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func parseFlags(args []string) (*Flags, error) {
|
|
||||||
fs := flag.NewFlagSet("build", flag.ContinueOnError)
|
|
||||||
fs.SetOutput(os.Stderr)
|
|
||||||
|
|
||||||
flags := &Flags{}
|
|
||||||
|
|
||||||
fs.StringVar(&flags.Arch, "arch", "amd64", "Target architecture (amd64, arm64)")
|
|
||||||
fs.StringVar(&flags.Output, "output", "", "Output archive path (default: /tmp/orama-<version>-linux-<arch>.tar.gz)")
|
|
||||||
fs.BoolVar(&flags.Verbose, "verbose", false, "Verbose output")
|
|
||||||
fs.BoolVar(&flags.Sign, "sign", false, "Sign the manifest with rootwallet (requires rw in PATH)")
|
|
||||||
|
|
||||||
if err := fs.Parse(args); err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
|
|
||||||
return flags, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// findProjectRoot walks up from the current directory looking for go.mod.
|
|
||||||
func findProjectRoot() (string, error) {
|
|
||||||
dir, err := os.Getwd()
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
|
|
||||||
for {
|
|
||||||
if _, err := os.Stat(filepath.Join(dir, "go.mod")); err == nil {
|
|
||||||
// Verify it's the network project
|
|
||||||
if _, err := os.Stat(filepath.Join(dir, "cmd", "cli")); err == nil {
|
|
||||||
return dir, nil
|
|
||||||
}
|
|
||||||
}
|
|
||||||
parent := filepath.Dir(dir)
|
|
||||||
if parent == dir {
|
|
||||||
break
|
|
||||||
}
|
|
||||||
dir = parent
|
|
||||||
}
|
|
||||||
|
|
||||||
return "", fmt.Errorf("could not find project root (no go.mod with cmd/cli found)")
|
|
||||||
}
|
|
||||||
|
|
||||||
// detectHostArch returns the host architecture in Go naming convention.
|
|
||||||
func detectHostArch() string {
|
|
||||||
return runtime.GOARCH
|
|
||||||
}
|
|
||||||
@ -1,80 +0,0 @@
|
|||||||
package cluster
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleCommand handles cluster subcommands.
|
|
||||||
func HandleCommand(args []string) {
|
|
||||||
if len(args) == 0 {
|
|
||||||
ShowHelp()
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
subcommand := args[0]
|
|
||||||
subargs := args[1:]
|
|
||||||
|
|
||||||
switch subcommand {
|
|
||||||
case "status":
|
|
||||||
HandleStatus(subargs)
|
|
||||||
case "health":
|
|
||||||
HandleHealth(subargs)
|
|
||||||
case "rqlite":
|
|
||||||
HandleRQLite(subargs)
|
|
||||||
case "watch":
|
|
||||||
HandleWatch(subargs)
|
|
||||||
case "help":
|
|
||||||
ShowHelp()
|
|
||||||
default:
|
|
||||||
fmt.Fprintf(os.Stderr, "Unknown cluster subcommand: %s\n", subcommand)
|
|
||||||
ShowHelp()
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// hasFlag checks if a flag is present in the args slice.
|
|
||||||
func hasFlag(args []string, flag string) bool {
|
|
||||||
for _, a := range args {
|
|
||||||
if a == flag {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
// getFlagValue returns the value of a flag from the args slice.
|
|
||||||
// Returns empty string if the flag is not found or has no value.
|
|
||||||
func getFlagValue(args []string, flag string) string {
|
|
||||||
for i, a := range args {
|
|
||||||
if a == flag && i+1 < len(args) {
|
|
||||||
return args[i+1]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
// ShowHelp displays help information for cluster commands.
|
|
||||||
func ShowHelp() {
|
|
||||||
fmt.Printf("Cluster Management Commands\n\n")
|
|
||||||
fmt.Printf("Usage: orama cluster <subcommand> [options]\n\n")
|
|
||||||
fmt.Printf("Subcommands:\n")
|
|
||||||
fmt.Printf(" status - Show cluster node status (RQLite + Olric)\n")
|
|
||||||
fmt.Printf(" Options:\n")
|
|
||||||
fmt.Printf(" --all - SSH into all nodes from nodes.conf (TODO)\n")
|
|
||||||
fmt.Printf(" health - Run cluster health checks\n")
|
|
||||||
fmt.Printf(" rqlite <subcommand> - RQLite-specific commands\n")
|
|
||||||
fmt.Printf(" status - Show detailed Raft state for local node\n")
|
|
||||||
fmt.Printf(" voters - Show current voter list\n")
|
|
||||||
fmt.Printf(" backup [--output FILE] - Trigger manual backup\n")
|
|
||||||
fmt.Printf(" watch - Live cluster status monitor\n")
|
|
||||||
fmt.Printf(" Options:\n")
|
|
||||||
fmt.Printf(" --interval SECONDS - Refresh interval (default: 10)\n\n")
|
|
||||||
fmt.Printf("Examples:\n")
|
|
||||||
fmt.Printf(" orama cluster status\n")
|
|
||||||
fmt.Printf(" orama cluster health\n")
|
|
||||||
fmt.Printf(" orama cluster rqlite status\n")
|
|
||||||
fmt.Printf(" orama cluster rqlite voters\n")
|
|
||||||
fmt.Printf(" orama cluster rqlite backup --output /tmp/backup.db\n")
|
|
||||||
fmt.Printf(" orama cluster watch --interval 5\n")
|
|
||||||
}
|
|
||||||
@ -1,244 +0,0 @@
|
|||||||
package cluster
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"os"
|
|
||||||
)
|
|
||||||
|
|
||||||
// checkResult represents the outcome of a single health check.
|
|
||||||
type checkResult struct {
|
|
||||||
Name string
|
|
||||||
Status string // "PASS", "FAIL", "WARN"
|
|
||||||
Detail string
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleHealth handles the "orama cluster health" command.
|
|
||||||
func HandleHealth(args []string) {
|
|
||||||
fmt.Printf("Cluster Health Check\n")
|
|
||||||
fmt.Printf("====================\n\n")
|
|
||||||
|
|
||||||
var results []checkResult
|
|
||||||
|
|
||||||
// Check 1: RQLite reachable
|
|
||||||
status, err := queryRQLiteStatus()
|
|
||||||
if err != nil {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "RQLite reachable",
|
|
||||||
Status: "FAIL",
|
|
||||||
Detail: fmt.Sprintf("Cannot connect to RQLite: %v", err),
|
|
||||||
})
|
|
||||||
printHealthResults(results)
|
|
||||||
os.Exit(1)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "RQLite reachable",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: fmt.Sprintf("HTTP API responding on %s", status.HTTP.Address),
|
|
||||||
})
|
|
||||||
|
|
||||||
// Check 2: Raft state is leader or follower (not candidate or shutdown)
|
|
||||||
raftState := status.Store.Raft.State
|
|
||||||
switch raftState {
|
|
||||||
case "Leader", "Follower":
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Raft state healthy",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: fmt.Sprintf("Node is %s", raftState),
|
|
||||||
})
|
|
||||||
case "Candidate":
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Raft state healthy",
|
|
||||||
Status: "WARN",
|
|
||||||
Detail: "Node is Candidate (election in progress)",
|
|
||||||
})
|
|
||||||
default:
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Raft state healthy",
|
|
||||||
Status: "FAIL",
|
|
||||||
Detail: fmt.Sprintf("Node is in unexpected state: %s", raftState),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check 3: Leader exists
|
|
||||||
if status.Store.Raft.Leader != "" {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Leader exists",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: fmt.Sprintf("Leader: %s", status.Store.Raft.Leader),
|
|
||||||
})
|
|
||||||
} else {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Leader exists",
|
|
||||||
Status: "FAIL",
|
|
||||||
Detail: "No leader detected in Raft cluster",
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check 4: Applied index is advancing (commit == applied means caught up)
|
|
||||||
if status.Store.Raft.AppliedIndex >= status.Store.Raft.CommitIndex {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Log replication",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: fmt.Sprintf("Applied index (%d) >= commit index (%d)",
|
|
||||||
status.Store.Raft.AppliedIndex, status.Store.Raft.CommitIndex),
|
|
||||||
})
|
|
||||||
} else {
|
|
||||||
lag := status.Store.Raft.CommitIndex - status.Store.Raft.AppliedIndex
|
|
||||||
severity := "WARN"
|
|
||||||
if lag > 1000 {
|
|
||||||
severity = "FAIL"
|
|
||||||
}
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Log replication",
|
|
||||||
Status: severity,
|
|
||||||
Detail: fmt.Sprintf("Applied index (%d) behind commit index (%d) by %d entries",
|
|
||||||
status.Store.Raft.AppliedIndex, status.Store.Raft.CommitIndex, lag),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check 5: Query nodes to validate cluster membership
|
|
||||||
nodes, err := queryRQLiteNodes(true)
|
|
||||||
if err != nil {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Cluster nodes reachable",
|
|
||||||
Status: "FAIL",
|
|
||||||
Detail: fmt.Sprintf("Cannot query /nodes: %v", err),
|
|
||||||
})
|
|
||||||
} else {
|
|
||||||
totalNodes := len(nodes)
|
|
||||||
voters := 0
|
|
||||||
nonVoters := 0
|
|
||||||
reachable := 0
|
|
||||||
leaders := 0
|
|
||||||
|
|
||||||
for _, node := range nodes {
|
|
||||||
if node.Voter {
|
|
||||||
voters++
|
|
||||||
} else {
|
|
||||||
nonVoters++
|
|
||||||
}
|
|
||||||
if node.Reachable {
|
|
||||||
reachable++
|
|
||||||
}
|
|
||||||
if node.Leader {
|
|
||||||
leaders++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check 5a: Node count
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Cluster membership",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: fmt.Sprintf("%d nodes (%d voters, %d non-voters)", totalNodes, voters, nonVoters),
|
|
||||||
})
|
|
||||||
|
|
||||||
// Check 5b: All nodes reachable
|
|
||||||
if reachable == totalNodes {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "All nodes reachable",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: fmt.Sprintf("%d/%d nodes reachable", reachable, totalNodes),
|
|
||||||
})
|
|
||||||
} else {
|
|
||||||
unreachable := totalNodes - reachable
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "All nodes reachable",
|
|
||||||
Status: "WARN",
|
|
||||||
Detail: fmt.Sprintf("%d/%d nodes reachable (%d unreachable)", reachable, totalNodes, unreachable),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check 5c: Exactly one leader
|
|
||||||
if leaders == 1 {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Single leader",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: "Exactly 1 leader in cluster",
|
|
||||||
})
|
|
||||||
} else if leaders == 0 {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Single leader",
|
|
||||||
Status: "FAIL",
|
|
||||||
Detail: "No leader found among nodes",
|
|
||||||
})
|
|
||||||
} else {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Single leader",
|
|
||||||
Status: "FAIL",
|
|
||||||
Detail: fmt.Sprintf("Multiple leaders detected: %d (split-brain?)", leaders),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// Check 5d: Quorum check (majority of voters must be reachable)
|
|
||||||
quorum := (voters / 2) + 1
|
|
||||||
reachableVoters := 0
|
|
||||||
for _, node := range nodes {
|
|
||||||
if node.Voter && node.Reachable {
|
|
||||||
reachableVoters++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if reachableVoters >= quorum {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Quorum healthy",
|
|
||||||
Status: "PASS",
|
|
||||||
Detail: fmt.Sprintf("%d/%d voters reachable (quorum requires %d)", reachableVoters, voters, quorum),
|
|
||||||
})
|
|
||||||
} else {
|
|
||||||
results = append(results, checkResult{
|
|
||||||
Name: "Quorum healthy",
|
|
||||||
Status: "FAIL",
|
|
||||||
Detail: fmt.Sprintf("%d/%d voters reachable (quorum requires %d)", reachableVoters, voters, quorum),
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
printHealthResults(results)
|
|
||||||
|
|
||||||
// Exit with non-zero if any failures
|
|
||||||
for _, r := range results {
|
|
||||||
if r.Status == "FAIL" {
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// printHealthResults prints the health check results in a formatted table.
|
|
||||||
func printHealthResults(results []checkResult) {
|
|
||||||
// Find the longest check name for alignment
|
|
||||||
maxName := 0
|
|
||||||
for _, r := range results {
|
|
||||||
if len(r.Name) > maxName {
|
|
||||||
maxName = len(r.Name)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, r := range results {
|
|
||||||
indicator := " "
|
|
||||||
switch r.Status {
|
|
||||||
case "PASS":
|
|
||||||
indicator = "PASS"
|
|
||||||
case "FAIL":
|
|
||||||
indicator = "FAIL"
|
|
||||||
case "WARN":
|
|
||||||
indicator = "WARN"
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf(" [%s] %-*s %s\n", indicator, maxName, r.Name, r.Detail)
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
|
|
||||||
// Summary
|
|
||||||
pass, fail, warn := 0, 0, 0
|
|
||||||
for _, r := range results {
|
|
||||||
switch r.Status {
|
|
||||||
case "PASS":
|
|
||||||
pass++
|
|
||||||
case "FAIL":
|
|
||||||
fail++
|
|
||||||
case "WARN":
|
|
||||||
warn++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
fmt.Printf("Summary: %d passed, %d failed, %d warnings\n", pass, fail, warn)
|
|
||||||
}
|
|
||||||
@ -1,187 +0,0 @@
|
|||||||
package cluster
|
|
||||||
|
|
||||||
import (
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HandleRQLite handles the "orama cluster rqlite" subcommand group.
|
|
||||||
func HandleRQLite(args []string) {
|
|
||||||
if len(args) == 0 {
|
|
||||||
showRQLiteHelp()
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
subcommand := args[0]
|
|
||||||
subargs := args[1:]
|
|
||||||
|
|
||||||
switch subcommand {
|
|
||||||
case "status":
|
|
||||||
handleRQLiteStatus()
|
|
||||||
case "voters":
|
|
||||||
handleRQLiteVoters()
|
|
||||||
case "backup":
|
|
||||||
handleRQLiteBackup(subargs)
|
|
||||||
case "help":
|
|
||||||
showRQLiteHelp()
|
|
||||||
default:
|
|
||||||
fmt.Fprintf(os.Stderr, "Unknown rqlite subcommand: %s\n", subcommand)
|
|
||||||
showRQLiteHelp()
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// handleRQLiteStatus shows detailed Raft state for the local node.
|
|
||||||
func handleRQLiteStatus() {
|
|
||||||
fmt.Printf("RQLite Raft Status\n")
|
|
||||||
fmt.Printf("==================\n\n")
|
|
||||||
|
|
||||||
status, err := queryRQLiteStatus()
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("Node Configuration\n")
|
|
||||||
fmt.Printf(" Node ID: %s\n", status.Store.NodeID)
|
|
||||||
fmt.Printf(" Raft Address: %s\n", status.Store.Address)
|
|
||||||
fmt.Printf(" HTTP Address: %s\n", status.HTTP.Address)
|
|
||||||
fmt.Printf(" Data Directory: %s\n", status.Store.Dir)
|
|
||||||
fmt.Println()
|
|
||||||
|
|
||||||
fmt.Printf("Raft State\n")
|
|
||||||
fmt.Printf(" State: %s\n", strings.ToUpper(status.Store.Raft.State))
|
|
||||||
fmt.Printf(" Current Term: %d\n", status.Store.Raft.Term)
|
|
||||||
fmt.Printf(" Applied Index: %d\n", status.Store.Raft.AppliedIndex)
|
|
||||||
fmt.Printf(" Commit Index: %d\n", status.Store.Raft.CommitIndex)
|
|
||||||
fmt.Printf(" Leader: %s\n", status.Store.Raft.Leader)
|
|
||||||
|
|
||||||
if status.Store.Raft.AppliedIndex < status.Store.Raft.CommitIndex {
|
|
||||||
lag := status.Store.Raft.CommitIndex - status.Store.Raft.AppliedIndex
|
|
||||||
fmt.Printf(" Replication Lag: %d entries behind\n", lag)
|
|
||||||
} else {
|
|
||||||
fmt.Printf(" Replication Lag: none (fully caught up)\n")
|
|
||||||
}
|
|
||||||
|
|
||||||
if status.Node.Uptime != "" {
|
|
||||||
fmt.Printf(" Uptime: %s\n", status.Node.Uptime)
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
}
|
|
||||||
|
|
||||||
// handleRQLiteVoters shows the current voter list from /nodes.
|
|
||||||
func handleRQLiteVoters() {
|
|
||||||
fmt.Printf("RQLite Cluster Voters\n")
|
|
||||||
fmt.Printf("=====================\n\n")
|
|
||||||
|
|
||||||
nodes, err := queryRQLiteNodes(true)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
voters := 0
|
|
||||||
nonVoters := 0
|
|
||||||
|
|
||||||
fmt.Printf("%-20s %-30s %-8s %-10s %-10s\n",
|
|
||||||
"NODE ID", "ADDRESS", "ROLE", "LEADER", "REACHABLE")
|
|
||||||
fmt.Printf("%-20s %-30s %-8s %-10s %-10s\n",
|
|
||||||
strings.Repeat("-", 20),
|
|
||||||
strings.Repeat("-", 30),
|
|
||||||
strings.Repeat("-", 8),
|
|
||||||
strings.Repeat("-", 10),
|
|
||||||
strings.Repeat("-", 10))
|
|
||||||
|
|
||||||
for id, node := range nodes {
|
|
||||||
nodeID := id
|
|
||||||
if len(nodeID) > 20 {
|
|
||||||
nodeID = nodeID[:17] + "..."
|
|
||||||
}
|
|
||||||
|
|
||||||
role := "non-voter"
|
|
||||||
if node.Voter {
|
|
||||||
role = "voter"
|
|
||||||
voters++
|
|
||||||
} else {
|
|
||||||
nonVoters++
|
|
||||||
}
|
|
||||||
|
|
||||||
leader := "no"
|
|
||||||
if node.Leader {
|
|
||||||
leader = "yes"
|
|
||||||
}
|
|
||||||
|
|
||||||
reachable := "no"
|
|
||||||
if node.Reachable {
|
|
||||||
reachable = "yes"
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("%-20s %-30s %-8s %-10s %-10s\n",
|
|
||||||
nodeID, node.Address, role, leader, reachable)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("\nTotal: %d voters, %d non-voters\n", voters, nonVoters)
|
|
||||||
quorum := (voters / 2) + 1
|
|
||||||
fmt.Printf("Quorum requirement: %d/%d voters\n", quorum, voters)
|
|
||||||
}
|
|
||||||
|
|
||||||
// handleRQLiteBackup triggers a manual backup via the RQLite backup endpoint.
|
|
||||||
func handleRQLiteBackup(args []string) {
|
|
||||||
outputFile := getFlagValue(args, "--output")
|
|
||||||
if outputFile == "" {
|
|
||||||
outputFile = fmt.Sprintf("rqlite-backup-%s.db", time.Now().Format("20060102-150405"))
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("RQLite Backup\n")
|
|
||||||
fmt.Printf("=============\n\n")
|
|
||||||
fmt.Printf("Requesting backup from %s/db/backup ...\n", rqliteBaseURL)
|
|
||||||
|
|
||||||
client := &http.Client{Timeout: 60 * time.Second}
|
|
||||||
resp, err := client.Get(rqliteBaseURL + "/db/backup")
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: cannot connect to RQLite: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
body, _ := io.ReadAll(resp.Body)
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: backup request returned HTTP %d: %s\n", resp.StatusCode, string(body))
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
outFile, err := os.Create(outputFile)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: cannot create output file: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
defer outFile.Close()
|
|
||||||
|
|
||||||
written, err := io.Copy(outFile, resp.Body)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error: failed to write backup: %v\n", err)
|
|
||||||
os.Exit(1)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("Backup saved to: %s (%d bytes)\n", outputFile, written)
|
|
||||||
}
|
|
||||||
|
|
||||||
// showRQLiteHelp displays help for rqlite subcommands.
|
|
||||||
func showRQLiteHelp() {
|
|
||||||
fmt.Printf("RQLite Commands\n\n")
|
|
||||||
fmt.Printf("Usage: orama cluster rqlite <subcommand> [options]\n\n")
|
|
||||||
fmt.Printf("Subcommands:\n")
|
|
||||||
fmt.Printf(" status - Show detailed Raft state for local node\n")
|
|
||||||
fmt.Printf(" voters - Show current voter list from cluster\n")
|
|
||||||
fmt.Printf(" backup - Trigger manual database backup\n")
|
|
||||||
fmt.Printf(" Options:\n")
|
|
||||||
fmt.Printf(" --output FILE - Output file path (default: rqlite-backup-<timestamp>.db)\n\n")
|
|
||||||
fmt.Printf("Examples:\n")
|
|
||||||
fmt.Printf(" orama cluster rqlite status\n")
|
|
||||||
fmt.Printf(" orama cluster rqlite voters\n")
|
|
||||||
fmt.Printf(" orama cluster rqlite backup --output /tmp/backup.db\n")
|
|
||||||
}
|
|
||||||
@ -1,248 +0,0 @@
|
|||||||
package cluster
|
|
||||||
|
|
||||||
import (
|
|
||||||
"encoding/json"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
|
||||||
rqliteBaseURL = "http://localhost:5001"
|
|
||||||
httpTimeout = 10 * time.Second
|
|
||||||
)
|
|
||||||
|
|
||||||
// rqliteStatus represents the relevant fields from the RQLite /status endpoint.
|
|
||||||
type rqliteStatus struct {
|
|
||||||
Store struct {
|
|
||||||
Raft struct {
|
|
||||||
State string `json:"state"`
|
|
||||||
AppliedIndex uint64 `json:"applied_index"`
|
|
||||||
CommitIndex uint64 `json:"commit_index"`
|
|
||||||
Term uint64 `json:"current_term"`
|
|
||||||
Leader string `json:"leader"`
|
|
||||||
} `json:"raft"`
|
|
||||||
Dir string `json:"dir"`
|
|
||||||
NodeID string `json:"node_id"`
|
|
||||||
Address string `json:"addr"`
|
|
||||||
} `json:"store"`
|
|
||||||
HTTP struct {
|
|
||||||
Address string `json:"addr"`
|
|
||||||
} `json:"http"`
|
|
||||||
Node struct {
|
|
||||||
Uptime string `json:"uptime"`
|
|
||||||
} `json:"node"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// rqliteNode represents a node from the /nodes endpoint.
|
|
||||||
type rqliteNode struct {
|
|
||||||
ID string `json:"id"`
|
|
||||||
Address string `json:"addr"`
|
|
||||||
Leader bool `json:"leader"`
|
|
||||||
Voter bool `json:"voter"`
|
|
||||||
Reachable bool `json:"reachable"`
|
|
||||||
Time float64 `json:"time"`
|
|
||||||
TimeS string `json:"time_s"`
|
|
||||||
}
|
|
||||||
|
|
||||||
// HandleStatus handles the "orama cluster status" command.
|
|
||||||
func HandleStatus(args []string) {
|
|
||||||
if hasFlag(args, "--all") {
|
|
||||||
fmt.Printf("Remote node aggregation via SSH is not yet implemented.\n")
|
|
||||||
fmt.Printf("Currently showing local node status only.\n\n")
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("Cluster Status\n")
|
|
||||||
fmt.Printf("==============\n\n")
|
|
||||||
|
|
||||||
// Query RQLite status
|
|
||||||
status, err := queryRQLiteStatus()
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error querying RQLite status: %v\n", err)
|
|
||||||
fmt.Printf("RQLite may not be running on this node.\n\n")
|
|
||||||
} else {
|
|
||||||
printLocalStatus(status)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Query RQLite nodes
|
|
||||||
nodes, err := queryRQLiteNodes(true)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintf(os.Stderr, "Error querying RQLite nodes: %v\n", err)
|
|
||||||
} else {
|
|
||||||
printNodesTable(nodes)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Query Olric status (best-effort)
|
|
||||||
printOlricStatus()
|
|
||||||
}
|
|
||||||
|
|
||||||
// queryRQLiteStatus queries the local RQLite /status endpoint.
|
|
||||||
func queryRQLiteStatus() (*rqliteStatus, error) {
|
|
||||||
client := &http.Client{Timeout: httpTimeout}
|
|
||||||
resp, err := client.Get(rqliteBaseURL + "/status")
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("connect to RQLite: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
return nil, fmt.Errorf("unexpected status code: %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var status rqliteStatus
|
|
||||||
if err := json.Unmarshal(body, &status); err != nil {
|
|
||||||
return nil, fmt.Errorf("parse response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return &status, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// queryRQLiteNodes queries the local RQLite /nodes endpoint.
|
|
||||||
// If includeNonVoters is true, appends ?nonvoters to the query.
|
|
||||||
func queryRQLiteNodes(includeNonVoters bool) (map[string]*rqliteNode, error) {
|
|
||||||
client := &http.Client{Timeout: httpTimeout}
|
|
||||||
|
|
||||||
url := rqliteBaseURL + "/nodes"
|
|
||||||
if includeNonVoters {
|
|
||||||
url += "?nonvoters"
|
|
||||||
}
|
|
||||||
|
|
||||||
resp, err := client.Get(url)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("connect to RQLite: %w", err)
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
return nil, fmt.Errorf("unexpected status code: %d", resp.StatusCode)
|
|
||||||
}
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var nodes map[string]*rqliteNode
|
|
||||||
if err := json.Unmarshal(body, &nodes); err != nil {
|
|
||||||
return nil, fmt.Errorf("parse response: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return nodes, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
// printLocalStatus prints the local node's RQLite status.
|
|
||||||
func printLocalStatus(s *rqliteStatus) {
|
|
||||||
fmt.Printf("Local Node\n")
|
|
||||||
fmt.Printf(" Node ID: %s\n", s.Store.NodeID)
|
|
||||||
fmt.Printf(" Raft Address: %s\n", s.Store.Address)
|
|
||||||
fmt.Printf(" HTTP Address: %s\n", s.HTTP.Address)
|
|
||||||
fmt.Printf(" Raft State: %s\n", strings.ToUpper(s.Store.Raft.State))
|
|
||||||
fmt.Printf(" Raft Term: %d\n", s.Store.Raft.Term)
|
|
||||||
fmt.Printf(" Applied Index: %d\n", s.Store.Raft.AppliedIndex)
|
|
||||||
fmt.Printf(" Commit Index: %d\n", s.Store.Raft.CommitIndex)
|
|
||||||
fmt.Printf(" Leader: %s\n", s.Store.Raft.Leader)
|
|
||||||
if s.Node.Uptime != "" {
|
|
||||||
fmt.Printf(" Uptime: %s\n", s.Node.Uptime)
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
}
|
|
||||||
|
|
||||||
// printNodesTable prints a formatted table of all cluster nodes.
|
|
||||||
func printNodesTable(nodes map[string]*rqliteNode) {
|
|
||||||
if len(nodes) == 0 {
|
|
||||||
fmt.Printf("No nodes found in cluster.\n\n")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("Cluster Nodes (%d total)\n", len(nodes))
|
|
||||||
fmt.Printf("%-20s %-30s %-8s %-10s %-10s %-12s\n",
|
|
||||||
"NODE ID", "ADDRESS", "VOTER", "LEADER", "REACHABLE", "LATENCY")
|
|
||||||
fmt.Printf("%-20s %-30s %-8s %-10s %-10s %-12s\n",
|
|
||||||
strings.Repeat("-", 20),
|
|
||||||
strings.Repeat("-", 30),
|
|
||||||
strings.Repeat("-", 8),
|
|
||||||
strings.Repeat("-", 10),
|
|
||||||
strings.Repeat("-", 10),
|
|
||||||
strings.Repeat("-", 12))
|
|
||||||
|
|
||||||
for id, node := range nodes {
|
|
||||||
nodeID := id
|
|
||||||
if len(nodeID) > 20 {
|
|
||||||
nodeID = nodeID[:17] + "..."
|
|
||||||
}
|
|
||||||
|
|
||||||
voter := "no"
|
|
||||||
if node.Voter {
|
|
||||||
voter = "yes"
|
|
||||||
}
|
|
||||||
|
|
||||||
leader := "no"
|
|
||||||
if node.Leader {
|
|
||||||
leader = "yes"
|
|
||||||
}
|
|
||||||
|
|
||||||
reachable := "no"
|
|
||||||
if node.Reachable {
|
|
||||||
reachable = "yes"
|
|
||||||
}
|
|
||||||
|
|
||||||
latency := "-"
|
|
||||||
if node.TimeS != "" {
|
|
||||||
latency = node.TimeS
|
|
||||||
} else if node.Time > 0 {
|
|
||||||
latency = fmt.Sprintf("%.3fs", node.Time)
|
|
||||||
}
|
|
||||||
|
|
||||||
fmt.Printf("%-20s %-30s %-8s %-10s %-10s %-12s\n",
|
|
||||||
nodeID, node.Address, voter, leader, reachable, latency)
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
}
|
|
||||||
|
|
||||||
// printOlricStatus attempts to query the local Olric status endpoint.
|
|
||||||
func printOlricStatus() {
|
|
||||||
client := &http.Client{Timeout: 5 * time.Second}
|
|
||||||
resp, err := client.Get("http://localhost:3320/")
|
|
||||||
if err != nil {
|
|
||||||
fmt.Printf("Olric: not reachable on localhost:3320 (%v)\n\n", err)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
|
|
||||||
body, err := io.ReadAll(resp.Body)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Printf("Olric: reachable but could not read response\n\n")
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if resp.StatusCode == http.StatusOK {
|
|
||||||
fmt.Printf("Olric: reachable (HTTP %d)\n", resp.StatusCode)
|
|
||||||
// Try to parse as JSON for a nicer display
|
|
||||||
var data map[string]interface{}
|
|
||||||
if err := json.Unmarshal(body, &data); err == nil {
|
|
||||||
for key, val := range data {
|
|
||||||
fmt.Printf(" %s: %v\n", key, val)
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
// Not JSON, print raw (truncated)
|
|
||||||
raw := strings.TrimSpace(string(body))
|
|
||||||
if len(raw) > 200 {
|
|
||||||
raw = raw[:200] + "..."
|
|
||||||
}
|
|
||||||
if raw != "" {
|
|
||||||
fmt.Printf(" Response: %s\n", raw)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
} else {
|
|
||||||
fmt.Printf("Olric: reachable but returned HTTP %d\n", resp.StatusCode)
|
|
||||||
}
|
|
||||||
fmt.Println()
|
|
||||||
}
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
x
Reference in New Issue
Block a user