# tuneharness

> Deterministic small-model training pipeline and GPU marketplace harness with deterministic deploy gates, post-boot bf16 compute preflights, 30-min idle watchdogs, and append-only cost ledgers.

tuneharness is an open engineering framework and operational harness built by RoamingPigs Inc. for renting untrusted, spot-market GPU instances (such as Vast.ai and RunPod) safely, cheaply, and deterministically.

## Core Capabilities

- **Preflight Verification**: Runs a 9.4-second renter-side bf16 tensor core sanity check immediately upon SSH connection. Verifies memory bandwidth, GEMM correctness, and thermal stability before fine-tuning starts. Defective hosts are torn down immediately for full refund.
- **Fail-Safe Watchdogs**: A dual-layer watchdog (renter-side daemon and host-side systemd timer) terminates instances after 30 minutes of idle GPU activity or network disconnects, preventing runaway billing.
- **Deterministic Quality Gates**: Every trained checkpoint must satisfy validation perplexity delta <= 0.05, pass benchmark evaluation suites, and retain conversational formatting adherence before artifacts are accepted into production.
- **Append-Only Cost Ledger**: Every instance spin-up, spot price fluctuation, run duration, failure mode, and dollar cost is recorded in an immutable ledger. 121 rentals recorded to date with $41.62 total compute spend and 62.0% failure absorption.
- **Provider Neutral**: Abstract provider interface supporting Vast.ai and RunPod with pluggable adapters.

## Canonical Pages

- Overview: https://tuneharness.com/
- Architecture & Seams: https://tuneharness.com/architecture/
- Live Cost Ledger: https://tuneharness.com/ledger/
- Eval Gates & Simulator: https://tuneharness.com/evals/
- Living Whitepaper: https://tuneharness.com/whitepaper/
- Sitemap: https://tuneharness.com/sitemap.xml

## Engineering Principles

1. Rented compute is untrusted until verified by tenant-owned tests.
2. Training jobs must fail fast rather than waste spot budget on throttled hardware.
3. Quality gates are non-negotiable: models that fail perplexity or eval thresholds are discarded.
4. Billing transparency requires every rental event to be published in the ledger.

## Contact

RoamingPigs Inc.
Website: https://roamingpigs.com
Author: Cisco Caceres