Training providers

The classifier does not know where training runs. A TrainingProvider accepts a versioned job and returns a candidate artifact plus resource evidence.

Local

import { createClassifier, localTrainer } from "@swapai/core";

const classifier = createClassifier({
  // ...
  training: localTrainer(),
});

Use local training on a suitably provisioned development machine or self-hosted runner. It uses the same runner contract and hash verification as remote training.

Runpod

import { runpodTrainer } from "@swapai/core/runpod";

const training = runpodTrainer({
  apiKey: process.env.RUNPOD_API_KEY!,
  maximumCostUsd: 1,
  maximumRuntimeMinutes: 30,
  cloud: "SECURE",
  sshPrivateKey: "/run/secrets/runpod-ssh-key",
  registerSshPublicKeyForTraining: true,
});

The adapter uses Runpod's REST API v2, including its v2 create and terminate Pod operations. Keep the API key in the controller environment or secret store; never put it in application source, a training bundle, logs or browser code.

apiKey and maximumCostUsd are required. maximumRuntimeMinutes defaults to 30. You can also set image, an ordered gpuTypes preference, cloud and sshPrivateKey. Set registerSshPublicKeyForTraining: true when the configured key is not already registered on the Runpod account. SwapAI registers that exact public key before creating a paid Pod, preserves existing account keys, then removes only the key it added after verified Pod cleanup. A registration failure therefore happens before spend. The supported adapter path is Runpod Secure Cloud only: cloud accepts "SECURE"; Community Cloud is rejected. A created Pod must advertise CUDA 12.8 or newer and expose full direct SSH on its published 22/tcp port.

The 0.6.9 package defaults to ghcr.io/smccamley/swapai-trainer:0.6.9. The published release image digest is sha256:bc6d8830e93f4df02d858859e0550b5c2d85de6cc433ec4bb9d73479a50b0148.

Allocation and retry behavior

GPU types are tried in the configured order. Allocation handles Runpod responses deliberately:

• 400 and 403 move to the next GPU type;
• 429 honours Retry-After, and 5xx responses use bounded retries on the same GPU;
• 422 and other 4xx responses fail immediately because retrying the same request would hide a configuration error.

Pod-list reconciliation follows every REST v2 page. A terminal Pod fails fast instead of waiting for SSH. SwapAI waits for full direct SSH on the Pod's published 22/tcp port before transferring the runner bundle. Runpod's basic proxy SSH supports neither SCP nor SFTP, so SwapAI never uses that proxy for training automation. If direct SSH does not appear within 12 minutes, or does not accept the registered key within the following 10 minutes, the run fails with a bounded error and verified Pod cleanup.

Provider HTTP requests have a bounded timeout. The paid training deadline propagates an AbortSignal through HTTP polling and every SSH or SCP command, so an expired run does not leave controller I/O running in the background. Custom command adapters receive the same optional signal.

Cost and cleanup

Immediately after creating a Pod, the adapter derives a paid training window bounded by both maximumCostUsd and maximumRuntimeMinutes, with five minutes reserved for cleanup. The cost ceiling includes reported compute and a conservative charge for the 30 GB container storage, not GPU price alone. maximumRuntimeMinutes must therefore be greater than 5 and cannot exceed 360. A Pod that leaves no positive training window is terminated without running the job. During a run SwapAI records:

• the exact Runpod pod ID;
• start and finish timestamps;
• estimated accrued compute and container-storage cost;
• cleanup status and failure message, including verified termination.

Cost is recorded after verified cleanup for successful and failed provider runs. A failed run therefore retains both its nested failure cause and its measured cost instead of losing either piece of evidence.

Cleanup deletes the recorded pod, not resources found by a broad name search. Call reconcileTraining() after controller startup to reconcile incomplete runs and retry exact-pod cleanup. An unverified deletion remains a visible operational failure.

Custom provider

Implement TrainingProvider to use AWS, another GPU host or your own queue:

const training: TrainingProvider = {
  name: "internal-gpu-queue",
  async train(job) {
    // Send the immutable runner bundle, wait for one result,
    // and return the complete artifact and provider run identity.
  },
};

The adapter owns infrastructure creation and cleanup. SwapAI owns dataset revisions, the runner contract, artifact verification, protected evaluation and promotion.