Skip to content
Tenzro
← All tutorials
Tutorial · Inference and training

Run a trainer node

Contribute GPU time to decentralised training runs on Network 1, including chat fine-tuning on agent trajectories, and earn TNZO per finalized round.

Advanced30 min

Training on Tenzro is data-parallel with decoupled outer synchronisation. Sponsors post training runs; trainers run the inner loop on a shard of the data and submit an outer gradient; a syncer aggregates the gradients and finalizes each round on chain. Every round is replayable from its seed, so anyone can re-run a round and check the result. Trainers earn TNZO for the rounds they contribute to.

Runs can use the supervised objective, reinforcement-learning post-training, or the chat fine-tuning objective, which trains on chat and tool-call conversations with the loss on assistant turns. That last one is how agents are fine-tuned on their own trajectories.

Prerequisites

  • A Linux machine with a GPU and a node with a hardware-rooted identity. See Join Network 1 as a provider.
  • Python 3.10 or newer, or Docker.
  • TNZO for the trainer bond. Check the minimum with tenzro provider bond params.

1. Install the trainer runtime

The node binary stays lean; the Python trainer is installed only on machines that train. Either build the trainer image, which bundles the runtime on top of the node:

bash
git clone https://github.com/tenzro/tenzro-network.git
cd tenzro-network
docker build -f Dockerfile.trainer -t tenzro-trainer-node .

Or install the reference trainer into a virtual environment next to an existing node:

bash
python3 -m venv ~/.venvs/tenzro-trainer
~/.venvs/tenzro-trainer/bin/pip install -e 'integrations/trainer[language,vision]'

Extras select what you can train: language, vision, timeseries, and confidential for runs whose data is sealed to a TEE.

2. Enable the trainer daemon

Add a [training] section to the node's config file:

toml
[training]
enabled = true
venv_path = "/home/tenzro/.venvs/tenzro-trainer"
max_concurrent_trainers = 1

With the trainer image, enabled = true is enough; the image already points the node at its runtime.

3. Start the node and check the trainer identity

bash
tenzro-node --roles ai --data-dir ./data --config ./node.toml
tenzro train daemon-status

Expected output reports running: true, the number of live trainers and a trainer_did of the form did:tenzro:machine:trainer:<node-address>.

The trainer's signing key is derived from the node's hardware-rooted identity. There is no second secret to manage, and every gradient you submit is attributable to your machine across restarts, which is what makes reward attribution stable.

4. Bond as a trainer

bash
tenzro provider bond post --did <trainer-did> --address <your-address> --amount <tnzo>
tenzro provider register --type trainer --did <trainer-did>

5. Find runs

bash
tenzro train list-runs
tenzro train get-run --task-id <task-id>

get-run shows the run's modality, objective, trust tier, current round and enrolled trainers. Runs come in three tiers: Open, Verified (the trainer attests its environment) and Confidential (the data is sealed to the trainer's TEE and never reaches the host in cleartext).

6. Enroll and train

The daemon discovers runs that are enrolling or training and provisions a trainer for each, up to max_concurrent_trainers. There is no manual assignment step. To enroll in a specific run yourself:

bash
tenzro train enroll-trainer --task-id <task-id> --trainer-did <trainer-did>

For a Confidential run, pass --attestation ./attestation.json. The report is verified against the vendor's roots and must commit to the enclave key the sponsor sealed the data to.

Each round the trainer fetches its shard (natively from tenzro://blob/<hash> URIs on the data plane), runs the inner loop the run's objective selects, and submits its outer gradient. For a chat fine-tuning run, the shard is JSONL in the OpenAI chat shape, one conversation per line:

json
{"messages":[{"role":"user","content":"Book a table for two."},{"role":"assistant","tool_calls":[{"type":"function","function":{"name":"book_table","arguments":"{\"party\":2}"}}]}],"tools":[{"type":"function","function":{"name":"book_table"}}]}

7. Follow rounds and receipts

bash
tenzro train get-run --task-id <task-id>
tenzro train get-receipt --task-id <task-id>

The syncer aggregates submitted gradients with the run's aggregation rule, applies the outer optimiser and commits the round on chain. Because each round is replayable from its seed, a gradient that does not match a re-execution can be challenged and thrown out.

8. Get paid

Rewards for a finalized round are paid in TNZO to the address bound to your trainer identity.

bash
tenzro wallet balance --address <your-address>

Next steps