Inputs & Outputs
The config-driven input/output contract
trainer is configured entirely through config.yaml. It uses config.yaml rather than a separate inputs.yaml because the file describes a full training run profile, not just upstream inputs. Treat config.yaml as the source of truth before launching a run, and validate it with bash scripts/dryrun.sh.
Inputs
Read from config.yaml → runtime_info.input:
| Name | Description | Source |
|---|---|---|
source.type | Where the LF dataset comes from: harbor_job | hf_lf | local_lf | external |
source | Per-type fields: scaffold+job_dir (harbor), hf_hub_url plus optional hf_file_name/subset/split (hf), lf_path (local) | dependency / external |
conversion | max_instances, exclude_repos_file, data_name | external |
dataset | LLaMA-Factory dataset registration (auto-derived from data_name) | derived |
model | Base model path and trust_remote_code | external |
training | Stage, deepspeed, template, cutoff, batch size, lr, epochs, … | external |
infrastructure | n_gpus_per_node | external |
experiment | WandB mode and run name | external |
credentials | WandB API key, hf_token (private hf_lf datasets) | external |
When source.type is harbor_job, source.job_dir is the handoff from tracer, wired via meta_info.dependencies.from.source.job_dir (tracer.output.raw_trajectories_dir), mirrored by tracer's own dependencies.to.raw_trajectories_dir. For hf_lf, setting hf_file_name downloads and registers only that file; leaving it empty loads the Hub dataset config/split at train time. local_lf registers lf_path directly. See Input sources.
Active runtime values
Excerpted from config.yaml:
source:
type: hf_lf
hf_hub_url: "SWE-Lego/samples_for_llama_factory_sft"
hf_file_name: "jierun_glm52_openswe_mirror_score4_proto_oh_sdk_512_for_qwen3_5.json"
hf_subset: ""
hf_split: "train"
conversion:
max_instances: 0
exclude_repos_file: artifacts/data/excluded_repos.txt
data_name: "glm52_openswe_mirror_score4_proto_oh_sdk_512_qwen3_5"
model:
model_name_or_path: Qwen/Qwen3.5-35B-A3B-Base
trust_remote_code: true
training:
stage: sft
finetuning_type: full
deepspeed: artifacts/training_config/deepspeed/ds_z3_config.json
template: qwen3_5
cutoff_len: 131072
rope_scaling: null
max_samples: 1000000
output_dir: "qwen3_5_35b_a3b_base_glm52_openswe_mirror_score4_proto_oh_sdk_512_gbs64pbs1acc8_lr5e-5_epo3"
save_strategy: steps
save_steps: 200
overwrite_output_dir: false
save_only_model: true
per_device_train_batch_size: 1
gradient_accumulation_steps: 8
learning_rate: 5.0e-5
num_train_epochs: 3.0
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
flash_attn: fa2
infrastructure:
n_gpus_per_node: 8
experiment:
wandb_mode: offlineSee Configuration for what each training field controls.
Outputs
Written to config.yaml → runtime_info.output:
| Output | Path / value | Consumer |
|---|---|---|
checkpoint_path | artifacts/model/<run>/checkpoint-<step>/ | rl, evaluator |
training_metrics | final_loss, train_runtime, total_steps | reporting |
artifacts | train_results.json, training_loss.png, console log | reporting |
training_curves | wandb run id, when available | reporting |
See Results & Artifacts for the full layout, and Config Variants for running experiments without disturbing the active profile.
Environment
The training environment is a uv venv at meta_info.environment.sft_uv (default artifacts/env/lf, Python 3.12). Recreate it with bash scripts/install_env.sh. It is excluded from the repo, so its contents are not version-controlled.