SEGoS-data

Data assets for SE-GoS: Self-Evolving Graph-of-Skills for Skill Library at Scale (paper: arXiv:2609.08228, code: PKUfudawei/SEGoS).

This dataset hosts only what SE-GoS itself produces or repackages for convenience. The GoS skill libraries and prebuilt workspaces it builds on are not redistributed here — scripts/download_data.sh fetches them from the upstream GoS dataset davidliuk/graph-of-skills-data (davidliuk, 2026-04).

Contents

Assets are grouped by the experiment they belong to, so a single experiment's inputs can be fetched without pulling the rest.

Path	Used by
tasks/skillsbench_tasks.tar.gz	every table — 87 SkillsBench dockerized coding tasks (1.0 GB unpacked)
coldstart/segos_coldstart_skills1000.json	every table — the static (round-0) substrate: 1,000 skills, 863 semantic edges
alfworld/coldstart/segos_coldstart_skills200.json	the ALFWorld table — the static web substrate: 200 skills, 149 semantic edges
full87/graphs/segos_evolved_skills1000_round1.json	main table (the SE-GoS cell) and multi-round table round 1 — 1,118 edges: 863 semantic, 228 workflow, 27 avoid
full87/graphs/segos_evolved_skills1000_round2.json	multi-round table round 2 — 1,375 edges (previous chain, see below)
full87/graphs/segos_evolved_skills1000_round3.json	multi-round table round 3 — 1,502 edges (previous chain, see below)
heldout/graphs/segos_evolved_skills1000_heldout_train50.json	held-out table — evolved on the 50 training tasks only — 999 edges: 863 semantic, 127 workflow, 9 avoid
full87/evolution/segos_evolution_deltas.tar.gz	the L1/L2/L3 deltas for all eight full-87 cells, plus the parsed signals
heldout/evolution/segos_evolution_deltas.tar.gz	the L1/L2/L3 deltas behind the held-out graph
full87/traces/segos_traces_round*.tar.gz	per-trial job trees for the full-87 runs (round-0 train, rounds 1-3 evals)
full87/traces/segos_traces_ablation_*.tar.gz	per-trial job trees for the six $2^3$ factorial cells other than the full one — L1, L2, L3, L1L2, L1L3, L2L3, 174 scored attempts each
heldout/traces/segos_traces_heldout_eval.tar.gz	per-trial job trees for the held-out run (37 eval tasks)

Each trace archive holds one directory per task attempt with result.json, config.json, and the agent transcript.

The tarballs contain one top-level directory; extract with tar -xzf <archive> --strip-components=1. The graph JSONs and the deltas archive are what download_data.sh unpacks for you.

Regeneration status (2026-09-15)

round1 and heldout_train50 were regenerated from the static training traces with the avoid relation enabled at its default evidence bar, plus the non-contradiction invariant. Only their edge sets and weights changed; their description rewrites were carried over, because the node update calls an LLM and is not reproducible offline. Round 1 carries six rewritten descriptions and the held-out graph three.

• round 1: 1,102 → 1,118 edges (the workflow set also shifted from 239 to 228 because eleven trials of the training job were re-run after the original deltas were computed)
• held-out: 990 → 999 edges

The reward, token and runtime columns of the paper were measured on the previous graphs and have not been re-measured; a full re-run is pending. The six full87/traces/segos_traces_ablation_*.tar.gz archives belong to that same measurement generation, so their cells and the full one are comparable with each other and with the paper's columns, but not with the regenerated round-1 graph.

round2 and round3 are deliberately left as they were. Their evolution consumes the round-1 evaluation traces, which were produced on the previous round-1 graph, so they cannot be regenerated until that re-run happens. Their deltas inside full87/evolution/segos_evolution_deltas.tar.gz are the previous chain's, unchanged; everything else in that archive is current.

Statistics in this release

evolution/multiround/summary.json inside full87/evolution/segos_evolution_deltas.tar.gz records two numbers per round. R is the pooled mean reward over all 174 scored attempts of that round's eval job, and T_M is the mean input tokens per attempt over the same 174 attempts (n_attempts records the denominator). Both are recomputable directly from the per-trial results in full87/traces/, and both match the reward and token columns the paper reports for rounds 2 and 3.

An earlier release computed T_M by first collapsing the attempts of a task into one entry, which averaged 87 samples instead of 174 and kept whichever attempt the file order happened to reach last. Those values (3.701 and 3.892) are not a quantity the paper reports and have been replaced (3.664 and 3.708).

Note that the reward, token, and runtime columns of the paper were measured on the graphs as they stood before the round-1 and held-out regeneration described above; re-measuring them on the regenerated graphs is pending.

The cold-start graph

SE-GoS does not start from GoS's LLM-validated typed graph. It starts from a deterministic semantic-only graph: every skill linked to its top-1 neighbour by signature-token overlap, no LLM pass and no embedding service.

coldstart/segos_coldstart_skills1000.json is exactly that substrate — the graph behind the static (round-0) row of every table in the paper (1,000 nodes / 863 edges). It is reproducible from the upstream GoS workspace in one command:

cd evaluation/skillsbench
PYTHONPATH=$PWD python -m evo.rebuild_graph \
  --igraph ../../data/gos_workspace/skills_1000_v1/graph_igraph_data.pklz \
  --edge-types sem --sem-metric token --semantic-k 1 \
  --out generated/shared/graphskills_bundle_semonly_token_k1_1000.json

--hnsw is not needed: the token metric reads node attributes from the official igraph pickle, so no embedding index or API key is involved.

alfworld/coldstart/segos_coldstart_skills200.json is the same substrate built over the 200-skill library (200 nodes / 149 edges) and is what the ALFWorld runs retrieve from:

cd evaluation/skillsbench
PYTHONPATH=$PWD python -m evo.rebuild_graph \
  --igraph ../../data/gos_workspace/skills_200_v1/graph_igraph_data.pklz.official \
  --edge-types sem --sem-metric token --semantic-k 1 \
  --out ../alfworld/generated/shared/alfworld_coldstart_skills200_k1_token.json

Both are fetched by download_data.sh (--coldstart and --alfworld respectively); neither is committed to the code repository.

The evolved graphs

The multi-round and held-out tables run on graphs that SE-GoS produced from execution traces, not on the cold start:

• full87/graphs/segos_evolved_skills1000_round{1,2,3}.json — the full-87 protocol, one L1/L2/L3 pass per round. Round 1 is the SE-GoS cell of the main table (1,102 edges); rounds 2 and 3 re-evolve and re-measure all 87 tasks.
• heldout/graphs/segos_evolved_skills1000_heldout_train50.json — evolved on the 50 training tasks of the disjoint 50/37 split only (990 edges), then measured on the 37 held-out tasks. A different artifact from round 1 despite the similar name; the two are never interchangeable.

full87/evolution/segos_evolution_deltas.tar.gz carries the L1/L2/L3 deltas and the parsed signals the graphs were built from, so the evolution itself can be recomputed offline (no agent runs). */traces/*.tar.gz carries the raw per-trial job trees if you want to replay the runs.

Usage

git clone https://github.com/PKUfudawei/SEGoS.git && cd SEGoS
./scripts/download_data.sh              # everything except traces
./scripts/download_data.sh --tasks      # SkillsBench tasks only
./scripts/download_data.sh --coldstart  # cold-start graph only
./scripts/download_data.sh --alfworld   # ALFWorld cold-start graph only
./scripts/download_data.sh --evolved    # evolved graphs + deltas (multi-round / held-out)
./scripts/download_data.sh --traces     # per-trial job trees for those runs (~1.7 GB)

Fetch one experiment's assets directly, without the rest of the dataset:

hf download PKUfudawei/SEGoS-data --include "heldout/*" --repo-type dataset
hf download PKUfudawei/SEGoS-data --include "full87/graphs/*" "coldstart/*" --repo-type dataset

GOS_HF_REPO overrides the upstream GoS dataset used for skill libraries and prebuilt workspaces; SEGOS_HF_REPO overrides this dataset.

Provenance

• SkillsBench tasks — from benchflow-ai/skillsbench. download_data.sh falls back to a sparse checkout of that repo if the archive here is unavailable.
• Cold-start graph — produced by this project from the upstream GoS workspace (see the rebuild_graph commands above). Two sizes are hosted: the 1,000-skill substrate behind the SkillsBench tables, and the 200-skill substrate used by the ALFWorld runs.
• Evolved graphs and deltas — produced by this project's evolution pass over traces from the SkillsBench runs. The graphs are the artifacts the tables are measured on; the deltas are the auditable record of what each update changed.
• Per-trial traces — the raw job trees those runs wrote: one directory per task attempt, holding result.json, config.json, and the agent transcript.
• Skill libraries / prebuilt workspaces — not hosted here; fetched from davidliuk/graph-of-skills-data. Note those *_v1 workspace archives hold undirected graphs from an earlier GoS cleanup implementation and reproduce older results only.