Commands
Core
| Command | Description |
|---|---|
eigenpal workflow list|ls [options] | List workflows the caller can read. |
eigenpal workflow pull [options] <workflow-id> | Download the YAML definition of the workflow at its current version. |
eigenpal workflow push [options] | Create or update a workflow from a YAML file. |
eigenpal workflow move [options] <workflow-id> | Move a workflow to a folder path, creating folders as needed |
eigenpal workflow validate [options] [path] | Local-only validation. Without [path]: runs the templated three-way check (./workflow.yaml + ./evaluators.yaml + ./dataset/). With [path] pointing at a YAML file: validates just that workflow.yaml. For per-noun targeting use evaluators validate or dataset validate. |
eigenpal workflow clear-local [options] [examples...] | Delete local execution artifacts under ./dataset/examples/. Keeps the latest run per example by default. |
Evaluators
| Command | Description |
|---|---|
eigenpal workflow evaluators pull [options] <workflow-id> | Download the workflow’s evaluators YAML. |
eigenpal workflow evaluators push [options] <workflow-id> | Overwrite the workflow’s evaluator config from a YAML file. |
eigenpal workflow evaluators validate [options] [path] | Validate an evaluators YAML file against the EvalConfig schema. Defaults to ./evaluators.yaml. |
Dataset
| Command | Description |
|---|---|
eigenpal workflow dataset list|ls [options] <workflow-id> | List eval examples for the workflow. |
eigenpal workflow dataset pull [options] <workflow-id> | Download the workflow’s dataset as a ZIP archive. |
eigenpal workflow dataset push [options] <workflow-id> | Replace or extend the workflow’s dataset from a ZIP or folder. |
eigenpal workflow dataset example create [options] <workflow-id> | Create one eval example without re-uploading the dataset. |
eigenpal workflow dataset example update [options] <workflow-id> <exampleId> | Patch one eval example. Omitted flags are left alone. |
eigenpal workflow dataset example delete [options] <workflow-id> <exampleId> | Delete one eval example by id. Non-TTY shells require —yes. |
eigenpal workflow dataset example get [options] <workflow-id> <exampleId> | Fetch one eval example with full triggerInput, expectedOutput, and metadata. |
eigenpal workflow dataset validate [options] [path] | Validate a dataset folder against the examples/<name>/{input,expected,meta} convention. Defaults to ./dataset/. |
Experiment
| Command | Description |
|---|---|
eigenpal workflow experiment|exp list|ls [options] <workflow-id> | List executions for the workflow, newest first. |
eigenpal workflow experiment|exp run [options] <workflow-id> | Start a batch eval against the workflow’s dataset. |
eigenpal workflow experiment|exp status [options] <workflow-id> <batchId> | Aggregate progress for a batch by batchId. |
eigenpal workflow experiment|exp cancel [options] <workflow-id> <batchId> | Cancel every execution in a batch. Idempotent. |
eigenpal workflow experiment|exp results [options] <workflow-id> [batchId] | Download eval results in CSV or JSON. |
eigenpal workflow experiment|exp compare|diff [options] <batchIdA> <batchIdB> | Diff eval scores between two experiment batches. |
eigenpal workflow experiment|exp watch [options] <workflow-id> <batchId> | Poll until terminal, then auto-pull results, replaces status --watch + results --out. |
Versions
| Command | Description |
|---|---|
eigenpal workflow versions list|ls [options] <workflow-id> | List historical workflow versions, newest first. |
eigenpal workflow versions restore [options] <workflow-id> <versionId> | Restore the workflow to a previous version. |
Step-type
| Command | Description |
|---|---|
eigenpal workflow step-type list|ls [options] | List every step type the deployment supports. |
eigenpal workflow step-type get [options] <type> | Return the full schema and behavioral docs for one step type. |
Evaluator-type
| Command | Description |
|---|---|
eigenpal workflow evaluator-type list|ls [options] | List every evaluator type with a one-line description. |
eigenpal workflow evaluator-type get [options] <type> | Fetch the JSON Schema for one evaluator type. Pipe through jq to inspect specific fields. |
Step
| Command | Description |
|---|---|
eigenpal workflow step exec [options] <type> | DISABLED, local mimic runners removed pending server-side redesign (EIG-104). Use run or workflow experiment run instead. |
Details
eigenpal workflow list|ls [options]
List workflows the caller can read.
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--search <q> | no | Filter by name | |
--limit <n> | no | 50 | Page size |
--offset <n> | no | 0 | Page offset |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow pull [options] <workflow-id>
Download the YAML definition of the workflow at its current version.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--out <path> | no | Write YAML to file instead of stdout | |
--base-url <url> | no | Server base URL |
eigenpal workflow push [options]
Create or update a workflow from a YAML file.
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--file <yaml> | no | Path to YAML file | |
--workflow-id <id> | no | Update existing workflow (default: create new) | |
--bump <level> | no | Auto-bump from the server’s current version: patch | minor | major. Mutually exclusive with --version and with a top-level version: in the YAML. | |
--set-version <semver> | no | Push at this exact semver (e.g. 1.4.0). Mutually exclusive with --bump and with a top-level version: in the YAML. (Named --set-version to avoid the global -v, --version flag.) | |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow move [options] <workflow-id>
Move a workflow to a folder path, creating folders as needed
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--folder <path> | yes | Target folder path (/ for root) | |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow validate [options] [path]
Local-only validation. Without [path]: runs the templated three-way check (./workflow.yaml + ./evaluators.yaml + ./dataset/). With [path] pointing at a YAML file: validates just that workflow.yaml. For per-noun targeting use evaluators validate or dataset validate.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
path | no | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--dir <path> | no | Project root (defaults to cwd; resolves the three default paths from here) |
eigenpal workflow clear-local [options] [examples...]
Delete local execution artifacts under ./dataset/examples/. Keeps the latest run per example by default.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
examples | no | yes |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--dir <dir> | no | Local eigenpal directory | |
--all | no | false | Remove all artifacts, including the latest kept by default |
eigenpal workflow evaluators pull [options] <workflow-id>
Download the workflow’s evaluators YAML.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--out <path> | no | Write YAML to file instead of stdout | |
--base-url <url> | no | Server base URL |
eigenpal workflow evaluators push [options] <workflow-id>
Overwrite the workflow’s evaluator config from a YAML file.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--file <yaml> | yes | Path to evaluators YAML file | |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow evaluators validate [options] [path]
Validate an evaluators YAML file against the EvalConfig schema. Defaults to ./evaluators.yaml.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
path | no | no |
eigenpal workflow dataset list|ls [options] <workflow-id>
List eval examples for the workflow.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--limit <n> | no | 100 | Page size |
--offset <n> | no | 0 | Page offset |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow dataset pull [options] <workflow-id>
Download the workflow’s dataset as a ZIP archive.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--out <zip> | no | Write the dataset ZIP to this path. When omitted, the binary streams to stdout. | |
--example-id <id> | no | [] | Export only this example (repeatable). When omitted, the whole dataset is exported. |
--base-url <url> | no | Server base URL |
eigenpal workflow dataset push [options] <workflow-id>
Replace or extend the workflow’s dataset from a ZIP or folder.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--file <path> | yes | Path to a dataset ZIP file or a dataset/ folder (folder is zipped in memory before upload) | |
--mode <append|replace> | no | "append" | Import mode (default: append) |
--yes | no | false | Skip the destructive confirmation prompt for —mode replace (use in CI) |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow dataset example create [options] <workflow-id>
Create one eval example without re-uploading the dataset.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--name <name> | yes | Example name (1–64 chars) | |
--input-json <json> | no | Trigger input as a JSON literal | |
--input-file <path> | no | Trigger input from a JSON file (or - for stdin) | |
--expected-json <json> | no | Expected output as a JSON literal | |
--expected-file <path> | no | Expected output from a JSON file (or - for stdin) | |
--annotation <text> | no | Free-form annotation | |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow dataset example update [options] <workflow-id> <exampleId>
Patch one eval example. Omitted flags are left alone.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
exampleId | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--name <name> | no | Rename the example (1–64 chars) | |
--input-json <json> | no | Replace trigger input with this JSON literal | |
--input-file <path> | no | Replace trigger input from a JSON file (or - for stdin) | |
--expected-json <json> | no | Replace expected output with this JSON literal | |
--expected-file <path> | no | Replace expected output from a JSON file (or - for stdin) | |
--annotation <text> | no | Replace annotation; pass empty string to clear | |
--row-order <n> | no | Reorder the row (0-based) | |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow dataset example delete [options] <workflow-id> <exampleId>
Delete one eval example by id. Non-TTY shells require —yes.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
exampleId | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--yes | no | false | Required for non-TTY shells; explicit acknowledgment that this is destructive |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow dataset example get [options] <workflow-id> <exampleId>
Fetch one eval example with full triggerInput, expectedOutput, and metadata.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
exampleId | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow dataset validate [options] [path]
Validate a dataset folder against the examples/<name>/{input,expected,meta} convention. Defaults to ./dataset/.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
path | no | no |
eigenpal workflow experiment|exp list|ls [options] <workflow-id>
List executions for the workflow, newest first.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--batch-id <id> | no | Filter by batch | |
--limit <n> | no | 50 | Page size |
--offset <n> | no | 0 | Page offset |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow experiment|exp run [options] <workflow-id>
Start a batch eval against the workflow’s dataset.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--example-id <id> | no | [] | Run only this example (repeatable) |
--wait | no | false | Poll until terminal; non-zero exit on passRate < 1.0 |
--interval <n> | no | 10 | Polling interval in seconds (default 10) |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow experiment|exp status [options] <workflow-id> <batchId>
Aggregate progress for a batch by batchId.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
batchId | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--watch | no | false | Poll until every execution reaches a terminal state (completed/failed/cancelled/rejected), then exit |
--short | no | false | Single-line plain-text summary on stdout (e.g. 6/6 done failed=0 cancelled=0 rejected=0). Pipe-friendly for monitoring loops; mutually exclusive with —watch. |
--interval <seconds> | no | 5 | Poll interval in seconds when —watch is set (default 5) |
--max-wait <seconds> | no | 1800 | Hard ceiling for —watch in seconds (default 1800 = 30 min) |
--include <kinds> | no | "" | Comma-separated extras to attach when —watch terminates: payload (full per-execution snapshot, can be hundreds of KB) |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow experiment|exp cancel [options] <workflow-id> <batchId>
Cancel every execution in a batch. Idempotent.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
batchId | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--yes | no | Required for non-TTY shells (CI, pipes). Acts immediately, no prompt. | |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow experiment|exp results [options] <workflow-id> [batchId]
Download eval results in CSV or JSON.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
batchId | no | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--format <csv|json> | yes | Output format | |
--out <path> | no | Output file. When omitted, the binary streams to stdout. | |
--base-url <url> | no | Server base URL |
eigenpal workflow experiment|exp compare|diff [options] <batchIdA> <batchIdB>
Diff eval scores between two experiment batches.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
batchIdA | yes | no | |
batchIdB | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--sort <abs-delta-desc|delta-asc|delta-desc|name> | no | "abs-delta-desc" | Row sort order (default: biggest movers first) |
--regression-threshold <n> | no | 0.05 | Δ below this is flagged as a regression (default 0.05) |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow experiment|exp watch [options] <workflow-id> <batchId>
Poll until terminal, then auto-pull results, replaces status --watch + results --out.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
batchId | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--interval <seconds> | no | 5 | Poll interval in seconds (default 5) |
--max-wait <seconds> | no | 1800 | Hard ceiling in seconds (default 1800 = 30 min) |
--pull-on-complete <path> | no | Destination for the results file. Default: ./results-<batchId>.<format> | |
--format <csv|json> | no | "json" | Results export format (default json) |
--no-pull | no | Skip auto-pulling results on terminal (just watch) | |
--base-url <url> | no | Server base URL |
eigenpal workflow versions list|ls [options] <workflow-id>
List historical workflow versions, newest first.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--limit <n> | no | 50 | Page size |
--offset <n> | no | 0 | Page offset |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow versions restore [options] <workflow-id> <versionId>
Restore the workflow to a previous version.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
workflow-id | yes | no | |
versionId | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow step-type list|ls [options]
List every step type the deployment supports.
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--search <q> | no | Filter | |
--limit <n> | no | 50 | Page size |
--offset <n> | no | 0 | Page offset |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow step-type get [options] <type>
Return the full schema and behavioral docs for one step type.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
type | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--out <path> | no | Write to file |
eigenpal workflow evaluator-type list|ls [options]
List every evaluator type with a one-line description.
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--search <q> | no | Filter by type, name, or description | |
--limit <n> | no | 50 | Page size |
--offset <n> | no | 0 | Page offset |
--base-url <url> | no | Server base URL | |
--json | no | Output the raw server response as JSON |
eigenpal workflow evaluator-type get [options] <type>
Fetch the JSON Schema for one evaluator type. Pipe through jq to inspect specific fields.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
type | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--out <path> | no | Write to file |
eigenpal workflow step exec [options] <type>
DISABLED, local mimic runners removed pending server-side redesign (EIG-104). Use run or workflow experiment run instead.
Arguments
| Name | Required | Variadic | Description |
|---|---|---|---|
type | yes | no |
Options
| Flag | Required | Default | Description |
|---|---|---|---|
--config-json <json> | no | (unused; kept for back-compat parsing) | |
--config-file <path> | no | (unused) | |
--inputs <pairs...> | no | (unused) | |
--output-schema <path> | no | (unused) |