Semantic operators

SemTransform

Extract, normalize, rewrite, or structure each input row.

SemTransform produces one semantic result per distinct primary-key value. It is appropriate when the model must read a value and return structured fields or a rewritten value.

SemTransform(
    name="score-rate-sentiment",
    input_spaces=("headlines",),
    output_space="sentiment",
    primary_key="headline",
    prompt=(
        "Using the headline only, return yield_sentiment and slope_sentiment "
        "as -1, 0, or 1, plus a concise rationale for each score."
    ),
    emits=("yield_sentiment", "slope_sentiment",
           "yield_rationale", "slope_rationale"),
    provider="openai",
    model="gpt-5",
    reasoning_effort="low",
    batch_size=12,
    max_output_tokens=4096,
)

Parameter	Default	Meaning
`input_spaces`	required	The Space or Spaces read by the task.
`output_space`	required	The Space that receives the task output.
`name`	generated	A stable task name used in reviews, sessions, and metrics.
`primary_key`	id	The semantically meaningful field that identifies a value for this task. Rows with the same primary key represent one task value and produce one task result. Use Union first when every source identity must remain attached.
`evidence`	none	Optional supporting fields shown to the semantic model. Do not repeat a primary-key field or include irrelevant metadata.
`prompt`	required	The criterion or transformation written in plain language.
`provider / model`	required	The model used for this task's data decisions.
`reasoning_effort`	low	OpenAI GPT-5-family reasoning effort. Use none for the lowest-latency, lowest-reasoning-cost path when the task contract is precise; increase it only after representative quality evidence justifies the added time and spend.
`include_reason`	true	Whether every model decision includes item-local visible evidence. The compiled prompt defaults to at most ten words; an explicit prompt request for longer visible reasoning is preserved. This is generation guidance; returned reasons are never truncated. Set false only for a validated task where result and confidence suffice.
`confidence_threshold`	operator default	Exact boundary between unresolved and accepted evidence. Relifold compiles this value and operator-specific confidence guidance into the effective judge prompt.
`function_parameters`	registered gate defaults	Typed values for a generated semantic pair predicate such as merge_only_if, ancestor_only_if, match_only_if, or relate_only_if. The predicate declares the schema and finite candidates; reviewed evidence can then support parameter advice. Closed library gates do not take these bindings.
`max_llm_calls`	unlimited	A hard task-level call ceiling. The task stops instead of silently exceeding it.
`max_cost_usd`	unlimited	A hard task-level model-cost ceiling. A session-level ceiling may be stricter.
`max_output_tokens`	2048	Maximum generated tokens per model request. Increase it when one legitimate batch cannot fit its structured result.

Parameter	Default	Meaning
`batch_size`	operator default	Number of distinct values requested together. Larger batches reduce overhead but need more output space and can make individual results less reliable.
`emits`	none	Expected output column names included in the task instruction and version schema.

Design guidance

Keep the primary key short and semantically central. Do not send IDs, URLs, timestamps, or full article bodies unless they materially improve the requested transformation. If two rows have the same primary key, Relifold transforms that task value once and emits one task result. To retain both source rows, exact-consolidate them first:

Union(
    name="certify-headline-identity",
    input_spaces=("headline-records",),
    output_space="headline-values",
    key="headline",
    merge_policy=MergePolicy(
        per_column={"article_ids": "union"},
        default="last",
    ),
)

SemTransform(
    name="score-headline",
    input_spaces=("headline-values",),
    output_space="headline-sentiment",
    primary_key="headline",
    evidence=(),
    ...
)

The semantic task sees the meaningful headline once; article_ids remains attached for deterministic expansion or mapping afterward and is not repeated as model evidence.

Meaning-bearing outputs. When an emitted field becomes a downstream identity or retrieval signature, require it to preserve every distinction that changes identity. For an event label, a reader seeing only the label should be able to identify the principal subject and the concrete action, incident, policy, object, dispute, or other development. Generic presentation words such as news, roundup, update, or coverage cannot replace those anchors. If one row supports multiple distinct values, declare a collection and Flatten it, or require one faithful scalar that retains the distinguishing details; never invent details absent from the evidence.

Fallback rows. A result below confidence_threshold, or an item still unresolved after retries, preserves the source row without inventing the declared semantic fields. Before using an emitted field as an exact grouping key or downstream primary key, add a deterministic total fallback, or explicitly filter unresolved rows when omission is allowed by the output contract.

Output size. If one batch legitimately returns many fields or long rationales, increase max_output_tokens. A cut-off structured response counts as a failed call after retries and appears in session observability.

Research coverage. A draft research run reports declared semantic-field coverage separately from output row count, together with failed judge calls. Source rows preserved as fallbacks still count as output rows but do not count as having the declared fields; manually review the produced values before accepting their quality.