intelligence.zenith_tune.evaluators.regex

Regex-based value extractor.

This module provides a flexible regex-based extractor that can extract objective values from log output using configurable patterns.

RegexEvaluator Objects

class RegexEvaluator(TuningEvaluator)

Extract objective values from text using regular expressions.

This extractor searches for patterns in the input text and extracts numeric values. It supports multiple matches with various aggregation methods.

Example:

Extract loss value (minimize by default)

extractor = RegexEvaluator( pattern=r"loss:\s*([\d.]+)", selector="last" ) value = extractor.evaluate("epoch 1 loss: 0.5\nepoch 2 loss: 0.3")

Returns 0.3 (last match)

Extract throughput (maximize)

extractor = RegexEvaluator( pattern=r"throughput:\s*([\d.]+)", selector="mean", direction=Direction.MAXIMIZE )

__init__

def __init__(pattern: str,
             selector: SelectorType = "last",
             direction: Union[Direction, str] = Direction.MINIMIZE)

Initialize the regex extractor.

Arguments:

• pattern - Regular expression pattern with one capture group for the value.
• selector - How to select from multiple matches:
  • "first": Use the first match
  • "last": Use the last match (default)
  • "min": Use the minimum value
  • "max": Use the maximum value
  • "mean": Use the mean of all values
  • "median": Use the median of all values
• direction - Optimization direction. Can be Direction enum or string ("minimize" or "maximize").

Raises:

• ValueError - If pattern is invalid or doesn't have exactly one group.

evaluate

def evaluate(stdout: str, metadata: dict[str, Any]) -> float

Extract the objective value from stdout using regex.

Arguments:

• stdout - The stdout output from the command execution.
• metadata - Trial metadata (unused).

Returns:

The extracted objective value.

Raises:

• ValueError - If no matches found.