No-Code Tuning Usage

No-code tuning is a tool for running hyperparameter tuning from the command line without writing Python scripts.

Overview

No-code tuning consists of three components: Preset, Strategy, and Evaluator. By combining these components according to your tuning objective, you can easily run tuning.

Component	Role
Preset	Defines the parameter search space and how to apply parameters to a command
Strategy	Determines how to explore the search space
Evaluator	Extracts the objective function value from command output

tip

When Strategy and Evaluator are not specified, recommended components are automatically selected depending on the Preset. If you are unsure which Strategy and Evaluator to use, specifying only the Preset allows you to start tuning immediately.

What is a Preset?

A Preset is a template that bundles tuning settings for a specific framework. Each Preset defines framework-specific knowledge as shown below, allowing users to run tuning without worrying about framework details simply by specifying a Preset.

• What to tune: The hyperparameters to optimize for the framework and their search ranges
• How to apply: How to pass the explored parameters to the command in the framework-specific argument format

Currently available Presets:

Preset	Description	Default Strategy	Default Evaluator
megatron	Parameter set for tuning with Megatron family frameworks	megatron-heuristic	megatron

What is a Strategy?

A Strategy is the component that determines how to explore the parameter search space. Given the search space defined by a Preset, it proposes the next combination of parameters to try. There are two types: preset-specific Strategies that can only be used with certain Presets, and general-purpose Strategies that can be used with any Preset.

Currently available preset-specific Strategies:

Strategy	Description
megatron-heuristic	Efficiently explores parameters for the megatron Preset based on parallelization strategy knowledge
megatron-staged-blackbox	Performs staged black-box search of parameters for the megatron Preset

Currently available general-purpose Strategies (see General-Purpose Strategies for details):

Strategy	Description
random	Samples parameters randomly
grid	Exhaustively searches all parameter combinations
optuna	Samples according to Optuna's Sampler

What is an Evaluator?

An Evaluator is the component that extracts the objective function value from command execution results. It extracts a numerical value from stdout or execution time, and tuning proceeds to minimize (or maximize) that value. Like Strategies, there are preset-specific Evaluators and general-purpose Evaluators.

Currently available preset-specific Evaluators:

Evaluator	Description
megatron	Maximizes the training throughput value TFLOP/s/GPU for the megatron Preset

Currently available general-purpose Evaluators:

Evaluator	Description
duration	Minimizes command execution time

CLI

No-code tuning is executed through the zenithtune command. There are three subcommands:

Subcommand	Description
optimize	Run tuning
apply	Apply the best parameters from tuning results and run the command
analyze	Analyze and visualize tuning results

optimize: Run Tuning

zenithtune optimize --preset <preset_name> [options] -- <command_to_tune>

Arguments:

Argument	Description	Default
--preset	Preset name to use (required)	-
--strategy	Strategy name	Preset's recommended Strategy
--evaluator	Evaluator name	Preset's recommended Evaluator
--n-trials	Number of trials	10
--output-dir	Output directory for results	outputs
--study-name	Optuna study name	study_YYYYMMDD_HHMMSS
--db-path	Path to existing DB for resuming	-
--skip-default	Skip the baseline trial	False
--timeout	Static timeout in seconds (see below)	None
--timeout-dynamic	Dynamic timeout factor (see below)	None (default when specified: 1.5)
--args	Comma-separated key=value pairs passed to the preset	-
--skip-duplicate	Skip re-execution of duplicate parameter sets	True
--list	List registered presets, strategies, and evaluators	-
--dev	Enable demo components (dev: prefixed)	False

Examples:

# Tune with dev:demo preset for 10 trials
zenithtune optimize --preset dev:demo --dev --n-trials 10 -- python train.py --epochs 3

# Resume tuning from an existing study with 20 additional trials
zenithtune optimize --preset dev:demo --dev --n-trials 20 --db-path outputs/my_study/study.db -- python train.py --epochs 3

# List registered presets, strategies, and evaluators
zenithtune optimize --list --dev

warning

Place a separator -- between the zenithtune command and the command to be tuned.

apply: Apply Best Parameters

Retrieves the best parameters from tuning results (DB file) and runs the command with them applied.

zenithtune apply --db-path <db_file_path> --preset <preset_name> [--args <key=value,...>] -- <command>

Examples:

# Apply tuning results and run the command
zenithtune apply --db-path outputs/my_study/study.db --preset dev:demo --dev -- python train.py --epochs 100

The apply subcommand runs the command once with the best parameters from the DB specified by --db-path. You can modify the command for production use, such as specifying a longer --epochs than during tuning.

warning

The --preset and --args specified during optimize must also be specified when using apply.

analyze: Analyze Tuning Results

Used when you want to analyze results from an existing study DB. The same analysis as the analyze subcommand is automatically run upon completion of optimize, so this is useful when analysis results have been deleted or when the optimize command was interrupted.

zenithtune analyze <db_file_path>

Examples:

zenithtune analyze outputs/my_study/study.db

The following files are generated in the same directory as the DB file:

• history.png: Progression of the tuning target (objective function value) across trials
• timeline.png: Timeline of each trial
• importances.png: Parameter importance
• contour.html: Contour plot between parameters

Workflow Example

Here is a typical workflow using the dev:demo preset. dev:demo is a demo preset that lets you try the workflow without GPUs or training frameworks. For actual tuning, use a preset such as megatron (see Megatron Preset).

# 1. Check available presets
zenithtune optimize --list --dev

# 2. Run 10 trials of tuning
zenithtune optimize --preset dev:demo --dev --n-trials 10 -- python train.py --epochs 3

# 3. Analyze results
zenithtune analyze outputs/study_20260305_120000/study.db

# 4. Run production with best parameters
zenithtune apply --db-path outputs/study_20260305_120000/study.db --preset dev:demo --dev -- python train.py --epochs 100

Useful Features

Here are some useful features to enhance tuning effectiveness.

Timeout

Trials that exceed the timeout are recorded as failed (FAIL) and the next trial begins. For performance tuning, limiting significantly slow trials reduces the total tuning time. There are two types of timeout: static timeout and dynamic timeout.

Static Timeout

Use --timeout to specify a fixed number of seconds. The same timeout applies to all trials.

# Timeout each trial at 1200 seconds (20 minutes)
zenithtune optimize --preset <preset_name> --timeout 1200 -- python train.py

Dynamic Timeout

Use --timeout-dynamic to automatically adjust the timeout based on the best trial's duration. The timeout is set to the best trial's duration multiplied by a factor (default: 1.5). If no trial has completed yet (initial trials), no timeout is applied.

# Dynamic timeout (default factor: 1.5)
zenithtune optimize --preset <preset_name> --timeout-dynamic -- python train.py

# Specify the factor (allow up to 3x the best trial's duration)
zenithtune optimize --preset <preset_name> --timeout-dynamic 3.0 -- python train.py

tip

Dynamic timeout is useful when execution time varies significantly depending on parameter combinations. It shortens overall tuning time by terminating obviously slow trials early.

Combining Static and Dynamic Timeout

--timeout and --timeout-dynamic can be used together. When both are specified, the effective timeout for each trial is the minimum of the static and dynamic timeouts.

# Use static 1200s (20 min) as an upper bound, with dynamic timeout (factor 2.0) for early termination
zenithtune optimize --preset <preset_name> --timeout 1200 --timeout-dynamic 2.0 -- python train.py

This allows dynamic timeout to efficiently terminate slow trials while the static timeout acts as an absolute upper bound, even when the first trial is extremely slow.

Baseline Trial

When running optimize, the original command without tuning parameters is executed first by default (baseline trial). This allows you to check the improvement rate from tuning in analyze.

If the baseline trial is not needed, use --skip-default to skip it.

zenithtune optimize --preset dev:demo --dev --skip-default -- python train.py

Using the Python API

You can also use the PresetTuner API from Python code instead of the CLI.

from aibooster.intelligence.zenith_tune.tuners.preset import PresetTuner

# Run tuning
tuner = PresetTuner(
    command="python train.py --epochs 3",
    preset="dev:demo",
)
best_value, best_params = tuner.optimize(n_trials=10, dynamic_timeout=1.5)

# Analyze
tuner.analyze()

# Apply best parameters and run
returncode = PresetTuner.apply_from_db(
    db_path="outputs/my_study/study.db",
    preset="dev:demo",
    command="python train.py --epochs 100",
)

Customization

For creating custom presets and evaluators, see Customization.