intelligence.zenith_tune.utils

staging_directory

def staging_directory(command: str, old_path: str, new_path: str) -> str

Replace directory paths in command and copy directory for distributed training.

Only rank 0 performs the copy operation.

Arguments:

• command - The shell command string
• old_path - Path to replace
• new_path - New path to use

Returns:

Modified command string

Raises:

• FileNotFoundError - If old_path doesn't exist
• OSError - If copying fails

replace_params_to_file

def replace_params_to_file(input_filepath: str, output_filepath: str,
                           params: Dict[str, Any]) -> None

Replace parameters in a file with given values.

Arguments:

• input_filepath str - Path to the input file.
• output_filepath str - Path to the output file.
• params Dict[str, Any] - Dictionary of parameters to replace.

get_hf_model_config

def get_hf_model_config(model_name_or_path: str) -> Dict[str, Any]

Get model config.json from a HuggingFace model.

If model_name_or_path is a local directory, reads config.json directly. Otherwise, downloads config.json from HuggingFace Hub.

Arguments:

• model_name_or_path - HuggingFace model ID (e.g. "meta-llama/Llama-2-7b-hf") or path to a local directory containing config.json.

Returns:

Dictionary of the model's config.json contents.

Raises:

• FileNotFoundError - If local path has no config.json.
• huggingface_hub.errors.RepositoryNotFoundError - If the model ID does not exist or is private.
• huggingface_hub.errors.EntryNotFoundError - If the repository does not contain config.json.
• requests.exceptions.ConnectionError - If the Hub is unreachable.