dlgforge.pipeline.hf_push

Hugging Face export preparation and push helpers.

HFPushSettings(enabled, auto_push_on_run, repo_id, repo_type, export_dir, include_run_state, private, commit_message, source_file, clean_remote, generate_stats, stats_file, generate_plots, plots_dir, output_columns) dataclass

Resolved Hugging Face export/push settings.

Parameters:

Name	Type	Description	Default
enabled	bool	Boolean flag that controls optional behavior.	required
auto_push_on_run	bool	bool value used by this operation.	required
repo_id	str	str value used by this operation.	required
repo_type	str	str value used by this operation.	required
export_dir	Path	Path value used by this operation.	required
include_run_state	bool	bool value used by this operation.	required
private	bool	bool value used by this operation.	required
commit_message	str	str value used by this operation.	required
source_file	str	Filesystem path used by this operation.	required
clean_remote	bool	Boolean flag that controls optional behavior.	required
generate_stats	bool	bool value used by this operation.	required
stats_file	str	str value used by this operation.	required
generate_plots	bool	bool value used by this operation.	required
plots_dir	str	str value used by this operation.	required
output_columns	Dict[str, str]	Dict[str, str] value used by this operation.	required

Raises:

Type	Description
Exception	Construction may raise when required dependencies or inputs are invalid.

Side Effects / I/O: - May read from or write to local filesystem artifacts. - May perform network, model, or distributed runtime operations.

Preconditions / Invariants: - Instantiate and use through documented public methods.

Examples:

>>> from dlgforge.pipeline.hf_push import HFPushSettings
>>> HFPushSettings(...)

HFPushOptions(repo_id='', repo_type='', source_dir='', export_dir='', include_run_state=False, token=None, commit_message='', prepare_export=True, push=True, clean_remote=False) dataclass

CLI override options for export/push operations.

Parameters:

Name	Type	Description	Default
repo_id	str	str value used by this operation.	''
repo_type	str	str value used by this operation.	''
source_dir	str	str value used by this operation.	''
export_dir	str	str value used by this operation.	''
include_run_state	bool	bool value used by this operation.	False
token	Optional[str]	Optional[str] value used by this operation.	None
commit_message	str	str value used by this operation.	''
prepare_export	bool	Boolean flag that controls optional behavior.	True
push	bool	Boolean flag that controls optional behavior.	True
clean_remote	bool	Boolean flag that controls optional behavior.	False

Raises:

Type	Description
Exception	Construction may raise when required dependencies or inputs are invalid.

Side Effects / I/O: - May read from or write to local filesystem artifacts. - May perform network, model, or distributed runtime operations.

Preconditions / Invariants: - Instantiate and use through documented public methods.

Examples:

>>> from dlgforge.pipeline.hf_push import HFPushOptions
>>> HFPushOptions(...)

run_push(config_path, options)

Prepare export artifacts and optionally push them to the Hub.

Parameters:

Name	Type	Description	Default
config_path	str	Path to a configuration file.	required
options	HFPushOptions	Configuration mapping that controls runtime behavior.	required

Returns:

Name	Type	Description
None	None	No value is returned.

Raises:

Type	Description
FileNotFoundError	Raised when validation or runtime requirements are not met.

Side Effects / I/O: - May read from or write to local filesystem artifacts. - May perform network, model, or distributed runtime operations.

Preconditions / Invariants: - Callers should provide arguments matching annotated types and expected data contracts.

Examples:

>>> from dlgforge.pipeline.hf_push import run_push
>>> run_push(...)

maybe_auto_push_after_run(cfg, output_paths)

Conditionally execute auto push after run.

Parameters:

Name	Type	Description	Default
cfg	Dict[str, Any]	Configuration mapping that controls runtime behavior.	required
output_paths	OutputPaths	Filesystem path used by this operation.	required

Returns:

Name	Type	Description
None	None	No value is returned.

Raises:

Type	Description
Exception	Propagates unexpected runtime errors from downstream calls.

Side Effects / I/O: - May read from or write to local filesystem artifacts. - May perform network, model, or distributed runtime operations.

Preconditions / Invariants: - Callers should provide arguments matching annotated types and expected data contracts.

Examples:

>>> from dlgforge.pipeline.hf_push import maybe_auto_push_after_run
>>> maybe_auto_push_after_run(...)

resolve_hf_push_settings(cfg, project_root)

Resolve hf push settings.

Parameters:

Name	Type	Description	Default
cfg	Dict[str, Any]	Configuration mapping that controls runtime behavior.	required
project_root	Path	Resolved project directory context.	required

Returns:

Name	Type	Description
HFPushSettings	HFPushSettings	Resolved value after applying defaults and normalization rules.

Raises:

Type	Description
Exception	Propagates unexpected runtime errors from downstream calls.

Side Effects / I/O: - May read from or write to local filesystem artifacts. - May perform network, model, or distributed runtime operations.

Preconditions / Invariants: - Callers should provide arguments matching annotated types and expected data contracts.

Examples:

>>> from dlgforge.pipeline.hf_push import resolve_hf_push_settings
>>> resolve_hf_push_settings(...)

prepare_export(source_dir, export_dir, source_file, include_run_state, repo_id, generate_stats=False, stats_file='dataset_stats.json', generate_plots=False, plots_dir='plots', output_columns=None)

Prepare export.

Parameters:

Name	Type	Description	Default
source_dir	Path	Path value used by this operation.	required
export_dir	Path	Path value used by this operation.	required
source_file	str	Filesystem path used by this operation.	required
include_run_state	bool	bool value used by this operation.	required
repo_id	str | None	str | None value used by this operation.	required
generate_stats	bool	bool value used by this operation.	False
stats_file	str	str value used by this operation.	'dataset_stats.json'
generate_plots	bool	bool value used by this operation.	False
plots_dir	str	str value used by this operation.	'plots'
output_columns	Optional[Dict[str, str]]	Optional[Dict[str, str]] value used by this operation.	None

Returns:

Name	Type	Description
Path	Path	Value produced by this API.

Raises:

Type	Description
FileNotFoundError	Raised when validation or runtime requirements are not met.

Side Effects / I/O: - May read from or write to local filesystem artifacts. - May perform network, model, or distributed runtime operations.

Preconditions / Invariants: - Callers should provide arguments matching annotated types and expected data contracts.

Examples:

>>> from dlgforge.pipeline.hf_push import prepare_export
>>> prepare_export(...)

push_to_hub(export_dir, repo_id, repo_type, token, commit_message, private=True, clean_remote=False)

Push to hub.

Parameters:

Name	Type	Description	Default
export_dir	Path	Path value used by this operation.	required
repo_id	str	str value used by this operation.	required
repo_type	str	str value used by this operation.	required
token	Optional[str]	Optional[str] value used by this operation.	required
commit_message	str	str value used by this operation.	required
private	bool	bool value used by this operation.	True
clean_remote	bool	Boolean flag that controls optional behavior.	False

Returns:

Name	Type	Description
None	None	No value is returned.

Raises:

Type	Description
ValueError	Raised when validation or runtime requirements are not met.
ImportError	Raised when validation or runtime requirements are not met.

Side Effects / I/O: - May read from or write to local filesystem artifacts. - May perform network, model, or distributed runtime operations.

Preconditions / Invariants: - Callers should provide arguments matching annotated types and expected data contracts.

Examples:

>>> from dlgforge.pipeline.hf_push import push_to_hub
>>> push_to_hub(...)