agentlab.experiments.launch_exp

Functions

find_incomplete(study_dir[, include_errors])

Find all incomplete experiments for relaunching.

import_object(path)

non_dummy_count(exp_args_list)

noop(*args, **kwargs)

run_experiments(n_jobs, exp_args_list, study_dir)

Run a list of ExpArgs in parallel.
agentlab.experiments.launch_exp.find_incomplete(study_dir: str | Path, include_errors=True)

Find all incomplete experiments for relaunching.

Note: completed experiments are kept but are replaced by dummy exp_args with nothing to run. This help keeping the dependencies between tasks.

Parameters:

  • study_dir – Path The directory where the experiments are saved.

  • include_errors – str Find all incomplete experiments and relaunch them. - “incomplete_only”: relaunch only the incomplete experiments. - “incomplete_or_error”: relaunch incomplete or errors.

Returns:

list[ExpArgs]

List of ExpArgs objects to relaunch.

Raises:

ValueError – If the study_dir does not exist.

agentlab.experiments.launch_exp.import_object(path: str)
agentlab.experiments.launch_exp.non_dummy_count(exp_args_list: list[ExpArgs]) → int
agentlab.experiments.launch_exp.noop(*args, **kwargs)
agentlab.experiments.launch_exp.run_experiments(n_jobs, exp_args_list: list[ExpArgs], study_dir, parallel_backend='ray', avg_step_timeout=60)

Run a list of ExpArgs in parallel.

To ensure optimal parallelism, make sure ExpArgs.depend_on is set correctly and the backend is set to dask.

Parameters:

  • n_jobs – int Number of parallel jobs.

  • exp_args_list – list[ExpArgs] List of ExpArgs objects.

  • study_dir – Path Directory where the experiments will be saved.

  • parallel_backend – str Parallel backend to use. Either “joblib”, “ray” or “sequential”. The only backend that supports webarena graph dependencies correctly is ray or sequential.

  • avg_step_timeout – int Will raise a TimeoutError if the episode is not finished after env_args.max_steps * avg_step_timeout seconds.

Raises:

ValueError – If the parallel_backend is not recognized.