Branch: refs/heads/ehb54-issue-915 Home: https://github.com/ehb54/us3lims_common Commit: a6f9e2ef5a75d86857bb1a34acafa0e6573a7b62 https://github.com/ehb54/us3lims_common/commit/a6f9e2ef5a75d86857bb1a34acafa... Author: ehb54 <brookes@uthscsa.edu> Date: 2026-06-20 (Sat, 20 Jun 2026) Changed paths: M class/submit_local.php M global_config.php.template Log Message: ----------- Validate sbatch submission status before recording a gfacID submit_job() never checked the exit status of the sbatch/qsub exec() call, and parsed the job ID positionally from the first output line assuming it was always "Submitted batch job <N>". Under scheduler load, sbatch can instead fail with e.g. "sbatch: error: Batch job submission failed: Socket timed out on send/recv operation" - the positional parse of that line happens to land on the literal word "job", which then got stored as gfacID and tracked through the rest of the pipeline as if a real job existed, eventually surfacing much later and opaquely as "Failed data fetch" during cleanup. Validate the sbatch output against /^Submitted batch job\s+(\d+)/ and check exec()'s exit status before accepting a job ID. Retry the submission with exponential backoff on failure (configurable via $global_sbatch_submit_retries / $global_sbatch_submit_retry_wait_seconds in global_config.php, overridable per-cluster), since this class of failure is transient and load-related. If all retries are exhausted, update_db() now marks the request SUBMIT_TIMEOUT with the real error instead of recording an empty/bogus gfacID and launching a jobmonitor to watch a job that was never submitted. Fixes ehb54/ultrascan-tickets#915 To unsubscribe from these emails, change your notification settings at https://github.com/ehb54/us3lims_common/settings/notifications