Fix bash job scripts: require_command checks, curl temp file leak, batched MySQL inserts#187
Conversation
…batch MySQL inserts Co-authored-by: sfreeman422 <16405652+sfreeman422@users.noreply.github.com> Agent-Logs-Url: https://github.com/dev-chat/mocker/sessions/1c571688-b133-4945-9edc-a6d11e23df1d
There was a problem hiding this comment.
Pull request overview
This PR tightens reliability and performance of the cron-executed bash job scripts under packages/jobs/ by adding dependency validation, preventing temp file leaks on curl failures, and reducing MySQL connection overhead via batched inserts.
Changes:
- Add
require_commandchecks inhealth-jobforcurl,grep, andmktemp. - Ensure
fun-fact-jobcleans up its curl response temp file even when curl fails. - Batch
pricing-jobMySQL inserts into a single transaction instead of per-rowmysql_querycalls.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| packages/jobs/health-job/script.sh | Adds require_command helper and validates required CLI dependencies in main(). |
| packages/jobs/fun-fact-job/script.sh | Prevents temp file leakage on curl failure by avoiding set -e exit and cleaning up on error. |
| packages/jobs/pricing-job/script.sh | Switches to transactional, batched inserts to reduce per-row MySQL invocations. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| mysql_query "START TRANSACTION; | ||
| ${sql_batch}COMMIT;" || { echo 'Batch insert failed; transaction has been rolled back' >&2; return 1; } |
There was a problem hiding this comment.
Building the entire INSERT workload into sql_batch and passing it as a single mysql -e argument can hit OS ARG_MAX/"Argument list too long" limits and/or MySQL max_allowed_packet as teams/items grow, causing the job to fail before any SQL runs. Consider streaming the SQL to mysql via stdin (or a temp file) and/or chunking into smaller batches (e.g., per team or fixed-size batches) to avoid unbounded query size and quadratic string-concatenation overhead.
| mysql_query "START TRANSACTION; | |
| ${sql_batch}COMMIT;" || { echo 'Batch insert failed; transaction has been rolled back' >&2; return 1; } | |
| { | |
| echo 'START TRANSACTION;' | |
| printf '%s' "${sql_batch}" | |
| echo 'COMMIT;' | |
| } | mysql --host="${MYSQL_HOST}" --user="${MYSQL_USER}" --password="${MYSQL_PASSWORD}" "${MYSQL_DATABASE}" \ | |
| || { echo 'Batch insert failed; transaction has been rolled back' >&2; return 1; } |
* Refactored jobs to use bash instead of python for portability * Update packages/jobs/pricing-job/script.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Fix bash job scripts: require_command checks, curl temp file leak, batched MySQL inserts (#187) * Initial plan * Fix code review feedback: require_command checks, temp file cleanup, batch MySQL inserts Co-authored-by: sfreeman422 <16405652+sfreeman422@users.noreply.github.com> Agent-Logs-Url: https://github.com/dev-chat/mocker/sessions/1c571688-b133-4945-9edc-a6d11e23df1d --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfreeman422 <16405652+sfreeman422@users.noreply.github.com> --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: sfreeman422 <16405652+sfreeman422@users.noreply.github.com>
Three unresolved code review issues in the bash job scripts: missing dependency validation in
health-job, a temp file leak on curl failure infun-fact-job, and per-row MySQL connections inpricing-job.Changes
health-job/script.sh: Addedrequire_commandfunction (matching the pattern in other jobs) with checks forcurl,grep, andmktempinmain().fun-fact-job/script.sh: Fixed temp file leak insend_slack_message— curl now uses|| truewith an emptyresponse_codeguard that cleans up the temp file and returns early on failure, consistent with the health-job pattern.pricing-job/script.sh: Replaced O(teams × items) individualmysql_querycalls with a single batched invocation wrapped in a transaction:📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.