Parallelize CI Jobs

Goal

Achieve a target runtime reduction through parallel job execution

How to Run

Run this loop in your preferred coding agent to automatically parallelize CI jobs and reduce pipeline runtime.

1. 01

   Identify CI Configuration File

   Locate your project's CI configuration file (e.g., .github/workflows/*.yml, .gitlab-ci.yml)

2. 02

   Run Check Command

   Execute 'pipeline metrics' to get baseline performance data

3. 03

   Apply Parallelization Changes

   Implement suggested parallel job configurations based on agent analysis

4. 04

   Verify Results

   Re-run pipeline metrics to confirm runtime improvement

5. 05

   Iterate Until Target

   Continue loop iterations until target runtime is achieved or max iterations reached

Workflow Steps

1. 01

   Analyze Current Pipeline Structure

   Identify sequential jobs and dependencies in the CI configuration

2. 02

   Detect Parallelization Opportunities

   Find independent jobs that can run concurrently without conflicts

3. 03

   Modify CI Configuration

   Update workflow file to enable parallel execution for identified jobs

4. 04

   Execute Pipeline Check

   Run pipeline metrics to measure current runtime performance

5. 05

   Evaluate Improvement

   Compare new metrics against target threshold to determine success

6. 06

   Decide Next Action

   If target not met and iterations remain, return to analysis; otherwise exit loop

Start the "Parallelize CI Jobs" loop.

Goal: Achieve a target runtime reduction through parallel job execution
Max iterations: 10
Between iterations run: pipeline metrics
Exit when: Runtime reduced to target


Analyze the current CI pipeline configuration to identify sequential jobs that can be executed in parallel. For each job, check if it has dependencies on other jobs and whether those dependencies prevent parallel execution. Modify the CI configuration to maximize parallel job runs while maintaining proper build integrity. After making changes, run 'pipeline metrics' to measure the runtime improvement. If the runtime has been reduced to the target threshold, exit the loop. Otherwise, continue iterating up to the maximum allowed iterations. Always preserve existing test dependencies and ensure that parallelization does not introduce race conditions or inconsistent artifacts.

Self-pace this loop. After each iteration, run `pipeline metrics` and evaluate the output, and only continue if the exit condition is not met (Runtime reduced to target). Stop when the exit condition passes or 10 iterations are reached. Give a short status update each pass.