Complete Guide: Force Terminate Stuck Jenkins Builds

When a Jenkins build gets stuck in the “aborting” state, you can force terminate it using several methods: try the UI’s “forcibly terminate running steps” link first, then use the script console with build.doStop() and build.doKill() commands, or manually remove the build directory. For stubborn builds, you may also need to check thread dumps, kill hanging processes on the agent, or restart the Jenkins agent to completely resolve the stuck state.

Contents

• Understanding the Stuck Aborting State
• Method 1: UI-Based Forcible Termination
• Method 2: Script Console Commands
• Method 3: Manual Build Directory Removal
• Method 4: Process-Level Termination
• Prevention and Best Practices
• Troubleshooting Tips

Understanding the Stuck Aborting State

Jenkins builds can get stuck in the “aborting” state for several reasons. When you click the “Abort” button on a build, Jenkins sends a signal to stop the running processes, but sometimes the processes don’t respond properly or become “zombie” processes that continue running despite the abort signal.

Common causes of stuck aborting builds:

• Deadlocked processes in the build script
• Hanging external applications or services
• Insufficient permissions to terminate processes

• Agent connectivity issues
• Resource exhaustion on the agent machine

According to Stack Overflow discussions, this is particularly common with pipeline jobs running on Ubuntu agents where the console log shows “Aborting build” indefinitely without actually stopping the execution.

Method 1: UI-Based Forcible Termination

Start with the simplest method using the Jenkins web interface before moving to more drastic measures.

Step-by-Step UI Termination:

1. Initial Abort Attempt: Click the red “X” (Abort) button on the build page
2. Wait for 30 seconds: After the initial abort attempt, Jenkins should provide additional options
3. Look for “forcibly terminate” link: A link should appear in the console output saying “Click here to forcibly terminate running steps”
4. Click the first link: This attempts to stop the running steps more aggressively
5. Check for second link: If the first termination doesn’t work, another link “Click here to forcibly kill entire build” should appear
6. Click the second link: This performs a hard kill of the entire build

Important: As noted in the CloudBees documentation, if you see these links and click them but the build still doesn’t stop, you’ll need to proceed to more aggressive methods.

Method 2: Script Console Commands

If the UI methods fail, use the Jenkins Script Console to force-terminate the build programmatically.

Accessing the Script Console:
Navigate to http://your-jenkins-url/script to access the Groovy script console.

Basic Termination Script:

def jobName = "your-job-name"
def buildNumber = 123 // Replace with your actual build number

def job = Jenkins.instance.getItemByFullName(jobName)
def build = job.getBuildByNumber(buildNumber)

// First attempt graceful stop
build.doStop()

// If that doesn't work, force kill
build.doKill()

Advanced Script for Multiple Stuck Builds:

// Kill all currently running builds
Jenkins.instance.getAllItems(Job.class).findAll { it.isBuilding() }.each { job ->
    def jobName = job.toString()
    def val = jobName.split(/\[|\]/)
    if(val.size() > 1) {
        def actualJobName = val[1].trim()
        def job = Jenkins.instance.getItemByFullName(actualJobName)
        job.builds.each { build ->
            if(build.isBuilding()) {
                println "Stopping build ${build.fullDisplayName}"
                build.doStop()
                build.doKill()
            }
        }
    }
}

For Multibranch Pipelines:

def multibranchPipelineProjectName = "your-multibranch-project-name"
Jenkins.instance.getItemByFullName(multibranchPipelineProjectName).getItems().each { repository ->
    repository.getItems().each { branch ->
        branch.builds.each { build ->
            if (build.getResult().equals(null)) {
                println "Killing stuck build: ${build.fullDisplayName}"
                build.doStop()
                build.doKill()
            }
        }
    }
}

Note: As explained in the Mirantis documentation, the build.doStop() method attempts a graceful termination while build.doKill() forces an immediate termination.

Method 3: Manual Build Directory Removal

When script console methods fail, manually removing the build directory can resolve the issue.

Steps for Manual Directory Removal:

1. Find the build directory location:

   • Default location: $JENKINS_HOME/jobs/[job-name]/builds/[build-number]
   • For pipeline jobs: $JENKINS_HOME/jobs/[job-name]/builds/[build-number]/workspace

2. Stop Jenkins service (optional but recommended):

   bash

   sudo systemctl stop jenkins
   

3. Move or remove the problematic build directory:

   bash

   # Move the directory to prevent resumption on restart
   sudo mv $JENKINS_HOME/jobs/[job-name]/builds/[build-number] /tmp/
   
   # Or delete it completely
   sudo rm -rf $JENKINS_HOME/jobs/[job-name]/builds/[build-number]
   

4. Restart Jenkins service:

   bash

   sudo systemctl start jenkins
   

Warning: As stated in the CloudBees documentation, this method can be used to prevent any Pipeline build from resuming when Jenkins is starting up, but you should be careful not to remove directories of builds that are still running properly.

Method 4: Process-Level Termination

If the Jenkins methods don’t work, you may need to terminate the underlying processes directly on the agent machine.

Finding and Killing Hanging Processes:

1. Connect to the Ubuntu agent machine via SSH

2. Find the hanging process:

   bash

   # Find Jenkins-related processes
   ps aux | grep jenkins
   
   # Find processes by build number or job name
   ps aux | grep "your-build-number"
   ps aux | grep "your-job-name"
   
   # Check for zombie processes
   ps aux | grep Z
   

3. Kill the process group (more effective than single process kill):

   bash

   # Find the process ID (PID)
   PID=$(pgrep -f "your-build-identifier")
   
   # Kill the entire process group
   kill -TERM -$PID
   
   # If that doesn't work, force kill
   kill -KILL -$PID
   

4. For stubborn hanging applications:

   bash

   # Kill all processes related to the hanging build
   pkill -f "build-identifier"
   
   # Use killall for broader matching
   killall -9 "hanging-application-name"
   

Advanced Process Management:
According to the GitHub gist, you can create more robust process cleanup scripts:

#!/bin/bash
set -euf -o pipefail

echo "Process group id is $$"

OK=0
trap 'if [[ "$OK" != "1" ]]; then echo "---"; echo "TRAP terminating, kill all processes with parent $$"; trap - SIGTERM && pkill -P $$; fi' SIGINT SIGTERM EXIT

SECONDS=0
echo "---"
"$@" &
PID=$!

set +e
wait $PID
CODE=$?
set -e

OK=1
echo "---"
echo "Process exited with code $CODE, took $SECONDS seconds"
exit $CODE

Prevention and Best Practices

To avoid builds getting stuck in the aborting state, implement these preventive measures:

1. Implement Proper Cleanup in Scripts:

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                script {
                    // Ensure proper cleanup even on abort
                    try {
                        // Your build steps here
                    } catch (Exception e) {
                        // Cleanup code
                        sh 'rm -rf /tmp/build-artifacts'
                        throw e
                    }
                }
            }
        }
    }
}

2. Use Timeout Mechanisms:

timeout(time: 30, unit: 'MINUTES') {
    // Your build steps that might hang
}

3. Implement Process Monitoring:

• Regularly check thread dumps: http://yourserver/jenkins/threadDump
• Monitor agent health and resource usage
• Set up alerts for stuck builds

4. Configure Proper Signal Handling:
As noted in the Jenkins JIRA issue, ensure your scripts handle SIGTERM properly so they can clean up resources when Jenkins attempts to abort them.

5. Regular Maintenance:

• Periodically restart Jenkins agents
• Monitor and clean up temporary build artifacts
• Keep Jenkins and plugins updated

Troubleshooting Tips

When facing stuck builds, follow this systematic approach:

Diagnostic Steps:

1. Check Jenkins thread dump: Navigate to http://your-jenkins-url/threadDump and look for executor threads
2. Monitor agent resources: Use top, htop, or htop to check CPU, memory, and disk usage
3. Check network connectivity: Verify agent-to-master communication
4. Review logs: Check both Jenkins master and agent logs for error messages

Common Error Patterns:

• Disk space issues: Builds can hang when there’s insufficient disk space for temporary files
• Network timeouts: Builds may appear stuck due to connectivity issues
• Resource exhaustion: High CPU or memory usage can cause processes to hang
• Permission issues: Insufficient permissions to clean up build artifacts

Recovery Sequence:

1. Try UI-based termination first
2. If that fails, use script console commands
3. If still stuck, try manual process termination on the agent
4. As a last resort, remove build directories and restart services
5. After recovery, analyze the root cause to prevent recurrence

Expert Tip: According to experienced Jenkins administrators, the combination of build.doStop() followed by build.doKill() in the script console is the most reliable method for terminating stuck builds, as it attempts both graceful and forceful termination in sequence.

Sources

1. Jenkins Official Documentation - Aborting a build
2. Stack Overflow - How to stop an unstoppable zombie job on Jenkins
3. Stack Overflow - Jenkins build stuck in “aborting” state
4. CloudBees Documentation - Stuck Pipeline troubleshooting
5. Mirantis Documentation - Abort a hung build in Jenkins
6. Goldfish Tips - How do I kill a stuck Jenkins build?
7. GitHub Gist - Stop & Kill Jenkins stuck build
8. Jenkins JIRA - graceful job termination issue

Conclusion

Force-terminating a Jenkins build stuck in the “aborting” state requires a systematic approach starting with UI-based methods and progressing to more aggressive techniques when necessary. Always begin with the “forcibly terminate running steps” link in the web interface, then proceed to script console commands using build.doStop() and build.doKill(). For persistent issues, manually removing build directories or terminating processes at the OS level may be required. To prevent future occurrences, implement proper timeout mechanisms, cleanup procedures in your build scripts, and regular system monitoring. Remember that each method carries risks, so always backup important data before performing destructive operations and document your troubleshooting process for future reference.