Skip to content

Fix mutation bugs #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
varuniy merged 4 commits into from
May 19, 2026
Merged

Fix mutation bugs #103

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d310ec3
Only run mutation analysis if needed (specify through -s flag), and t…
May 19, 2026
3ecbe25
Reformat python file
May 19, 2026
662321c
Continued to clean up trim_mutants.py
May 19, 2026
0a0124c
Fixed buggy build files
May 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 35 additions & 20 deletions scripts/mutation-evosuite.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,15 @@
#------------------------------------------------------------------------------
# Options (command-line arguments):
#------------------------------------------------------------------------------
USAGE_STRING="usage: mutation-evosuite.sh [-o RESULTS_CSV] [-t total_time] [-c time_per_class] [-n num_iterations] [-r] [-v] [-h] TEST-CASE-NAME
USAGE_STRING="usage: mutation-evosuite.sh [-o RESULTS_CSV] [-t total_time] [-c time_per_class] [-n num_iterations] [-s] [-r] [-v] [-h] TEST-CASE-NAME
-o N Write experiment results to this CSV file (N should end in '.csv').
If the file does not exist, a header row will be created automatically.
Paths are not allowed; only a filename may be given.
-t N Total time limit for test generation (in seconds).
-c N Per-class time limit (in seconds, default: 2s/class).
Mutually exclusive with -t.
-n N Number of iterations to run the experiment (default: 1).
-s Skip mutation analysis (only run test generation and coverage).
-r Redirect logs and diagnostics to results/result/mutation_output.txt.
-v Enables verbose mode.
-h Displays this help message.
Expand Down Expand Up @@ -83,10 +84,11 @@ fi
NUM_LOOP=1 # Number of experiment runs (10 in GRT paper)
VERBOSE=0 # Verbose option
REDIRECT=0 # Redirect output to mutation_output.txt
SKIP_MUTATION=0 # Skip mutation analysis
UUID=$(uuidgen) # Generate a unique identifier per instance

# Parse command-line arguments
while getopts ":hvro:t:c:n:" opt; do
while getopts ":hvrso:t:c:n:" opt; do
case ${opt} in
h)
# Display help message
Expand All @@ -101,6 +103,10 @@ while getopts ":hvro:t:c:n:" opt; do
# Redirect output to a log file
REDIRECT=1
;;
s)
# Skip mutation analysis
SKIP_MUTATION=1
;;
o)
RESULTS_CSV="$OPTARG"
;;
Expand Down Expand Up @@ -490,6 +496,13 @@ for i in $(seq 1 "$NUM_LOOP"); do
"$MAJOR_HOME"/bin/ant -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dmutator="mml:$MAJOR_HOME/mml/all.mml.bin" -Dsrc="$JAVA_SRC_DIR" -Dtargetdir="$COVERAGE_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" compile.mutation
"$MAJOR_HOME"/bin/ant -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dmutator="mml:$MAJOR_HOME/mml/all.mml.bin" -Dsrc="$JAVA_SRC_DIR" -Dtargetdir="$COVERAGE_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" compile.jacoco

PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
if [ -z "$PYTHON_EXECUTABLE" ]; then
echo "Error: Python is not installed." >&2
exit 2
fi
"$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/trim_mutants.py "$RESULT_DIR/mutants.log"

Comment on lines +499 to +505
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

-s mode still requires Python, so skip path can fail before coverage runs.

trim_mutants.py (and runner conversion) are mutation-only steps, but they run before the skip gate. That makes -s fail on machines without Python.

🐛 Proposed fix 
-  PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
-  if [ -z "$PYTHON_EXECUTABLE" ]; then
-    echo "Error: Python is not installed." >&2
-    exit 2
-  fi
-  "$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/trim_mutants.py "$RESULT_DIR/mutants.log"
+  if [[ "$SKIP_MUTATION" -eq 0 ]]; then
+    PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
+    if [ -z "$PYTHON_EXECUTABLE" ]; then
+      echo "Error: Python is required for mutation trimming/runner conversion." >&2
+      exit 2
+    fi
+    "$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/trim_mutants.py "$RESULT_DIR/mutants.log"
+  fi
@@
-  if [ "$SUBJECT_PROGRAM" == "jdom-1.0" ]; then
+  if [[ "$SKIP_MUTATION" -eq 0 && "$SUBJECT_PROGRAM" == "jdom-1.0" ]]; then
     "$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/convert_test_runners.py "$TEST_DIRECTORY" --mode evosuite-to-randoop
   fi

Also applies to: 545-547

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/mutation-evosuite.sh` around lines 499 - 505, The script currently
always runs trim_mutants.py using $PYTHON_EXECUTABLE (and similarly at lines
545-547), which makes the -s (skip mutation) mode fail on machines without
Python; guard these mutation-only steps so they only run when not in skip mode:
detect the existing skip/skip-mutation flag used by the script (the option that
implements -s) and wrap the "$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/trim_mutants.py
"$RESULT_DIR/mutants.log" calls in an if block that runs them only when skip
mode is false, and otherwise skip them (and do not try to locate Python). Ensure
you update both occurrences.

echo
echo "Compiling tests..."
if [[ "$VERBOSE" -eq 1 ]]; then
Expand Down Expand Up @@ -530,28 +543,30 @@ for i in $(seq 1 "$NUM_LOOP"); do
# resulting in a lot of methods being excluded from mutation analysis.
# We use a Python script to convert the tests.
if [ "$SUBJECT_PROGRAM" == "jdom-1.0" ]; then
PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
if [ -z "$PYTHON_EXECUTABLE" ]; then
echo "Error: Python is not installed." >&2
exit 2
fi
"$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/convert_test_runners.py "$TEST_DIRECTORY" --mode evosuite-to-randoop
fi

echo
echo "Running tests with mutation analysis..."
if [[ "$VERBOSE" -eq 1 ]]; then
echo command:
echo "$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test
# Run mutation analysis unless -s flag is set
if [[ "$SKIP_MUTATION" -eq 0 ]]; then
echo
echo "Running tests with mutation analysis..."
if [[ "$VERBOSE" -eq 1 ]]; then
echo command:
echo "$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test
fi
echo
"$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test

# Calculate Mutation Score
mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")
Comment on lines +561 to +564
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Handle zero generated mutants before computing mutation score.

If mutants_generated is 0, bc exits on division by zero and aborts the run due to set -e.

🐛 Proposed fix 
     mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
     mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
-    mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
-    mutation_score=$(printf "%.2f" "$mutation_score")
+    if [[ "$mutants_generated" =~ ^[0-9]+$ ]] && [[ "$mutants_generated" -gt 0 ]]; then
+      mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
+      mutation_score=$(printf "%.2f" "$mutation_score")
+    else
+      mutation_score="0.00"
+    fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")
mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
if [[ "$mutants_generated" =~ ^[0-9]+$ ]] && [[ "$mutants_generated" -gt 0 ]]; then
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")
else
mutation_score="0.00"
fi
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/mutation-evosuite.sh` around lines 561 - 564, The script computes
mutation_score using mutants_generated and mutants_killed which will cause a
division-by-zero crash when mutants_generated is 0; update the logic around
mutants_generated/mutants_killed in scripts/mutation-evosuite.sh to check if
mutants_generated is zero before calling bc—if zero, set mutation_score to a
safe default (e.g., "0.00" or "N/A") and skip the bc division, otherwise perform
the existing bc calculation and printf formatting; reference the variables
mutants_generated, mutants_killed and mutation_score when implementing the
conditional guard.

else
echo
echo "Skipping mutation analysis (use -s flag)."
mutation_score="N/A"
fi
echo
"$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test

# Calculate Mutation Score
mutants_generated=$(awk -F, 'NR==2 {print $1}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")

echo "Instruction Coverage: $instruction_coverage%"
echo "Branch Coverage: $branch_coverage%"
Expand Down
64 changes: 41 additions & 23 deletions scripts/mutation-randoop.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
#------------------------------------------------------------------------------
# Options (command-line arguments):
#------------------------------------------------------------------------------
USAGE_STRING="usage: mutation-randoop.sh [-f features] [-o RESULTS_CSV] [-t total_time] [-c time_per_class] [-n num_iterations] [-r] [-v] [-h] TEST-CASE-NAME
USAGE_STRING="usage: mutation-randoop.sh [-f features] [-o RESULTS_CSV] [-t total_time] [-c time_per_class] [-n num_iterations] [-s] [-r] [-v] [-h] TEST-CASE-NAME
-f Specify the Randoop features to use.
Available features: BASELINE, BLOODHOUND, ORIENTEERING, DETECTIVE, GRT_FUZZING, ELEPHANT_BRAIN, CONSTANT_MINING.
example usage: -f BASELINE,BLOODHOUND
Expand All @@ -34,6 +34,7 @@ USAGE_STRING="usage: mutation-randoop.sh [-f features] [-o RESULTS_CSV] [-t tota
-c N Per-class time limit (in seconds, default: 2s/class).
Mutually exclusive with -t.
-n N Number of iterations to run the experiment (default: 1).
-s Skip mutation analysis (only run test generation and coverage).
-r Redirect logs and diagnostics to results/result/mutation_output.txt.
-v Enables verbose mode.
-h Displays this help message.
Expand Down Expand Up @@ -92,10 +93,11 @@ fi
NUM_LOOP=1 # Number of experiment runs (10 in GRT paper)
VERBOSE=0 # Verbose option
REDIRECT=0 # Redirect output to mutation_output.txt
SKIP_MUTATION=0 # Skip mutation analysis
UUID=$(uuidgen) # Generate a unique identifier per instance

# Parse command-line arguments
while getopts ":hvrf:o:t:c:n:" opt; do
while getopts ":hvrsf:o:t:c:n:" opt; do
case ${opt} in
h)
# Display help message
Expand All @@ -110,6 +112,10 @@ while getopts ":hvrf:o:t:c:n:" opt; do
# Redirect output to a log file
REDIRECT=1
;;
s)
# Skip mutation analysis
SKIP_MUTATION=1
;;
f)
FEATURES_OPT="$OPTARG"
;;
Expand Down Expand Up @@ -161,10 +167,10 @@ declare -A FEATURE_FLAGS
FEATURE_FLAGS=(
["BLOODHOUND"]="--method-selection=BLOODHOUND"
["ORIENTEERING"]="--input-selection=ORIENTEERING"
["DETECTIVE"]="--demand-driven=true"
["DETECTIVE"]="--call-non-sut-methods=true"
["GRT_FUZZING"]="--grt-fuzzing=true"
["ELEPHANT_BRAIN"]="--cast-to-run-time-type=true"
["CONSTANT_MINING"]="--constant-mining=true"
["CONSTANT_MINING"]="--literal-tfidf=true --literal-tfidf-probability=1.0 --include-superclass-literals=true"
["BASELINE"]=""
)

Expand All @@ -174,7 +180,10 @@ for feat in "${RANDOOP_FEATURES[@]}"; do
if [[ "${FEATURE_FLAGS[$feat]+exists}" ]]; then
flag="${FEATURE_FLAGS[$feat]}"
if [[ -n "$flag" ]]; then
EXPANDED_FEATURE_FLAGS+=("$flag")
# Split multi-argument feature strings into separate array entries
# shellcheck disable=SC2206
split_flags=($flag)
EXPANDED_FEATURE_FLAGS+=("${split_flags[@]}")
fi
else
echo "${SCRIPT_NAME}: error: unknown feature '$feat'"
Expand Down Expand Up @@ -610,6 +619,13 @@ for i in $(seq 1 "$NUM_LOOP"); do
"$MAJOR_HOME"/bin/ant -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dmutator="mml:$MAJOR_HOME/mml/all.mml.bin" -Dsrc="$JAVA_SRC_DIR" -Dtargetdir="$COVERAGE_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" compile.mutation
"$MAJOR_HOME"/bin/ant -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dmutator="mml:$MAJOR_HOME/mml/all.mml.bin" -Dsrc="$JAVA_SRC_DIR" -Dtargetdir="$COVERAGE_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" compile.jacoco

PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
if [ -z "$PYTHON_EXECUTABLE" ]; then
echo "Error: Python is not installed." >&2
exit 2
fi
"$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/trim_mutants.py "$RESULT_DIR/mutants.log"

Comment on lines +622 to +628
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

-s mode still hard-depends on Python in the pre-mutation path.

Skip mode should allow generation + coverage without mutation tooling, but trim_mutants.py/runner conversion still execute before the skip gate.

🐛 Proposed fix 
-  PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
-  if [ -z "$PYTHON_EXECUTABLE" ]; then
-    echo "Error: Python is not installed." >&2
-    exit 2
-  fi
-  "$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/trim_mutants.py "$RESULT_DIR/mutants.log"
+  if [[ "$SKIP_MUTATION" -eq 0 ]]; then
+    PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
+    if [ -z "$PYTHON_EXECUTABLE" ]; then
+      echo "Error: Python is required for mutation trimming/runner conversion." >&2
+      exit 2
+    fi
+    "$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/trim_mutants.py "$RESULT_DIR/mutants.log"
+  fi
@@
-  if [ "$SUBJECT_PROGRAM" == "hamcrest-core-1.3" ]; then
+  if [[ "$SKIP_MUTATION" -eq 0 && "$SUBJECT_PROGRAM" == "hamcrest-core-1.3" ]]; then
     "$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/convert_test_runners.py "$TEST_DIRECTORY" --mode randoop-to-evosuite
   fi

Also applies to: 671-673

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/mutation-randoop.sh` around lines 622 - 628, The pre-mutation steps
still invoke trim_mutants.py (and the runner-conversion step) unconditionally,
causing a hard Python dependency even when skip mode (-s) is requested; wrap the
calls that use PYTHON_EXECUTABLE (the trim_mutants.py invocation and the
subsequent "runner conversion" call mentioned around the same section) in a
conditional that checks the skip-mode flag (the variable set when -s is passed)
and only runs those Python-dependent commands when skip mode is not enabled,
otherwise skip them (and optionally print a short info message); also ensure the
PYTHON_EXECUTABLE lookup remains where used so Python is not required when skip
mode is active.

echo
echo "Compiling tests..."
if [[ "$VERBOSE" -eq 1 ]]; then
Expand Down Expand Up @@ -653,28 +669,30 @@ for i in $(seq 1 "$NUM_LOOP"); do
# proper isolation and compatibility for accurate mutant coverage.

if [ "$SUBJECT_PROGRAM" == "hamcrest-core-1.3" ]; then
PYTHON_EXECUTABLE=$(command -v python3 2> /dev/null || command -v python 2> /dev/null)
if [ -z "$PYTHON_EXECUTABLE" ]; then
echo "Error: Python is not installed." >&2
exit 1
fi
"$PYTHON_EXECUTABLE" "$SCRIPT_DIR"/convert_test_runners.py "$TEST_DIRECTORY" --mode randoop-to-evosuite
fi

echo
echo "Running tests with mutation analysis..."
if [[ "$VERBOSE" -eq 1 ]]; then
echo command:
echo "$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test
# Run mutation analysis unless -s flag is set
if [[ "$SKIP_MUTATION" -eq 0 ]]; then
echo
echo "Running tests with mutation analysis..."
if [[ "$VERBOSE" -eq 1 ]]; then
echo command:
echo "$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test
fi
echo
"$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test

# Calculate Mutation Score
mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")
Comment on lines +687 to +690
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Guard mutation score calculation when generated mutants is zero.

A zero value in summary.csv causes division-by-zero and aborts the script under set -e.

🐛 Proposed fix 
     mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
     mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
-    mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
-    mutation_score=$(printf "%.2f" "$mutation_score")
+    if [[ "$mutants_generated" =~ ^[0-9]+$ ]] && [[ "$mutants_generated" -gt 0 ]]; then
+      mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
+      mutation_score=$(printf "%.2f" "$mutation_score")
+    else
+      mutation_score="0.00"
+    fi
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")
mutants_generated=$(awk -F, 'NR==2 {print $2}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
if [[ "$mutants_generated" =~ ^[0-9]+$ ]] && [[ "$mutants_generated" -gt 0 ]]; then
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")
else
mutation_score="0.00"
fi
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/mutation-randoop.sh` around lines 687 - 690, The division-by-zero
happens when mutants_generated (from summary.csv) is zero; update the logic in
scripts/mutation-randoop.sh around the variables mutants_generated,
mutants_killed and mutation_score to guard the calculation: check if
mutants_generated is empty or equals 0 before running the bc division, and if so
set mutation_score to a safe default like "0.00" (or "N/A") instead of
performing the division; only compute mutation_score using bc/printf when
mutants_generated is non-zero.

else
echo
echo "Skipping mutation analysis (use -s flag)."
mutation_score="N/A"
fi
echo
"$MAJOR_HOME"/bin/"$ANT" -f "$SCRIPT_DIR"/program-config/"$1"/${buildfile} -Dbasedir="$SCRIPT_DIR" -Dbindir="$SCRIPT_DIR/build/bin/$FILE_SUFFIX" -Dresultdir="$RESULT_DIR" -Dtest="$TEST_DIRECTORY" -Dlibdir="$SCRIPT_DIR/build/lib/$UUID" mutation.test

# Calculate Mutation Score
mutants_generated=$(awk -F, 'NR==2 {print $1}' "$RESULT_DIR"/summary.csv)
mutants_killed=$(awk -F, 'NR==2 {print $4}' "$RESULT_DIR"/summary.csv)
mutation_score=$(echo "scale=4; $mutants_killed / $mutants_generated * 100" | bc)
mutation_score=$(printf "%.2f" "$mutation_score")

echo "Instruction Coverage: $instruction_coverage%"
echo "Branch Coverage: $branch_coverage%"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/ClassViewer-5.0.5b/build-evosuite.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@
testOrder="sort_methods"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/ClassViewer-5.0.5b/build-randoop.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@
excludeFailingTests="false"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/JSAP-2.1/build-evosuite.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@
testOrder="sort_methods"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/JSAP-2.1/build-randoop.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,7 @@
excludeFailingTests="false"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/a4j-1.0b/build-evosuite.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@
testOrder="sort_methods"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/a4j-1.0b/build-randoop.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,7 @@
excludeFailingTests="false"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/asm-5.0.1/build-evosuite.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@
testOrder="sort_methods"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/asm-5.0.1/build-randoop.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@
excludeFailingTests="false"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/bcel-5.2/build-evosuite.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@
testOrder="sort_methods"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
1 change: 1 addition & 0 deletions scripts/program-config/bcel-5.2/build-randoop.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@
excludeFailingTests="false"
serializedMapsFile="${resultdir}/preprocessing.ser"
mutantsLogFile="${resultdir}/mutants.log"
excludeMutantsFile="${resultdir}/exclude_mutants.txt"
summaryFile="${resultdir}/summary.csv"
mutantDetailsFile="${resultdir}/details.csv"
covMapFile="${resultdir}/covMap.csv"
Expand Down
Loading