iFixFlakies

This plugin automatically generates patches for order-dependent tests by finding and utilizing helper test methods in the test suite. This plugin builds upon code from iDFlakies (https://github.com/idflakies/iDFlakies), and relies on detected order-dependent tests that iDFlakies finds.

More details about iFixFlakies can be found in its paper and website.

Quickstart

First, follow the instructions for iDFlakies to detect order-dependent tests and obtain passing and failing orders for them. Specifically, you will need the .dtfixingtools directory generated by iDFlakies.

Next, build the iFixFlakies plugin within this repository (i.e., running mvn install inside this project's directory). After building the iFixFlakies plugin, you will then need to add it to the Maven project you ran iDFlakies on. (If you had modified the pom.xml to run iDFlakies, you will notice the two modifications are quite similar. You will need to change the existing modification for iDFlakies to be like this modification for iFixFlakies.)

Automatically setting up the pom.xml for iFixFlakies

Run the following command to automatically setup the pom.xml for iFixFlakies. Note that if you made changes to the pom.xml for iDFlakies you will first need to remove those changes before you run the following command.

bash pom-modify/modify-project.sh path_to_maven_project

Manually setting up the pom.xml for iFixFlakies

Copy the following plugin into the Maven project's pom.xml. You do not need to perform this step if you have already completed the instructions in Automatically setting up the pom.xml for iFixFlakies.

<build>
    ...
    <plugins>
        ...
        <plugin>
            <groupId>edu.illinois.cs</groupId>
            <artifactId>testrunner-maven-plugin</artifactId>
            <version>1.0</version>
            <dependencies>
                <dependency>
                    <groupId>edu.illinois.cs</groupId>
                    <artifactId>ifixflakies</artifactId>
                    <version>1.0.0-SNAPSHOT</version>
                </dependency>
            </dependencies>
            <configuration>
                <className>edu.illinois.cs.dt.tools.fixer.CleanerFixerPlugin</className>
            </configuration>
        </plugin>
    </plugins>
</build>

Running iFixFlakies

After the pom.xml is setup for the iFixFlakies plugin, run the following command on the Maven project:

mvn testrunner:testplugin

Cite

If you use iFixFlakies, please cite our corresponding ESEC/FSE paper:

@inproceedings{ShiETAL2019FSE,
    author    = {Shi, August and Lam, Wing and Oei, Reed and Xie, Tao and Marinov, Darko},
    title     = {{iFixFlakies}: {A} Framework for Automatically Fixing Order-Dependent Flaky Tests},
    booktitle = {ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
    pages     = {545--555},
    year      = {2019},
}