labs-template

C program (main.c) for automated testing of other C programs without the need of actual unit tests or any additional libraries. The program relies solely on four .txt files (one automatically generated by the program) that define the test's input, output and expected output. The program that is up for testing should be written inside the my_main() function of main.c. main.c writes a summary of the test results onto the console (screenshots/).

You may be wondering: Why? Why would you make your life more complicated when you can just write some unit tests with some C library. Well, this may be intuitive thinking for you and me, programmers with some experience under their belt, but for beginners that just started learning programming with C, not so much. The concept of testing is fairly new to the beginners, let alone unit testing, libraries (and their subsequent installations), running the tests, interpreting the results, etc.

This program is meant to shield beginner programmers from the details about testing, similar to how web applications for solving code problems hide the testing details from their users. This program is flexible enough that it let's you test any code problem in C by just following the steps defined in the Usage section.

TIP for instructors: you can combine the main.c file with a Code Blocks project file (.cbp) to enable starting this program in the Code Blocks IDE, which is usually the most used IDE for beginner programming in C. Why do this? - To make it even easier for the beginner programmers.

Usage

1. Download the main.c file
2. Create the 3 files: input.txt, input_test.txt and expected.txt
3. Write the program up for testing inside my_main()
4. Compile and run main.c
5. Inspect the summary

main.c

The main program which does the actual testing. It contains the program up for testing (my_main()), the logic for testing (main()), it reads/writes to files, writes the summary to the console, ...

input.txt

File that contains the tests input. The first line is the number of tests, then subsequently each line represents a combination of values for an input of a certain test. main.c actually redirects the stdin (keyboard) to this file before executing my_main(). This characteristic allows for the programmer to use the standard scanf(...) function in my_main() and creates the illusion that it reads the input values from the keyboard, when in reality it reads from this file. Again, this shields the programmer of my_main() from certain details.

Example input.txt file:

3
1 2 3
4 5 6
7 8 9

The example file tells main.c that it has three tests to run, i.e. it needs to call my_main() three times. The first call of scanf("%d", &var) in my_main() will read the value '1' into 'var', the second will read '2', etc.

NOTE: the new lines (\n) in this file are not needed, but are recommended for improved readability.

output.txt

File that is generated by main.c which contains the output generated by the test's executions (output from my_main()). main.c actually redirects the stdout (console) to this file before executing my_main(). This characteristic allows for the programmer to use the standard printf(...) function in my_main() and creates the illusion that it writes its output to the console, when in reality it writes to this file. Again, this shields the programmer of my_main() from certain details. We need to distinguish the outputs from different test cases, so we wrap each output from each execution of 'my_main()' between "STARTTEST\n" and "ENDTEST\n".

Example output.txt file:

STARTTEST
Result: 6
ENDTEST
STARTTEST
Result: 15
ENDTEST
STARTTEST
Result: 27
ENDTEST

From the example file we can conclude that there were 3 tests, i.e. 3 executions of my_main(). The output of the first execution is "Result: 6", from the second: "Result: 15", etc.

expected.txt

File that contains the expected output of each test in the program. This file has the same format as output.txt. main.c concludes that a test case has passed if the lines in output.txt match the lines in expected.txt for that test case.

Example expected.txt file:

STARTTEST
Result: 6
ENDTEST
STARTTEST
Result: 15
ENDTEST
STARTTEST
Result: 24
ENDTEST

If we take the example output.txt and compare it to this example expected.txt, we can conclude that test cases no. 1 & 2 have passed, but not test case no. 3

NOTE: The new lines (\n) are mandatory both in output.txt and in expected.txt because main.c reads line by line from these files.

NOTE: Also the order of the tests both in output.txt and expected.txt is really important, so we have to make sure that they follow the same order.

input_test.txt

File that contains the tests input, but in the format of the output.txt/expected.txt files. This file is used for displaying the input for the tests to the summary. Why not use input.txt you say? - Well we cannot rely on that file because we cannot distinguish between the input from one test case and another. But isn't the input of each test case in one line in input.txt you say? - Not necessarily. The input can be multiple lines (think of matrix). Also, \n is not mandatory for input.txt.

Example input_test.txt file (analogous to input.txt):

STARTTEST
1 2 3
ENDTEST
STARTTEST
4 5 6
ENDTEST
STARTTEST
7 8 9
ENDTEST

my_main()

int my_main(){
    // your code goes here
    // always put '\n' on the last executed printf()

    printf("Hello world!\n");

    return 0;
}

// don't change anything below this line
...

The function that represents the code that needs to be tested. This function simulates the main() function in regular programs.

WARNING: There is an unresolved bug that makes the program behave in a unpredictable way if there isn't a \n on the last executed printf(...) in my_main().

NOTE: All .txt must be in the same directory as the main.c program

Summary screenshots

• Passed test
• Failed test
• Not all tests passed
• All tests passed

Auto generating .txt files

You thought it was over huh? We programmers cannot rest until everything that can be automated is automated. So I present to you the generate/ folder. In this folder you can find a C program (generate.c) that will generate the expected.txt and input_test.txt files from the input.txt file.

Contributors

• Dejan Slamkov (LinkedIn, GitHub)