Regression Tests
Running Regressions
Running the M5 regression tests is the recommended way to make sure that any changes to the simulator still comply with M5. Regression Testing is performed using SCons to help guide which tests are run. All current tests are located in the: 'm5/tests/' directory.
The basic usage for running a regression is:
% scons build/<arch>_<mode>/tests/<binary_type>/<test_directory>
Here is an example of running all of the 'quick' regression tests for the ALPHA architecture in syscall-emulation (SE) mode.
% scons build/ALPHA_SE/tests/debug/quick ... **** build/ALPHA_SE/tests/debug/quick/00.hello/alpha/linux/o3-timing PASSED! ... **** build/ALPHA_SE/tests/debug/quick/00.hello/alpha/linux/simple-atomic PASSED! ... **** build/ALPHA_SE/tests/debug/quick/00.hello/alpha/linux/simple-timing PASSED! ... **** build/ALPHA_SE/tests/debug/quick/00.hello/alpha/tru64/o3-timing PASSED! ...
Creating Your Own Regressions
Adding a regression test is done by adding a new directory to the 'm5/tests' directory and filling that directory up with the appropriate reference files. Every regression test needs these files in order to compare the current changes of M5 with the 'known working state:
- simout - Standard Output Stream
- simerr - Standard Error Stream
- config.ini - M5 Configuration File
- stats.txt - Statistics From M5 Simulation
One needs to run the M5 with the appropriate binary in order to get the aforementioned simulation files. For example, a command line to collect simulation data for a hello world regression test might be:
% build/ALPHA_SE/m5.fast -re configs/example/se.py --cmd=tests/test-progs/hello/bin/alpha/hello
Creating the 'hello world' regression test directory and copying over the reference output might then follow this command line sequence:
% mkdir tests/quick/00.hello % mkdir tests/quick/00.hello/ref % cp config.ini tests/quick/00.hello/ref % cp stats.txt tests/quick/00.hello/ref % cp simout tests/quick/00.hello/ref % cp simerr tests/quick/00.hello/ref
The last thing one needs to do is add a 'test.py' configuration file that specifies the workload for the regression test. For this 'hello world' example, the 'test.py' file would just contain:
root.system.cpu.workload = LiveProcess(cmd='hello', executable = binpath('hello'))
Once the test.py is copied to the 'hello world' regression test directory, the individual regression test can be run:
% cp test.py tests/quick/00.hello % scons build/ALPHA_SE/tests/debug/quick/00.hello
Updating Regression Tests
Once you've made and verified changes to the simulator, you can update any regression tests with the "update" option:
% scons update_ref=y build/ALPHA_SE/tests/debug/quick/00.hello
