Difference between revisions of "Compiling M5"

From gem5
Jump to: navigation, search
m (Added "Compiling with runtime checks")
 
(59 intermediate revisions by 7 users not shown)
Line 1: Line 1:
M5 runs on Linux and OpenBSD, but should be easily portable to other Unix-like OSes. Cross-endian support has been mostly added, however this has not been extensively tested. All the syscall emulation (_SE) targets support running regardless of host/target endianess, however at present ALPHA_FS does not fully support big endian machines.  
+
gem5 runs on Linux and Mac OS X, but should be easily portable to other Unix-like OSes. At times in the past gem5 has worked on OpenBSD and Microsoft Windows (under Cygwin), but these platforms are not regularly tested. Cygwin in particular is no longer actively supported; if you must run on a Windows host, we recommend installing Linux (e.g., Ubuntu Server) under a VM and running gem5 there.  Free virtualization solutions such as VirtualBox and VMware Player work well for this usage.
  
 +
Cross-endian support has been mostly added, however this has not been extensively tested. Running a program with syscall emulation is supported regardless of host/target endianess, however full-system simulation may have cross-endian issues (ALPHA full-system is known not to work on big endian machines).
  
=== Required Software ===
+
To build gem5, you need to ensure you have all the [[dependencies]] installed.
To build M5, you will need the following software:
 
  
* [http://http://gcc.gnu.org/ g++] version 3.0 or newer. Once upon a time, M5 built with g++ 2.95, but it probably doesn't anymore.
+
== Possible Targets ==
* [http://www.pythohn.org Python], version 2.4 or newer.
+
gem5 can build many binaries each for a different guest architecture. The currently available architectures are '''ALPHA''', '''ARM''', '''MIPS''', '''POWER''', '''SPARC''', and '''X86'''. In addition there is a '''NULL''' architecture.
* [http://www.scons.org SCons], version 0.96.91 or newer.
 
* [http://www.swig.com SWIG], version 1.3.28 or newer.
 
  
SCons is a powerful replacement for make. See here[http://www.scons.org], to download SCons, and here for installation instructions. Note that you can install SCons in your home directory using the '--prefix=' option (see here for details) or use SCons Local if you do not have administrator privileges on the machine you are trying to compile on.
+
For each possible architecture and mode, several different executables can be built:
 +
*'''gem5.debug''' - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly.
 +
*'''gem5.opt'''  - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target.
 +
*'''gem5.prof''' - This binary is like the opt target, however it also includes profiling support suitable for use with gprof.
 +
*'''gem5.perf''' - Similar to prof, this target is aimed for CPU and heap profiling using the google perftools.
 +
*'''gem5.fast''' - This binary is the fastest binary and all debugging support is removed from the binary (including trace support). By default it also uses Link Time Optimization
  
The regression tester and a few utility scripts are written in Perl, but Perl is not strictly necessary to build the simulator.
+
== Compiling ==
  
 +
Starting in the root of the source tree, you can build gem5 using a command of the form:
  
=== Possible Targets ===
+
% scons build/<arch>/gem5.<binary>
M5 can build many binaries each for a different guest architecture, simulation mode, and use. The currently available architectures are: '''ALPHA''', '''SPARC''', and '''MIPS''', which are self explanatory.  
 
  
Each of these can built in two simulation modes:
+
where the items between &lt;&gt; are the architectures, modes and binaries listed aboveFor example:
*'''FS''' - Full System mode. In this mode a complete system is simulated including a kernel, I/O devices, etc. It currently only works with the '''ALPHA''' architecture.
 
*'''SE''' - Syscall Emulation mode. This mode simulates statically compiled binaries by functionally emulating any syscall they make. .
 
 
 
Possible Binaries
 
*'''m5.debug''' - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly. (-g3 -gdwarf-2 -O0).
 
*'''m5.opt'''  - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target. (-g -O3)
 
*'''m5.prof''' - This binary is like the opt target, however it also includes profiling support suitable for use with gprof. (-O3 -g -pg).
 
*'''m5.fast''' - This binary is the fastest binary and all debugging support is removed from the binary (trace flags for example).  (-O3 -DNDEBUG)
 
 
 
=== Source Tree ===
 
 
 
Once you have obtained and unpacked the M5 sources, the root of your of your M5 source tree should have three directories:
 
 
 
*'''src''' - This directory contains the sources for the simulator itself.
 
*'''ext''' - This directory contains external packages used by M5 that are not commonly installed on systems.
 
*'''configs''' - This directory contains sample script files that can be used to run M5 and as a starting place for your own configurations.
 
*'''docs''' - This directory has the doxygen output in it and any other interesting bits documentation we have
 
*'''util''' - This directory has random programs that are used in conjunction with M5 such as Python scripts to process data in an DB, submit jobs to a batch processing system, etc.
 
*'''system''' - This directory contains platform dependent code such as BIOS, hypervisor, that someone using M5 may want to modify.
 
 
 
 
 
There are multiple architectures, modes and binaries that M5 can build.
 
*Architectures
 
*; ALPHA : Alpha ISA
 
*; SPARC : SPARC ISA
 
*; MIPS : MIPS ISA
 
*Types
 
*; FS : A binary for full system simulation. This currently only works with the ALPHA target.
 
*; SE : A binary for syscall emulation.
 
*Target
 
*; debug : A binary with no optimization and all debugging features (g++ -ggdb -g3 -O0 -DDEBUG)
 
*; opt : A binary with optimization and all debugging features  (g++ -ggdb -g3 -O5 -DDEBUG)
 
*; fast : A binary with optimization and no debugging features (g++ -ggdb -g3 -O5 -DNO_DEBUG)
 
*; prof : A binary with optimization that generates profile output suitable to use with gprof
 
Starting in the root of this tree, you can build M5 and test your build using the following commands.
 
  
 
<pre>
 
<pre>
% cd m5
+
% cd gem5
% scons build/<arch>_<type>/m5.<target>
+
% scons build/ALPHA/gem5.debug
  
 
scons: Reading SConscript files ...
 
scons: Reading SConscript files ...
 
Checking for C header file fenv.h... yes
 
Checking for C header file fenv.h... yes
Building in /tmp/newmem/build/ALPHA_FS
+
Building in /tmp/gem5/build/ALPHA
Options file /tmp/newmem/build/options/ALPHA_FS not found,
+
Options file /tmp/gem5/build/options/ALPHA not found,
   using defaults in build_opts/ALPHA_FS
+
   using defaults in build_opts/ALPHA
Compiling in ALPHA_FS with MySQL support.
+
Compiling in ALPHA with MySQL support.
 
scons: done reading SConscript files.
 
scons: done reading SConscript files.
 
scons: Building targets ...
 
scons: Building targets ...
g++ -o build/ALPHA_FS/base/circlebuf.do -c -pipe -fno-strict-aliasing
+
g++ -o build/ALPHA/base/circlebuf.do -c -pipe -fno-strict-aliasing
 
     -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
 
     -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
 
     -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
 
     -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
     -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA_FS
+
     -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA
     build/ALPHA_FS/base/circlebuf.cc
+
     build/ALPHA/base/circlebuf.cc
 
...
 
...
  
 
</pre>
 
</pre>
  
If your output looked like the above, then congratulations, you've complied M5!  
+
If your output looked like the above, then congratulations, you've compiled gem5! The final binary is located at the path you specified in the argument to scons, e.g., build/ALPHA/gem5.debug.  For more build options and further details about the build system, see the [[SCons build system]] page.
 +
 
 +
== Installing full system files ==
 +
 
 +
If you want to run the full-system version (including the full-system regression tests), you will also need to download the full-system files (disk images and binaries) from the [[Download]] page.
 +
 
 +
The path to these files is determined in configs/common/SysPaths.py.  There are a couple of default paths hard-coded into this script; you can place the system files at one of those paths, edit SysPaths.py to change those paths, or override the paths in that file by setting your M5_PATH environment variable. If this is not done correctly you will see an error like <code>ImportError: Can't find a path to system files.</code> when you first attempt to run the simulator in full-system mode.
 +
 
 +
Note that the default path, <code>/dist/m5/system</code>, is designed for environments where you have root (sudo) access (to create <code>/dist</code>) and want the files in a place where they can be shared by multiple users.  If both of these are true, you can follow this example to put the system files at the default location:
 +
 
 +
<pre>
 +
% sudo mkdir -p /dist/m5/system
 +
% cd /dist/m5/system
 +
% sudo tar vxfj &lt;path&gt;/m5_system_2.0b3.tar.bz2
 +
% sudo mv m5_system_2.0b3/* . ; sudo rmdir m5_system_2.0b3/
 +
% sudo chgrp -R <grp> /dist  # where <grp> is a group that contains all the m5 users
 +
</pre>
 +
 
 +
In most cases, it's simplest to put the files wherever is convenient and then set M5_PATH to point to them.
 +
 
 +
== Testing your build ==
 +
 
 +
Once you've compiled gem5, you can verify that the build worked by running regression tests. Regression tests are also run via scons.  The command to run all tests for a particular is constructed as follows:
 +
 
 +
% scons build/<target>/tests/<binary>
 +
 
 +
For example, to run the regression tests on ALPHA/gem5.opt, type:
 +
 
 +
% scons build/ALPHA/tests/opt
 +
 
 +
The regression framework is integrated into the scons build process, so the command above will (re)build ALPHA/gem5.opt if necessary before running the tests.  Also thanks to scons's dependence tracking, tests will be re-run only if the binary has been rebuilt since the last time the test was run.  If the previous test run is still valid (as far as scons can tell), only a brief pass/fail message will be printed out based on the result of that previous test, rather than the full output and statistics diff that is printed when the test is actually executed.
 +
 
 +
Regression tests are further subdivided into three categories ("quick", "medium", and "long") based on runtime.  You can run only the tests in a particular category by adding that category name to the target path, e.g.:
 +
 
 +
% scons build/ALPHA/tests/opt/quick
 +
 
 +
(Note that currently the "medium" category is empty; all of the tests are "quick" or "long".)
 +
 
 +
Specific tests can be run by appending the test name:
 +
 
 +
% scons build/ALPHA/tests/opt/quick/fs/10.linux-boot
 +
 
 +
For more details, see [[Regression Tests]].
 +
 
 +
== Compiling with runtime checks ==
  
Each configuration is built in a separate subdirectory of m5/build. For each configuration, the various flavors of binaries are built in the same directory. The default binary that you built above is ALPHA_SE/m5.debug. You can build other binaries by specifying them on the scons command line, e.g. '<code>scons ALPHA_SE/m5.opt' or 'scons ALPHA_FS/m5.debug</code>'.
+
GCC and clang both have the ability to compile in 'sanitizers' that ensure certain invariants at runtime. These invariants can check for memory leaks, invalid memory accesses, and other undefined behaviors. Currently, [https://github.com/google/sanitizers/wiki/AddressSanitizer AddressSanitizer] and [https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html UndefinedBehaviorSanitizer] are supported with the flags <code>--with-asan</code> and <code>--with-ubsan</code> respectively. When ASAN is enabled, tcmalloc must be disabled (<code>--without-tcmalloc</code>), because it confuses ASAN's shadow memory.

Latest revision as of 10:07, 24 September 2019

gem5 runs on Linux and Mac OS X, but should be easily portable to other Unix-like OSes. At times in the past gem5 has worked on OpenBSD and Microsoft Windows (under Cygwin), but these platforms are not regularly tested. Cygwin in particular is no longer actively supported; if you must run on a Windows host, we recommend installing Linux (e.g., Ubuntu Server) under a VM and running gem5 there. Free virtualization solutions such as VirtualBox and VMware Player work well for this usage.

Cross-endian support has been mostly added, however this has not been extensively tested. Running a program with syscall emulation is supported regardless of host/target endianess, however full-system simulation may have cross-endian issues (ALPHA full-system is known not to work on big endian machines).

To build gem5, you need to ensure you have all the dependencies installed.

Possible Targets

gem5 can build many binaries each for a different guest architecture. The currently available architectures are ALPHA, ARM, MIPS, POWER, SPARC, and X86. In addition there is a NULL architecture.

For each possible architecture and mode, several different executables can be built:

  • gem5.debug - A binary used for debugging without any optimizations. Since no optimizations are done this binary is compiled the fastest, however since no optimizations are done it executes very slowly.
  • gem5.opt - A binary with debugging and optimization. This binary executes much faster than the debug binary and still provides all the debugging facility of the debug version. However when debugging source code it can be more difficult to use that the debug target.
  • gem5.prof - This binary is like the opt target, however it also includes profiling support suitable for use with gprof.
  • gem5.perf - Similar to prof, this target is aimed for CPU and heap profiling using the google perftools.
  • gem5.fast - This binary is the fastest binary and all debugging support is removed from the binary (including trace support). By default it also uses Link Time Optimization

Compiling

Starting in the root of the source tree, you can build gem5 using a command of the form:

% scons build/<arch>/gem5.<binary>

where the items between <> are the architectures, modes and binaries listed above. For example:

% cd gem5
% scons build/ALPHA/gem5.debug

scons: Reading SConscript files ...
Checking for C header file fenv.h... yes
Building in /tmp/gem5/build/ALPHA
Options file /tmp/gem5/build/options/ALPHA not found,
  using defaults in build_opts/ALPHA
Compiling in ALPHA with MySQL support.
scons: done reading SConscript files.
scons: Building targets ...
g++ -o build/ALPHA/base/circlebuf.do -c -pipe -fno-strict-aliasing
    -Wall -Wno-sign-compare -Werror -Wundef -g3 -gdwarf-2 -O0
    -DTHE_ISA=ALPHA_ISA -DDEBUG -Iext/dnet -I/usr/include/python2.4
    -Ibuild/libelf/include -I/usr/include/mysql -Ibuild/ALPHA
    build/ALPHA/base/circlebuf.cc
...

If your output looked like the above, then congratulations, you've compiled gem5! The final binary is located at the path you specified in the argument to scons, e.g., build/ALPHA/gem5.debug. For more build options and further details about the build system, see the SCons build system page.

Installing full system files

If you want to run the full-system version (including the full-system regression tests), you will also need to download the full-system files (disk images and binaries) from the Download page.

The path to these files is determined in configs/common/SysPaths.py. There are a couple of default paths hard-coded into this script; you can place the system files at one of those paths, edit SysPaths.py to change those paths, or override the paths in that file by setting your M5_PATH environment variable. If this is not done correctly you will see an error like ImportError: Can't find a path to system files. when you first attempt to run the simulator in full-system mode.

Note that the default path, /dist/m5/system, is designed for environments where you have root (sudo) access (to create /dist) and want the files in a place where they can be shared by multiple users. If both of these are true, you can follow this example to put the system files at the default location:

% sudo mkdir -p /dist/m5/system
% cd /dist/m5/system
% sudo tar vxfj <path>/m5_system_2.0b3.tar.bz2
% sudo mv m5_system_2.0b3/* . ; sudo rmdir m5_system_2.0b3/
% sudo chgrp -R <grp> /dist  # where <grp> is a group that contains all the m5 users

In most cases, it's simplest to put the files wherever is convenient and then set M5_PATH to point to them.

Testing your build

Once you've compiled gem5, you can verify that the build worked by running regression tests. Regression tests are also run via scons. The command to run all tests for a particular is constructed as follows:

% scons build/<target>/tests/<binary>

For example, to run the regression tests on ALPHA/gem5.opt, type:

% scons build/ALPHA/tests/opt

The regression framework is integrated into the scons build process, so the command above will (re)build ALPHA/gem5.opt if necessary before running the tests. Also thanks to scons's dependence tracking, tests will be re-run only if the binary has been rebuilt since the last time the test was run. If the previous test run is still valid (as far as scons can tell), only a brief pass/fail message will be printed out based on the result of that previous test, rather than the full output and statistics diff that is printed when the test is actually executed.

Regression tests are further subdivided into three categories ("quick", "medium", and "long") based on runtime. You can run only the tests in a particular category by adding that category name to the target path, e.g.:

% scons build/ALPHA/tests/opt/quick

(Note that currently the "medium" category is empty; all of the tests are "quick" or "long".)

Specific tests can be run by appending the test name:

% scons build/ALPHA/tests/opt/quick/fs/10.linux-boot

For more details, see Regression Tests.

Compiling with runtime checks

GCC and clang both have the ability to compile in 'sanitizers' that ensure certain invariants at runtime. These invariants can check for memory leaks, invalid memory accesses, and other undefined behaviors. Currently, AddressSanitizer and UndefinedBehaviorSanitizer are supported with the flags --with-asan and --with-ubsan respectively. When ASAN is enabled, tcmalloc must be disabled (--without-tcmalloc), because it confuses ASAN's shadow memory.