Difference between revisions of "SPEC CPU2006 benchmarks"
(Added a bit to the cross-compiler and building SPEC benchmarks instrucitons.) |
m (→Create the SPEC CPU2006 processes for M5 SE mode) |
||
(14 intermediate revisions by 4 users not shown) | |||
Line 20: | Line 20: | ||
Then follow the steps in the [http://www.kegel.com/crosstool/current/doc/crosstool-howto.html crosstool-howto] page to build the cross compiler. | Then follow the steps in the [http://www.kegel.com/crosstool/current/doc/crosstool-howto.html crosstool-howto] page to build the cross compiler. | ||
− | ===== Build the | + | ===== Build the SPEC CPU2006 alpha binaries ===== |
− | Install the | + | Install the SPEC CPU2006 from DVD and modify the CC, CXX, and FC in config/alpha.cfg. If alpha.cfg is not in the config/ directory, you can use linux32-i386-gcc42.cfg and modify the CC, CXX, and FC variables. |
<pre> | <pre> | ||
For example: | For example: | ||
− | CC = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-gcc | + | CC = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-gcc |
− | CXX = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-g++ | + | CXX = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-g++ |
FC = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-gfortran | FC = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-gfortran | ||
</pre> | </pre> | ||
Line 37: | Line 37: | ||
</pre> | </pre> | ||
− | + | ===== Create the SPEC CPU2006 processes for M5 SE mode===== | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | ===== Create the | ||
A good reference for the correct command line options can be found here: | A good reference for the correct command line options can be found here: | ||
− | [http:// | + | [http://kejo.be/ELIS/spec_cpu2006/spec_cpu2006_command_lines.html SPEC_CPU2006_Commands]. |
For your convenience, here is our benchmark python file for the M5 SE mode. | For your convenience, here is our benchmark python file for the M5 SE mode. | ||
Line 466: | Line 391: | ||
</pre> | </pre> | ||
− | ===== The | + | ===== The SPEC CPU2006 testing dataset results ===== |
− | We use the quard-core Xeon 2.5GHz with 16G memory machine. The operation system is 64bits CentOS 5.2. The timing results are from the simple cpu model and | + | We use the quard-core Xeon 2.5GHz with 16G memory machine. The operation system is 64bits CentOS 5.2. The timing results are from the simple cpu model and SPEC CPU2006 testing data set. |
{| cellspacing="0" border="1" | {| cellspacing="0" border="1" | ||
Line 730: | Line 655: | ||
===== Trouble shooting ===== | ===== Trouble shooting ===== | ||
− | You may encounter errors while executing the | + | You may encounter errors while executing the SPEC CPU2006, and these two errors are common on 32-bits machine. |
1. terminate called after throwing an instance of 'std::bad_alloc': | 1. terminate called after throwing an instance of 'std::bad_alloc': | ||
Line 742: | Line 667: | ||
<pre> | <pre> | ||
− | If you have any questions, please email to M5 Mailing-List | + | If you have any questions, please email to M5 Mailing-List. |
</pre> | </pre> |
Latest revision as of 17:09, 17 September 2016
This is a work in-progress. Everyone should feel free to extend this page with their experiences to help new users get started.
Contents
Input sets and Binaries
We can't provide the binaries or input files because of licensing restrictions, but It's not hard to build the binaries by yourself. In this short article, we will share our experiences about what we have done so far.
Build the cross-compiler for alpha machine
It is suggested that you use crosstool-ng available here, a new tool based on Dan Kegel's cross tool that is more up to date. Follow the instructions available on that page for building the cross-compiler.
If you do not wish to build this tool yourself, you may be able to use one of the pre-compiled cross-compilers available on the M5 Download Page.
Or, you can use the older crosstool. Download the crosstool-0.43.tar.gz from http://kegel.com/crosstool and modify these three lines in the demo-alpha.sh :
RESULT_TOP=where_you_want_to_put_the_compiler GCC_LANGUAGES="c,c++,fortran" eval `cat alpha.dat gcc-4.1.0-glibc-2.3.6.dat` sh all.sh --notest
Then follow the steps in the crosstool-howto page to build the cross compiler.
Build the SPEC CPU2006 alpha binaries
Install the SPEC CPU2006 from DVD and modify the CC, CXX, and FC in config/alpha.cfg. If alpha.cfg is not in the config/ directory, you can use linux32-i386-gcc42.cfg and modify the CC, CXX, and FC variables.
For example: CC = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-gcc CXX = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-g++ FC = /home/mjwu/crosstool/gcc-4.1.0-glibc-2.3.6/alpha/bin/alpha-gfortran
Then follow the instructions in the ./Docs/install-guide-unix.html to build the binaries
For example: runspec --config=alpha.cfg --action=build --tune=base bzip2
Create the SPEC CPU2006 processes for M5 SE mode
A good reference for the correct command line options can be found here: SPEC_CPU2006_Commands.
For your convenience, here is our benchmark python file for the M5 SE mode.
#Mybench.py #400.perlbench perlbench = LiveProcess() perlbench.executable = binary_dir+'400.perlbench_base.alpha-gcc' perlbench.cmd = [perlbench.executable] + ['-I./lib', 'attrs.pl'] perlbench.output = 'attrs.out' #401.bzip2 bzip2 = LiveProcess() bzip2.executable = binary_dir+'401.bzip2_base.alpha-gcc' data=data_dir+'401.bzip2/data/all/input/input.program' bzip2.cmd = [bzip2.executable] + [data, '1'] bzip2.output = 'input.program.out' #403.gcc gcc = LiveProcess() gcc.executable = binary_dir+'403.gcc_base.alpha-gcc' data=data_dir+'403.gcc/data/test/input/cccp.i' output='/import/home1/mjwu/work_spec2006/403.gcc/m5/cccp.s' gcc.cmd = [gcc.executable] + [data]+['-o',output] gcc.output = 'ccc.out' #410.bwaves bwaves = LiveProcess() bwaves.executable = binary_dir+'410.bwaves_base.alpha-gcc' bwaves.cmd = [bwaves.executable] #416.gamess gamess=LiveProcess() gamess.executable = binary_dir+'416.gamess_base.alpha-gcc' gamess.cmd = [gamess.executable] gamess.input='exam29.config' gamess.output='exam29.output' #429.mcf mcf = LiveProcess() mcf.executable = binary_dir+'429.mcf_base.alpha-gcc' data=data_dir+'429.mcf/data/test/input/inp.in' mcf.cmd = [mcf.executable] + [data] mcf.output = 'inp.out' #433.milc milc=LiveProcess() milc.executable = binary_dir+'433.milc_base.alpha-gcc' stdin=data_dir+'433.milc/data/test/input/su3imp.in' milc.cmd = [milc.executable] milc.input=stdin milc.output='su3imp.out' #434.zeusmp zeusmp=LiveProcess() zeusmp.executable = binary_dir+'434.zeusmp_base.alpha-gcc' zeusmp.cmd = [zeusmp.executable] zeusmp.output = 'zeusmp.stdout' #435.gromacs gromacs = LiveProcess() gromacs.executable = binary_dir+'435.gromacs_base.alpha-gcc' data=data_dir+'435.gromacs/data/test/input/gromacs.tpr' gromacs.cmd = [gromacs.executable] + ['-silent','-deffnm',data,'-nice','0'] #436.cactusADM cactusADM = LiveProcess() cactusADM.executable = binary_dir+'436.cactusADM_base.alpha-gcc' data=data_dir+'436.cactusADM/data/test/input/benchADM.par' cactusADM.cmd = [cactusADM.executable] + [data] cactusADM.output = 'benchADM.out' #437.leslie3d leslie3d=LiveProcess() leslie3d.executable = binary_dir+'437.leslie3d_base.alpha-gcc' stdin=data_dir+'437.leslie3d/data/test/input/leslie3d.in' leslie3d.cmd = [leslie3d.executable] leslie3d.input=stdin leslie3d.output='leslie3d.stdout' #444.namd namd = LiveProcess() namd.executable = binary_dir+'444.namd_base.alpha-gcc' input=data_dir+'444.namd/data/all/input/namd.input' namd.cmd = [namd.executable] + ['--input',input,'--iterations','1','--output','namd.out'] namd.output='namd.stdout' #445.gobmk gobmk=LiveProcess() gobmk.executable = binary_dir+'445.gobmk_base.alpha-gcc' stdin=data_dir+'445.gobmk/data/test/input/capture.tst' gobmk.cmd = [gobmk.executable]+['--quiet','--mode','gtp'] gobmk.input=stdin gobmk.output='capture.out' #447.dealII dealII=LiveProcess() dealII.executable = binary_dir+'447.dealII_base.alpha-gcc' dealII.cmd = [gobmk.executable]+['8'] dealII.output='log' #450.soplex soplex=LiveProcess() soplex.executable = binary_dir+'450.soplex_base.alpha-gcc' data=data_dir+'450.soplex/data/test/input/test.mps' soplex.cmd = [soplex.executable]+['-m10000',data] soplex.output = 'test.out' #453.povray povray=LiveProcess() povray.executable = binary_dir+'453.povray_base.alpha-gcc' data=data_dir+'453.povray/data/test/input/SPEC-benchmark-test.ini' #povray.cmd = [povray.executable]+['SPEC-benchmark-test.ini'] povray.cmd = [povray.executable]+[data] povray.output = 'SPEC-benchmark-test.stdout' #454.calculix calculix=LiveProcess() calculix.executable = binary_dir+'454.calculix_base.alpha-gcc' data='/import/RaidHome/mjwu/work_spec2006/454.calculix/m5/beampic' calculix.cmd = [calculix.executable]+['-i',data] calculix.output = 'beampic.log' #456.hmmer hmmer=LiveProcess() hmmer.executable = binary_dir+'456.hmmer_base.alpha-gcc' data=data_dir+'456.hmmer/data/test/input/bombesin.hmm' hmmer.cmd = [hmmer.executable]+['--fixed', '0', '--mean', '325', '--num', '5000', '--sd', '200', '--seed', '0', data] hmmer.output = 'bombesin.out' #458.sjeng sjeng=LiveProcess() sjeng.executable = binary_dir+'458.sjeng_base.alpha-gcc' data=data_dir+'458.sjeng/data/test/input/test.txt' sjeng.cmd = [sjeng.executable]+[data] sjeng.output = 'test.out' #459.GemsFDTD GemsFDTD=LiveProcess() GemsFDTD.executable = binary_dir+'459.GemsFDTD_base.alpha-gcc' GemsFDTD.cmd = [GemsFDTD.executable] GemsFDTD.output = 'test.log' #462.libquantum libquantum=LiveProcess() libquantum.executable = binary_dir+'462.libquantum_base.alpha-gcc' libquantum.cmd = [libquantum.executable],'33','5' libquantum.output = 'test.out' #464.h264ref h264ref=LiveProcess() h264ref.executable = binary_dir+'464.h264ref_base.alpha-gcc' data=data_dir+'464.h264ref/data/test/input/foreman_test_encoder_baseline.cfg' h264ref.cmd = [h264ref.executable]+['-d',data] h264ref.output = 'foreman_test_encoder_baseline.out' #470.lbm lbm=LiveProcess() lbm.executable = binary_dir+'470.lbm_base.alpha-gcc' data=data_dir+'470.lbm/data/test/input/100_100_130_cf_a.of' lbm.cmd = [lbm.executable]+['20', 'reference.dat', '0', '1' ,data] lbm.output = 'lbm.out' #471.omnetpp omnetpp=LiveProcess() omnetpp.executable = binary_dir+'471.omnetpp_base.alpha-gcc' data=data_dir+'471.omnetpp/data/test/input/omnetpp.ini' omnetpp.cmd = [omnetpp.executable]+[data] omnetpp.output = 'omnetpp.log' #473.astar astar=LiveProcess() astar.executable = binary_dir+'473.astar_base.alpha-gcc' astar.cmd = [astar.executable]+['lake.cfg'] astar.output = 'lake.out' #481.wrf wrf=LiveProcess() wrf.executable = binary_dir+'481.wrf_base.alpha-gcc' wrf.cmd = [wrf.executable]+['namelist.input'] wrf.output = 'rsl.out.0000' #482.sphinx sphinx3=LiveProcess() sphinx3.executable = binary_dir+'482.sphinx_livepretend_base.alpha-gcc' sphinx3.cmd = [sphinx3.executable]+['ctlfile', '.', 'args.an4'] sphinx3.output = 'an4.out' #483.xalancbmk xalancbmk=LiveProcess() xalancbmk.executable = binary_dir+'483.Xalan_base.alpha-gcc' xalancbmk.cmd = [xalancbmk.executable]+['-v','test.xml','xalanc.xsl'] xalancbmk.output = 'test.out' #998.specrand specrand_i=LiveProcess() specrand_i.executable = binary_dir+'998.specrand_base.alpha-gcc' specrand_i.cmd = [specrand_i.executable] + ['324342','24239'] specrand_i.output = 'rand.24239.out' #999.specrand specrand_f=LiveProcess() specrand_f.executable = binary_dir+'999.specrand_base.alpha-gcc' specrand_f.cmd = [specrand_i.executable] + ['324342','24239'] specrand_f.output = 'rand.24239.out'
M5 python configure script
Here is our system configuration python file for the M5 SE mode.
#cmp.py # Simple configuration script import m5 from m5.objects import * import os, optparse, sys m5.AddToPath('./configs') import Simulation from Caches import * import Mybench # Get paths we might need. It's expected this file is in m5/configs/example. config_path = os.path.dirname(os.path.abspath(__file__)) print config_path config_root = os.path.dirname(config_path)+"/configs" print config_root m5_root = os.path.dirname(config_root) print m5_root parser = optparse.OptionParser() # Benchmark options parser.add_option("-b", "--benchmark", default="", help="The benchmark to be loaded.") parser.add_option("-c", "--chkpt", default="", help="The checkpoint to load.") execfile(os.path.join(config_root, "configs", "Options.py")) (options, args) = parser.parse_args() if args: print "Error: script doesn't take any positional arguments" sys.exit(1) if options.benchmark == 'perlbench': process = Mybench.perlbench elif options.benchmark == 'bzip2': process = Mybench.bzip2 elif options.benchmark == 'gcc': process = Mybench.gcc elif options.benchmark == 'bwaves': process = Mybench.bwaves elif options.benchmark == 'gamess': process = Mybench.gamess elif options.benchmark == 'mcf': process = Mybench.mcf elif options.benchmark == 'milc': process = Mybench.milc elif options.benchmark == 'zeusmp': process = Mybench.zeusmp elif options.benchmark == 'gromacs': process = Mybench.gromacs elif options.benchmark == 'cactusADM': process = Mybench.cactusADM elif options.benchmark == 'leslie3d': process = Mybench.leslie3d elif options.benchmark == 'namd': process = Mybench.namd elif options.benchmark == 'gobmk': process = Mybench.gobmk; elif options.benchmark == 'dealII': process = Mybench.dealII elif options.benchmark == 'soplex': process = Mybench.soplex elif options.benchmark == 'povray': process = Mybench.povray elif options.benchmark == 'calculix': process = Mybench.calculix elif options.benchmark == 'hmmer': process = Mybench.hmmer elif options.benchmark == 'sjeng': process = Mybench.sjeng elif options.benchmark == 'GemsFDTD': process = Mybench.GemsFDTD elif options.benchmark == 'libquantum': process = Mybench.libquantum elif options.benchmark == 'h264ref': process = Mybench.h264ref elif options.benchmark == 'tonto': process = Mybench.tonto elif options.benchmark == 'lbm': process = Mybench.lbm elif options.benchmark == 'omnetpp': process = Mybench.omnetpp elif options.benchmark == 'astar': process = Mybench.astar elif options.benchmark == 'wrf': process = Mybench.wrf elif options.benchmark == 'sphinx3': process = Mybench.sphinx3 elif options.benchmark == 'xalancbmk': process = Mybench.xalancbmk elif options.benchmark == 'specrand_i': process = Mybench.specrand_i elif options.benchmark == 'specrand_f': process = Mybench.specrand_f if options.chkpt != "": process.chkpt = options.chkpt (CPUClass, test_mem_mode, FutureClass) = Simulation.setCPUClass(options) CPUClass.clock = '1.0GHz' #np = options.num_cpus np = 1 system = System(cpu = [CPUClass(cpu_id=i) for i in xrange(np)], physmem = PhysicalMemory(range=AddrRange("4096MB")), membus = Bus(), mem_mode = 'timing') system.physmem.port = system.membus.port for i in xrange(np): if options.caches: system.cpu[i].addPrivateSplitL1Caches(L1Cache(size = '64kB'), L1Cache(size = '64kB')) if options.l2cache: system.l2 = L2Cache(size='2MB') system.tol2bus = Bus() system.l2.cpu_side = system.tol2bus.port system.l2.mem_side = system.membus.port system.cpu[i].connectMemPorts(system.tol2bus) else: system.cpu[i].connectMemPorts(system.membus) system.cpu[i].workload = process[i] root = Root(system = system) Simulation.run(options, root, system, FutureClass)
The SPEC CPU2006 testing dataset results
We use the quard-core Xeon 2.5GHz with 16G memory machine. The operation system is 64bits CentOS 5.2. The timing results are from the simple cpu model and SPEC CPU2006 testing data set.
benchmark | datatype | language | input data | number of instructions | host seconds | comment |
400.perlbench | integer | C | attrs.out | - | - | fatal: fault (unalign) detected @ PC 0x12009cedc |
401.bzip2 | integer | C | input.program | 3171671617 | 1353.56 | o.k. |
403.gcc | integer | C | cccp.i | - | - | never end, but o.k. for smaller input |
410.bwaves | floating | Fortran | test | 119365801487 | 51703.94 | o.k. |
416.gamess | floating | Fortran | exam29 | - | - | abormal exit |
429.mcf | integer | C | inp.in | 5112705810 | 3386.09 | o.k. |
433.milc | floating | C | su3imp.in | 38027871822 | 18402.06 | output is different with the reference |
434.zeusmp | floating | Fortran | zmp_inp | 62107158516 | 27746.77 | output is different with the reference |
435.gromacs | floating | C/Fortran | gromacs.tpr | 10861507208 | 4457.33 | output is wrong, mremap has problem |
436.cactusADM | floating | C/Fortran | benchADM.par | - | - | fatal: fault (unalign) detected @ PC 0x120026614 |
437.leslie3d | floating | Fortran | leslie3d.in | 87402135744 | 41635.96 | o.k. |
444.namd | floating point | C++ | namd.input | 64449976020 | 26798.88 | o.k |
445.gobmk | integer | C | capture.tst | 494502991 | 260.29 | o.k. |
447.dealII | floating | C++ | 8 | - | - | output is wrong |
450.soplex | floating | C++ | test.mps | 72422927 | 31.95 | o.k. |
453.povray | floating point | C++ | test.ini | 3597778011 | 1737.24 | o.k. |
454.calculix | floating point | C | beampic.inp | 251699786 | 101.04 | o.k. |
456.hmmer | integer | C | bombesin.hmm | 2386768547 | 997.97 | o.k. |
458.sjeng | integer | C | test.txt | 21682684235 | 9406.80 | o.k. |
459.GemsFDTD | floating | Fortran | test.in | 11046857318 | 5289.88 | o.k. |
462.libquantum | integer | C | 33 5 | 292639209 | 111.64 | o.k. |
464.h264ref | integer | C | foreman_test_encoder_baseline.cfg | 154340641371 | 67426.00 | o.k. |
465.tonto | floating | Fortran | - | - | - | compile error |
470.lbm | floating | C | 100_100_130_cf_a.of | 7058506019 | 4599.69 | o.k. |
471.omnetpp | integer | C++ | omnetpp.ini | 2450821721 | 1153.36 | o.k. |
473.astar | integer | C++ | lake.cfg | 35796103621 | 16433.83 | output is different with the reference |
481.wrf | floating | C/Fortran | - | - | - | STOP wrf_abort. Need library |
482.sphinx3 | floating | C | args.an4 | 9352006427 | 4011.67 | o.k. |
483.xalancbmk | integer | C++ | test.xml | 501493417 | 276.77 | o.k. |
998.specrand | integer | C | 324342 24239 | 71348559 | 32.93 | o.k. |
999.specrand | floating | C | 324342 24239 | 71348559 | 31.58 | o.k. |
Trouble shooting
You may encounter errors while executing the SPEC CPU2006, and these two errors are common on 32-bits machine.
1. terminate called after throwing an instance of 'std::bad_alloc':
The M5 cannot allocate memory from you system. This happens a lot in the 32-bits machine, because only 3G memory can be used. To make life easier, you need a 64-bits machine.
2. bus error:
The same as above.
If you have any questions, please email to M5 Mailing-List.