Difference between revisions of "Documentation"

Revision as of 15:26, 4 March 2011 (view source) Rsen (talk | contribs) (→‎MI example) ← Older edit		Revision as of 15:26, 4 March 2011 (view source) Rsen (talk | contribs) (→‎DMASequencer) Newer edit →
Line 423:		Line 423:
	===== DMASequencer =====		===== DMASequencer =====
		+	''Derek will do it''
		+
	===== Memory Controller =====		===== Memory Controller =====

---

Revision as of 15:26, 4 March 2011

Outline for some new documentation. Maybe we should be doing this wit LaTeX and using pandoc (LaTeX->mediawiki converter)?

Getting Started

What is gem5?

gem5 is a modular discrete event driven computer system simulator platform. That means that:

1. gem5's components can be rearranged, parameterized, extended or replaced easily to suite your needs.
2. It simulates the passing of time as a series of discrete events.
3. It's intended use is to simulate one or more computer systems in various ways.
4. It's more than just a simulator, it's a simulator platform that lets you use as many of its premade components as you want to build up your own simulation system.

gem5 is written primarily in C++ and python and most components are provided under a BSD style license. It can simulate a complete system with devices and an operating system in full system mode (FS mode), or user space only programs where system services are provided directly by the simulator in syscall emulation mode (SE mode). There are varying levels of support for executing Alpha, ARM, MIPS, Power, SPARC, and 64 bit x86 binaries on CPU models including two simple single CPI models, an out of order model, and an in order pipelined model. A memory system can be flexibly built out of caches and interconnects. Recently the Ruby simulator has been integrated with gem5 to provide even better, more flexible memory system modeling.

There are many components and features not mentioned here, but from just this partial list it should be obvious that M5 is a sophisticated and capable simulation platform. Even with all gem5 can do today, active development continues through the support of individuals and some companies, and new features are added and existing features improved on a regular basis. For the most up to date information you can check out the project's website at www.gem5.org.

Pointer to gem5 Capabilities/Recipes

Getting a copy

M5's source code is managed using the mercurial revision control system. There are several repositories you may be interested in:

1. m5 – The main repository is where active development takes place.
2. m5-stable – A repository which lags behind “m5” repository but has basically the same contents. It’s usually better to use “m5” instead of “m5-stable”
3. encumbered – A repository for extensions to M5 that are under a different, more restrictive license. Currently this only includes support for SimpleScalar's EIO trace format.
4. linux-patches – A repository for patches to the linux kernel that modify it so it can be simulated more efficiently. These patches are optional, but it's a good idea to use them if possible to cut down on simulation run time.

To check out a copy, first, make sure you have mercurial installed on your system and that you can run the hg command. Then use hg clone to create your own local copy using the URL http://repo.m5sim.org/XXX where XXX is replaced by the name of the repository your interested in. For example, to check out the main repository, you'd use the command:

hg clone http://repo.m5sim.org/m5

You can find out more about mercurial and its commands using its built in help by running:

hg help

Building

M5 uses the scons build system which is based on python. To build the simulator binary, run scons from the top of the source directory with a target of the form build/<config>/<binary> where <config> is replaced with one of the predefined set of build parameters and <binary> is replaced with one of the possible m5 binary names. The predefined set of parameters determine build wide configuration settings that affect the behavior, composition, and capabilities of the binary being built. These include whether the simulator will run in FS or SE mode, if Ruby support is included, which ISA will be supported, which CPU models to build, and what coherence protocol Ruby should use. Examples are ARM_FS, X86_SE, and ALPHA_SE_MOESI_CMP_token. All of the available options can be found in the build_opts directory, and it should be fairly easy to see what each is for. We'll talk about the build system in more detail later. Valid binary names are m5.debug, m5.opt, m5.fast, and m5.prof. These binaries all have different properties suggested by their extension. m5.debug has optimization turned off to make debugging easier in tools like gdb, m5.opt has optimizations turned on but debug output and asserts left in, m5.fast removes those debugging tools, and m5.prof is built to use with gprof. Normally you'll want to use m5.opt. To build the simulator in syscall emulation mode with Alpha support, optimizations turned on, and debugging left in, you would run:

scons build/ALPHA_SE/m5.opt

In your source tree, you'd then find a new build/ALPHA_FS/ directory with the requested m5.opt in it. For the rest of this chapter we'll assume this is the binary you're using.

Running

Now that you've built M5, it's time to try running it. An M5 command line is composed of four parts, the binary itself, options for M5, a configuration script to run, and then finally options for the configuration script. Several example configuration scripts are provided in the “configs/example” directory and are generally pretty powerful. You are encouraged to make your own scripts, but these are a good starting point. The example script we'll use in SE mode is called se.py and sets up a basic SE mode simulation for us. We'll tell it to run the hello world binary provided in the M5 source tree.

build/ALPHA_SE/m5.opt configs/example/se.py -c tests/test-progs/hello/bin/alpha/linux/hello

This builds up a simulated system, tells it to run the binary found at the location specified, and kicks off the simulation. As the binary runs, it's output is sent to the console by default and looks like this:

M5 Simulator System

Copyright (c) 2001-2008
The Regents of The University of Michigan
All Rights Reserved

M5 compiled Feb 12 2011 20:43:55
M5 revision e00ef55a2c49 7933 default tip
M5 started Feb 12 2011 20:45:47
M5 executing on fajita
command line: build/ALPHA_SE/m5.opt configs/example/se.py -c tests/test-progs/hello/bin/alpha/linux/hello
Global frequency set at 1000000000000 ticks per second
0: system.remote_gdb.listener: listening for remote gdb #0 on port 7000
**** REAL SIMULATION ****

info: Entering event queue @ 0. Starting simulation...
info: Increasing stack size by one page.
Hello world!
hack: be nice to actually delete the event here
Exiting @ tick 3240000 because target called exit()

You can see a lot of output from the simulator itself, but the line “Hello world!” came from the simulated program. In this example we didn't provide any options to M5 itself. If we had, they would have gone on the command line between m5.opt and se.py. If you'd like to see what command line options are supported, you can pass the --help option to either M5 or the configuration script. The two groups of options are different, so make sure you keep track of whether they go before or after the configuration script.

Asking for help

M5 has two main mailing lists where you can ask for help or advice. m5-dev is for developers who are working on the main version of M5. This is the version that's distributed from the website and most likely what you'll base your own work off of. m5-users is a larger mailing list and is for people working on their own projects which are not, at least initially, going to be distributed as part of the official version of M5. Most of the time m5-users is the right mailing list to use. Most of the people on m5-dev are also on m5-users including all the main developers, and in addition many other members of the M5 community will see your post. That helps you because they might be able to answer your question, and it also helps them because they'll be able to see the answers people send you. To find more information about the mailing lists, to sign up, or to look through archived posts visit:

http://m5sim.org/wiki/index.php/Mailing_Lists

Cross-product status matrix

gem5 combines several different ISAs, system modes (SE or FS), CPU models and memory models under a single infrastructure. The cross-product of these combinations may or may not actually work. The following page describes the current status of these combinations.

http://m5sim.org/wiki/index.php/Status_Matrix

Source Code

Tour of the tree

These are the files and directories at the top of the tree:

AUTHORS LICENSE README RELEASE_NOTES SConstruct build_opts configs ext src system tests util

AUTHORS, LICENSE, README are files with general information about the simulator. AUTHORS is a list of people who have historically contributed to M5. LICENSE has the license terms that applies to M5 as a whole, unless overridden by a more specific license. README has some very basic information introducing M5 and explaining how to get started.

The SConstruct file is part of the build system, as is the build_opts directory. build_opts holds files that define default settings for build different build configurations. These include X86_FS and MIPS_SE, for instance.

The configs directory is for simulation configuration scripts which are written in python. These are described in more detail later. The files in this directory help make writing configurations easier by providing some basic prepackaged functionality, and include a few examples which can be used directly or as a starting point for your own scripts.

The ext directory is for things M5 depends on but which aren’t actually part of M5. Specifically these are for dependencies that are harder to find, not likely to be available, or where a particular version is needed.

The src directory is where most of M5 is located. This is where all of the C++ and python source that contributes to the M5 binary is kept, excluding components in the ext directory.

The system directory is for the source for low level software like firmware or bootloaders for use in simulated systems. Currently this includes Alpha’s PAL and console code, and a simple bootloader for ARM.

The tests directory stores files related to M5’s regression tests. These include the scripts that build up the configurations used in the tests and reference outputs. Simple hello world binaries are also stored here, but other binaries need to be downloaded separately.

Finally, in the util directory are utility scripts, programs and useful files which are not part of the M5 binary but are generally useful when working on M5.

Style rules

Generated files - where do they end up

.m5 config files

jobfile to run multiple jobs

Build System

M5's build system is based on Scons, an open source build system implemented in python. You can find more information about scons at www.scons.org. The main scons file is called SConstruct and is found in the root of the source tree. additional scons files are called SConscript and are found throughout the tree, usually near the files they're associated with.

Build targets

In M5, scons build targets are of the form <build dir>/<configuration>/<target>. The <build dir> part of the target is a directory path that ends in "build". Typically this is simply "build" by itself, but you can specify a directory called "build" located somewhere else instead. The <configuration> part selects a set of preset build configuration variables values which correspond to common . The possible options are the file names in the build_opts directory, and are discussed more below. The build targets can be regression tests which are explained in more detail below, or they can be different versions of the M5 binary. The name of the binary is "m5" with an extension that specifies what version to build. Currently supported versions are m5.debug, m5.opt, m5.fast, and m5.prof.

m5.debug has optimizations turned off. This ensures that variables won't be optimized out, functions won't be unexpectedly inlined, and control flow will won't behave in surprising ways. That makes this version easier to work with in tools like gdb, but without optimizations this version is significantly slower than the others. You should choose it when using tools like gdb and valgrind and don't want any details obscured, but other wise more optimized versions are recommended.

m5.opt has optimizations turned on and debugging functionality like asserts and DPRINTFs left in. This gives a good balance between the speed of the simulation and insight into what's happening in case something goes wrong. This version is best in most circumstances.

m5.fast has optimizations turned on and debugging functionality compiled out. This pulls out all the stops performance wise, but does so at the expense of run time error checking and the ability to turn on debug output. This version is recommended if you're very confident everything is working correctly and want to get peak performance from the simulator.

m5.prof is similar to m5.fast but also includes instrumentation that allows it to be used with the gprof profiling tool. This version is not needed very often, but can be used to identify the areas of M5 that should be focused on to improve performance.

These versions are summarized in the following table.

Binary name	Optimizations	Run time debugging support	Profiling support
m5.debug		X
m5.opt	X	X
m5.fast	X
m5.prof	X		X

Command line options

Scons will recognize the following command line options specific to M5.

Option	Effect
--color	Turn on colorized output
--no-color	Turn off colorized output

Environment variables

The following environment variables are imported from the host environment for use in scons:

Variable	Use
AS	Assembler command
AR	Archive tool command
CC	C compiler command
CXX	C++ compiler command
HOME	User's home directory
LD_LIBRARY_PATH	Path to search for library files
PATH	Path to search for programs
PYTHONPATH	Path to search for python files
RANLIB	Ranlib command
M5_CONFIG	Where to look for the special ".m5" directory
M5_DEFAULT_BINARY	The default build target which overrides the default default build/ALPHA_SE/m5.debug

Configuration variables

These configuration variables are used to control the way M5 is built. Some are global, affecting all configurations built in the same build directory. Some are sticky, which means they retain their value once set. The available options are summarized in the following tables.

Global sticky

Variable	Description	Default
CC	C Compiler	CC environment variable or value determined by scons
CXX	C++ Compiler	CXX environment variable or value determined by scons
BATCH	Use batch pool for build and tests	False
BATCH_CMD	Batch pool submission command	qdo
M5_BUILD_CACHE	Cache built objects in this directory	False
EXTRAS	Add extra directories to the compilation

Global non-sticky

Variable	Description	Default
VERBOSE	Print full tool command lines	False
update_ref	Update test reference outputs	False

Non-global sticky

Variable	Description	Default	Exported as config/*.hh
TARGET_ISA	Target ISA	alpha	X
FULL_SYSTEM	Full-system support	False	X
CPU_MODELS	CPU Models	AtomicSimpleCPU,InOrderCPU,O3CPU,TimingSimpleCPU
NO_FAST_ALLOC	Disable fast object allocator	False	X
FAST_ALLOC_DEBUG	Enable fast object allocator debugging	False	X
FAST_ALLOC_STATS	Enable fast object allocator statistics	False	X
EFENCE	Link with Electric Fence malloc debugger	False
SS_COMPATIBLE_FP	Make floating-point results compatible with SimpleScalar	False	X
USE_SSE2	Compile for SSE2 (-msse2) to get IEEE FP on x86 hosts	False
USE_MYSQL	Use MySQL for stats output	whether mysql_config was found	X
USE_POSIX_CLOCK	Use POSIX Clocks	whether posix clocks are available on this host	X
USE_FENV	Use <fenv.h> IEEE mode control	whether fenv.h was found on this host	X
USE_CHECKER	Use checker for detailed CPU models	False	X
CP_ANNOTATE	Enable critical path annotation capability	False	X
RUBY	Build with Ruby	False
PROTOCOL	Coherence protocol for Ruby	MI_example	X
NO_HTML	Do not create HTML files	False
NO_VECTOR_BOUNDS_CHECKS	Don't do bounds checks	True	X
GEMS_ROOT	Add debugging stuff to Ruby	the directory where GEMS is stored, currently src/mem	X

Setting configuration variable values

The first way you set configuration variable values is through the configuration name you choose as part of the build target. This file is loaded from build_opts and contains preset values for some of these variables which configures the build as the file name suggests. For instance, using the SPARC_FS configuration will turn on full system mode with the FULL_SYSTEM variable and set the TARGET_ISA to sparc.

It is important to note that the values in the file corresponding to the configuration you picked are -default- values and are only used if no directory already exists with its own values already in place. Those files are for defining reasonable starting points to configure M5 to behave the way you want it to, and are not intended to actively configure a particular build.

If you want to change a value after your build and configuration directory is already created, or if you want to override a value as it's created, you can specify the new values on the command line. The syntax is similar to setting environment variables at a shell prompt, but these go after the scons command. For example, to turn on RUBY support for an existing ALPHA_FS build, you could use the following command.

scons RUBY=True build/ALPHA_FS/m5.opt

It's often a good idea to add --help to the scons command line which will print out all of the configuration variables and what their values are. This way you can make sure everything is set up like you want, and that you don't have any typos in any variable names. If everything is as you expect, you can remove --help to actually start the build.

Running regressions

Adding files to the build

Using EXTRAS

SimObjects

The python side

Param types

Inheritance

Special attribute names

Rules for importing - how to get what

Pro tips - avoiding cycles, always descend from root, etc.

The C++ side

create functions

Stages of initialization

Header files to include

Configuration Scripts

SimObject hierarchy

Explanation of infrastructure scripts in configs

How to use se.py and fs.py.

How to use other top level scripts (what are they?)

Where M5 looks for files

General memory system

Ports system

Ports in general

Various port types

Packets

Requests

Atomic/Timing/Functional accesses

Two memory systems: Classic and Ruby

Classic memory system

Summarize functionality and highlight advantages and disadvantages. Uses ports to connect all components.

Ruby memory system

Summarize functionality and highlight advantages and disadvantages. Uses ports to connect cpus to the Ruby memory system. Uses message buffers to connect components within the memory system.

Classic memory system

MemObjects

Caches

Hooking them up

Parameters

Interconnects

Buses

Bridges

Anything else?

Coherence

Ruby

High level components of Ruby

  Need to talk about the overview of Ruby and what are the major components.

SLICC + Coherence protocols:

   Need to say what is SLICC and whats its purpose. 
   Talk about high level strcture of a typical coherence protocol file, that SLICC uses to generate code. 
   A simple example structure from protocol like MI_example can help here.

Protocol independent memory components

1. Cache Memory
2. Replacement Policies
3. Memory Controller
4. Rubyport

Arka will do it

Topologies and Networks

There is a separation between the network topology and the implementation.

Implementation of Ruby

  Low level details goes here. Need to explain code directory structure as well.

SLICC

   Explain functionality/ capability of SLICC
   Talk about
   AST, Symbols, Parser and code generation in some details but NO need to cover every file and/or functions. 
   Few examples should suffice.

Protocols

   Need to talk about each protocol being shipped. Need to talk about protocol specific configuration parameters.
   NO need to explain every action or every state/events, but need to give overall idea and how it works
   and assumptions (if any).

MI example

MI protocols don't support LL/SC semantics.

Rathijit will do it

MOESI_hammer

MOESI_CMP_token

Shoaib will do it

MOESI_CMP_directory

Rathijit will do it

MESI_CMP_directory

Arka will do it

Protocol Independent Memory components

System

Arka will do it

Sequencer

Arka will do it

CacheMemory

Arka will do it

DMASequencer

Derek will do it

Memory Controller

Topologies and Networks

Topology specification

Python files specify connections. Shortest path graph traversals program the routing tables.

Network implementation

1. SimpleNetwork
2. Garnet

Life of a memory request in Ruby

Cpu model generates a packet -> RubyPort converts it to a ruby request -> L1 cache controller converts it to a protocol specific message ...etc.

Arka will do it

Events

Event queue

EventManager objects (I don’t know a lot about these)

Event objects

Time sync

Devices

I/O device base classes

IDE

NICS

Timers

PCI devices

DMA devices

UARTs and serial terminals

Explanation of platforms and systems, how they’re related, and what they’re each for

Execution Basics

Predecoding

StaticInsts

Microcode support

ExecContext

ThreadContext

Faults

Architectural State

Registers

Register types - float, int, misc

Indexing - register spaces stuff

PCs

Address Translation

TLB management

Delayed translation

Page table walkers (defer to ISA discussion?)

Differences between SE and FS mode

CPUs

Simple CPU

Flow of execution

Sharing of code between atomic and timing

O3 CPU

Anatomy of the pipeline

DynInsts

Wires/delay

Squashing

Load/store handling

Renaming

etc.

InOrder (what to say here?)

Interrupts

Traditional handling with platform object

New, currently X86 only system

Interrupt sources/sinks/wires/pins

Interrupt messages

Interrupt managing objects in the CPU

ISA parser

Formats

operands

decode tree

let blocks

microcode assembler

microops

macroops

directives

rom object

Lots more stuff

Multiple ISA support

Switching header files

Specialized classes and functions

Matrix of supported ISA/mode/CPU combinations

Alpha Implementation

ARM Implementation

Supported features and modes

Thumb support

Special PC management

MIPS Implementation

Power Implementation

SPARC Implemenation

Supported features and modes

X86 Implementation

Microcode ISA

Macroop specialization

Platform objects

Physical memory space allocation

Microcode registers and other hidden implementation details

Supported features and modes

Pseudo Instructions

List of pseudo instructions and what they do

How to get at and use them from a binary, script, etc.

SE mode

Loading binaries

System call instruction handling

Implementation of various system calls

Limitations

Statistics (don’t know what to say here)

Summary: gem5 Capabilities/Recipes

Atomic mode

Only supported by the classic memory system.

Timing mode

Classic memory system

Ruby memory system

MOESI_hammer in ALPHA_SE mode running multiple SPEC benchmarks with Mesh topology

Highlight no DMA controllers

??? FS mode

Highlight DMA controllers and how they are connected in the network.

Atomic mode then timing mode

Only supported with the classic memory system.

Utility Code

Bitfield functions

BitUnion classes

FastAlloc

IntMath

panic, warn, etc., when to use what

random number generation

reference counting pointers

Debugging

Trace flags

State trace/Tracer objects

Remote GDB support

tracediff

Regressions

Unit tests

Development Tools/Contributing

Mercurial queues

Review board

Mercurial repo browser

External Dependencies

Required versions

Things in ext