Garnet2.0

[UNDER DEVELOPMENT].

More details of the gem5 Ruby Interconnection Network are here.

Garnet2.0: An On-Chip Network Model for Heterogeneous SoCs

Garnet2.0 is a detailed interconnection network model inside gem5. It builds upon the original Garnet model which was published in 2009.

If your use of Garnet contributes to a published paper, please cite the following paper:

@inproceedings{garnet,
  title={GARNET: A detailed on-chip network model inside a full-system simulator},
  author={Agarwal, Niket and Krishna, Tushar and Peh, Li-Shiuan and Jha, Niraj K},
  booktitle={Performance Analysis of Systems and Software, 2009. ISPASS 2009. IEEE International Symposium on},
  pages={33--42},
  year={2009},
  organization={IEEE}
}

Garnet2.0 provides a cycle-accurate micro-architectural implementation of an on-chip network router. It leverages the Topology and Routing infrastructure provided by gem5's ruby memory system model.

The default router is a state-of-the-art 1-cycle pipeline. There is support to add additional delay of any number of cycles in any router, by specifying it within the topology.

• Related Files:
  • src/mem/ruby/network/Network.py
  • src/mem/ruby/network/garnet2.0/GarnetNetwork.py

Invocation

The garnet networks can be enabled by adding --network=garnet2.0.

Configuration

Garnet uses the generic network parameters in Network.py:

• • number_of_virtual_networks: This is the maximum number of virtual networks. The actual number of active virtual networks is determined by the protocol.
  • control_msg_size: The size of control messages in bytes. Default is 8. m_data_msg_size in Network.cc is set to the block size in bytes + control_msg_size.

Additional parameters are specified in garnet2.0/GarnetNetwork.py:

• • ni_flit_size': flit size in bytes. Flits are the granularity at which information is sent from one router to the other. Default is 16 (=> 128 bits). [This default value of 16 results in control messages fitting within 1 flit, and data messages fitting within 5 flits]. Garnet requires the ni_flit_size to be the same as the bandwidth_factor (in network/BasicLink.py) as it does not model variable bandwidth within the network. This can also be set from the command line with --link-width-bits.
  • vcs_per_vnet': number of virtual channels (VC) per virtual network. Default is 4. This can also be set from the command line with --vcs-per-vnet.
  • buffers_per_data_vc: number of flit-buffers per VC in the data message class. Since data messages occupy 5 flits, this value can lie between 1-5. Default is 4.
  • buffers_per_ctrl_vc: number of flit-buffers per VC in the control message class. Since control messages occupy 1 flit, and a VC can only hold one message at a time, this value has to be 1. Default is 1.

Topology

Garnet2.0 leverages the Topology infrastructure provided by gem5's ruby memory system model. Any heterogeneous topology can be modeled. Each router in the topology file can be given an independent latency, which overrides the default. In addition, each link has 2 optional parameters: src_outport and dst_inport, which are strings with names of the output and input ports of the source and destination routers for each link. These can be used inside garnet2.0 to implement custom routing algorithms, as described next. For instance, in a Mesh, the west to east links have src_outport set to "west" and dst_inport" set to "east".

Routing

Garnet2.0 leverages the Routing]

• Additional features
  • Routing: Currently, garnet only models deterministic routing using the routing tables described earlier.
  • Modeling variable link bandwidth: The bandwidth_factor specifies the link bandwidth as the number of bytes per cycle per network link. ni_flit_size has to be same as this value. Links which have low bandwidth can be modeled by specifying a longer latency across them in the topology file (as explained earlier).
  • Multicast messages: The network modeled does not have hardware multi-cast support within the network. A multi-cast message gets broken into multiple uni-cast messages at the interface to the network.

Garnet fixed-pipeline network

The garnet fixed-pipeline models a classic 5-stage Virtual Channel router. The 5-stages are:

1. Buffer Write (BW) + Route Compute (RC): The incoming flit gets buffered and computes its output port.
2. VC Allocation (VA): All buffered flits allocate for VCs at the next routers. [The allocation occurs in a separable manner: First, each input VC chooses one output VC, choosing input arbiters, and places a request for it. Then, each output VC breaks conflicts via output arbiters]. All arbiters in ordered virtual networks are queueing to maintain point-to-point ordering. All other arbiters are round-robin.
3. Switch Allocation (SA): All buffered flits try to reserve the switch ports for the next cycle. [The allocation occurs in a separable manner: First, each input chooses one input VC, using input arbiters, which places a switch request. Then, each output port breaks conflicts via output arbiters]. All arbiters in ordered virtual networks are queueing to maintain point-to-point ordering. All other arbiters are round-robin.
4. Switch Traversal (ST): Flits that won SA traverse the crossbar switch.
5. Link Traversal (LT): Flits from the crossbar traverse links to reach the next routers.

The flow-control implemented is credit-based.

Garnet flexible-pipeline network

The garnet flexible-pipeline model should be used when one desires a router pipeline different than 5 stages (the 5 stages include the link traversal stage). All the components of a router (buffers, VC and switch allocators, switch etc) are modeled similar to the fixed-pipeline design, but the pipeline depth is not modeled, and comes as an input parameter number_of_pipe_stages. The flow-control is implemented by monitoring the availability of buffers at each output port before sending.