On-Chip Interconnect
& NoC Design

• Bandwidth ceiling — modern SoCs have 100+ IP blocks; the interconnect determines aggregate throughput between them
• Latency floor — cache miss to DRAM is dominated by NoC traversal + memory controller queuing, not DRAM access time
• Power budget — interconnect consumes 30–40% of total SoC dynamic power; wire energy dominates at advanced nodes
• Design reuse — a well-designed NoC allows IP blocks to be swapped, added, or removed without backend redesign

AMBA AHB / APB

An arbiter grants one manager access at a time. All subordinates share the same address/data wires.

Characteristics

• Simple — minimal area, easy verification
• Pipelined — AHB uses address + data phases
• Single-master bottleneck — only one transaction at a time on the bus
• Non-scalable — adding masters degrades throughput linearly

Where It Still Works

• APB — low-bandwidth peripheral register access (UART, SPI, I²C config registers)
• AHB-Lite — single-manager subsystems (sensor hubs, audio DSPs)
• MCUs with ≤4 bus masters and ≤1 GHz clock

Multiple concurrent transactions as long as they target different subordinates. Conflicts arbitrated per-subordinate port.

Crossbar Properties

• Parallel paths — M0→S2 and M1→S0 proceed simultaneously
• Low latency — single-hop, no store-and-forward
• Wire cost — grows as O(N×M); practical limit ~16×16 ports
• Centralized — all arbitration in one block; routing congestion

Examples

• Arm NIC-400 — configurable AXI crossbar for mid-range SoCs
• Synopsys DW_axi — DesignWare AXI interconnect matrix
• Most FPGA soft-core SoCs (Xilinx AXI Interconnect IP)

A NoC replaces ad-hoc global wires and centralized crossbars with a packet-switched network of distributed routers connected by point-to-point links.

Three Building Blocks

• Network Interface (NI) — converts IP transactions (AXI/CHI) to network packets and back; handles protocol isolation, clock/power domain crossing
• Router — receives packets, performs routing lookup, arbitrates between inputs, forwards to output port; typically 3–5 pipeline stages
• Link — physical wires between routers; can be pipelined for long distances; defines raw bandwidth (width × frequency)

Why NoC over Crossbar?

Simplified 4×4 mesh showing cross-point routers (XP), CPU clusters, SLC slices, DDR and PCIe

Used in AWS Graviton3/4, Ampere AmpereOne, NVIDIA Grace, and many HPC/networking SoCs.

CMN S3 (current): 32 CCG (CXL 3.0 gateway), 512-bit CXS Issue B, up to 256 cores with 512 MB SLC, confidential compute, Armv9.2. CMN S3AE (2025) targets automotive chiplet designs.

Manager → Subordinate & Subordinate → Manager

Each channel uses independent xVALID / xREADY handshake — channels decouple fully for maximum throughput.

Key AXI Features

• Burst transfers — FIXED, INCR, WRAP; up to 256 beats (AXI4)
• Out-of-order — responses can return in any order using transaction IDs
• Multiple outstanding — manager issues N requests before first response returns
• Write strobes — WSTRB enables per-byte lane masking
• Protection — AxPROT[2:0]: privileged/unprivileged, secure/non-secure, instruction/data
• Cache hints — AxCACHE[3:0]: bufferable, cacheable, read/write-allocate
• QoS — AxQOS[3:0]: 16 priority levels for interconnect arbitration

AXI-Lite: single-beat, no burst, no ID — for simple register interfaces. AXI-Stream: unidirectional data streaming (DMA, video, DSP pipes).

ACE extends AXI with three additional channels to support hardware cache coherency between multiple managers.

Additional ACE Channels

• AC (Snoop Address) — interconnect sends snoop request to cached managers
• CR (Snoop Response) — manager responds with cache line state
• CD (Snoop Data) — manager returns dirty data to interconnect

ACE Transaction Types

• ReadShared — read, others may retain copy
• ReadUnique — read with exclusive ownership
• CleanUnique — upgrade shared → exclusive
• WriteBack — evict dirty line to memory
• MakeInvalid — invalidate all other copies

ACE Variants

Layered Architecture

Node Types

• RN-F — Request Node, Full coherent (CPU)
• RN-I — Request Node, I/O coherent (GPU, DMA)
• HN-F — Home Node, Full (SLC + snoop filter)
• SN-F — Subordinate Node (memory controller)
• MN — Misc Node (debug, DVM)

Why CHI over ACE?

• Packet-based — replaces ACE's signal-level protocol with flits; reduces wire count by ~30%
• Scalable — credit-based flow control prevents congestion; supports 100+ agents
• Retry mechanism — if home node is busy, request can be retried without blocking the network
• Atomic ops — near-memory atomic operations (AtomicStore, AtomicCompare) reduce round-trips
• Data poisoning — end-to-end error signaling for RAS
• MTE — Memory Tagging Extension support (CHI Issue E)

MESI (no Owned) used in Intel; MOESI used in AMD. Arm CHI uses a similar model with Unique/Shared/Invalid + Dirty/Clean.

Snoop vs Directory

• Broadcast snoop — every coherent agent is queried on each miss; simple but O(N) traffic; used in ACE-based systems with ≤8 cores
• Directory / snoop filter — home node tracks which caches hold each line; targeted snoops; O(1) snoop traffic; essential for 16+ core systems
• Distributed directory — snoop filter state distributed across HN-F nodes in mesh (CMN-700); scales to 256+ cores

Traffic Classes

• Latency-sensitive — CPU demand loads; require low, predictable latency (e.g., <100 ns to DRAM)
• Latency-critical + BW — display/video pipeline; sustained bandwidth with deadline (frame drops if starved)
• Best-effort BW — bulk DMA, Ethernet, background copies; can tolerate jitter
• Signaling — inter-processor interrupts, config register accesses; tiny but urgent

QoS Mechanisms

• AxQOS priority — AXI 4-bit field (0–15) attached to every transaction; arbiters use it for prioritization
• Urgency escalation — dynamically raise priority of aging transactions to prevent starvation
• Bandwidth regulation — rate limiters at NI ports; prevent any single agent from monopolizing links
• Virtual Channels (VCs) — separate physical link into logically independent channels; prevent head-of-line blocking between traffic classes
• TDM slots — Time-Division Multiplexing guarantees deterministic bandwidth; used in safety-critical (automotive) NoCs

Arbitration Schemes

Flow Control

• Credit-based — sender tracks receiver buffer space via credits; used in CHI. Prevents buffer overflow without back-pressure stalls.
• Wormhole — packet split into flits; head flit reserves path, body/tail follow. Low buffer cost but can cause head-of-line blocking.
• Virtual Cut-Through (VCT) — forward as soon as enough buffer exists for entire packet. Higher throughput than wormhole, more buffers.
• Store-and-Forward — entire packet buffered before forwarding. Highest latency, simplest error handling.

Routing Strategies

• Deterministic (XY) — route first in X, then Y dimension; no adaptivity but deadlock-free by construction; simplest hardware
• Oblivious — fixed path per source-dest pair but can use multiple static routes; load-balanced
• Adaptive — choose path based on local congestion; higher throughput under non-uniform traffic; needs deadlock avoidance mechanism
• Source routing — full path embedded in packet header; no per-router routing tables; used in some custom NoCs

Deadlock Avoidance

• Dimension-ordered routing — XY routing on mesh creates an acyclic channel dependency graph → no deadlock
• Virtual channel classes — separate request/response/snoop into VCs; break circular dependencies between protocol message classes
• Turn model — forbid specific turns (e.g., west-first, negative-first) to eliminate cycles while allowing partial adaptivity
• Escape channels — designate one VC with deterministic routing as an "escape" for adaptive traffic that might deadlock

AMD's proprietary scalable interconnect connecting CCDs, IODs, and across sockets.

Key Design Decisions

• Chiplet-native — designed from the ground up for die-to-die communication between CCD and IOD chiplets
• NUMA-aware — each CCD has preferred memory channels; OS scheduler considers fabric topology
• Scalable — same fabric from 1-CCD Ryzen (8 cores) to 16-CCD EPYC (128+ cores), to 2-socket (256+ cores)
• Bandwidth — each IF link: ~40 GB/s per direction; EPYC 9004: 12 IF links = ~480 GB/s aggregate CCD↔IOD

Infinity Fabric also carries non-coherent traffic (PCIe, SATA, USB) between IOD and peripherals.

Ring (Sandy Bridge → Broadwell)

• Bidirectional ring connecting cores, LLC slices, memory controllers, and I/O agents
• Simple, low latency for ≤10 cores
• Bandwidth saturates at ~12–14 cores; latency grows linearly with core count
• Dual ring used in Haswell-EP (up to 18 cores) to mitigate, but fundamental limit remains

Mesh (Skylake-SP onwards)

• 2D mesh with XY routing; each tile = core + LLC slice + mesh stop
• Scales to 128+ P-cores (Granite Rapids) or 288 E-cores (Sierra Forest) per socket via chiplets
• Snoop filter replaces broadcast for coherency
• Sub-NUMA clustering: partition mesh into 1, 2, or 4 NUMA domains for latency optimization

Why the Transition?

Xeon Scalable (Skylake-SP): first Intel server mesh. Granite Rapids continues with mesh + EMIB chiplet fabric.

CXL (Compute Express Link)

• CXL.io — PCIe-based discovery and config (non-coherent)
• CXL.cache — device caches host memory with coherency
• CXL.mem — host-managed device memory (memory expansion / pooling)
• CXL 3.2 / 4.0 — peer-to-peer, fabric switches, global fabric addressing; 4.0 doubles bandwidth (PCIe 7 PHY)

CXL runs over PCIe PHY — reuses existing SerDes; supported in Arm CMN-700/S3 via AMBA CXS bridge.

Die-to-Die (D2D) NoC Extension

• UCIe 3.0 (2025) — 64 GT/s D2D PHY + protocol; extends NoC across chiplet boundaries with CXL 3.x/PCIe tunnelling
• BoW (Bunch of Wires) — OCP-defined low-latency parallel D2D for short reach
• Custom links — AMD IF, NVIDIA NV-HBI, Apple UltraFusion

NoC Power Techniques

• Clock gating — idle routers/links gate clocks; AMBA Q-Channel handshake for safe clock stop
• Power gating — entire NoC regions powered down; requires wake-up latency management
• DVFS per domain — NI adapters handle clock domain crossing; high-traffic paths run faster
• Link serialization — reduce wire count on low-BW links; trade latency for fewer active wires
• GALS — Globally Asynchronous, Locally Synchronous; each IP block has its own clock; NI handles synchronization

Functional Safety (ISO 26262)

• AMBA parity extension — AXI Issue G+ / CHI Issue E+ add interface parity for transient fault detection
• End-to-end protection — CRC or ECC on flits across the NoC; detects router/link errors
• Data poisoning — CHI poison bit marks corrupted data; consumer can handle gracefully
• TDM scheduling — deterministic time slots guarantee worst-case latency bounds for ASIL-rated paths
• Lockstep routers — duplicate router logic for ASIL-D; compare outputs cycle-by-cycle