CHI — Coherent Hub Interface

• ACE was a signal-level protocol — every master port physically owned AC/CR/CD wires running to the interconnect.
• At 2 clusters: fine. At 8 clusters: the snoop fan-out starts to hurt. At 32+ coherent agents: the wire count becomes unmanageable.
• For servers and high-end SoCs Arm needed a transport-agnostic, packet-based protocol: any coherent message becomes a packet that the interconnect routes over whatever topology fits (ring, mesh, hybrid).
• First delivered as AMBA 5 CHI Issue A in 2013 alongside Arm's CCN-504 interconnect; refined through CHI-B/C/D/E/F as Neoverse and CMN-600/650/700 scaled up.

• Every CHI link (a bidirectional port between two nodes) carries four virtual channels:
  • REQ — request from RN / HN (ReadShared, ReadUnique, WriteBack, CMO, Atomic…)
  • RSP — responses (CompAck, RetryAck, DBIDResp, DVMResp…)
  • SNP — snoops directed from HN to RN-F
  • DAT — cache-line data (bursts)
• Each channel has its own credit-based flow control — a producer must hold a credit for the target's buffer before sending.
• Packets are flits — small fixed chunks that fit in one NoC beat. A full 64 B cache line is typically 4 × 16 B flits on a 128-bit link.

• CHI inherits the five-state MOESI-variant from ACE, with slightly different names and tighter specification:
  • UC — UniqueClean (E)
  • UD — UniqueDirty (M)
  • SC — SharedClean (S)
  • SD — SharedDirty (O)
  • I — Invalid
• Additionally CHI-E introduced the partial-line concept (UCE — Unique Clean Empty, used when a line is reserved for write-allocate but not yet populated).
• The HN-F's directory entry stores coarse state per line: who holds it, and what overall shared/unique state the line is in — enough to determine which snoops to fan out.

DMT — Direct Memory Transfer

• For plain misses where no cache holds the line, HN-F asks SN-F (DRAM) to send data directly to the requester (RN-F), bypassing the HN-F's critical path.
• HN-F stays in the loop for tracking (response still comes back), but the data flit does not have to hop through it.

DCT — Direct Cache Transfer

• When a peer RN-F has the data (Shared or Unique), HN-F can ask that peer to forward it directly to the requester, again bypassing HN-F as a relay.
• Critical for cache-to-cache latency on contended lines.

• Each HN-F tile in CMN-600/650/700 includes a slice of the System Level Cache — a distributed L3/L4 shared across all RN-Fs.
• Functions:
  • Reduce DRAM traffic (hit in SLC = no SN-F access).
  • Act as a snoop-filter-equivalent — if a line is in SLC, track who also holds it.
  • Hold dirty data temporarily until an eviction forces write-back.
• Total SLC in a Neoverse V2 system can be ≥ 128 MB — 2 MB per HN-F × 64 HN-Fs on a 8×8 mesh.

Every Neoverse server SoC (N1, N2, V1, V2) uses one of these. AWS Graviton, Ampere Altra, NVIDIA Grace, Microsoft Cobalt — all CMN-based.

• Every CHI channel has a finite buffer at the receiver.
• Senders operate with credits: you don't send a packet unless you hold a credit for its channel at the target. Credits are returned after the packet is consumed.
• If the sender has no credit available, the packet waits. No packet is dropped.
• For request channels, a target that cannot accept right now responds RetryAck + PCrdGrant. The requester must later re-send carrying the granted credit (PCrdType). Bounded retry — no livelock.

• DVM on CHI is a dedicated request opcode (DVMOp) rather than a special AC-channel encoding.
• Flows:
  • An RN-F or RN-D (typically SMMU) sends a DVMOp REQ to a Misc Node (MN) — the DVM serialisation point.
  • MN forwards SnpDVMOp to every RN-F + RN-D in the DVM domain.
  • Each peer acknowledges; MN aggregates; requester gets final DVMComplete.
• Payload: TLBI address range, ASID, VMID, IC invalidate, sync barrier.
• CHI-E extended DVM to handle RME (Realm) invalidates separately.

• MPAM — Memory-system Partitioning and Monitoring — is Arm's answer to CAT (Cache Allocation Technology) on x86.
• CHI-D and CHI-E carry a PARTID on every transaction (plus PMG — Performance Monitoring Group).
• HN-F tiles enforce per-PARTID policies:
  • SLC way allocation quotas
  • Memory-bandwidth fair shares
  • Miss-rate telemetry
• Lets a hypervisor prevent a noisy-neighbour VM from thrashing the system cache or saturating memory bandwidth.

• Arm Confidential Compute Architecture (CCA, Armv9.2) introduces Realms — hypervisor-unreadable virtual machines.
• CHI-E adds attributes to support this:
  • 4-way security state per transaction — Root / Realm / Secure / Non-Secure.
  • Granular permission checks at the HN-F (Realm transactions can't read Secure memory, etc.).
  • SLC tagging ensures Realm lines aren't returned to Non-Secure readers.
  • DVM semantics extended to invalidate Realm TLB entries separately.

• Huge state space: 128+ RN-Fs, each with independent caches; each line has 5 states × N possible holders.
• Packet reordering: NoC may reorder between channels.
• Ordering rules are subtle: some transactions serialise at HN, others at RN; CompAck ordering vs DAT arrival; PD forwarding.
• Modern CHI verification is a mix of:
  • Formal (Jasper / Questa) on protocol properties at each interface.
  • UVM constrained-random on the mesh-level fabric.
  • Emulation (Palladium / Veloce) for system-level boot on real software.
  • Post-silicon coherency stress tests (memory ordering litmus).

Why CHI resists "minimal"

• CHI is packet-based with 4 virtual channels + credit flow control + a directory.
• The smallest compliant RN-F must: respond to every inbound SNP type, maintain per-line cache state, track outstanding TxnIDs, generate CompAck, and honour ordering domains. ~2–4 k flops before you add any actual caching.
• There is no "just wires" CHI. The protocol's coherency contract forbids it.

What can be minimal: an RN-I-ish bridge

A non-coherent CHI Request Node (no caches, no SNP channel participation) wrapped around an AXI4 master is the simplest agent that can plug into a CHI mesh — useful for DMA and legacy IP integration.

// Sketch: CHI RN-I built as a thin state-machine
// in front of an existing AXI4 master.
// No caches; never receives SNP on the RN-I port.
module chi_rni_axi_bridge (
  input  logic        clk, rstn,
  // --- CHI RN-I ports (to interconnect) ---
  input  logic        chi_req_credit,   // credit from XP
  output logic        chi_req_flitv,
  output logic [63:0] chi_req_flit,     // opcode + addr + TxnID
  input  logic        chi_rsp_flitv,
  input  logic [31:0] chi_rsp_flit,     // DBIDResp / Comp
  input  logic        chi_dat_flitv,
  input  logic [255:0] chi_dat_flit,    // DAT payload
  // --- AXI4 slave port (to the non-coherent master) ---
  input  logic        ARVALID, output logic ARREADY,
  input  logic [31:0] ARADDR,
  output logic        RVALID,  input  logic RREADY,
  output logic [31:0] RDATA,   output logic RLAST
  // ... AW/W/B tied off for brevity
);
  typedef enum logic [1:0] {IDLE, REQ, WAIT, RSP} st_t;
  st_t st, nxt;
  logic [7:0] txnid;

  always_ff @(posedge clk or negedge rstn)
    if (!rstn) {st, txnid} <= '0;
    else       {st, txnid} <= {nxt, txnid + (st==REQ)};

  always_comb begin
    nxt = st; chi_req_flitv = 1'b0;  ARREADY = 1'b0;
    RVALID = 1'b0; RLAST = 1'b0;
    unique case (st)
      IDLE: if (ARVALID)                 nxt = REQ;
      REQ : if (chi_req_credit) begin
              chi_req_flitv = 1'b1;      nxt = WAIT;
            end
      WAIT: if (chi_dat_flitv)           nxt = RSP;
      RSP : begin RVALID = 1'b1; RLAST = 1'b1;
              if (RREADY)                nxt = IDLE;
            end
    endcase
  end

  assign chi_req_flit = {8'h00 /*ReadNoSnp*/,
                         ARADDR, txnid, 16'h0};
  assign RDATA        = chi_dat_flit[31:0];
endmodule

A real product RN-I adds credit counters per channel, retry on RetryAck, and a proper outstanding-txn scoreboard — but this sketch fits on one page and exposes where the work is.

• "Why does CHI exist if ACE exists?" → ACE is signal-level; scales to ~8 coherent masters. CHI is packet-based; scales to 128+ by using a home-based directory instead of broadcast snoop.
• "What are the four CHI channels?" → REQ, RSP, SNP, DAT. Each credit-flow-controlled, independently ordered.
• "What's an HN-F?" → Home Node (Fully-coherent): owns a slice of the physical address space, holds a directory + System Level Cache slice, serialises coherent transactions for that slice.
• "What is DMT?" → Direct Memory Transfer: HN-F asks SN-F to send data straight to the requester, bypassing HN-F as a relay. Saves hops on cold misses.

• "What is DCT?" → Direct Cache Transfer: on a peer hit, the snoop responder forwards data directly to the new requester instead of via HN-F.
• "Where does the System Level Cache live?" → Distributed across HN-F tiles; each tile holds a slice. Address-hashed.
• "What does MPAM do?" → Per-transaction PARTID lets the interconnect enforce SLC way allocations and memory bandwidth shares per partition (cloud tenant / VM).
• "Why CHI-E?" → Added RME / Realm attributes so CCA (confidential compute) can enforce its four-way security state end-to-end through the interconnect.