Microarchitecture & Core Details

• RAS everywhere — ECC on L1/L2/L3, poison handling, containable aborts, SDEI for fatal-error signalling.
• Larger TLBs + TLB hierarchy — servers touch much larger working sets; N2 has ~2-3× the L2 TLB of Cortex-A710.
• Server-tuned prefetchers — favour large strides, DRAM-bandwidth-aware throttling.
• CHI-B (N2+) egress — not DSU: each core sits on a CMN mesh tile directly.
• MPAM v1 — allocate cache ways + memory bandwidth per partition ID. Mandatory for multi-tenant cloud.
• Clock/voltage optimised for 2.5-3.3 GHz sustained — not 3.5+ GHz burst. Better perf/W at scale.

• Based on Cortex-X1. 5-wide decode, 8-wide issue, deeper OoO.
• SVE: 2 × 256-bit FP/int pipes (twice the vector throughput of a 2 × 128-bit Neoverse V2). Best for dense FP kernels.
• SMT2 — two hardware threads per core. First Arm IP core with SMT.
• Larger ROB (~320 entries), richer prefetcher.
• Private L2 up to 2 MB, 8-way.
• bf16 / INT8 dot-product / matmul — v8.6-A extensions targeting ML.
• First silicon: AWS Graviton 3 (Nov 2021) — 64 cores, 7-chiplet design, DDR5-4800, PCIe Gen 5.

• Based on Cortex-A710 (mobile flagship, v9-A). 5-wide decode, 8-wide issue, 4 × 128-bit SVE2.
• Mandatory SVE2 — first Neoverse with vector-length-agnostic ops.
• Armv9-A features: RME / CCA (confidential compute), BTI, MTE.
• Private L2 up to 1 MB. CHI-B mesh egress.
• MPAM v1, SPE (statistical profiling), ETE/TRBE (embedded trace).
• Sweet spot: 64-128 cores per socket at 2.7-3.2 GHz, DDR5 / LPDDR5X, CXL 2.0.
• Canonical silicon: Microsoft Cobalt 100 (128 cores, Azure), Alibaba Yitian 710 (128 cores, 2021 early adopter), AWS Graviton 4 (96 × N2 + some V2 variants).

• Based on Cortex-X3. 6-wide decode. Biggest IPC jump in Neoverse history at the time (~35% SPECint over V1).
• SIMD: 4 × 128-bit SVE2 (not 2 × 256 like V1) — better for dispatch-limited code, matches V1 in total FLOPs.
• SMT2 retained.
• Larger ROB (~384 entries), refined TAGE predictor.
• Private L2 up to 2 MB, 8-way.
• Canonical silicon: NVIDIA Grace CPU — 72 × V2 cores per chiplet, 480 GB LPDDR5X on-package, NVLink-C2C to Hopper/Blackwell GPU at 900 GB/s coherent.

• N3 — successor to N2. ~20% IPC gain, ~25% energy reduction.
  • Ported A720-class core
  • Improved BF16/INT8 matmul path — "CPU AI inference" story
  • 4 × 128-bit SVE2 retained
• V3 — successor to V2. ~35% SPECint gain, bigger per-core resources.
  • X4-class front-end, ~384-entry ROB, wider dispatch
  • New "data-dependent prefetcher" — latency-hiding for pointer-chasing workloads (DB B-trees)
  • CCA hardware acceleration — near-zero overhead for Realms
• Delivered as CSS N3 / CSS V3 — pre-integrated with CMN S3 mesh + GIC-700 + SMMU-700.

• V1 and V2 (and V3) implement SMT2 — two hardware threads per physical core.
• Each thread has its own:
  • Architectural register file
  • Rename map
  • RSB
  • PMU counters
• Shared:
  • Execution units
  • L1/L2 caches
  • Branch predictor tables
  • TLBs
• Each thread presents as a separate PE to the OS, with its own MPIDR Aff0.

• Servers run 24/7 for years. RAS features turn "crash" into "detected, contained, reported."
• Arm RAS architecture (v8.2-A onward) defines a standard framework:
  • Error records per PE (ERXADDR_EL1, ERXSTATUS_EL1)
  • Severity levels: UEC (uncorrected, contained), UER (recoverable), UEO (outer), DE (deferred)
  • SError + SDEI signalling to EL3 / EL2 / EL1
  • Poison propagation: bad cache line carries a 1-bit "poison" until consumed
• Neoverse cores implement ECC on:
  • L1 D-cache, L2, L3 tags + data
  • TLB entries
  • RF (parity on register file)
• Memory scrubbing at CMN / memory controller level.

• Armv8.4-A optional, mandatory from Neoverse N2.
• Each memory access is tagged with a PartID (partition ID) + PMG (monitoring group ID).
• At each MPAM-aware point (L2, L3, CMN, memory controller), resources are partitioned by PartID:
  • Cache way/capacity allocation (cache-set like Intel CAT)
  • Bandwidth quotas (min / max BW per partition)
  • Monitoring counters for observed use
• OS (Linux resctrl) or hypervisor assigns PartIDs per process / VM.
• The enabler for multi-tenant cloud QoS — noisy-neighbour workloads can be capped.

// Linux MPAM (resctrl mount)
// Create a partition for tenant A
mount -t resctrl resctrl /sys/fs/resctrl

mkdir /sys/fs/resctrl/tenantA

echo "L3:0=00ff00;MB:0=50" \
     > /sys/fs/resctrl/tenantA/schemata

// L3 mask = 00ff00 → 8 ways allocated
// MB     = 50     → 50% of memory bw
// CPUs 4-7 belong to this partition
echo "4-7" > /sys/fs/resctrl/tenantA/cpus

MPAM was designed to match Intel CAT/CDP/MBA — same hyperscaler-friendly partition model, Arm-architectural instead of Intel-proprietary.

• L1 — 4-way, VIPT tag, 4-cycle hit.
• L2 — 8-way, private, 8-12 cycle hit. ECC single-bit correct, double-bit detect.
• Mesh / SLC — distributed across HN-F home nodes. Each hop ~3-5 cycles. 48-80 cycle miss-to-DRAM latency total.
• DRAM — 8-12 ch DDR5 4800/5600 typical, up to 12 ch DDR5-6000 on Graviton 4 and Cobalt 100. HBM3 on Grace.

• LDXR/STXR exclusive-monitor atomics scale poorly beyond ~16 cores — multiple exclusive locks ping-pong, livelock can ensue.
• LSE atomics (CAS, LDADD, SWP) in Neoverse are implemented as near-atomics at the HN-F: the home node for that cache line executes the RMW on behalf of the requester, without transferring the line.
• This scales to 128+ cores with near-zero tail latency — the hot counter / lock simply queues at the HN-F.
• Linux -moutline-atomics flag (glibc 2.32+) compiles binaries that pick LSE at runtime.
• On N1+, LSE cuts a mutex-heavy benchmark (Memcached, Redis SET heavy) by 20-40% versus the LDXR/STXR equivalent.

// LSE atomic vs legacy LDXR/STXR

// Legacy — can livelock
retry:
    ldxr   w1, [x0]
    add    w1, w1, #1
    stxr   w2, w1, [x0]
    cbnz   w2, retry

// LSE (Neoverse) — single op, HN-F does it
    ldadd  w3, w1, [x0]      // atomic fetch-add
// round-trip: 1 REQ → HN-F, 1 RSP back
// no need to own line in Unique state

The CHI protocol at the HN-F has dedicated atomic RMW opcodes; the home node's cache controller executes them without the requester ever taking the line.

• Typical sustained clock: 2.5 – 3.3 GHz. Cortex-X can hit 3.5+ GHz on mobile thanks to bursty workload; servers want sustained & thermally stable.
• Per-core DVFS domains (N1+). Idle cores drop to very low power.
• Graviton 3 reports ~60 W TDP for 64 cores (~1 W/core at full load). Xeon 6430 at 64 cores: ~250 W.
• Power-state hints:
  • WFI / WFE / WFIT / WFET
  • PSCI CPU_SUSPEND
  • Per-core clock gating
  • L3 way power-down on idle
• AMU (Activity Monitors v8.4-A) gives the OS a hardware view of "busy cycles" per core; Linux cpufreq uses it for better-than-nominal frequency decisions.

• Linux perf + perf record -e work on Neoverse just like x86. Core events selected by Arm standard numbers.
• Must-know events on Neoverse:
  • cpu_cycles, inst_retired
  • stall_backend_mem — memory-system stalls (CMN / LLC misses)
  • stall_frontend — I-cache or BP misses
  • br_mis_pred_retired
  • l2d_cache_refill / l3d_cache_refill
• SPE — perf record -e arm_spe// records per-instruction samples with latency + attribution.
• AmperePerf, NVIDIA NsightCS — tooling layered on top of PMU + SPE.

• "Why is V1 SVE 2×256 but V2 is 4×128?" → same total FLOPs, different dispatch shape. V1 favours long-vector HPC (BLAS); V2 favours short-vector dispatch density (cloud + DB).
• "What makes Neoverse different from Cortex-A76?" → same core plus RAS, larger TLBs, MPAM, CHI mesh egress, server-tuned prefetchers, server clock/voltage curves.
• "Why SMT on V but not N?" → HPC + DB workloads memory-stall; SMT hides that. Cloud scale-out is already parallel across cores; SMT just causes cache pressure.

• "How do LSE atomics scale better than LDXR/STXR?" → RMW executed at the home node (HN-F) in CHI; no cache line transfer to the requester. Scales to 128+ cores.
• "What is MPAM used for?" → multi-tenant cache + bandwidth QoS. Linux resctrl-style. Cloud operators stop noisy-neighbour VMs.
• "What is poison in RAS?" → when an ECC error is uncorrectable, the cache line carries a 1-bit flag. Process only gets SError if/when it reads that exact line. Contained, not fatal.
• "Graviton 2 vs 3 — main differences?" → N1 → V1 (SVE + SMT2), DDR4 → DDR5, PCIe 4 → 5, chiplet packaging, ~25-30% better SPECint.