Optical Matrix Multiply: Precision and Conversions

Topics We'll Cover

• The Operation and Its Reuse
• Encoding Values in Light
• Storing the Weights
• Noise and Precision
• The Photon Floor
• Interactive: Energy per MAC
• Why Converters Dominate
• Static Power and Utilisation
• Scaling the Array
• Calibration and Drift
• The Digital Yardstick
• What to Take Away

The Operation and Its Reuse

A matrix–vector product y = Wx with an N × N matrix is N² multiply–accumulates (MACs). Done optically, it needs N input values converted into light (DAC samples), N outputs converted back (ADC samples), and the N² weights held in the optics. Everything in between is free.

• Reuse is the whole argument. Each converted input is used by N MACs, and each converted output collects N. A 64 × 64 engine shares every conversion between 32 MACs on average; a 1,024 × 1,024 one between 512. The converter energy per MAC falls as 1/N.
• The same scaling at model level: Anderson et al. find that the optical energy per MAC of an optical Transformer scales as 1/d with the model width d (arXiv:2302.10360; TMLR 2024), the root of their projections in deck 03.
• Weights must stay put. If W changes for every vector, each of its N² values needs a conversion too, and reuse disappears. That is the line between weight-times-activation products (static weights) and activation-times-activation ones such as attention scores (weight-masked and data-by-data products).

Encoding Values in Light

• Every encoding is analogue. A value is a voltage on a modulator and a current from a detector; its precision is whatever the noise allows, not a number of stored bits (slide 04).
• The modulators that write inputs are those of OptSiP 02: Mach–Zehnder and ring modulators, at GHz rates and femtojoules to picojoules per sample including the driver.

Storing the Weights

• Heaters are the default and the problem: an N × N mesh holds about N² phases, each drawing a fraction of a π-shift's power whether or not it computes. That power is paid per MAC at the vector rate (slide 08).
• Non-volatile weights remove static power but offer few levels and slow, energetic writes, so they suit models whose weights never change in service.

Noise and Precision

An optical matrix multiplier's output carries noise from many sources: detector shot noise and thermal noise, laser intensity noise, crosstalk between channels, phase and splitting errors (OptAI 01's interactive), drift, and the converters' quantisation. Together they set an effective number of bits (ENOB) for the whole chain.

• Floating-point workloads need a noise level, not exactness: matching INT8's rounding error needs about 8 ENOB and BF16's about 10, by the rule worked through in FOptInf 01, slide 09 (the noise rule; crest factor).
• Where photonics is expected to win: Nahmias et al. conclude that photonic MACs have their advantages “in the limit of large processor sizes (>100 µm), large vector sizes (N > 500), and low noise precision (≤4 bits)”, with potential gains over digital electronics in energy (>10²), speed (>10³) and compute density (>10²) (IEEE JSTQE 26, 2020).
• Precision can be bought per layer: repeating a pass and averaging buys half a bit per doubling (dynamic precision by repeated passes). Garg et al. choose precision layer by layer and report energy cuts of up to 89% for ResNet-50 and 24% for BERT (arXiv:2102.06365).

The Photon Floor

Even a perfect detector counts photons, and photon counts fluctuate: with n photons, the relative noise is 1/√n. To make that noise no larger than the quantisation noise of an E-bit converter on a full-scale signal (2^−E·2/√12), a detector needs about n = 3 · 4^E photons per sample. The interactive uses this as its laser floor.

Arithmetic: n = 3 · 4^E; energy n·hν, with hν = 1.28 × 10⁻¹⁹ J at 1550 nm; before loss and laser efficiency, and shared by the N MACs that feed one detector.

• Each extra bit quadruples the photons. Low-precision networks therefore have a very low optical floor: Hamerly et al.'s theoretical limit is as low as 50 zJ per MAC (Phys. Rev. X 9, 021032, 2019), and Wang et al. reached about 90% digit accuracy at about 0.66 photons per multiplication and about 99% at about 3.1 (Nature Commun. 13, 123, 2022).
• Those experiments work below one photon per multiplication because each detection sums many multiplications, and because a classifier tolerates noisy activations. A layer that needs 8 bits at every output cannot go that low.

Interactive: Energy per MAC

Energy per MAC of an optical N × N matrix–vector engine, term by term: input DACs and output ADCs at Walden FoM × 2^ENOB each, shared by N MACs (Walden figure of merit); weight loads if the weights change; the laser at the photon floor of slide 05, raised by optical loss and divided by wall-plug efficiency; and static power per weight cell at the vector rate. The converter FoMs, laser efficiency, loss and static power are illustrative defaults; the digital reference default is Horowitz's 8-bit multiply plus add at 45 nm (slide 11).

Limits: no memory or data-movement energy (often the largest term in a real system, deck 03); no modulator drivers, TIAs or nonlinearities beyond what the converter FoMs absorb; the laser sits exactly at the shot-noise floor, which real systems exceed. It is a lower bound for the optical side and a fair yardstick for what conversion costs.

Why Converters Dominate

Converter energy per sample is roughly the Walden figure of merit times 2^ENOB, so each extra bit doubles it (conversion-bound optics, with the table in FOptInf 01, slide 11).

• What converters achieve: the best Walden figures of merit in Murmann's ADC survey are below a femtojoule per conversion step, at low sample rates (0.4 fJ was the best reported in 2019, per a survey of SAR converters); at the GHz rates an optical engine wants, the survey's frontier is about 11 fJ at 10 GS/s, 23 fJ at 20 GS/s and 70 fJ at 60 GS/s (as read from the survey by an Analog Devices article; survey at github.com/bmurmann/ADC-survey).
• System models agree: Andrulis et al.'s architecture-level model of photonic accelerators finds that “cross-domain conversion and DRAM can consume a significant portion of photonic system energy” (arXiv:2405.07266).
• The optics is the cheap part. The interactive's defaults put the laser at the photon floor well below the converters at 8 bits; at 12 bits both grow by 16× (converters) and 256× (photons), and the advantage over digital closes.

Static Power and Utilisation

• Static power is paid per weight per vector. A weight cell drawing P whatever it does costs P/f per use at vector rate f: 1 mW at 1 GHz is 1 pJ per MAC, already more than the digital 8-bit reference of slide 11. Heaters must therefore draw far less than a milliwatt each, or weights must be non-volatile (slide 03).
• Lasers burn power whether or not work arrives, as do heaters, bias circuits and converters left running (power in photonic compute; static and dynamic power). An engine idle half the time doubles its static energy per useful MAC.
• Utilisation needs a workload that keeps the matrix full: large, static weight matrices, batched inputs, and a memory system that delivers them at the vector rate (deck 03, slide 05).

Scaling the Array

• The tension: reuse wants N large (slide 01) and Nahmias et al. put the advantage above N ≈ 500 (slide 04), while loss, error and footprint of integrated arrays grow with N. Lightmatter's 2025 system used four 128 × 128 tensor cores in one package (deck 04), a sign of where integrated arrays stand.
• Tiling a large matrix onto a smaller array works, but each tile pays its own conversions, so the effective N for reuse is the array's, not the model's.
• Loss costs laser power exponentially: every 3 dB doubles it (the interactive's loss slider), the same arithmetic as the link budget of OptSiP 03.

Calibration and Drift

• No two devices are alike: nanometre variations shift every phase (fabrication variation), so meshes are characterised and calibrated after fabrication. Miller showed that a self-configuring mesh can absorb split ratios anywhere from 85:15 to 15:85 using only external sources and detectors (Optica 2, 747, 2015); Pai et al. showed how to initialise mesh programming so it converges (Phys. Rev. Applied 11, 064044, 2019).
• Temperature moves everything: silicon's index changes by 1.8 × 10⁻⁴ per kelvin (thermo-optic effect), so calibration must be held against heat from the chip itself, with monitors and control loops that cost power (thermal tuning).
• Training can absorb what calibration misses: in-situ and physics-aware training (OptAI 01, slide 11).
• Programmable photonic circuits in general, and their control, are reviewed by Bogaerts et al. (Nature 586, 207, 2020).

The Digital Yardstick

Source: Horowitz, “Computing's energy problem (and what we can do about it)” (ISSCC 2014).

• An 8-bit MAC at 45 nm is about 0.23 pJ (multiply plus add): the interactive's default reference. Newer nodes are lower; the reference to beat moves every generation.
• A modern accelerator, at board level: an H100 SXM is specified at 1,979 dense FP8 teraFLOPS (3,958 with sparsity) at up to 700 W (NVIDIA; vendor figures), which works out at about 0.7 pJ per MAC including everything on the board at peak. Anderson et al. use 300 fJ/MAC as their state-of-the-art digital reference.
• Memory dwarfs arithmetic: one DRAM access costs thousands of 8-bit MACs. An optical engine that makes MACs cheaper but still fetches its weights from DRAM saves little (deck 03, slide 05).

What to Take Away

• An N × N optical product shares each conversion between N MACs; reuse, and therefore large N and static weights, is the whole energy argument.
• Precision is a noise level, set by the whole analogue chain: about 8 ENOB for INT8-like error. Analyses place the photonic advantage at large N (> 500) and low precision (≤ 4 bits).
• The photon floor is small (about 25 fJ per detection at 8 bits, shared by N MACs) but quadruples per bit; converters dominate at Walden FoM × 2^ENOB per sample.
• Static power kills unless weights are non-volatile or heaters draw far under a milliwatt: 1 mW per weight at 1 GHz is 1 pJ per MAC.
• The yardstick: about 0.23 pJ per 8-bit MAC at 45 nm, less at newer nodes; DRAM access costs thousands of MACs, so memory, not arithmetic, often decides.
• Next: deck 03 asks where, given all this, optics can win.