As AI Moves from Training to Inference, Optics Moves Closer to the Chip

Imec researchers argue that co-packaged optics will not be enough for future AI systems, pushing the industry toward 2.5D and eventually 3D optical I/O.

By Pat Brans, EE Times | July 9, 2026

For years, the AI infrastructure story has been dominated by training—bigger models, larger GPU clusters, and data centers built to absorb the computational load. But as those models move into large-scale use, inference is turning connectivity into one of the central bottlenecks in AI system design.

“Inference is the moment when users use the model,” Peter Ossieur, portfolio director at imec and professor at Ghent University, told EE Times. “So, it needs to be reactive.”

That makes inference a different problem from training. Training large models is enormously compute-intensive, but it is less directly tied to the end user’s experience of waiting for an answer. Inference must generate tokens quickly and repeatedly, often for many users at once, involving long context windows, retrieval-augmented generation, reasoning steps, multimodal inputs, and multi-agent architectures.  

Inference is not just a compute problem. It’s a memory, network, bandwidth, latency, and energy problem.

To read the full article, click here