Capturing the WORLD in a Blink: Revolutionizing the CAMERA GAME

Cameras – which are, nowadays, our eyes on the world, preserving our memories as well as capturing our moments – have been at the forefront of the technological race. But as we push forward with greater speed, so too must our machines – the ones that seek to think and react to the world like we do. But now a step-change has occurred: a new sort of camera that can detect threats in a fraction of the time and using a tiny fraction of the memory.

The Next Evolution: CAMERAS That See Like Us

This leap of the imagination can be traced back to Elon Musk’s observation that the camera is the only sense that humans share with a machine. Inspired by this, researchers at the University of Zurich have designed a camera system that brings us one step closer to the wonder of human sight. Using an array of high-definition Sony IMX490 cameras, the system is able to reduce perceived delay to levels that are truly encouraging for applications in automotive safety and beyond.

The Human Vision Challenge

To truly grasp the significance of these cameras’ leap forward, you need to grasp the concept of perceptual latency, which refers to the lapse of time between when something is detected and when it is processed. That eon, which makes all the difference between safety and disaster when you’re hurtling down the highway, can be fractions of a second, which traditional RGB cameras struggle to match in quickly changing scenes, not to mention human eyesight.

Bridging Latency Gaps with Innovation

The University of Zurich team, made up of Daniel Gehrig, Davide Scaramuzza and others, started to reduce this latency. They turned to high-speed cameras that register, although at enormous speed, fewer frames per second. But this too was causing an avalanche of data. The solution turned out to be event cameras. Unlike regular cameras whose pixels sense brightness in a rate-based manner (ie, they continuously sense brightness during the same time interval), event cameras use so-called smart pixels that fire only when there’s a change in brightness, thus massively reducing data requirements.

Hybrid Systems: When Two Worlds Collide

And the real beauty of what Gehrig and Scaramuzza achieved is that they used a hybrid system: by combining an event camera with a standard one, they circumvented the limitations of each. Event cameras pick up fast-moving objects well, but standard cameras keep static, or slow-moving, objects visible. That means the event-camera-plus-ordinary-camera combo could give reaction times an important boost to systems that need to provide a fast response to what they see, like autonomous vehicles.

Autonomous Vehicles: Steering Towards Safer Futures

Picture one of these advance cameras mounted on an autonomous car, cruising through the city streets with less latency and more efficient data processing than ever before. In the event of a hung-up pedestrian, the difference could be the reason for a close call or a crash that the pedestrian lives through. As the safety of pedestrians becomes an ethical concern and perhaps eventually a legal one, technology is merging with ethics and psychology to create a vision for a car that will not only see the world but comprehend it and react to it with the speed of the human brain.

Event CAMERAS: Peering into the Future

Importantly, the implications of these innovations go well beyond the realm of autonomous vehicles. Because it requires far less data to generate useful information than the alternatives, event cameras hold the promise of enabling entirely new classes of applications in high-speed visual processing. Any domain where it’s important to detect visual changes rapidly across a broad field of view could benefit, including video surveillance and visual quality control for manufacturing.

The Essence of CAMERAS: Origins and Evolution

As we conclude this journey through a transformative leap in camera technology, it is fitting to return to what, at their very origin, cameras were all about. From