Raspberry Pi AI Camera, the intelligent eye developed by Sony

The Raspberry Pi now has an AI-boosted eye. If the famous micro-PC already benefits from numerous classic camera modules, Sony's semiconductor division (SSS) has just announced the launch of a module integrating a slightly different sensor.

Called IMX500, this 12 Mpx sensor launched in 2020 represented a huge novelty: it integrated not only memory (stacked sensor) but also and above all a processor capable of carrying out inference operations. In mainstream jargon, it's called an AI processor, DSP, or accelerator (yes, they're usually the same thing).

And that comes at the right time for the Raspberry Pi, since it is not equipped with one. And that its CPU and CPU (integrated into its SoC) do not offer very good performance in AI uses.

If it had a first life as a component purchasable individually, this IMX500 is therefore integrated into a complete module comprising not only an optic, but also the Raspberry Pi foundation's in-house microcontroller, the RP2040. But then, is it a Sony or Raspberry product? Both.

If Sony is so involved in a product which will be sold under the official flag of “Raspberry Pi AI Camera”, it is because the Japanese firm now has minority shares in Raspberry Pi Ltd, the commercial branch backed by the foundation which bears the project. Sony already manufactured the Raspberry Pi in its factories – that of Wales for the European market – and the Japanese are now second in terms of development.

A complete solution

Concretely, the Raspberry Pi AI camera consists of an optic covering 76° of field of vision (roughly: a 28 mm), the IMX500 sensor (12 Mpx) with its neural accelerator and its memory as well as an RP2040 which makes it possible to manage loading/changing the sensor firmware.

But not only has Sony integrated its sensor into a finished product, but it has also been developing AI chip development and control tools for four years – a platform called AITRIOS. After launching its sensor as a development platform, Sony Semiconductor Solutions now has a mass product capable of taking advantage of its components.

The question is obviously what can such a sensor be used for? To many things. From a technological point of view, the chip integrated into the sensor can carry out image classification, object or pose detection, as well as semantic segmentation (cutting an image into sky, subject, house, etc. into real time). All this, without overloading the processor responsible for controlling it – here the Raspberry Pi.

In terms of uses, we can cite robotics, surveillance, autonomous piloting of small vehicles, etc. A pilot project was launched in Rome, where cameras detect free parking spaces in real time.

This type of sensor has many advantages both in terms of energy and network discharge: no need to output the bitmap images from the sensor to analyze them. Only a small volume of already processed data comes out, which frees up bandwidth on a network. In addition to a potential latency advantage, this type of sensor can also offer a privacy advantage.

If it is configured not to transmit pixelated images, it can execute position recognition algorithms (a body which falls and remains inert for example) and transmit an alert without ever sharing a compromising, shocking image, etc.

The Raspberry Pi AI Camera is already available from official resellers at €80.40 including tax.