MANOUCHEHR RAFIE, Ph.D., Vice President of Advanced Technologies at Gyrfalcon Technology Inc. (GTI).
Nearly three quarters of all smartphones sold this year will be powered by AI and by 2025 it will rise to 100%, according to Strategy Analytics. Our phones use AI to optimize performance and provide virtual assistants, but perhaps the most intriguing use of AI is for camera sensors, which can make our smartphones “aware” of their surroundings. The industry understands the value of AI, edge computing, and smartphones, and chip vendors are making these their highest priority.
Smartphones are integrating AI-powered vision computing algorithms into their camera modules to address both human and computer-vision functional requirements. AI-powered cameras at the edge enable smartphone, automotive, computing, industrial, and IoT devices to redefine the way they process, restore, enhance, analyze, search, and share video and images. On-device integrated AI-camera sensor co-processor chips with their built-in high-processing power and memory allow the machine- and human-vision applications to operate much faster, more energy-efficiently, cost-effectively, and securely without sending any data to remote servers.
To execute computer vision algorithms on a camera module such as object detection, segmentation, or facial recognition, a separate (either on- or off-chip) vision processor is also required. Intelligent image sensor processors usually are assisted by both an image signal processing (ISP) capability as well as a neural network accelerator often with a deep learning-based Convolutional Neural Network (CNN) engine. Camera modules with a combined deep learning-based CNN engine and an ISP-like processor can be deployed in a variety of vision-based systems offering both enhanced viewing and computer-vision capabilities.
Over the past few years, mobile cameras in smartphones, surveillance devices, and robotic vehicles, including autonomous cars, have benefited from the integration of AI and ISP engines. Machine learning (ML) is used not only to enhance the quality of the video/images captured by cameras, but also to understand video contents like a human can detect, recognize and classify objects, events, and even actions in a frame. The application of this technology is massive, enabling everything from vision-based end-point devices to smartphones, automotive, robotic vehicles, automated surveillance, and beyond.
Read the full article in the December 2021 issue of Semiconductor Digest.