Ambarella unveiled two new AI chip families for 4K security cameras today as it pushes further into computer vision.

The Santa Clara, California-based company is introducing its new CV5S and CV52S security chips as the latest in its system-on-chip portfolio based on the CVflow architecture. The chips use an advanced 5-nanometer manufacturing process, where the width between circuits is five-billionths of a meter.

With the combination of new designs and better miniaturization from the manufacturing process, the SoCs can support simultaneous 4K encoding and advanced AI processing in a single low-power design, which provides great AI SoC performance per watt of power consumed, said Fermi Wang, CEO of Ambarella, in an interview with VentureBeat.

“If you have a camera, you want to cover a very big space, and then you want to use a 4K camera to cover all 360 degrees,” Wang said. “You want to have a single chip to talk to the 4K sensors. You want to process all of the videos together and analyze the video.”

The CV5S family targets security camera applications that require multiple sensors for 360-degree coverage, over a wide area and with a long range, such as outdoor city environments or large buildings. Ambarella designed the CV52S family for single-sensor security cameras with advanced AI performance that need to more clearly identify individuals or objects in a scene, including faces and license plate numbers over long distances, such as ITS traffic cameras.

Camera applications

The cameras can be used for detecting the faces of criminals in crowds or recognizing license plates. There are of course privacy concerns about that.

To address that, Ambarella said it is providing customers with several ways to address privacy, using CV5S and CV52S security families of edge AI vision SoCs. First, these new SoCs provide the performance required to do AI on-camera, which is inherently more private than streaming the video to a server for analysis. Less data transmission also means fewer points of entry for a hacker trying to access the video. With the Ambarella CV5S and CV52S SoCs, the AI analysis can be performed on the camera and then the video can be discarded, with just a signature or metadata of the face being used to find a match. No actual video needs to be stored or transmitted, which ensures total privacy.

In addition, these SoCs contain a very secure hardware cybersecurity block, which includes one-time-programmable (OTP) memory, Arm® TrustZones, DRAM scrambling and I/O virtualization. These features of our on-chip hardware cybersecurity block make it very difficult for a hacker to replace the firmware on the camera, providing another level of security and privacy at the system level.

Another privacy feature that the new SoCs provide is called privacy masking. This feature enables portions of the video (for example, a door or a window) to be blocked out before being encoded in the video stream. The blocked portions of the scene are not present in the recorded video, thus providing a privacy option for cameras that are facing private areas.

The cameras can also send an alert if a crowd is forming in a part of a city, and they can monitor packages left behind in stations, airports, and more. Applications also include managing traffic congestion, detecting vehicle accidents, automating speed control, locating missing or stolen vehicles, monitoring queues in retail environments, better managing retail product placement, enhancing warehouse tracking — generally providing more actionable intelligence at both the store and corporate levels. And these more invasive applications can be countered with the ability to set up privacy masks, like continuously obscuring certain portions of larger scenes, preserving privacy.

“We are generating revenue based on all of the CVflow family today,” Wang said. “If you look at all the possible applications that we can address here — our current security camera market, automotive markets — you can see that there’s a lot more opportunities. We’re talking about smart homes, smart cities, smart retail, and also in the future robotics. There are many, many applications that we can address moving forward.”

John Lorenz, senior technology and market analyst at Yole Développement, said in a statement that security system designers want higher resolution cameras, more channels, and faster AI. He said Ambarella’s new chips are competitive in the security chip market, which is expected to exceed $4 billion by 2025, with two-thirds of that being chips with AI capabilities.

The new CV5S SoC family supports multi-imager camera designs and can simultaneously process and encode four imager channels of up to 8 megapixels (MP), or 4K resolution, each at 30 frames per second, while performing advanced AI on each 4K imager. These SoCs double the encoding resolution and memory bandwidth while consuming 30% less power than Ambarella’s prior generation.

The new CV52S SoC family targets single-sensor security cameras and supports 4K resolution at 60fps, while providing four times the AI computer vision performance, two times the central processing unit (CPU) performance, and 50% more memory bandwidth than its predecessors. This increase in neural network (NN) performance enables more AI processing to be performed at the edge, instead of in the cloud.

“Because you do all the video analytics at the edge, the full video doesn’t need to leave the camera,” Wang said. “You only pass the data that you analyze along to the cloud.”

That’s important as you don’t want traffic from self-driving cars to clog up the wireless bandwidth connections to datacenters.

“The biggest difference approach is that we talked about in the past, we call it ‘algorithm first.’ Basically, when we do the video compression or video processing, or image processing, even and then the controller computer vision for the deep neural network or AI processor, we can see the flow, we try to determine the hardware architecture as well,” Wang said. “For the architecture, we look at what kind of algorithm we want to implement first. So then, after we go through all the areas of study, we understand how an application works, and the portion of the area that takes the most computation performance, and where we can optimize without losing the performance or accuracy of the algorithm.”

He added, “And after going through all the tradeoffs, we create the architecture not only try to deliver the best performance, but also deliver the best power consumption as well. I think we are very differentiated and can compete.”

In addition to security, there are many other AI-based internet of things (IoT) applications that can take advantage of the high resolution and advanced AI processing provided by these new SoC families. For example, smart cities can leverage high edge AI performance and image resolution for improved traffic management, accident detection, and automated speed control, as well as the rapid location of missing and stolen vehicles.

“If there’s an accident where there is traffic congestion, or if you need to find a missing vehicle, then you can get enough information to monitor all of the smart city requirements,” Wang said. “You need to do real-time management.”

Likewise, smart retail operations can use this resolution and advanced AI to better manage product placement, adjust cashier staffing for real-time line management, enhance warehouse product tracking, and provide more actionable intelligence at both the store and corporate levels.

The chip families share features such as a software development kit (SDK) for the security camera market, CVflow development tools, and dual Arm A76 1.6GHz CPUs with 1MB of L3 cache memory and a two times performance gain over prior generations. It also has enhanced image signal processing with high-dynamic range, ISO low-light, dewarping, and rotation performance.

They also have on-chip privacy masking to block out a portion of the captured scene, connector interfaces, on-chip cybersecurity hardware with secure boot, OTP, and Arm TrustZone technology. They can support up to 14 cameras and a variety of memory.

The CV5S and CV52S SoC families are expected to be available for sampling in October.

Origins

Ambarella was founded in 2005. It has evolved over the years from a video processor chip design firm to a designer of computer vision chips for a variety of markets. It started with video processors for cameras and video cameras. Then it transitioned to making AI-based chips for automobiles and security cameras.

“We went through many different markets, some good, some bad,” Wang said. “We did chips for camcorders and GoPro sports cameras, DJI drone cameras, and eventually there was no innovation in these markets.”

The markets that have lasted longer include security cameras, which require ever-increasing levels of resolution and quality, and automotive cameras, which started in 2011 and continue today as cars need more video sensors and AI processing to distinguish driving hazards.

“Over time, we believes that video analytics will become very important, where you can interpret what the computer vision shows,” Wang said.

Deep learning neural networks are necessary for that work, and that has put a lot of pressure on better AI processing while at the same time delivering better efficiency with low power consumption and lower costs.

“We start our CVflow family with 10-nanometer production, and today we go to five nanometers,” Wang said. “And also we build tons of different software, including a tool to convert any neural network designed by our customers.”

In the past several years, Ambarella has spent $500 million on research and development on computer vision, and it reported last year’s revenue for the segment at $25 million, Wang said. Analysts are expecting the company to hit $75 million in computer vision revenue in 2021.

“It’s not only just a product anymore,” Wang said. “It’s really a revenue generator for our customers. We proved that the investment was really important, and I’m glad we went through that.”

The company has about 40 mass-production customers now. And they are asking for better and better performance.

“If you use 8K performance to process multiple video streams at the same time, then it becomes a mainstream product,” Wang said. “In fact, I can say that in security cameras, people want to connect four to six to eight cameras to one single chip.”