Despite all the stories about big companies bailing out of CES 2022 amidst the latest surge in COVID-19 cases, the consumer electronics show in Las Vegas is still the place to be for robots, autonomous vehicles, smart gadgets, and their inventors — an opportunity to take stock of what’s required to build practical machine intelligence into a consumer product.

A sampling of the innovations VentureBeat heard about in advance briefings:

• Efforts to improve data integration within vehicles and consolidate computing capabilities (Sonatus) and establish a common operating system or framework for automotive computing (Apex.ai).
• An autonomous driving retrofit kit for existing vehicles, with software that eschews bleeding edge AI in favor of deterministic programs and practical constraints on where the vehicle will operate (Perrone Robotics).
• Food and retail delivery robots for use within airports and for curbside delivery (Ottonomy).
• Smart driver-facing cameras for cars and trucks that sound an alert when a driver is falling asleep or not watching the road, either for industrial fleets (SmartWitness) or as a consumer safety device (Xperi) or an enhancement dashcams (NextBase), where the ability to record road rage incidents or aggressive police road stops is an additional benefit.
• A software framework for running deep neural network AI models on small, power-constrained devices (Deeplite).
• A partnership between Fluent.ai, a specialist in voice AI, and the audio tech experts at Knowles Corp., on voice command applications for compact, low-power devices.
• Computer vision packed into compact devices for the visually impaired that can read — and extract meaning from — a full page of text at a glance, as well as a hearing aid that uses visual cues like reading the lips of the person the listener is talking with at a party to know which sounds to amplify (OrCam).

OrCam and Sonatus are among the companies no longer planning to travel to Las Vegas or announce products at CES, and it’s possible some of the other vendors VentureBeat interviewed in advance of the event will also be no-shows. Big names like Microsoft, Google, Intel, Amazon, and T-Mobile backed out in recent weeks. Augmented reality, virtual reality, and the metaverse will be topics of discussion that will have to proceed without Meta (the company formerly known as Facebook). Automotive tech will be a big theme of the event, but General Motors, BMW, and Mercedes-Benz decided not to make the drive (GM’s all-digital presence is still supposed to include a video keynote from CEO Mary Barra on Wednesday). On the other hand, some like Perrone Robotics had already shipped vehicles and a test track set up, indicating their commitment

Still, the Consumer Technology Association, which sponsors the event, determined that the show must go on. Despite the big company drama, exhibiting and networking at CES remains an opportunity for “thousands of smaller companies, entrepreneurs, and innovators who have made investments in building their exhibits and are counting on CES for their business, inspiration, and future,” CTA CEO Gary Shapiro wrote in an op-ed for the Las Vegas Review-Journal.

Driving AI progress

Although CES exhibitors pitched VentureBeat on everything, including sexual wellness products, VB sought out briefings related to the uses of data and AI that readers can learn from. In particular, consumer device makers tend to want to take advantage of the cloud for software and data updates without being dependent on the cloud — with the smarts of the smart device resident on the device itself. So they’re worth studying as pioneers in edge computing.

Much as enterprise tech may have to learn from the consumer technology world, the opposite is also true. For example, Jeffrey Chou, CEO, and founder at Sonatus, said that one way for computerized systems in automobiles to improve is by learning from the model of the enterprise data center. In other words, siloed software running on lots of little computers (electronics control units or ECUs in automotive jargon) needs to be simplified and tied together with middleware, which Sonatus provides. That unification has to happen while preserving real-time performance for vehicle safety systems and addressing new concerns like vehicle cybersecurity.

“There’s no short-term fix. The long-term fix is to do software right,” Chou said.

Apex.ai has a somewhat similar story about improving the software foundations for autonomous driving and other smart vehicle tech, which, in its case, involved a series of enhancements and optimizations for ROS, the open source Robotics Operating System (considered as more of a programming framework). “There isn’t a company in automotive that doesn’t use ROS for prototyping,” CEO Jan Becker said, adding that his company’s products help with turning successful prototypes into production products.

Auto processor consolidation is paving the way for more sophisticated software, according to Becker. “The trend that we see now — that Tesla introduced that a couple of years ago and everybody else is introducing that in the next three years — is having those more powerful central more central computers, for infotainment for driver systems, potentially for a gateway potentially then also for vehicle safety functions like ESP and ABS anti-lock braking,” he said.

At the same time, Becker noted it’s been years since self-driving car enthusiasts began predicting that robot taxis would be roaming the streets any day now. “The truth is, the problem is really, really hard. What our industry has begun to understand better in the last couple of years is which applications are commercially reasonable,” he said. For example, long before fully autonomous driving becomes available and affordable in passenger cars that need to be able to go anywhere, it can be practical for commercial vehicles navigating well-known and profitable routes.

Perrone Robotics is applying that approach to autonomous commercial vehicles that can navigate freight yards or circulate through urban or campus bus routes. Although it has partnerships with electric vehicle manufacturers like GreenPower Motor Company, Perrone also sells a retrofit kit that works with the pedals, transmission, and steering wheel of conventional vehicles to render them autonomous for low-speed operation over a known route. “There’s going to be a very long path to autonomy,” CEO Paul Perrone said. “My focus is on, here’s what you can do now.”

In fact, he’s a bit of a contrarian: rather than chasing bleeding edge AI applications, he leans toward “deterministic software,” the logic of which is easier to certify as safe for the operation of a vehicle. “You can’t just train it with some probabilistic learning system that will probably get it to its destination,” he said.

Meanwhile, Ottobot is capitalizing on automotive innovations, such as lidar rangefinders, for use in its delivery robots, which began navigating the concourses at Cleveland International Airport in December 2020. Ottobot also recently announced a partnership with restaurant tech company Presto for curbside and parking lot delivery of food orders with less labor required.

While taking advantage of autonomous vehicle tech, Ottobot has innovated in other directions to allow its bots to go where many other delivery bots can’t because, like cars, they rely on GPS navigation. To work within an airport, for example, Ottobot creates a software simulation of the floor plan. “We create a digital twin and then navigate within that,” CEO Ritukar Vijay said. The arrangement of sensors also needs to be different to navigate through crowds and see glass barriers.

Scaling down

While automobiles and robots are capturing increasing attention at CES, the show is best known for showcasing smaller gadgets.

When device manufacturers talk about embedding AI, typically that doesn’t mean expecting big AI models to run on the device. A gadget may or not embed some modest machine learning capabilities, but typically the training of the model occurs in the cloud while what gets installed on the device is a much more compact inferencing model for interpreting and acting on sensor data. Even with that simplification, optimizing software to run within the size, power, and processing constraints of a given device can be a steep challenge.

For example, OrCam’s assistive technologies for the blind and visually impaired are in form factors the size of magic or a clip-on camera for a pair of glasses. So while the vice president of R&D, Oren Tadmor, respects the AI processors from companies like Nvidia, “they’re not computers that we can ever dream of fitting into our devices,” he said. Instead, the company winds up working with specialized chipsets for vision processing.

At the same time, Tamdor says, OrCam has been able to take advantage of big advances in the state of the art for deep learning as it applies to computer vision, which has made problems like face recognition much easier to solve. OrCam is an Israeli company whose cofounders Amnon Shashua and Ziv Aviram also founded Mobileye, a leader in computer vision for collision avoidance and self-driving car technology.

“For computer vision, we can do anything, or almost anything, that a person can,” Tamdor said. “And it’s just a matter of finding, what are the features that our users can use?”

Software versus hardware optimization

Hardware-specific optimizations may sometimes be necessary, but that isn’t stopping software tool makers from trying to promote a more standardized approach to device programmability. “I think one of the exciting things here is the interplay between these two types of optimizations,” said Davis Sawyer, cofounder, and chief product officer at Deeplite. “Where the two meet up, that’s where we see 400 to 500% increases over one or the other on their own.”

At CES, Deeplite announced the Deeplite Runtime software development kit for creating efficient deep learning models based on Pytorch, particularly for computer vision applications. Where the company’s previous Deeplite Neutrino product worked with GPUs and other types of processors, the new Deeplite Runtime is specifically for compiling applications to run on ARM processors, which are among the most popular on smart devices.

“Given the prevalence of things like ARM CPUs, the familiarity with developers, and also the low power profile for battery-powered devices, that’s where I think there [are] a lot of opportunit[ies] created,” Sawyer said.

Fluent.ai, a device software player focused on voice command systems, aims to be “as hardware agnostic as possible,” CEO Probal Lala said. However, some hardware partners prove to be easier to work with than others. At CES, Fluent.ai is announcing a partnership with audio tech specialist Knowles, and they’ll be jointly demoing voice-controlled earbuds.

For Knowles, the attraction is that Fluent.ai’s software operates efficiently, without being dependent on cloud services or the power and network capacity required to access them. “They offer a large command set, the largest I’ve ever seen that’s completely offline,” said Raj Senguttuvan, director of strategic marketing for audio and sensing solutions at Knowles. That opens up a wide range of entertainment and business application opportunities, he said.

Fluent’s key optimization is that it shortcuts the common voice application pattern of translating voice to text and then doing further processing on the text. Instead, the software does its pattern matching by working with the audio data directly.

For smart tech innovation, just add imagination

The increasing variety of base technologies, including AI capabilities, ought to get you thinking about business opportunities.

“I’m a big believer that the technology doesn’t mean anything to the end-user without a little imagination as to how it is going to improve their lives,” Richard Browning, chief sales and marketing officer at NextBase, a maker of car dash cams.

For NextBase, that means re-imagining how the dashcam can move beyond being just a mobile security camera you can use to share crash footage with your insurance company. Just the challenge of producing good video under conditions that can range from glaring daylight to rainy daylight is steep enough and requires some AI image processing power, Browning says. The NextBase IQ product being announced at the show and readied to ship in September takes that capability further to also provide driver assistance (recognizing when other drivers are behaving badly) and spatial awareness (anticipating accidents so they can be recorded more completely).

The addition of an inside-facing camera allows the system to detect and warn drowsy or distracted drivers, but it also allows for capturing video evidence that wouldn’t be captured by a front-facing camera such as road rage or “aggressive road stop” incidents. With a voice command, the device can be toggled into “witness” mode to record exactly how you behave when a cop walks up to the vehicle and asks for your license and insurance.

When in witness mode, whether triggered by voice command or sensors detecting that an accident has occurred, the video is transmitted to a cloud account for later review. Previous versions of NextBase’s products required the driver to manually download video data to their phone.

With these and other features, the NextBase IQ has almost outgrown the “dashcam” category as it was previously defined — except the company can’t figure out what else to call it, other than a “smart dashcam,” Browning says. “People understand what ‘smart’ is these days — they’ve got [a] smart home, smart security, smart health — it’s a product that is connected and intelligent.”

That will be a large part of what CES 2022 is about.