Solution Notes
March 19, 2025
There has been a seemingly infinite amount written about generative AI as the technology has burst on the scene, effortlessly performing tasks that people formerly believed impossible for computers. While generative AI will dramatically change the nature of office work, there is a form of AI that is likely to bring even more significant changes to society. People sometimes call this “Physical AI”, AI that is applied directly to interacting with the physical world. The most visible example is self-driving cars. But equally remarkable, and equally useful physical AI systems are being developed in many business segments beyond transportation. Physical automation based on AI control is being deployed in factories, health care facilities, warehouses, agricultural facilities, and eventually will be deployed in nursing homes and private residences. Many of these deployments will be in outdoor facilities, including car lots, storage facilities, lumber yards, ports, and farms. In many cases, the work will be performed by robots, perhaps even humanoid robots. More typically, the appearance of the “robots” will not be remarkable. However, their intelligence and flexibility to adapt to the situation, whether it be an out of spec part, an unusual piece of fruit, or an injured person, will be dramatic.
Physical AI shares some of its requirements and underlying technology with generative AI. However, it has many requirements that are significantly different. In fact, the AI powerhouse NVIDIA has seen fit to create several product families, including Jetson, Deepstream, and Omniverse, just for this set of applications. Next week’s NVIDIA GTC includes no less than 163 sessions (out of 847 sessions total) related to these three physical AI product families. Central to these technologies is the notion of AI at the edge. It is easy to understand the excitement looking at the applications for physical AI, but does it really require a different set of technologies?
Let’s make the discussion a bit more concrete by looking at a specific example that Ramen Networks has experience with, a food processing plant. While the example is specific to the food industry, the activities, requirements, and infrastructure are remarkably consistent across most industries. The flow of activities can be summarised as follows:
Outdoor warehousing of raw farm products at the facility: Food products are shipped from farms to the processing plant. Many ingredients can be safely stored outside for a period of time, reducing the cost to build and cool storage facilities. While advantageous, this introduces additional challenges in tracking inventory, monitoring conditions and activity (physical security), and achieving reliable communication across significant distances to outdoor locations with no infrastructure. The solution to these problems is automated inventory tracking, AI processed video surveillance, and long range wireless communications.
Movement of the farm products to the processing lines: The raw farm products need to be moved to the start of the processing lines. This is not a trivial task due to the volumes of food these plants handle, and the around the clock nature of the operation of many of the plants. A good solution that balances flexibility with efficiency are AI enabled Automated Guided Vehicles (AGVs). In this case, these most conveniently navigate by sight using cameras. They operate in real time, depending on low latency and requiring the highest level of reliability to keep the lines running and avoid accidents and even potential injury.
Automated production lines: Food processing has long used highly automated production lines that can clean, prepare, cook, and package foods. Key requirements include monitoring for line stop, jams, overflows, etc. Supervisory Control and Data Acquisition (SCADA) is often employed, and requires reliably gathering measurements, moving them to a central location at the facility, and returning AI based or human controls and action requests to the equipment.
Hand work at kitchen slabs: Some tasks in food preparation have proven difficult to automate, so work is still performed by hand. These areas need to be monitored for food safety, quality, and productivity. Cameras followed by AI processing are ideal for this application.
Packing and stacking: Once the food preparation is done, the finished product needs to be boxed, stacked, and eventually loaded. Automated Mobile Robots (AMRs) can do this task autonomously and flexibly if they are equipped with AI. Again, reliable wireless communications that can carry video streams and provide low latency are required for full centralized monitoring and control.
While the food processing flow is completed, the processing plant operation is not. The following additional tasks need to be supported as well:
Monitoring of production equipment: The production equipment needs to be monitored for malfunctions or pending maintenance. This is additionally complicated for equipment that can move such as AGVs and AMRs. Sensors such as temperature and vibration should also be augmented with AI powered video based observation to catch unusual forms of failure that might not be detected by traditional sensors.
Overall equipment efficiency calculation (OEE): It is the overall efficiency of the processing line, and its utilization of expensive assets and labor, that determines profit and loss for the owner. Sophisticated AI algorithms have been developed for assessing OEE. What is required is gathering video from throughout the facility in order to perform this analysis.
Ramen Networks has built systems like this, and has expertise in both the networking components and the AI components. The approaches described in the rest of the paper are based on real world learnings.
With a concrete example of an “uncarpeted enterprise” in mind, the importance of AI at the edge becomes clear. AI at the edge has a number of advantages that are required to make the level of automation described in the example practical.
The reliability requirements in an automated factory are extremely high. A single failing piece of equipment can stop the entire production line, and line stoppages burn money quickly. People reading this article may have a distorted view of the reliability and capability of internet access due to being in an urban environment. Many production facilities, whether factories or food processing plants are in rural environments. Studies have shown that internet reliability and capabilities are significantly worse in rural areas. For example, Scout Nelson of Nebraska Ag Connection reports that while only 11% of urban residents report their internet connection as “somewhat to not dependable”, 22% of those in open country, and 28% on farms report their internet connection is somewhat to not dependable.1
Edge AI, coupled with the right local infrastructure, has a number of advantages regarding reliability.
Although not all the applications described in our example food processing facility require low latency, many of them do. Robots and automated vehicles require real time decisions and actions, measured on the order of milliseconds. Communicating through the internet inherently accumulates delay, on top of whatever delay already exists in the local network. Even if the cloud AI processing were instantaneous (which it is not), the communication delays alone are problematic. And, to paraphrase a number of common sayings, it is not the latency that kills, it is the variance of the latency.
The following graph, taken from a paper published by researchers at the Stevens Institute of Technology and the University of Nevada2, compares the measured latency observed with Edge services vs. five different cloud service providers.
If we look at the median (50th percentile) difference in latency we see approximately 30-9 = 21ms lower latency for local services. However, if we look at the 90th percentile measurement, the difference becomes approximately 100-23 = 77ms lower latency for local services. Keep in mind that these latencies do not include processing or other delays, and 90th percentile is nowhere near the reliability point required for these applications.
As with reliability, edge services provide the owner of the facility much more control and consistency regarding latency. A dramatic example of this can be seen in the following graph from researchers at Tianjin University3 which shows the measured latency accessing the cloud vs. time of day.
The figure shows the substantial increase in latency in the evening hours, presumably due to the high load placed on the internet by streaming video and other recreational web uses. Keep in mind that this graph shows only average latencies. As the average latency goes up, the variance in latency also inherently rises, making the difference in worst case latency between daytime and evening even greater.
This problem can be mitigated if edge AI processing is used. In that case the load on the network can be known, and will be much more consistent versus time of day and day of week. Network planning can ensure sufficient capacity, reliability, and thereby consistently low latency as required by the real time applications being used.
Cloud cost is a real concern when applying AI across an entire facility for AI based automation. A key element for all the activities described in the food processing facility was video capture. Whether it is the robots themselves, or the various monitors, AI provides the tools to use video to conveniently perform complex tasks. In a way, this should be of no surprise. Human beings perform nearly all of our everyday tasks on the basis of vision, and AI can now do the same.
However, high resolution video is inherently high bit rate, and with so many cameras it adds up. Our example facility included seven major tasks. It is not unreasonable to imagine an average of 10 cameras for each of these task categories throughout a large facility, for a total of around 70 video streams in a facility. Even with a modest video data rate of 4Mb/s, the aggregate bandwidth required is 280Mb/s.
Such a high aggregate bandwidth, flowing continuously 24/7, can create two cost problems if it is moved to the cloud for analysis. First, internet service, to a potentially remote site, supporting such high bandwidth will be expensive. The high bandwidth, together with 24/7 continuous operation may even trigger extra data allowance fees. Second, cloud providers usually charge for data ingestion (data transfer), putting an additional per byte fee from the cloud provider on top of ISP charges. The end result is the cloud processing of all these data streams can be very expensive, considering just the data transfer costs alone.
Edge AI eliminates these internet and cloud communication expenses. Keep in mind that all the data would need to flow through the local network when processed in the cloud, so local communications infrastructure requirements are similar or less when performing AI on the edge vs in the cloud.
Cost for the compute power to perform AI is more nuanced between edge and cloud. Cloud compute centers are remarkably efficient, and competition has driven reasonable pricing. However, edge AI compute has some advantages. Just as there are often fundamental financial advantages to owning rather than renting, owning local processing resources may prove to be more economical long term than continuously paying for shared resources. And, in many circumstances, there is some amount of excess compute available in the local equipment that can be applied to the AI processing.
Along with advantages in reliability, latency, and cost, edge AI provides additional benefits:
Control: Edge AI provides the owner with much greater control over a wide range of aspects:
Distributed architecture: AI at the edge is inherently a more distributed architecture, often with computing at the location of each sensor or device. Edge AI does a better job of preserving the independence of functions and distributing network and compute loads.
Better security and trust: With edge AI, data does not need to travel over public networks or be processed in shared compute resource environments.
Of course, the security of the local network, local data, and local processing elements need to be ensured for the edge AI approach to provide its advantages in security and trust. For example, Ramen Networks includes a suite of security related features, including built in protection against cyber-attacks and ransomware, end-to-end encryption and authentication, and the ability to generate security audits and compliance reports including PCI & HIPAA. All of Ramen’s deployments are based on a “zero trust network access” philosophy. This level of protection of the local resources is equally important whether using cloud or edge AI.
With the previous background, it becomes fairly clear some of the properties of a good automation system. First, AI at the Edge should play a significant role to increase reliability, reduce latency, decrease cost, improve control, and enhance security and trust.
It is clear that a reliable and high capacity communication system is required to form the backbone of the system. The communication system should have the following properties:
Currently there are two wireless technologies that can play an effective role in meeting these requirements. WiFi is inexpensive and supports high data rates and high overall aggregate capacity. However, it can struggle with latency and achieving sufficient range and coverage. Cellular technology (LTE) has recently become affordable through the notion of “private networks.” These are small cellular systems that are owned by the facility, and utilize free spectrum in the CBRS band. Cellular technology is particularly strong in providing low latency, long range, and robust coverage. Almost magically, the two technologies complement each other. A hybrid system that utilizes both is therefore the best bet for meeting all of the requirements for an automation system.
The downside to a hybrid system is more complexity and management issues, and potential problems for devices roaming from one network to another. But this can be overcome with the right tools. For example, Ramen Networks unifies the management and control of their cellular and WiFi networks, so they look like one seamless network. Ramen also ensures that devices will operate seamlessly across both networks as desired to achieve the advantages of both networks.
In addition to the network itself, it is helpful to architect the system with a centralized control point. Ramen Networks calls this an “EdgeGateway.” This component consists of containerised mobile core services, security and business policy enforcement points, and the ability to segment traffic by flows and VLANs. The EdgeGateway is a key element in managing the data locally.
Beyond selecting the communications technology, a facility owner has options in how to buy it. A traditional method is to purchase the gear up front, then create the necessary IT department to run it. A new option has become popular in the last few years, purchasing the network as a service. Commonly called Networking as a Service, and often abbreviated as NaaS, the owner contracts with a provider who will then install and operate the network for a monthly fee. This provides a number of benefits:
The economics of a NaaS system can be quite attractive. While saving the large upfront investment, it still provides a flat rate (non-metered) service. The cost will be the same independent of how much data is transferred, critical in this type of facility due to the huge amounts of data being transferred.
Finally, edge AI software and hardware will be required, particularly for processing video streams from cameras and robots. Chip companies, such as Nvidia, Qualcomm, and Broadcom, have created a wide range of products oriented to these applications. Nvidia has organized these into families, and some of the families to consider include:
Similar to the networking technology, it may be possible to purchase the edge AI capability as a service, perhaps even from the NaaS vendor. Ramen Networks can supply both the networking solution and the AI infrastructure based on Nvidia Jetson (hardware platform) and DeepStream (software SDK) technology. Along with the NaaS advantages listed previously, obtaining the edge AI and communications from the same source simplifies the purchase, and ensures that the compute and communications infrastructure will work seamlessly. Vertical integration with the network stack provides coordinated operation, and the potential for using the AI capability in support of the networking system. For example, Ramen Networks uses the available AI processing to perform anomaly detection for cyber security.
The AI as a Service (AIaaS) vendor may provide application specific AI software, or it can be obtained from system integrators that are familiar with a particular application space. Whatever software system is chosen, it is important that it includes infrastructure to manage software at the edge, including the ability to securely do monitoring and SW upgrades at the site from remote. Flexibility is also an important property, including support for many different types of sensors, and an edge stack that can host applications coming from multiple vendors.
Physical AI is an attractive technology to help automate a wide range of facilities including factories, warehouses, agricultural facilities, and health care facilities. Physical AI hardware and software is now readily available automating a wide range of tasks including robotic transport and assembly, automated alerting of production line issues, and sophisticated inventory monitoring and management. Edge AI, as compared to cloud based AI, offers the advantages of increased reliability, reduced latency, decreased cost, improved control, and enhanced security and trust. Key to automating such facilities, whether based on edge AI or cloud AI, is the local communications infrastructure. A hybrid mix of WiFi and Cellular wireless networking is the best solution to meet all the requirements: long range robust coverage, low latency, high throughput, and high overall capacity. A convenient way to obtain such a system is through a Network as a Service (NaaS) vendor, enabling low upfront costs, experienced planning and installation, effortless maintenance and operation, 24/7 support, and professional grade security.
______________________________________________________________
1“Nebraska Internet Reliability - Rural vs Urban,” Scout Nelson, Nebraska Ag Connection, https://nebraskaagconnection.com/news/nebraska-internet-reliability-rural-vs-urban
2“Latency Comparison of Cloud Datacenters and Edge Servers,” Batyr Chryyev, Engin Arslan, Mehmet Hadi Gunes, Stevens Institute of Technology, University of Nevada, Reno, https://par.nsf.gov/servlets/purl/10184999.
3“How Far Have Edge Clouds Gone? A Spatial-Temporal Analysis of Edge Network Latency in the Wild”, Heng Zhang, et al., Tianjin University, Tianjin, China, https://xumengwei.github.io/files/IWQoS23-edge-latency.pdf. .
To view the video, please fill in the following information.
To read this blog, please fill in the following information.
Explore all of the ways Ramen can benefit your business. Reach out to learn more.
Contact Us
.svg)
