In the rapidly evolving field of artificial intelligence, Computer Vision stands out as a transformative technology with the potential to revolutionize various industries. This article clarifies several aspects of Computer Vision’s potential, focusing on the benefits and challenges of processing data close to the source. To gain deeper insights, Torsten Reidt, AI Engineer of the Cactai team, shares his expertise on the subject.
In recent years, one of the main trends in the development of Artificial Intelligence (AI) can be summarized as “the bigger, the better”. Models increased in size (parameter count) due to the availability of massive amounts of data, the development of specialized hardware such as GPUs and TPUs and the availability of computing resources. Recently, we have observed a different, very interesting trend: smaller models requiring less computational power allow for inference “at the edge”.
This blog will explain what this means when applied to computer vision. We will also briefly overview the differences between AI in the cloud and AI at the edge. To refresh our memory, computer vision is a sub-field of machine learning (ML) that focuses on interpreting and understanding information from image data, such as detecting a car in an image taken by a traffic surveillance camera.
Training or Inference
Training and inference are two critical phases in the lifecycle of an AI model. The choice of where to perform these phases—whether on the cloud or at the edge—depends on various factors such as computational requirements, data privacy, latency needs, and infrastructure costs. While Training (feeding large amounts of data into a machine learning model and adjusting the model parameters to improve its accuracy) is computationally intensive, it requires significant processing power, memory, and storage. Therefore, it is often done in the cloud.
Inference is the phase where the trained model makes predictions or decisions based on new data. This phase is less resource-intensive than training but still requires efficient processing. In this case, Edge AI is a plausible option for enabling a large number of applications.
AI at the edge vs AI at the cloud
In general terms, “cloud“ and “at the edge” or “Edge AI” refer to where the AI models are deployed and executed. An example of an edge device would be an industrial inspection camera. These cameras usually detect specific features in products, processing the visual data in real-time to enable quick decision-making in a production environment.
Edge AI is often preferable for applications requiring immediate responses, while cloud AI is suitable for less time-sensitive tasks. On top of that, Edge AI is advantageous when handling sensitive data, as it keeps data local.
Now that we have clarified the term “at the edge”, we compare AI at the edge with AI in the cloud for some characteristics to distil the idea of “at the edge” even further:
| Characteristics | At the edge | Cloud Deployment |
| Processing | Real-time processing on local devices | Centralized processing in cloud data centers |
| Latency | Milliseconds to seconds | Seconds to minutes (depending on network connectivity) |
| Bandwidth | Minimal data transmission required | Large amounts of data transmitted to the cloud |
| Power Consumption | Optimized for low power consumption | High power consumption in cloud data centers |
| Security / Privacy | Data processed locally, reducing risk of data breaches | Data transmitted to the cloud, increasing risk of data breaches |
| Maintenance | Device-level maintenance and updates required | Centralized maintenance and updates in the cloud |
| Offline Capability | Can operate offline or with intermittent connectivity | Requires constant internet connectivity |
Advantages and disadvantages of AI at the edge
What are the reasons for deploying a vision AI system at the edge and what are the limitations or challenges one might encounter doing so? We have already seen the differences between AI at the edge and AI in the cloud, let’s highlight some advantages of AI at the edge:
- Real-time Processing: By processing the image data closer to the source, data transmission times are reduced which reduces overall latency.
- Increased Autonomy: Devices with AI systems can operate independently, even in the absence of a network connection, making them suitable for remote or offline applications.
Doing inference at the edge implies on the other hand certain challenges:
- Limited Computing Resources: Edge devices usually have limited computing resources, which restricts the model selection.
- Maintenance and Updates: Edge devices can be difficult to maintain and update, especially when deployed in remote or hard-to-reach locations.
Available models and frameworks for vision AI systems at the edge
What are the reasons for deploying a vision AI system at the edge and what are the limitations or challenges one might encounter doing so? We have already seen the differences between AI at the edge and AI in the cloud, let’s highlight some advantages of AI at the edge:
- Real-time Processing: By processing the image data closer to the source, data transmission times are reduced which reduces overall latency.
- Increased Autonomy: Devices with AI systems can operate independently, even in the absence of a network connection, making them suitable for remote or offline applications.
Doing inference at the edge implies on the other hand certain challenges:
- Limited Computing Resources: Edge devices usually have limited computing resources, which restricts the model selection.
- Maintenance and Updates: Edge devices can be difficult to maintain and update, especially when deployed in remote or hard-to-reach locations.
Available models and frameworks for vision AI systems at the edge
Before we take a look at the available hardware, we want to focus on some of the software options. The main focus here lies on using software or AI models optimized for less computational power, less power consumption or reduced memory availability. As for models, we want to mention some popular choices:
- YOLO (You OnlyLookOnce) comes in different sizes. This makes it easy to choose the model tailored to the available hardware
- MobileNet, tailored to the use on hardware with limited computational resources.
- ShuffleNet, designed for mobile devices.
- SqueezeNet, streamlined model, and very lightweight.
Frameworks:
- PyTorch Mobile, training and deployment for iOS and Android mobile devices.
- TensorFlowLite, a lightweight version of TensorFlow, is optimized for edge devices.
- OpenVino, for optimization and deploying deep learning models on different hardware.
- OpenCV, A widely used computer vision library that provides a lot of functionalities for image and video processing.
Available hardware for vision AI systems at the edge
With hardware, we are referring to components necessary for the deployment and use of the AI models. Other necessary devices such as data storage or connectivity models are not considered in the following overview of devices.
- NVIDIA Jetson Modules. A series of modules offering high performance and low power consumption.
- ASUS Tinker Edge T which uses Google Coral modules.
- Foundational Developer Kits from Intel
- Raspberry PI, probably one of the most popular devices for at-the-edge applications
In addition to the above-mentioned hardware, the increasing calculation capacities of modern smartphones make it possible to deploy AI models directly on these devices. Smartphones capable of using AI on devices are among others the iPhone 15 Pro or the iPad Pro. Most top-of-the-line smartphones from major manufacturers are capable of deploying AI models on the device. Check out our related article concerning AI on mobile devices!
Other key components
Technologies such as CMOS Cameras have enabled Computer Vision at the edge in various applications. CMOS (Complementary Metal-Oxide-Semiconductor) cameras are a prevalent type of digital imaging technology, widely used in devices ranging from smartphones to industrial machine vision systems. The core component of these cameras is the CMOS sensor, which converts light into electrical signals to create digital images. These cameras offer:
- Low Power Consumption: CMOS sensors use less power compared to other sensors, making them suitable for battery-powered devices.
- Cost-Effective: Manufacturing CMOS sensors is less expensive due to the integration of additional circuitry on the same chip (e.g., amplifiers, analogue-to-digital converters), reducing overall production costs.
- High-Speed Operation: These sensors can read out data at high speeds, which is beneficial for applications requiring fast image capture and processing.
- Dynamic Range and Sensitivity: Advances in CMOS technology have improved their dynamic range and sensitivity, making them suitable for a wider range of lighting conditions.
Conclusion
Computer vision and AI at the edge offer multiple benefits and become a key feature for several applications. The reduced latency to take decisions, enhanced privacy by avoiding information exchange through multiple networks, and the lower bandwidth required are key features for various applications:
- Autonomous Vehicles: Edge computing allows for real-time processing of visual data from cameras and sensors in autonomous cars, enabling quicker decision-making and improving safety.
- Smart Surveillance: Edge-based computer vision systems can analyze video feeds locally, detecting suspicious activities and alerting authorities without the need for constant data streaming to a central server.
- Industrial Automation: In manufacturing, edge devices equipped with computer vision can inspect products, identify defects, and ensure quality control in real-time, improving efficiency and reducing downtime.
- Retail Analytics: Edge computing can be used in retail environments to monitor customer behaviour, manage inventory, and optimize store layouts by processing data from in-store cameras and sensors.
- Agriculture: In precision agriculture, edge devices can analyze images from drones or ground-based sensors to monitor crop health, detect pests, and optimize resource usage.
- Smart Buildings: Edge computing can enhance building management systems by processing data from cameras and sensors in real-time to monitor occupancy, optimize energy usage, improve security, and manage HVAC (Heating, Ventilating, Air Conditioning) systems efficiently. For example, edge devices can detect unoccupied rooms and adjust lighting and temperature settings to save energy.
Curious about the latest AI trends? Follow this link to discover more fascinating articles on our blog. At Cactus, we excel in all these capabilities and are ready to meet your needs!
By Torsten Reidt, AI Engineer of the Cactai Team! 🌵
