Computer Vision Applications: The Development and Deployment Processes

‘Computer Vision’ (CV) refers to processing visual data as a human would with their eyes, so that we can make conclusions about what is in an image. Once we know what is in an image, we can make our application respond, much like a human would when processing visual data. This is what enables technology like self-driving cars.

Developing a CV application involves building machine learning models and integrating these models with external libraries as well as various hardware, including operating systems, accelerators, and cameras. Getting all of these components to work in harmony is exactly what alwaysAI does, and we do it with abstraction and containerization. alwaysAI employs abstraction with two key components: our command line interface (CLI) and our application programming interface (API), edgeIQ.

A CLI is a tool that can be used to interact with software. If you open up cmd.exe or PowerShell on Windows, or Terminal on Mac, you’ll see a prompt that accepts text input and enables you to interact with your operating system: navigate directories, move files, copy input, open a text editor… This is the same way the alwaysAI CLI works: the alwaysAI software recognizes these commands and performs the desired functions: build the application, download computer vision models, configure your project, even accessing the annotation and training software and packaging your application as it’s own Docker image, is all done with just a handful of base commands. This means you only need to know a dozen commands to be able to annotate data, train a model, build your application, and deploy it to a device.

 An API also simplifies the development process, by serving as a kind of negotiator between a complicated backend and an end user. It defines what kinds of requests can be made by the user and how data is returned. This simplifies usage for the user, by hiding some of the complexities involved in these calls, and helps ensure the only appropriate requests can be made. You’ll see examples of this simplification throughout the rest of this article!

Finally, alwaysAI leverages containerization to make deploying computer vision applications as painless as possible. Containerization refers to the process of bundling up software and dependencies in one self-contained unit, which can then be executed in different environments. alwaysAI uses different Docker images to enable deployment on different devices, including Raspberry Pi and Jetson Nano. Another form of containerization you may already be familiar with is a virtual environment. This is popular with Python applications, but they only containerize Python dependencies. alwaysAI utilizes both of these tools to help containerize computer vision applications. By using a Python virtual environment inside a Docker container, all application and system dependencies can be bundled together and the application will work as a self-contained unit.

Whether a computer vision application works in a real-world environment depends first and foremost on the model it uses and this depends the quality of the data that the computer vision model is trained on; garbage in garbage out! If your goal is to get a prototype up and running quickly, it may be sufficient to choose from one of the high-quality, pre-trained models on the alwaysAI model catalog. Typically, different models may use different serialization formats or require different pre-processing, which means even integrating a pre-built model into an application can be challenging, especially if you have to change models or use more than one model in an application. All the models in the catalog have been fit into the alwaysAI workflow, so you don’t have to worry about compatibility, you can easily swap one out for another or use different computer vision models together.  However, we know that more specialized projects require custom models, so we have also built an object detection model training tool into our platform. This means you don’t have to worry about installing and configuring complex training packages yourself! You can even collect and annotate your own datasets using alwaysAI, so you can build your own CV applications from data all the way to deployment. If you want to read more about CV models, you can check out this article.

Once you have your computer vision model, you still need to get visual data into your application and feed it into your model. Getting video streams set up on your laptop or another device can be another pain point, and researching which libraries to use in your application, becoming familiar with different packages and modules can require a steep learning curve and large time investment as well. Here at alwaysAI, we recognize the immense value that developers receive from using existing computer vision libraries, like OpenCV. OpenCV enables developers to access over 2500 algorithms to support the development of computer vision applications, however installing this package from scratch can be daunting. alwaysAI complements OpenCV, wrapping some of the most common functionality like reading and writing data, and manipulating input frames, directly into the alwaysAI application programming interface (API). And you can simply import cv2 into any alwaysAI app to access any of OpenCV’s other functionality, no installation required. You can install and import your other favorite Python libraries into your applications with alwaysAI as well.

What is the edge? In general, edge devices are small, lightweight devices on which a computer vision application can be deployed and run. Many edge devices today even have a Graphical Processing Unit (GPU), or Visual Processing Unit (VPU), which enable usage of a greater range of models and application complexity. In this context, an edge device is a device such as a Raspberry Pi, NVIDIA Jetson devices like the Jetson Nano or Xavier NX, or various Internet of Things (Iot) devices, which are edge devices that have some capability to sense or assess, and possibly interact with, the environment in which they are used. 

Why deploy to the edge? While an internet connection can be used to deploy the application, once the application is on the edge device, there is no need for it to have cloud connectivity to function. This means that any inferencing the application does is on the edge device itself, not in the cloud, drastically reducing the time it takes for those inferences to be turned into actions by the application. For certain use cases, such as self-driving vehicles or security cameras, this is imperative. Aside from the risk that data may be lost being sent to and from the cloud, the additional time required for using a cloud approach can mean not responding to a task in time, which can be catastrophic for tasks like autonomous driving. Additionally, because the data doesn’t need to travel to the cloud with edge deployment, all data can stay in a closed circuit on the device itself, which is much more secure, and, without the need for cloud processing, much less expensive.

Building computer vision applications and running them on edge devices can be a challenge as well if not using an edge-based development tool like alwaysAI. Our goal is to make the process of building and deploying computer vision apps to edge devices as easy as possible. alwaysAI apps are built in Python and can run natively on Mac and Windows, as well as in our containerized edge runtime environment, which is optimized for different architectures commonly used in edge devices, such as 32 and 64 bit ARM architectures. While alwaysAI enables developers to write their code on an edge device, such as a Raspberry Pi or Jetson Nano, the real benefit of using alwaysAI comes with building your application on a regular development machine, and then running that application on an edge device. Edge devices can quickly overheat without a fan and heatsink, and even with these additions, they likely can’t handle running an IDE or support having a web browser with lots of open tabs, as is usually the case when writing application code. And as we mentioned before, installing a library like OpenCV on a Raspberry Pi requires building it from source; this is because many libraries don’t have compiled packages for ARM architectures. With alwaysAI, you get the benefit of developing on your regular machine, and deploying your app to the edge.