On April 1, 2019, I received an email from Udacity telling me that I won a scholarship and the message reads something like this:
Congratulations! We're very happy to offer you a Nanodegree program in the Intel ® Edge AI Scholarship. Out of thousand of students in the Scholarship OpenVINO Foundational Course, your progress in the course and dedication in the community really stood out. We’ve been impressed and we’re excited for you to continue the journey!
Actually, that is true. Out of about 11,000 participants I managed to be in one of 750 scholarship awardees for the brand new Intel® Edge AI for IoT Developers Nanodegree. And guess what? I also got a badge!
To be honest with you, this is not the first time that I got a scholarship from Udacity. I was also part of Secure and Private AI scholarship from Facebook and Bertelsmann Technology Scholarship, to which I earned Deep Learning and Cloud Devops Engineer Nanodegree seats respectively. In this article, I want to share my experience working on the three projects and how I managed to graduate near the program deadline (the day of this writing).
Program Background
The data science or machine learning projects prioritizes modeling and deployment of applications as part of their end-to-end process. A lot of times, machine learning application deployment needs a lot of compute resources in order for them to be accessed by stakeholders. Not to mention the cost of modeling which may need advanced compute environments with high powered CPUs and GPUs. And what if the project must be deployed in a site with limited to no connectivity but expects to deliver high accuracy and performance with real-time processing requirements? What if the organization has sensitive information that neededs to be processed but cannot risk running them to cloud for privacy reasons? This is where local processing or processing on the edge is important compare to running it the application in the cloud which can be problematic for scenarios mentioned above. If processing can be done in-device, needing connectivity to the cloud is unnecessary which will address issues of latency and privacy. Enter Intel ® OpenVINO. With this tool, we leverage pre-trained models available in the Intel ® Distribution of OpenVINO Toolkit™, convert and optimize other models with their own implementation of Model Optimizer, and perform inference with their implementation of the Inference Engine.
My Experience
Edge AI Fundamentals with OpenVINO™
For the first part of the course, Instructor Michael Virgo introduces AI at the edge and how OpenVINO Toolkit™ can be used to interesting edge AI projects like handling video streams and building a web application with MQTT communication layer. In the course, I learned about the parts that make up the toolkit like the wide array of pre-trained models that are directly available for use within the OpenVINO toolkit, the Model Optimizer that allows users to take models trained from industry-standard machine learning frameworks like TensorFlow, ONNX, Caffe, MXNet, and Kaldi with ONNX being able to suppport other frameworks such as PyTorch and CNTK, perform inference using the Inference Engine and finally, simulating a real-time application by using the model that is already in the Intermediate Representation (meaning it can be used by the inference engine for inference) to deploy an edge application.
That being said, the first project is Deploy a People Counter App at the Edge. Basically, the project requires the student to leverage pre-trained models from different industry-standard machine learning frameworks to detect persons per frame and create a logic that coounts the number of people going in and out of the frame.
The first project is a little bit challenging in that, you need to explore pre-trained models to detect persons in the frame and devise a logic that would count them correctly since OpenVINO spits out predictions on a per-frame basis. Apart from it being challenging, it is rewarding at the same time because this simulates what to expect as an IoT engineer. And to add to the excitement, the app can be run on the local machine via webcam and you can actually see it counting people in action, changing values in the dashboard real-time.
Choosing the Right Hardware
The second part of the course delves deeper into the process of creating machine learning-enabled edge application. A lot of times, developers only concerns themselves to the implementation of the application to make, without realizing that there are several constraints that must be taken care of like cost of hardware, electrical consumption and the power that you may get from acquiring the hardware. It is through these combinations you will be able to realize an application that, not only solves the problem at hand but also considers what the stakeholders require. That being said, Intel devised a framework that would enable seamless collaboration between data scientists, IoT engineers and stakeholders in designing and deploying and edge AI system:
Aside from the design, this lesson also gives an overview of compatible hardware offered by Intel from CPUs, Integrated GPUs, Visual Processing Units (VPUs) and Field-Programmable Gate Arrays (FPGAs), comparing the pros and cons between them. Finally, this chapter includes an introduction to Intel DevCloud, a cloud service that lets you benchmark the performance of each hardware and their combinations without actually having to acquire each of them.
For this module's project entitled Smart Queuing System, they are more of a case study albeit it also builds on top of the first project. Students are tasked to complete the Python and job submission scripts to communicate with the DevCloud right within the Udacity workspace. Students must be able to assess requirements from each of the scenarios given (manufacturing, retail and transportation) and be able to recommend hardware solutions based on the constraints of price, power and performance.
This project gleans towards the expected career of an IoT engineer in my opinion. To be able to extract meaningful information from clients, test combinations of hardware solutions that fits and deliver an application using the selected hardware solution.
Optimization Techniques and Tools
Many appliications, like I said before needs to process real-time data streams from myriad of sensors. And since we solved the majority of latency problems by running the model and inference in-device without needing external cloud solution, one aspect of how we can still improve the performance of the overall application is through software optimization. And this is what the final lesson is all about, optimizing models by (1) reducing model operations through removing network nodes or turning them off altogether, (2) reducing model size by replacing dense layers and convolutions with specialized layers and (3) using VTune amplifier to drill down to the performance of the application and find hotspots for bottlenecks.
The final project, Computer Pointer Controller is perhaps the most challenging of all projects. Basically, the task is to use multiple object detection and recognition models from Intel Pretrained Models and Intel hardware to create an application that controls mouse pointer aided only by gaze. Controlling the mouse using only your gaze? Amazing right?
In this project, I managed to controll the mouse using pyautogui package from Python and get the coordinates to which the mouse pointer is heading with the use of four models from Intel Model Zoo - (1) face detection, (2) landmark detection to detect eyes, (3) head pose detection to detect head orientation and (4) gaze detection to determine which direction does face in the camera feed is gazing. I also learned on how to toggle each of the bounding boxes on and off depending on user's preference.
To review, this project is worthy of the course's culmination. It is challenging, yes but the course materials really helped you getting up to speed on how to create virtual environments, provide detailed documentation, use multiple sources of inputs, implement inference data pipeline in a real time fashion, visualize outputs as needed and make use of the available Intel hardware to run the application locally.
Conclusion
I am incredibly proud to be one of 750 lucky scholarship awardees of this amazing nanodegree. I have here my projects uploaded in my GitHub- Deploy a People Counter App at the Edge, Smart Queuing System and Computer Pointer Controller. This is definitely not the end of my journey exploring OpenVINO let alone Internet of Things. Stay tuned for more.
Since this is the first time that I make a blog setting up the whole thing, you probably want to know how I managed to get three scholarships from Udacity. I am excited to tell my story in the upcoming days.