Send image notification on person detection — using OpenCV and Python

In this article we will see how to alert an user on detection of a person in the camera view. We will use a deep learning model for person detection and remaining work of cropping the detected person and sending it in a mail will be taken care of by OpenCV and Python. All the steps covered below can be found in the github repo: object_detection_notifier

Step 1: Setting up a virtual environment

Assuming you will be well versed in setting up of virtual environment in ubuntu or windows, am skipping this part. But still if you are new to setting up virtual environments, follow this link . Once the virtual environment is ready, run up the “requirements.txt” that you obtained by cloning the object_detection_notifier using the following command

pip3 install -r requirements.txt

Step 2: Download the deep learning model

We can obtain the deep learning model from tensorflow.org by running the command “wget "http://download.tensorflow.org/models/object_detection/ssd_mobilenet_v2_coco_2018_03_29.tar.gz"in the terminal or by copy pasting the link in any browser of your choice. Once the tar file is downloaded, extract it. We will find a file named “frozen_inference_graph.pb”, which is our deep learning model that we are going to use

Step 3: Enabling two step verification

For sending mail using python, we will use the “smtp” module. To send mail using the smtp module we need to enable two step verification which I have already written in this post: Two step verification. Also you can find how to send mail with attachment in that post, which will constitute a part of code for this project

Step 4: Sending a image as a email attachment using Python

Step 5: Object (person) detection

Now in Step 3, since we have a function defined to send a mail with attachment, now our job is to just trigger the send_mail function once our object is detected.

We read the model using cv2.dnn module as shown below and pass the model and configuration file as shown below (go through the full code hosted at github, if you can’t comprehend the below line)

model = cv2.dnn.readNet(model=dl_model,config=config,framework="Tensorflow")

We then use the following piece of code to read the input video frame by frame and resize it to (300,300) as our model required the input size to be (300,300). The cv2.dnn.blobFromImage is one of the preprocessing step before we pass our image to the model.

Now our “output” variable contains all the detection along with class id, confidence and bounding box co-ordinates.

Observe the below gist

From the above gist if we want to detect person alone, then we can modify line 5 (“if confidence > 0.5)” as below

if confidence > 0.5 & detection[1] == 0:
(because detection[1] contains the class id and class id '0' stands for person)