Smart Door

In this lesson we will explore building a smart auto door using facial and eye detection, presenting how modern intelligent security systems work, and learning more about AI applications like computer vision.

Prerequisite

You must have a basic understanding of coding with Python, have completed the Servo lesson, and know more about the OpenCV library, Also in this lesson we will leverage the BrainSense kit’s servo-driven door.

OpenCV object detection algorithm

Haar-cascade Detection in OpenCV is a smart way for computers to recognize faces and eyes in images. It’s like a digital detective that has been trained to spot specific patterns. Using a series of quick tests, called classifiers, it efficiently scans images to identify these features. Imagine it as a sliding window that swiftly sifts through the picture, focusing on areas with potential face or eye shapes. This method, integrated into OpenCV, is widely used in applications like facial recognition and gaze tracking, showcasing how artificial intelligence and computer vision can work together to make our digital devices smarter, you can learn more about Haar-cascade Detection from here.

The Implementation

Ensure you have Python 3.10 installed on your computer. Additionally, install the required libraries using the following commands:

pip install opencv-python 
pip install DUELink

The following code example will use pre-trained Haar cascade models to detect faces and eyes in an image. First, we need to download the necessary XML files 'haarcascade_frontalface_alt.xml' from here and 'haarcascade_eye_tree_eyeglasses.xml' from here, then place the two XML files in the project folder.

import cv2 as cv

# Function to detect faces and display them in the frame
def detectAndDisplay(frame):
    face_detected = False

    # Convert the frame to grayscale
    frame_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)

    # Equalize the histogram of the grayscale frame
    frame_gray = cv.equalizeHist(frame_gray)

    # Detect faces using Haar cascades
    faces = face_cascade.detectMultiScale(frame_gray)

    # Loop through each detected face
    for (x, y, w, h) in faces:
        center = (x + w // 2, y + h // 2)

        # Draw an ellipse around the detected face
        frame = cv.ellipse(frame, center, (w // 2, h // 2), 0, 0, 360, (255, 0, 255), 4)

        # Extract the region of interest (ROI) for eyes detection
        faceROI = frame_gray[y:y + h, x:x + w]

        # Detect eyes within the face ROI
        eyes = eyes_cascade.detectMultiScale(faceROI)

        # Check if two eyes are detected
        if len(eyes) == 2:
            face_detected = True

            # Loop through each detected eye and draw a circle around it
            for (x2, y2, w2, h2) in eyes:
                eye_center = (x + x2 + w2 // 2, y + y2 + h2 // 2)
                radius = int(round((w2 + h2) * 0.25))
                frame = cv.circle(frame, eye_center, radius, (255, 0, 0), 4)

    # Display the frame with face detection
    cv.imshow('Capture - Face detection', frame)

    return face_detected

Now, Write a method to control the Servo’s movement based on the boolean variable ‘face_detected’: open the door if true, and close it if false.

from DUELink.DUELinkController import DUELinkController

# Get the available port for DUELink communication
available_port = DUELinkController.GetConnectionPort()

# Initialize DUELinkController with the available port
BrainPad = DUELinkController(available_port)

# Define the servo pin for controlling the door
servo_pin = 10

# Function to open or close the door based on face detection
def opendoor(face):
    if face:
        # If a face is detected, open the door (set servo to 90 degrees)
        BrainPad.Servo.Set(servo_pin, 90)
    else:
        # If no face is detected, close the door (set servo to 0 degrees)
        BrainPad.Servo.Set(servo_pin, 0)

Finally, let’s put everything together, load the XML files, and write the main Loop that captures the frames and performs face detection.

import cv2 as cv
from DUELink.DUELinkController import DUELinkController

# Get the available port for DUELink communication
available_port = DUELinkController.GetConnectionPort()

# Initialize DUELinkController with the available port
BrainPad = DUELinkController(available_port)

# Define the servo pin for controlling the door
servo_pin = 10


# Function to detect faces and display them in the frame
def detectAndDisplay(frame):
    face_detected = False

    # Convert the frame to grayscale
    frame_gray = cv.cvtColor(frame, cv.COLOR_BGR2GRAY)

    # Equalize the histogram of the grayscale frame
    frame_gray = cv.equalizeHist(frame_gray)

    # Detect faces using Haar cascades
    faces = face_cascade.detectMultiScale(frame_gray)

    # Loop through each detected face
    for (x, y, w, h) in faces:
        center = (x + w // 2, y + h // 2)

        # Draw an ellipse around the detected face
        frame = cv.ellipse(frame, center, (w // 2, h // 2), 0, 0, 360, (255, 0, 255), 4)

        # Extract the region of interest (ROI) for eyes detection
        faceROI = frame_gray[y:y + h, x:x + w]

        # Detect eyes within the face ROI
        eyes = eyes_cascade.detectMultiScale(faceROI)

        # Check if two eyes are detected
        if len(eyes) == 2:
            face_detected = True

            # Loop through each detected eye and draw a circle around it
            for (x2, y2, w2, h2) in eyes:
                eye_center = (x + x2 + w2 // 2, y + y2 + h2 // 2)
                radius = int(round((w2 + h2) * 0.25))
                frame = cv.circle(frame, eye_center, radius, (255, 0, 0), 4)

    # Display the frame with face detection
    cv.imshow('Capture - Face detection', frame)

    return face_detected


# Function to open or close the door based on face detection
def opendoor(face):
    if face:
        # If a face is detected, open the door (set servo to 90 degrees)
        BrainPad.Servo.Set(servo_pin, 90)
    else:
        # If no face is detected, close the door (set servo to 0 degrees)
        BrainPad.Servo.Set(servo_pin, 0)


# Initialize Haar cascades for face and eyes detection
face_cascade = cv.CascadeClassifier()
eyes_cascade = cv.CascadeClassifier()

# Load Haar cascades from XML files
if not face_cascade.load(cv.samples.findFile('haarcascade_frontalface_alt.xml')):
    print('--(!)Error loading face cascade')
    exit(0)
if not eyes_cascade.load(cv.samples.findFile('haarcascade_eye_tree_eyeglasses.xml')):
    print('--(!)Error loading eyes cascade')
    exit(0)

# Open a video capture stream (camera index 0)
cap = cv.VideoCapture(0)
if not cap.isOpened:
    print('--(!)Error opening video capture')
    exit(0)

# Continuous loop to capture frames and perform face detection
while True:
    ret, frame = cap.read()

    # Check if a frame is captured
    if frame is None:
        print('--(!) No captured frame -- Break!')
        break

    # Perform face detection and open/close the door accordingly
    opendoor(detectAndDisplay(frame))

    # Break the loop if the 'Esc' key is pressed
    if cv.waitKey(10) == 27:
        break

The Result

The output of the code, utilizing the video stream from a built-in webcam as input, is as follows:

What’s Next?

Can you figure out other computer vision applications we can do with Python and the BrainSense kit? Jump back to see all BrainSense kit modules.

BrainStorm

Who knew AI with hardware could be so creative and powerful?! although the algorithm we used in this lesson is considered an easy old way to detect faces and eyes, can you imagine what applications we can do by merging the latest AI software and hardware technology?