Module 10: Detecting, Analyzing and Recognizing faces

 

Options for Face Detection, Analysis, and Recognition

There are two cognitive services that you can use to build solutions that detect faces in images.

The Computer Vision service

The Computer Vision service enables you to detect human faces in an image, returning a bounding box for its location. It also returns some facial feature information about the detected face; specifically, predictions for:

• Gender

• Age

The Face service

The Face service offers more comprehensive facial analysis capabilities than the Computer Vision service, including:

• Face detection (with bounding box).

• Comprehensive facial feature analysis (including age, gender, emotional state, head pose, hair color, presence of facial hair, presence of spectacles, and others).

• Face comparison and verification.

• Facial recognition.

Considerations for Face Detection and Facial Recognition

While all applications of artificial intelligence require considerations for responsible and ethical use, system that rely on facial data can be particularly problematic.

When building a solution that uses facial data, considerations include (but are not limited to):

• Data privacy and security. Facial data is personally identifiable, and should be considered sensitive and private. You should ensure that you have implemented adequate protection for facial data used for model training and inferencing.

• Transparency. Ensure that users are informed about how their facial data will be used, and who will have access to it.

• Fairness and inclusiveness. Ensure that you face-based system cannot be used in a manner that is prejudicial to individuals based on their appearance, or to unfairly target individuals.

Face Detection with the Computer Vision Service

To detect and analyze faces with the Computer Vision service, call the Analyze Image REST function (or equivalent SDK method), specifying Faces as one of the visual features to be returned.

In images that contain one or more faces, the response includes details of their location in the image and the predicted age and gender of the detected person, like this:

{ "faces": [ { "age": 32, "gender": "Female", "faceRectangle": { "top": 225, "left": 237, "width": 130, "height": 130 } }, { "age": 29, "gender": "Female", "faceRectangle": { "top": 309, "left": 534, "width": 119, "height": 119 } } ] }

The Face Service

The Face service provides comprehensive facial detection, analysis, and recognition capabilities. It provides functionality that you can use for:

• Face detection - for each detected face, the results include an ID that identifies the face and the bounding box coordinates indicating its location in the image.

• Face attribute analysis - you can return a wide range of facial attributes, including:

  • Age (estimated age of the person in years)

  • Gender (Male, Female, or Genderless)

  • Emotions (normalized confidence scores for happiness, sadness, neutral, anger, contempt, disgust, surprise, and fear)

  • Smile (numeric value from 0 (no smile) to 1 (full smile))

  • Head pose (pitch, roll, and yaw orientation in 3D space)

  • Hair (visibility, baldness, hair color)

  • Facial hair (presence and length)

  • Glasses (NoGlasses, ReadingGlasses, Sunglasses, or Swimming Goggles)

  • Makeup (Boolean values for eyeMakeup and lipMakeup)

  • Blur (low, medium, or high)

  • Exposure (underExposure, goodExposure, or overExposure)

  • Noise (visual noise in the image)

  • Occlusion (objects obscuring the face)

• Facial landmark location - coordinates for key landmarks in relation to facial features (for example, eye corners, pupils, tip of nose, and so on)

• Face comparison - you can compare faces across multiple images for similarity (to find individuals with similar facial features) and verification (to determine that a face in one image is the same person as a face in another image)

• Facial recognition - you can train a model with a collection of faces belonging to specific individuals, and use the model to identify those people in new images.

You can provision Face as a single-service resource, or you can use the Face API in a multi-service Cognitive Services resource.

Detected Face Identification

When a face is detected by the Face service, an ID is assigned to it and retained in the service resource for a period of 24 hours. The ID is a GUID, with no indication of the individual's identity other than their facial features.

While the detected face ID is cached, subsequent images can be used to compare the new faces to the cached identity and determine if they are similar (in other words, they share similar facial features) or to verify that the same person appears in two images.

This ability to compare faces anonymously can be useful in systems where it's important to confirm that the same person is present on two occasions, without the need to know the actual identity of the person. For example, by taking images of people as they enter and leave a secured space to verify that everyone who entered leaves.

Persisted Face Recognition

For scenarios where you need to positively identify individuals, you can train a facial recognition model using face images.

To train a facial recognition model with the Face service:

1. Create a Person Group that defines the set of individuals you want to identify (for example, employees).

2. Add a Person to the Person Group for each individual you want to identify.

3. Add detected faces from multiple images to each person, preferably in a variety of poses. The IDs of these faces will no longer expire after 24 hours (so they're now referred to a persisted faces).

4. Train the model.

The trained model is stored in your Face (or Cognitive Services) resource, and can be used by client applications to:

• Identify individuals in images.

• Verify the identify of a detected face.

• Analyze new images to find faces that are similar to a known, persisted face.

LAB

Detect, Analyze, and Recognize Faces

The ability to detect, analyze, and recognize human faces is a core AI capability. In this exercise, you'll explore two Azure Cognitive Services that you can use to work with faces in images: the Computer Vision service, and the Face service.