Project #8 Experimental Camera:

WHO ARE YOU PEOPLE

Description

This experimental camera is an homage to the dear childhood friend Patrick Star, as well as a play on the idea of our digital bodies being unconscious and overly conscious of being seen & tracked at all times. I thought "Who are you people" relates heavily with the idea of "liquid surveillance" Xin introduced during the lecture on Consentful Interface. Who are the people watching us? How long are they watching for, and who is to follow next?

Design Process

For this project, I most wanted to play with facial recognition and specifically, facial expression recognition.

Concept 1: Christmas Lights

The first concept started out quite simple. Detecting the occurrence of blinks using either top and bottom eye positions or the disappearance of the pupil coordinate, the lights in the background would light up with each blink.

Concept 2: Emoji Generator

The second concept was the most ambitious one – one that would recognize blinks, smiles, and other movements of the mouth to replicate the facial expresssion in a simple, emoji-like forms.

I tried out how these simple facial expression detection would look like, and quickly sketched them in P5.

Problem encountered: clmtrkr.js wasn't detecting my face properly most of the time, let alone get the coordinates of facial features correctly. Even when my face was correctly detected, it seemed like the coordinates of top of eye/lip always stayed higher than the bottom points (ie. it was as if the top of eye, pupil, bottom of eye coordinates were consistently stacked vertically, even when I blinked really hard. Same thing with the mouth – even when I placed my bottom lip above the top lip, it couldn't consistently recognize a closed vs open mouth).

After a few more attempts at getting the facial expression recognition right, I moved onto implementing my third concept:

Concept 3: WHO ARE YOU PEOPLE

I have been wanting to incorporate the arctangent eye movements in my P5 project for while. I did incoporate it to my Bitmap game (eyes tracking the player's movement), but the visual effect of the eye movement was minimal given the context of the game. Playing with the idea of unconscious and/or overly conscious surveillance, I wanted to throw in a bunch of eyeballs and have it follow the person in the frame.

This reminded me of one of the most iconic scenes from Spongebob, where Patrick returns home with groceries and an ice cream in his hands and finds a bunch of people hiding under his shell-roofed home.

Photos

Reflection

Facial recognition that felt simple enough (ie. blink, mouth opening & closing) seemed difficult to be managed by the resources we had. I don't know if it's my face specifically, but clmtrkr.js had trouble keeping the positions in the correct places to begin with. With more time, I'd explore other libraries or platforms that allow for a more accurate and consistent face tracking to try making the interactive filters I dreamed of making with the concepts.

Additionally, my computer was panting hard while working on this project. I'd love to be able to try more strenuous effects when I get a new computer in the new year!