By Lucas Wu (sw6889@nyu.edu) and Noah Black (nsb8333@nyu.edu).
Be Afraid is a Halloween-themed interactive experience that combines computer vision and physical actuators to reward users for successfully emoting,as judged by artificial intelligence, in a frightened or surprised way.
Ideation
Lucas had the initial idea: a trick-or-treating themed experience, incorporating some kind of emotion recognition software. Eventually we settled on this: a camera is pointed at the user, and the user is standing in front of a feed that displays how “scared” they are. When a user makes a sufficiently frightened or surprised face, they will be “rewarded” with candy. The candy will be dispensed by a servo. The title of the piece was inspired by the classic film The Fly.
Input – Computation – Output
| Input | Computation | Output |
| 1. Face (to emote) 2. Keyboard (to enter name) | The user’s expression is fed to an face emotion classifier to predict how afraid or surprised they are. The servo is activated, or not, based on this prediction. | 1. A real-time feed of the user’s face and a meter indicated their “fright” level. 2. Candy |
Coding
Code here: https://github.com/noah-black/face
We started with a Google Co-Lab to test out a few off-the-shelf face-emotion libraries. The ones that only picked a single label (“happy,” “scared,” etc.) felt too binary for a game. We landed on a model that, besides just an emotional state, also predicted an “arousal” level, which would allow us to distinguish between different levels of fear.
The GUI for the first prototype was pure OpenCV: green boxes, a tiny bar, and text on top of the camera feed. It was ugly, so we moved the interface to Qt, which looked a lot better, and gave us a lot more flexibility. We also split the code into three parts—camera/prediction, game rules, and UI—so we could tweak the visuals without breaking game state, and vice-versa. We added a simple game loop: a user enters their name, either “wins” or does not.
We worked for a while on how to capture the kinds of face we wanted people to make. We started with a blend of many signals; it was fussy and felt arbitrary. The version that clicked used a simple metric: the maximum of the normalized confidence scores for the “fear” and “surprise” emotions. We implemented some configurable hysteresis. The metric is displayed to the user as an on-screen “fright meter.” Later we made the speed of the rise depend on the “arousal” metric mentioned earlier. That made it feel like you’re really pushing on the bar with your expression.
Responsiveness mattered. We noticed the bar moved in tiny “steps,” which turned out to be the cadence of the model updates. To make motion feel continuous we switched to time-based updates: every frame the bar moves by rate × dt. We also ran the model on a background thread, avoided per-frame allocations, and kept smoothing light. The result feels snappier—when you drop from “scared” to a grin, the bar starts falling right away.
The game grew from a single “be scared” round into three modes: Be Afraid (fill and hold), Keep Calm (same input, opposite goal—don’t let it fill), and Free Mode (a playground that just mirrors the classifier so people can try expressions). We augmented the skinny side bar with a full-screen rising tint that reads in peripheral vision, added chunky retro type, and built summaries with the player’s best “scared” photos and a leaderboard (with thumbnails). We left simple hooks for physical feedback: an LED strip for color cues (blue calm → red danger), and a servo to drop candy or trigger a gotcha moment in Keep Calm.
Visual Design
Concept art by Lucas:
Fabrication
First Attempt
In our first attempt, Lucas tried to build a base for the candy dispenser by laser cutting plywood. However, he forgot to account for the kerfing tolerance when designing the plywood pieces, and at that time he also hadn’t fully solved the problem of securing the servo motor.
Second Attempt
In the second attempt, we used a thicker, solid wood base and secured the structure with screws. We also designed a simple mechanism to fix the servo motor, making it easier to control the release of the candy.
