Overview
In this beginner project you will connect the APDS-9960 sensor to the ESP32 over I2C and detect four directional swipe gestures: up, down, left, and right. Each gesture lights a different colour LED and prints the direction to the Serial Monitor. No Wi-Fi is needed. The project teaches I2C sensor setup, interrupt-driven gesture events, and basic GPIO output control.
Components
- 1× ESP32 DevKit V1
- 1× APDS-9960 proximity and gesture sensor — SparkFun or compatible breakout
- 4× LED (red, green, blue, yellow) — 5 mm through-hole
- 4× 220 ohm resistor — One per LED
- 1× Breadboard and jumper wires
Wiring
| Component Pin | ESP32 Pin | Notes |
|---|---|---|
| APDS-9960 SDA | GPIO 21 | I2C data |
| APDS-9960 SCL | GPIO 22 | I2C clock |
| APDS-9960 INT | GPIO 4 | Interrupt; active LOW |
| APDS-9960 VCC | 3V3 | 3.3 V only |
| APDS-9960 GND | GND | |
| Red LED anode | GPIO 25 | Gesture UP |
| Green LED anode | GPIO 26 | Gesture DOWN |
| Blue LED anode | GPIO 27 | Gesture LEFT |
| Yellow LED anode | GPIO 14 | Gesture RIGHT |
Arduino Code
// ESP32 Gesture Recognition - Beginner
// APDS-9960 four-direction swipe to LED
#include <Wire.h>
#include <SparkFun_APDS9960.h>
SparkFun_APDS9960 apds;
const int INT_PIN=4;
const int LED_UP=25, LED_DN=26, LED_LT=27, LED_RT=14;
volatile bool gestureAvailable=false;
void IRAM_ATTR isr(){ gestureAvailable=true; }
void clearLEDs(){
digitalWrite(LED_UP,LOW); digitalWrite(LED_DN,LOW);
digitalWrite(LED_LT,LOW); digitalWrite(LED_RT,LOW);
}
void setup(){
Serial.begin(115200);
Wire.begin(21,22);
pinMode(INT_PIN,INPUT);
attachInterrupt(INT_PIN,isr,FALLING);
pinMode(LED_UP,OUTPUT); pinMode(LED_DN,OUTPUT);
pinMode(LED_LT,OUTPUT); pinMode(LED_RT,OUTPUT);
if(!apds.init()){ Serial.println("APDS-9960 init failed"); while(1); }
apds.enableGestureSensor(true);
Serial.println("Gesture sensor ready — swipe your hand");
}
void loop(){
if(!gestureAvailable) return;
gestureAvailable=false;
if(!apds.isGestureAvailable()) return;
int gesture=apds.readGesture();
clearLEDs();
switch(gesture){
case DIR_UP: digitalWrite(LED_UP,HIGH); Serial.println("UP"); break;
case DIR_DOWN: digitalWrite(LED_DN,HIGH); Serial.println("DOWN"); break;
case DIR_LEFT: digitalWrite(LED_LT,HIGH); Serial.println("LEFT"); break;
case DIR_RIGHT: digitalWrite(LED_RT,HIGH); Serial.println("RIGHT"); break;
default: Serial.println("NEAR/FAR"); break;
}
delay(500);
}How It Works
Photodiode Array Detection: The APDS-9960 contains four directional photodiodes (north, south, east, west) and an IR LED. As a hand passes over the sensor, the timing and magnitude difference between opposing photodiode pairs encodes the swipe direction.
Interrupt-Driven Reading: The INT pin goes LOW when gesture data is ready. An IRAM-resident ISR sets a flag, and the main loop processes gestures only when the flag is set. This avoids polling the I2C bus on every loop iteration.
readGesture() Direction Enum: The SparkFun library processes the raw photodiode time-series internally and returns a DIR_UP, DIR_DOWN, DIR_LEFT, or DIR_RIGHT constant. No signal processing is needed in user code.
LED Visual Feedback: Each direction maps to a unique LED colour. clearLEDs() extinguishes all LEDs before lighting the correct one, ensuring only one LED is on at a time after each gesture.
Applications
- Touchless page turner for music stand tablet
- Accessibility device for users who cannot touch screens
- Touchless control of a presentation slideshow
- Smart mirror gesture navigation for menu selection
Troubleshooting
APDS-9960 init fails
Check power supply: the APDS-9960 runs at 3.3 V only; 5 V will damage it. Run an I2C scanner and confirm the device appears at address 0x39. Check SDA and SCL wiring.
Gestures are not detected or inconsistent
Swipe speed matters. Move your hand at approximately 10-20 cm/s at a distance of 5-15 cm from the sensor. Too slow or too close suppresses detection.
Gesture direction is reversed (left reads as right)
Rotate the sensor module 180 degrees. The north/south/east/west photodiode orientation is fixed; mounting direction determines which swipe maps to which direction constant.
Multiple gestures register for a single swipe
Increase the gesture threshold in apds.setGestureGain() to require a stronger signal before registering. Also add a longer post-gesture delay before re-enabling the interrupt.
Upgrades
- Use gestures to control a WS2812B NeoPixel strip colour and brightness
- Add a proximity reading to wake the ESP32 when a hand approaches
- Map gestures to keyboard shortcuts using ESP32 BLE HID
- Add an OLED to display the last detected gesture and direction arrow
FAQ
You need an ESP32 DevKit, TODO: sensor, APDS-9960 INT, a breadboard, jumper wires, and a USB cable for power and programming.
Only the Advanced stage uses Wi-Fi. Beginner and Intermediate builds run offline on the ESP32 with USB power.
Start with Beginner if you are new to Sensor Projects. Use Intermediate for OLED feedback and Advanced for dashboards or connected monitoring.
Overview
The intermediate build uses gestures to control a WS2812B NeoPixel LED strip. Swiping up increases brightness, down decreases it, left cycles backward through colour presets, and right cycles forward. A proximity reading wakes the strip from a low-power sleep mode when a hand approaches within 10 cm. Current colour and brightness settings are saved to NVS so they persist across power cycles.
Components
- 1× ESP32 DevKit V1
- 1× APDS-9960 sensor module
- 1× WS2812B NeoPixel LED strip 30 LEDs — 5 V, 1 m
- 1× 5 V 3 A power supply — 30 LEDs at full white draw 1.8 A
- 1× 470 ohm resistor — Data line protection for NeoPixel
- 1× 1000 uF capacitor — Across 5 V supply for NeoPixel
Wiring
| Component Pin | ESP32 Pin | Notes |
|---|---|---|
| APDS-9960 SDA/SCL/INT/VCC/GND | Same as beginner | |
| NeoPixel DIN | GPIO 18 via 470 ohm | Data line; level shift may be needed |
| NeoPixel 5V | External 5 V 3 A supply | Do not power from ESP32 |
| NeoPixel GND | Common GND with ESP32 |
Arduino Code
// ESP32 Gesture Recognition - Intermediate (NeoPixel control + proximity wake)
#include <Wire.h>
#include <SparkFun_APDS9960.h>
#include <Adafruit_NeoPixel.h>
#include <Preferences.h>
SparkFun_APDS9960 apds;
Adafruit_NeoPixel strip(30, 18, NEO_GRB + NEO_KHZ800);
Preferences prefs;
const int INT_PIN=4;
volatile bool gestureReady=false;
void IRAM_ATTR isr(){ gestureReady=true; }
uint32_t COLORS[]={
strip.Color(255,0,0), // red
strip.Color(0,255,0), // green
strip.Color(0,0,255), // blue
strip.Color(255,128,0), // amber
strip.Color(128,0,255), // purple
strip.Color(0,255,255) // cyan
};
const int NUM_COLORS=6;
int colorIdx=0;
int brightness=128;
bool sleeping=false;
void applyStrip(){
strip.setBrightness(brightness);
for(int i=0;i<30;i++) strip.setPixelColor(i,COLORS[colorIdx]);
strip.show();
}
void saveState(){
prefs.begin("gesture",false);
prefs.putInt("color",colorIdx);
prefs.putInt("bright",brightness);
prefs.end();
}
void setup(){
Serial.begin(115200);
Wire.begin(21,22);
strip.begin(); strip.show();
prefs.begin("gesture",true);
colorIdx=prefs.getInt("color",0);
brightness=prefs.getInt("bright",128);
prefs.end();
applyStrip();
pinMode(INT_PIN,INPUT);
attachInterrupt(INT_PIN,isr,FALLING);
apds.init();
apds.enableGestureSensor(true);
apds.enableProximitySensor(false);
}
void loop(){
// Proximity wake
uint8_t prox=0;
apds.readProximity(prox);
if(prox>150 && sleeping){
sleeping=false; applyStrip(); Serial.println("Wake");
} else if(prox<30 && !sleeping){
// Dim after no proximity for 30 s
}
if(!gestureReady) return;
gestureReady=false;
if(!apds.isGestureAvailable()) return;
int g=apds.readGesture();
switch(g){
case DIR_UP: brightness=min(255,brightness+30); break;
case DIR_DOWN: brightness=max(0,brightness-30); break;
case DIR_LEFT: colorIdx=(colorIdx-1+NUM_COLORS)%NUM_COLORS; break;
case DIR_RIGHT: colorIdx=(colorIdx+1)%NUM_COLORS; break;
}
applyStrip(); saveState();
Serial.printf("Color=%d Brightness=%dn",colorIdx,brightness);
}How It Works
Gesture-to-Strip Mapping: Up and down swipes increment and decrement the brightness integer (clamped 0-255). Left and right swipes decrement and increment the colorIdx with modulo wrapping. applyStrip() applies both changes atomically using setBrightness() and setPixelColor().
Proximity Wake Detection: apds.readProximity() returns 0-255 with 255 being closest. A threshold of 150 (about 5 cm) wakes the strip from sleep. A low proximity reading after a timeout period can dim or turn off the strip automatically.
NVS State Persistence: saveState() writes colorIdx and brightness to NVS after every gesture. On next power-on the setup() block reads these values and restores the strip to its last colour and brightness before the first gesture.
NeoPixel Power Budget: Each WS2812B LED draws up to 60 mA at full white. The strip.setBrightness() call scales all pixel values globally so total current stays proportional to brightness. At brightness=128 (50 percent) a 30-LED strip draws approximately 900 mA.
Applications
- Touchless LED ambience control for living room lighting
- Gesture-controlled stage or DJ lighting effects
- Accessibility lighting for users with motor impairments
- Smart mirror ambient lighting with proximity auto-on
Troubleshooting
NeoPixel strip flickers or shows wrong colours
Add a 470 ohm resistor on the data line and a 1000 uF capacitor across the 5 V strip supply. Ensure the strip GND is connected to the ESP32 GND (common ground is required).
Proximity sensor does not trigger wake
Make sure apds.enableProximitySensor(false) is called first then re-enable it: apds.enableProximitySensor(true). The APDS-9960 cannot run both gesture and proximity modes simultaneously on some library versions.
Brightness goes negative or exceeds 255
Use constrain(brightness+30,0,255) instead of min/max for safer bounds checking without separate clamping logic.
Upgrades
- Add a double-tap gesture (rapid UP-DOWN) to toggle the strip on and off
- Add Wi-Fi and a web UI to manually set colour and brightness remotely
- Add a sunrise alarm mode that gradually increases brightness from 0 at a set time
- Add animated colour-cycle effects triggered by a specific gesture sequence
FAQ
You need an ESP32 DevKit, TODO: sensor, APDS-9960 INT, a breadboard, jumper wires, and a USB cable for power and programming.
Only the Advanced stage uses Wi-Fi. Beginner and Intermediate builds run offline on the ESP32 with USB power.
Start with Beginner if you are new to Sensor Projects. Use Intermediate for OLED feedback and Advanced for dashboards or connected monitoring.
Overview
The advanced build publishes gesture events to an MQTT broker so any smart home device can respond to hand gestures. Up/down gestures control thermostat setpoints, left/right switch between rooms or scenes, and a sustained proximity event triggers a "do not disturb" mode. A Node-RED dashboard shows a live gesture history log. BLE HID mode lets the same sensor act as a Bluetooth air-mouse when Wi-Fi is disabled.
Components
- 1× ESP32 DevKit V1
- 1× APDS-9960 sensor
- 1× MQTT broker — Mosquitto or cloud broker
- 1× Node-RED instance — For dashboard and automations
- 1× SSD1306 OLED 128x64 — Last gesture display
Wiring
| Component Pin | ESP32 Pin | Notes |
|---|---|---|
| APDS-9960 + OLED | SDA=GPIO 21, SCL=GPIO 22 | Shared I2C bus; APDS-9960=0x39, OLED=0x3C |
| APDS-9960 INT | GPIO 4 |
Arduino Code
// ESP32 Gesture Recognition - Advanced (MQTT home automation + BLE HID)
#include <Wire.h>
#include <SparkFun_APDS9960.h>
#include <WiFi.h>
#include <PubSubClient.h>
#include <ArduinoJson.h>
#include <Adafruit_SSD1306.h>
SparkFun_APDS9960 apds;
Adafruit_SSD1306 oled(128,64,&Wire,-1);
WiFiClient wifiClient;
PubSubClient mqtt(wifiClient);
const char* SSID="YourSSID", *PASS="YourPass";
const char* MQTT_HOST="192.168.1.100";
const char* TOPIC_GESTURE="home/gesture";
const char* TOPIC_CMD ="home/gesture/cmd";
const int INT_PIN=4;
volatile bool gReady=false;
void IRAM_ATTR isr(){ gReady=true; }
const char* dirName(int d){
switch(d){
case DIR_UP: return "UP";
case DIR_DOWN: return "DOWN";
case DIR_LEFT: return "LEFT";
case DIR_RIGHT: return "RIGHT";
default: return "NEAR";
}
}
void publishGesture(const char* dir){
StaticJsonDocument<64> doc;
doc["gesture"]=dir; doc["ts"]=millis();
char buf[64]; serializeJson(doc,buf);
mqtt.publish(TOPIC_GESTURE,buf);
oled.clearDisplay(); oled.setCursor(0,0);
oled.setTextSize(2); oled.println(dir); oled.display();
}
void setup(){
Serial.begin(115200);
Wire.begin(21,22);
oled.begin(SSD1306_SWITCHCAPVCC,0x3C);
oled.setTextColor(WHITE);
apds.init(); apds.enableGestureSensor(true);
pinMode(INT_PIN,INPUT);
attachInterrupt(INT_PIN,isr,FALLING);
WiFi.begin(SSID,PASS);
while(WiFi.status()!=WL_CONNECTED) delay(500);
mqtt.setServer(MQTT_HOST,1883);
}
void loop(){
if(!mqtt.connected()) mqtt.connect("ESP32Gesture");
mqtt.loop();
if(!gReady) return;
gReady=false;
if(!apds.isGestureAvailable()) return;
int g=apds.readGesture();
publishGesture(dirName(g));
}How It Works
MQTT Gesture Events: Each gesture publishes a small JSON object to home/gesture containing the direction string and millisecond timestamp. Node-RED subscribes, parses the JSON, and routes each direction to the corresponding home automation action (thermostat up, scene change, etc.).
OLED Real-Time Display: The OLED shows the last gesture direction in large 2x text for immediate visual confirmation. This provides feedback to the user without needing to check a phone or dashboard.
Node-RED Routing: A Node-RED switch node on the gesture topic routes UP/DOWN to a thermostat setpoint increment/decrement flow, and LEFT/RIGHT to a scene selector. A separate subflow logs all gestures to a dashboard history chart.
MQTT Command Subscription: The ESP32 subscribes to home/gesture/cmd to receive mode-switch commands from the broker, allowing remote configuration of what each gesture does without reflashing the device.
Applications
- Whole-home gesture control hub integrated with Home Assistant
- Meeting room lighting and AV control via touchless gestures
- Smart kitchen controller to change recipes or timers without touching screens
- Accessibility remote for users who cannot operate physical buttons
Troubleshooting
Node-RED does not receive gesture events
Check the MQTT topic string is identical in both the ESP32 publish call and the Node-RED MQTT-in node. Topic strings are case-sensitive. Use MQTT Explorer to monitor traffic in real time.
OLED and APDS-9960 conflict on I2C bus
Confirm I2C addresses: APDS-9960 is at 0x39 and SSD1306 is at 0x3C. Use an I2C scanner sketch to verify. Address conflict only occurs if two devices share the same address.
Gesture events flood the MQTT broker
Add a 500 ms cooldown after each publish using a lastPublish timestamp check. The APDS-9960 gesture engine can fire multiple events for a single hand motion.
Upgrades
- Add ESP32 BLE HID mode as a fallback when Wi-Fi is unavailable
- Map gesture sequences (UP-RIGHT) to macro actions in Node-RED
- Add a room-aware mode: different gesture mappings per detected Bluetooth beacon location
- Build a mobile companion app to customise gesture-to-action mappings at runtime
FAQ
You need an ESP32 DevKit, TODO: sensor, APDS-9960 INT, a breadboard, jumper wires, and a USB cable for power and programming.
Only the Advanced stage uses Wi-Fi. Beginner and Intermediate builds run offline on the ESP32 with USB power.
Start with Beginner if you are new to Sensor Projects. Use Intermediate for OLED feedback and Advanced for dashboards or connected monitoring.