GeoSwipe
A heritage-exploration globe you steer with your bare hands.
- Role
- Full-stack engineering, computer vision and AI integration
- Timeline
- Jul 2025 – Sep 2026
- Stack
- React 19, Express 5, MongoDB, Three.js, MediaPipe, Socket.IO, MapLibre GL, Groq
- Recognition
- Runner-up, AI-Robo Festival 2026 · Exhibited at CIIA-5
- heritage sites, 36 UNESCO
- 126
- external APIs integrated
- 9
- cached lookup, down from 8.7 s
- 2 ms
- quiz questions
- 780
Overview
GeoSwipe puts 126 of India's heritage sites on a live 3D globe and map that you control with hand gestures through your webcam. Every monument opens a hub of live news, weather, 360° views, 3D models, quizzes and an AI guide, and friends can challenge each other to real-time quiz duels.
The problem
Heritage and geography are still taught from flat maps and lists of names. Students memorise places they never get to explore, and most map apps still expect a mouse, a keyboard and patience.
The approach
Put India's heritage on a living 3D globe, then take the mouse away. Hand tracking lets you fly, zoom and select in mid-air, while an LLM guide, live news and weather give every monument context.
AArchitecture
How it fits together
01Browser
- React 19 interface Globe, maps, monument hubs, trip planner
- MediaPipe hand tracking 9 gestures · 20 FPS · fully on-device
- Three.js + MapLibre GL 3D globe and satellite maps
02Express 5 API
- 25 REST endpoints Rate-limited proxy; API keys never reach the browser
- Socket.IO duel rooms 1v1 · 4 modes · 10 rounds
03Data & cache
- Two-tier cache In-memory map, then MongoDB, with per-resource TTLs
- API key pool Rotates on 429, 401 and 403 responses
- MongoDB Sites, quiz questions, flags, images, cache
04External services
- Groq LLM Heritage guide, safety assistant, trip planner
- Live context OpenWeatherMap, NewsAPI, Wikipedia, Unsplash
- Maps & places MapTiler, Overpass (OpenStreetMap)
BEngineering decisions
The calls that mattered
- 01
Moving hand tracking into the browser
The first version streamed base64-encoded webcam frames over a WebSocket to a Python worker. I ported the gesture classifier to JavaScript and ran MediaPipe's hand landmarker directly in the browser, on the GPU with a CPU fallback, so no camera frame ever leaves the device.
Before ~3.2 GB/h webcam data uploaded per userAfter 0 frames leave the device- 9 gesture classes
- 20 FPS inference cap
- 5-frame stability threshold
- Matched the original classifier on 20,000 generated poses with 0 mismatches
- 02
A two-tier cache that stretches free API tiers
Every visitor to the same monument should share one upstream call. Requests hit an in-memory map, then a MongoDB cache with per-resource lifetimes; concurrent misses are de-duplicated, stale answers are served when a provider fails, and a pool of API keys rotates on rate-limit or auth errors.
Before 8.7 s safe-places lookup, uncachedAfter 2 ms the same lookup, cached- News cached for 12 h
- Weather cached for 4 h, shared within ~1 km
- Map styles cached for 24 h
- Safe places cached for 7 days
- 03
Real-time 1v1 quiz duels
Two players join with a room code and race through ten rounds across four modes: flags, geography trivia, India-wide heritage and single-monument questions. Rooms live in memory, are capped and swept every minute, and each socket is rate-limited so nobody can exhaust the server.
- 4 game modes
- 10 rounds per match
- 500 concurrent rooms maximum
- 30 room actions per 10 seconds per socket
CIn the product
Screens
Outcome
Runner-up at AI-Robo Festival 2026 and exhibited at the national-level CIIA-5 innovation showcase at Nehru Science Centre, Mumbai. GeoSwipe is live on Cloudflare Pages.