A restaurant menu database indexing 800K+ restaurants and 60M+ menu items—built from scratch with infrastructure costs 90% lower than incumbent APIs.
The only viable API charged $0.02 per call—$16,000 just to populate the dataset once.
Collect and manage data from 800K+ restaurants and 60M+ menu items without collapsing under storage or processing costs.
Keep menus current by continuously detecting new restaurants and updating existing menus on a regular cycle.
Deliver search results fast enough to power real user-facing applications despite the dataset's size.
Avoid infrastructure costs that would push the project into the same pricing territory as incumbents.
"Can we design an end-to-end system—data collection, search, and API—that delivers national-scale menu data at a price point early-stage products can actually use?"
Instead of paying per request, I built a system that scrapes, normalizes, indexes, and serves menu data at national scale.
const pipeline = {
collection: 'Distributed Node.js workers',
storage: 'Elasticsearch indices',
refresh: '~6 hour full cycle',
api: 'Express.js REST endpoints'
};Turning the data pipeline into a developer-facing product with self-service access control and subscription management.
What this project achieved and what it demonstrates about building scalable, cost-efficient data infrastructure.