Lata's recipe site, and the index at the back of the book
Lata runs RasoiNet, a recipe website with 2 million recipes. (Lata, RasoiNet and every number here are made up for teaching.)
Her grandmother's cookbook taught her the first lesson. To find every mention of "salt" in 400 pages, you don't read the whole book. You flip to the index at the back: "salt: pages 12, 57, 203." A search engine is that back-of-book index, built for millions of pages, plus a way to rank the results.
RasoiNet's growth leads Lata through five systems, and each one is a common interview prompt:
| Lata's problem |
System |
The cookbook version |
| "Searching 'paneer butter' takes 9 seconds" |
Full-text search |
The index at the back |
| "Suggest recipes as people type" |
Autocomplete |
The index's first few letters, pre-sorted |
| "Index recipes from 50,000 food blogs" |
Web crawler |
Collecting the recipes to write the book |
| "Which server got slow at 7 pm?" |
Time-series storage |
The kitchen thermometer log |
| "Advertisers want accurate click bills" |
Ad-click aggregation |
Counting the coupons customers hand in |
This guide relies on Data platforms and Caching.
1. "Searching takes 9 seconds": full-text search
Lata's first search runs LIKE '%paneer%' over every recipe, which means reading all 2 million every time.
The inverted index
Instead of storing "document, then its words", store "word, then its documents":
| Term |
Postings list (document ID, positions) |
salt |
12 (pos 4, 19), 57 (pos 3), 203 (pos 88) |
pepper |
57 (pos 5), 88 (pos 2) |
Reading the table.
- Each row is one entry in the back-of-book index.
- The query
salt AND pepper is answered by intersecting the two lists. Walk both sorted lists together, and the only document in both is 57. Sorted lists make this intersection fast.
- Positions allow phrase queries. In document 57,
salt is at position 3 and pepper at position 5, so "salt and pepper" as a phrase matches there.
You've reached the free preview of this post.
Subscribe to the Blogs plan (₹99/mo) — or use any Algomi Suite, Career, Investor, or Trader plan — to keep reading.
See plans
