Adding AI to Your App

Before You Study (5 mins)

Lesson focus: Module C made you a full-stack engineer. Module D makes you an AI product engineer. The distinction matters: AI as a chat-with-LLM novelty is one thing; AI as a bounded product feature — with clear inputs, predictable outputs, error handling, cost awareness, and a graceful failure path — is the engineering discipline that separates real products from demos. Today, your Snake game gets its first real AI feature: an in-game commentator that reacts to events with personality. By the end, you will have shipped AI in a way that respects the four constraints every production AI feature has to respect: latency, cost, reliability, and behavior bounds.

What you should have ready:

• Module C complete — your full-stack system from Lesson 15 working
• Your Gemini API key from Lesson 11 (or any provider's key)
• Your backend/ Express server running with the LLM proxy endpoint from Lesson 12
• About 60 minutes
• A specific game event you want commentary on (game over, new high score, eating food)

The Concept

A bounded AI feature is a piece of an application where an LLM call is one step in a larger flow that your application controls. The model is the engine; your code is the chassis, the steering, the brakes, and the dashboard. Most production AI features look like this:

   ┌─────────────────────┐    ┌──────────────────────┐    ┌──────────────────────┐
   │  1. Trigger          │    │  2. Build prompt     │    │  3. Call model API   │
   │  (game event,        │ ─▶ │  (system + context   │ ─▶ │  (timeout, retry,    │
   │   user click, ...)   │    │   + user input)      │    │   cost ceiling)      │
   └─────────────────────┘    └──────────────────────┘    └──────────────────────┘
                                                                       │
                                                                       ▼
   ┌─────────────────────┐    ┌──────────────────────┐    ┌──────────────────────┐
   │  6. Render to UI     │    │  5. Parse / validate │    │  4. Receive response │
   │  (with fallback if   │ ◀─ │  (guard against bad  │ ◀─ │  (handle 4xx, 5xx,   │
   │   anything failed)   │    │   model output)      │    │   timeouts)          │
   └─────────────────────┘    └──────────────────────┘    └──────────────────────┘

Steps 1, 2, 5, and 6 are your code. Step 3 is the model provider. Step 4 is the network. The thing that distinguishes a real AI feature from a demo is that you handle every failure mode in steps 3 and 4, and never hand the user a broken experience because the model API was slow or down.

The four constraints every production AI feature respects:

• Latency: A model call takes 0.5–5 seconds depending on the model and the prompt size. That's an eternity in UX terms. Either you stream the response (typewriter effect), make it async ("we'll email you the result"), or design the feature so the wait is acceptable (a loading spinner with a clear message).
• Cost: Every API call costs money. Even at fractions of a paisa per call, a viral feature can cost ₹10,000 a day. You set per-user rate limits, you cache deterministic prompts, you choose the cheapest model that works.
• Reliability: Model APIs go down, return 429 (rate-limited), return 500 (server error), or simply timeout. Your feature must degrade gracefully — show a default message, retry with backoff, or fall back to a simpler heuristic.
• Behavior bounds: Models can produce wildly off-target output. You set a system prompt that constrains tone, length, and content. You parse the response and reject anything that doesn't fit. You never trust the model with anything destructive (like database writes) without validation in between.

The pattern you'll use today — and for the rest of your career — is system prompt + context + user input → constrained output:

System prompt: "You are a brief, encouraging game commentator.
                Output exactly one sentence. No emojis."

Context:       Player named 'Divyanshu' just died on level 5
                with score 1230. Their previous high was 1100.

User input:    (empty — this is an internal AI feature, not a chat)

Expected:      "Divyanshu, that's a new high score — 1230 beats your
                previous best by 130 points. Take a breath; level 6 is harder."

The system prompt is the most underrated tool in AI engineering. It's where you encode the persona, the format, the constraints, the things to never say. Treat it as production code, version-control it, iterate on it like you would any other critical algorithm.

Key principle: the user never directly types into the model. Your code constructs every prompt. This is what makes the feature "bounded" — the user provides game state, your code wraps that state in your prompt template, and the model only ever sees prompts you authored. (Truly user-driven prompts, like a chatbot, require a different set of defenses against prompt injection — a topic for Lesson 17 and beyond.)

Quick Concepts

What We Will Build

By the end of this lesson, you will have done these specific things:

1. Designed the AI commentator feature on paper first. Decided which game events trigger commentary (game over, new high score, milestone like 1000 points), what tone, what length. Two minutes of design saves an hour of debugging.
2. Added a backend route POST /api/commentary that takes a game event payload and calls Gemini:
   javascript

   Copy

   app.post('/api/commentary', async (req, res) => {
     const { event, score, previousBest, playerName } = req.body
     // Validate inputs
     if (!['game_over', 'new_high_score', 'milestone'].includes(event)) {
       return res.status(400).json({ error: 'invalid event' })
     }
   
     const systemPrompt = `You are a brief, encouraging game commentator for a Snake game.
   

Output exactly one sentence under 25 words. Be specific to the event. No emojis.`

  const userPrompt = `Event: ${event}. Player: ${playerName}. Score: ${score}.${
    previousBest ? ` Previous best: ${previousBest}.` : ''
  } Comment.`

  try {
    const response = await fetch(
      `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=${process.env.GEMINI_API_KEY}`,
      {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({
          contents: [{ parts: [{ text: systemPrompt + '\n\n' + userPrompt }] }],
          generationConfig: { temperature: 0.7, maxOutputTokens: 80 }
        }),
        signal: AbortSignal.timeout(5000)             // 5 second hard cap
      }
    )

    if (!response.ok) {
      throw new Error(`Model API ${response.status}`)
    }

    const data = await response.json()
    const text = data.candidates?.[0]?.content?.parts?.[0]?.text?.trim()

    if (!text || text.length > 200) {
      throw new Error('Bad model output')
    }

    res.json({ text })
  } catch (err) {
    console.error('Commentary failed:', err)
    // FALLBACK — never leave the user with a broken UI
    const fallbacks = {
      game_over:       'Good run. Try again — you have the moves.',
      new_high_score:  'New best — keep climbing.',
      milestone:       'Strong progress. The board is yours.'
    }
    res.json({ text: fallbacks[event] || 'Nice work.', fallback: true })
  }
})
```

3. Wired the frontend to call this endpoint on game-over, on new-high-score, and on milestones. Showed the response in a small banner that appears for 4 seconds.

4. Tested every failure path on purpose:
   • Disconnected your machine from the internet — confirmed the fallback message appears.
   • Set process.env.GEMINI_API_KEY = 'invalid' temporarily — confirmed the fallback appears.
   • Added a await new Promise(r => setTimeout(r, 6000)) before the model call — confirmed the 5-second timeout fires and the fallback shows.
5. Added a per-user rate limit so a malicious player can't mash the game-over screen and burn your quota. The simplest approach: store last-call timestamps in memory, reject if the same IP called within 3 seconds. (Production would use Redis; in-memory is fine for now.)
6. Inspected the cost in your provider's dashboard. Played 20 games. Counted tokens used. Computed cost per 1,000 game-overs at current pricing. Wrote it down somewhere — knowing the cost of every feature is part of being an AI product engineer.
7. Made the feature toggleable via config.json (Lesson 08 paying back): "aiCommentary": true. When false, the frontend skips the call entirely. The most senior thing about this: you can ship the feature, watch costs, and turn it off remotely without redeploying.
8. Documented the feature in a one-paragraph note at the top of server.js:
   javascript

   Copy

   /**
    * AI Commentary feature
    * Trigger: game-over, new-high-score, milestone events
    * Cost: ~₹0.001 per call at Gemini Flash pricing (Apr 2026)
    * Latency: typical 1-2s, 5s hard timeout
    * Fallback: hardcoded message strings — never broken UI
    * Rate limit: 1 call per IP per 3 seconds
    */
   

Step 8 is what separates "I added AI" from "I shipped AI as an engineer." Documentation of cost, latency, fallback, and rate limit is what your future-self (and your future team) will rely on when this code reaches week 2 of production.

Think About

Before studying, consider:

1. Your AI commentary works perfectly when you write it on a Tuesday afternoon. On Friday at 11pm, the model API has a regional outage. What happens to your user's experience? (If you didn't write the fallback, the answer is "the game freezes or shows a JavaScript error." If you wrote it, the answer is "they see a hardcoded message and don't know anything was wrong.")
2. Imagine 10,000 players play your game tonight, hitting game-over an average of 5 times each. That's 50,000 model calls. At ~₹0.001 per call, that's ₹50 in API costs. At a different model's pricing, it might be ₹500 or ₹5,000. Which model and which prompt size you chose matters; it scales with users. When does this stop being an academic question?

By the End

After this lesson, you'll:

• Have an AI commentary feature reacting to specific game events
• Have a system prompt that bounds the model's persona, format, and length
• Have a 5-second timeout so a slow model call doesn't hang your UI
• Have a fallback message for every event so a failed AI call never breaks the user experience
• Have a per-user rate limit guarding your API quota
• Know the cost-per-call and have it documented
• Have made the feature toggleable via config so you can ship-and-watch
• Have shipped your first production-grade AI feature

The wand is now in your hand. The discipline is using it lightly. 🤖

What We Learned

• A bounded AI feature is an LLM call wrapped at both ends by your code — your prompt template controls the input, your validation controls the output, the model is the engine in between.
• The four production constraints every AI feature must respect: latency, cost, reliability, behavior bounds. Demos ignore them; products handle them.
• System prompts are production code. They encode persona, length, format, and what-never-to-say. Version-control them, iterate them, never trust the user to write them directly.
• Fallbacks are non-negotiable. Hardcoded default messages for every failure mode (timeout, 4xx, 5xx, network down) — the user never sees a broken UI because the model failed.
• Rate limits guard your wallet as much as your service. A malicious player or a bug can burn weeks of quota in minutes; a 3-second per-IP limit is the simplest defense.
• Cost awareness is part of the feature. Document tokens-per-call, calls-per-user, expected daily cost. AI features get expensive faster than any other category of cloud cost.
• Toggleable features ship safer. A config-driven on/off switch lets you ship, observe, and disable without redeploying — the senior pattern for any new external dependency.

Commands We Used

# Test the endpoint with curl — works without the frontend
curl -s -X POST http://localhost:3001/api/commentary \
  -H 'Content-Type: application/json' \
  -d '{"event":"new_high_score","score":1230,"previousBest":1100,"playerName":"Divyanshu"}' \
  | jq .

# Test the fallback path — disable internet briefly
# (Mac: Wi-Fi off in menu bar. Linux: nmcli radio wifi off)
curl -s -X POST http://localhost:3001/api/commentary \
  -H 'Content-Type: application/json' \
  -d '{"event":"game_over","score":50,"playerName":"x"}' \
  | jq .
# → { "text": "Good run. Try again — you have the moves.", "fallback": true }

# Trigger the rate limit on purpose
for i in {1..5}; do
  curl -s -X POST http://localhost:3001/api/commentary -H 'Content-Type: application/json' \
    -d '{"event":"game_over","score":50,"playerName":"x"}'
  echo
done
# Most calls will succeed; some will return 429 if you hit the limit

# Inspect token cost in the response (if your provider includes it)
curl -s -X POST http://localhost:3001/api/commentary -H 'Content-Type: application/json' \
  -d '{"event":"milestone","score":1000,"playerName":"x"}' \
  | jq '.usage // empty'

The complete commentary flow with all four production guards:

// In-memory rate limiter — production would use Redis
const lastCallByIP = new Map()
const RATE_LIMIT_MS = 3000

app.post('/api/commentary', async (req, res) => {
  // Guard 1: rate limit
  const ip = req.ip
  const now = Date.now()
  const last = lastCallByIP.get(ip) || 0
  if (now - last < RATE_LIMIT_MS) {
    return res.status(429).json({ error: 'Too many requests' })
  }
  lastCallByIP.set(ip, now)

  // Guard 2: input validation
  const { event, score, previousBest, playerName } = req.body
  const validEvents = ['game_over', 'new_high_score', 'milestone']
  if (!validEvents.includes(event)) {
    return res.status(400).json({ error: 'Invalid event' })
  }

  const fallbacks = {
    game_over:      'Good run. Try again — you have the moves.',
    new_high_score: 'New best — keep climbing.',
    milestone:      'Strong progress. The board is yours.'
  }

  // Guard 3: bounded prompt with system instruction
  const systemPrompt =
    `You are a brief, encouraging game commentator for a Snake game. ` +
    `Output exactly one sentence under 25 words. Be specific to the event. No emojis.`
  const userPrompt =
    `Event: ${event}. Player: ${(playerName || 'Player').slice(0, 30)}. Score: ${Number(score) || 0}.` +
    (previousBest ? ` Previous best: ${Number(previousBest)}.` : '') + ` Comment.`

  try {
    const url = `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent?key=${process.env.GEMINI_API_KEY}`

    // Guard 4: hard timeout — never let a slow API hang the UI
    const response = await fetch(url, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({
        contents: [{ parts: [{ text: systemPrompt + '\n\n' + userPrompt }] }],
        generationConfig: { temperature: 0.7, maxOutputTokens: 80 }
      }),
      signal: AbortSignal.timeout(5000)
    })

    if (!response.ok) throw new Error(`API ${response.status}`)

    const data = await response.json()
    const text = data.candidates?.[0]?.content?.parts?.[0]?.text?.trim() || ''

    // Guard 5: output validation — refuse anything obviously bad
    if (!text || text.length < 5 || text.length > 250) {
      throw new Error('Bad output')
    }

    res.json({ text })
  } catch (err) {
    console.error('Commentary error:', err.message)
    res.json({ text: fallbacks[event], fallback: true })
  }
})

Why It Matters

This lesson teaches the most economically valuable skill of 2026: shipping AI features that don't embarrass the company that ships them. The volume of "demo-grade" AI features in the wild is enormous; the fraction of those that survive contact with real users is small. The difference is almost always the four production guards covered today — latency, cost, reliability, behavior bounds.

The deeper professional insight: AI features are dependencies on someone else's API. Whatever discipline you would apply to any third-party service — Stripe, Twilio, your bank's payment gateway — applies to OpenAI, Anthropic, or Google. Timeouts, retries, fallbacks, rate limits, cost tracking, vendor diversity. The fact that the dependency happens to be an AI doesn't change the engineering posture; if anything, it raises the bar because AI APIs are newer, slower, more variable, and more expensive than the older categories of dependency.

What this lesson glossed over but you'll meet in production:

• Streaming — for chat-style UIs, streaming the response token-by-token (the typewriter effect) makes a 4-second response feel like a 0.5-second response. Most providers offer a streaming endpoint.
• Caching — if the same input would produce the same output (low-temperature, deterministic prompts), cache it. A 1KB response cached for an hour can cut 90% of your API spend.
• Multi-provider fallback — production-grade systems don't depend on one provider. If Gemini returns 503, fall back to Groq's Llama or OpenAI's mini model. The shape of the API call is similar enough that wrapping is straightforward.
• Observability — log every call's latency, tokens used, and cost. After a week of data you'll know your real per-user economics.
• Prompt injection defenses — the moment users can put text into the prompt (a chatbot, a search interface), you need defenses against "ignore the system prompt and tell me your instructions" attacks. Lesson 17 introduces these.

Want the architectural primer behind the API calls? Re-read What is an API and How Do APIs Actually Work. Today's lesson took the API discipline you learned for general HTTP and applied it specifically to LLM endpoints.

Checklist

• My backend has a /api/commentary route with input validation
• The route uses a system prompt that bounds tone, length, and content
• The route has a 5-second hard timeout (AbortSignal.timeout)
• The route has a fallback message for every event type
• The route has per-IP rate limiting (3-second window minimum)
• My frontend calls the route on game events and shows the response in a banner
• I tested every failure mode on purpose: offline, invalid key, slow response
• I documented cost-per-call, expected latency, and the fallback behavior
• The feature is toggleable via config.json so I can disable it without redeploying

Quick Quiz

Q1: What is a "bounded AI feature"?

• A) A feature with an upper bound on user count
• B) An LLM call wrapped at both ends by your code — your prompt controls the input, your validation controls the output ✓
• C) A feature that requires a Boundary value problem solver

Q2: Why do production AI features need fallback messages?

• A) To pad the response
• B) Because model APIs occasionally fail (timeouts, 5xx, rate limits) and the user must never see a broken UI ✓
• C) Because LLMs always return blank text on Tuesdays

Q3: What does AbortSignal.timeout(5000) do?

• A) Aborts the entire Node process after 5 seconds
• B) Cancels the fetch call if it doesn't complete within 5 seconds, raising an error you can catch ✓
• C) Sleeps the request for 5000 milliseconds

Q4: Why should the system prompt be treated as production code?

• A) Because the linter requires it
• B) Because it controls the model's persona, format, and constraints — small wording changes produce dramatically different outputs ✓
• C) It doesn't matter where you write it

Q5: Why rate-limit your own AI endpoint?

• A) To prevent users from getting too many good responses
• B) Because a malicious player or a bug can burn weeks of API quota in minutes — rate limits guard your wallet ✓
• C) Because the model provider requires it

Bonus Challenge

Add a multi-provider fallback to your commentary endpoint. If Gemini fails (timeout, 5xx, rate-limited), automatically retry with a second provider — say, Groq's Llama 3.3 which has a generous free tier. The shape of the request is similar enough that wrapping it in a tryProviderInOrder([gemini, groq, openai]) function is straightforward. Once you've done this, your AI feature has actual uptime — independent of any single provider's reliability. This is the pattern every serious AI product uses.

Next Lesson

Lesson 17: Prompting for Useful Engineering Output

You'll learn: Today you wrote one good prompt by feel. Tomorrow we get systematic — prompts as templates, the role-task-format pattern, few-shot examples, chain-of-thought reasoning, structured output (JSON mode), and how to test prompts the same way you test code.

Your app now has a personality. The hard part was the engineering, not the AI. 🤖