How to build a game of your own city with an AI agent

• The one idea
• What you'll build
• What you need
• Step 1 · One loop
• Step 2 · Ground truth
• Step 3 · Graybox
• Step 4 · One landmark
• Step 5 · Ride one delivery
• Step 6 · The blind test
• Step 7 · Fix one thing
• Step 8 · Grow it
• Worked example: Dizengoff
• Mistakes to skip
• The kickoff prompt
• Gallery
• What it doesn't do yet

The one idea

You don't need to be a game engineer. You need a small first milestone, a method you can copy, and prompts you can paste. That's this page. My part was picking the scope, the gameplay, what mattered visually, and looking at every render. The agent wrote the Blender Python, wired up Three.js, wrote the tests and ran them.

What you'll build

A browser game with one loop the player repeats:

And the pipeline that makes the city look like your city:

What you need

• An agent with hands. Access to your project files, a terminal, Blender and a browser. If a tool isn't available it has to say so. Mine could run Blender headless, start a local server and drive a browser.
• The tools. Blender (I used 4.5), Node.js 22 or newer, a local HTTP server, a browser with WebGL.
• Map data and dated photos for one small area. Roads and bike lanes from OpenStreetMap, building footprints from your city's open data, street photos with a date you can defend.
• A small scope. One 100 to 200 m stretch or one square, and one repeatable task. Resist the whole city.

1Define one loop

Before any world exists, write down what the player does every thirty seconds and how they know they won or lost. Pick up an order, reach a destination, clear success and failure feedback. Nothing else.

You get: a one-page brief with loop, controls, success, failure, restart, and what's out of scope.

Build a browser-playable game about [local activity]. State the immediate objective, level objective, progression and lose condition. Keep the tone affectionate. Make the first playable loop coherent before adding content. Use an original, unbranded visual identity. List what the player does every 30 seconds and how they know they succeeded.

2Lock the ground truth

Pick one map origin (longitude, latitude), one reference camera with position, heading and field of view, and the footprint of one building. Pull roads and bike lanes from OpenStreetMap and footprints from your city's open data. Collect street photos and write down when each was taken.

Keep a ledger from the first download: URL, retrieval date, photo date, footprint and road IDs, and a column that says measured or estimated. The day you downloaded a map and the day a photo was taken are different things, and filenames lie.

For Tel Aviv the origin was longitude 34.774207, latitude 32.077963. In Blender, X is east, Y is north, Z is up. Three.js gets [x, z, -y]. Every asset lives on that one grid. It's a flat local approximation, which is fine for a compact district.

You get: a source folder with the ledger, the footprint IDs and the dated reference photo. Without one shared origin every asset lands somewhere slightly different and nobody recognizes the street.

Reconstruct [specific 100 to 200 m section] from real evidence. Gather street and bike-lane geometry, building footprints, dated street-level photos and a matching aerial. Record source URLs, IDs, retrieval dates and photo capture dates. Lock one hero camera with coordinates, heading and field of view. Distinguish measured geometry from photo estimates and invented detail. Do not start with a generic city and label it afterward.

3Graybox the real place

Plain boxes for the actual road, sidewalk and main buildings, at true scale, on the real footprints. Ride through it. Does a scooter fit? Do the curbs make sense? Does the block feel the right length? Fix scale here, while it's cheap.

You get: a screenshot of the graybox from the locked camera, next to the photo. I skipped this step once, modeled a whole corner of Rothschild on the wrong scale, and threw it away.

4Model one landmark in Blender

One Python module per landmark, with a build() function, its source data next to it, and named parts. Original geometry from the photos and the footprint. Compare silhouette, window widths, balcony rhythm, facade depth, colors and paving against the photo. Don't paste photos onto facades; it keeps the style consistent and you're not redistributing anyone's pictures.

Two things stay separate on purpose: parts that move (water, traffic lights, character limbs) stay as their own nodes, and materials that can change (scooter paint, skin tone, glass, metal, paving) keep their own material role so the game can swap them without touching geometry. Save the Python, the .blend, the .glb, a preview render and a short note on what you assumed.

You get: five files with the same base name, plus triangle count and dimensions. If the script can rebuild the asset from scratch, the asset is real. If it can't, you have a screenshot.

Build only [named building] in Blender from the supplied photos and footprint. Return a reproducible Python module, .blend, preview and a short source/assumption report. Use metres and the provided geographic origin. Preserve the actual plan; match facade rhythm, balconies, ground floor and signs. Keep articulated parts as separate nodes and keep material roles separate. No downloaded 3D assets. Do not edit the parent project or deploy anything. Give the parent exact orientation, dimensions and triangle count.

5Load it in Three.js and ride one delivery

GLTFLoader, camera, lights, shadows, scooter movement. Pickup, ride, drop, hit something, restart. Nothing else yet. The surfaces you ride on are the meshes the renderer draws, with ground-height and blocker queries on top, so what you see and where you can ride can't drift apart.

Water goes in GPU shaders on elapsed time, so it moves the same at any frame rate and the CPU never rebuilds the mesh. Repeated people and vehicles are instanced: one mesh, many copies, one draw call. That's how 487 pedestrians cost roughly what a few do.

You get: a screen recording of one complete loop in the browser.

6Ask "where is this?" before you show the answer

This is the step that makes the city recognizable, and most people do it backwards. Don't show the agent the photo and ask "does it match?" Run the review in two stages. First, a fresh reviewer that has seen nothing gets a render with a neutral filename and one question: which street is this, and which way are we looking? Only after it answers do you show the matched photo and ask for the differences. Show the photo first and you've contaminated the test.

My bar is three blind identifications in a row. A fresh AI critic is useful evidence. The real test is a local looking at the screen and saying "I ride there every day."

beforeafter

You get: a list of tells, ranked by how much each one hurts recognition.

Stage one. You are an independent critic. From this neutrally named render alone, identify the street or place and the viewing direction. State your confidence and what you based it on. Do not ask for hints.

Stage two, sent only after stage one is answered. Here is the dated reference photo from the same angle. List the differences in street geometry, scale, facade proportions, signs, cars, materials and lighting. Rank the three changes with the greatest recognition benefit. Do not reward labels for fixing incorrect geography.

7Fix one thing, then measure

Take the top tell and fix only that. Same camera, same coordinates. Rebuild, capture the same view, have a different critic compare. Then measure the real browser build before you add anything: frames per second, draw calls, triangles, with the device and method written next to the number.

Named parts make this safe. When the fountain looked wrong, the instruction was "change only fountain_metal and water_jet, keep the camera." The agent couldn't accidentally touch the hotel.

That one message turned the project. Foliage went on hold and the effort moved to street geometry, footprints, signs and cars. The agent optimizes what it can measure. You tell it what people actually recognize. Expect to send a message like this, and look at the render from the player's camera, not the aerial showcase, when you decide.

You get: before and after from the same camera, and one number with the device and method next to it.

Pick the highest-impact mismatch from the critique. Change only the responsible asset or system. Preserve coordinates and camera. Rebuild, capture the same view and have a different critic compare. Report the actual result even if the improvement is small. Measure performance using the same device and method as last time, and say what they were.

8Grow it one tested section at a time

Repeat steps 2 to 7 for the next block. Keep the same origin, the same file layout and the same review. Three things I'd decide early:

• Navigation. A graph of validated route segments with ground-height and blocker queries on the assembled world. Off the route, guide the player back to the trail instead of inventing a line through obstacles. Buses need their own dimensions and collision envelopes; a bus-sized box on a car's path clips through everything.
• Packaging. Keep one self-contained HTML file with the world embedded for review and sharing with friends. For players, split the runtime and the assets into separate content-hashed files. Mine went from a 43 MB page to a 56 KB shell plus about 29 MB of assets. GLB is the model format, gzip shrinks transfer, Draco shrinks geometry. Three different problems.
• Performance. Instancing for repeated actors, static grouping with spatial boundaries so culling still works, distance culling, a capped pixel ratio, limited shadows. Merge a whole city into one object and culling dies. Batching needs boundaries.

Keep a log at every milestone: the exact prompt, the sources, before and after, the validation result, one reusable lesson. It's what let me write this page.

After each meaningful milestone, save the exact prompt, sources, before/after render, script/build, validation result and one reusable lesson. Mark retrospective summaries as summaries. Keep concept art separate from actual screenshots. Never record a test as passed unless it ran.

Worked example: Dizengoff Square

The whole method on one square, small enough to follow.

The source packet

The municipal footprint for the Cinema Hotel, the OSM ways for the circular bike route and the roads that meet the square, a restoration architect's photo of the fountain and the hotel, and a street photo of the striped cafe corner with an unverified date, marked as such. Measured: the footprint, the road centerlines, the bike loop. Estimated from photos: balcony depth, ground-floor heights, awnings, furniture.

The coordinate conversion

Every source gets converted into the same local metres before anything is modeled:

east  = (longitude - originLongitude) * 111320 * cos(originLatitude in radians)
north = (latitude  - originLatitude)  * 111320
Blender position  = (east, north, elevation)      # X east, Y north, Z up
Three.js position = [east, elevation, -north]    # X east, Y up, Z south

This keeps the building, the street sign, the riding surface and the map marker in the same place: they were all born on the same grid.

The named parts

The square's export contains, among others: facade_cinema, facade_perimeter, fountain_metal, water_water, water_jet, dizengoff_world (the ground), plus separate signal, car and character assets.

The first blind review

The reviewer named Dizengoff, helped by the hotel lettering, and couldn't say which direction it was looking. The tells it listed became the next three fixes: fountain metal, the ring road width, the cafe corner. Then the lettering came off, because real businesses stay unbranded in the game.

Mistakes to skip

Each of these cost me at least a rebuild.

• Beauty before boxes. I modeled facades before the scale and the loop were right. Graybox first, every time.
• Naming the place in the review prompt. Early reviews were worthless because the critic already knew the answer. Two stages, blind first.
• Trusting filenames. Images saved as "aerial" and "plan" turned out to be historical photos and demolition scenes. Every image gets a date and a status the moment it lands.
• One giant HTML file for players. Perfect for review, wrong for players. Decide the packaging on day one.
• Looking at the aerial view. The player never sees it. Judge from the player's camera.

And four bugs you will meet, with the fix:

The kickoff prompt

Give this page to your agent (the Markdown download has everything, including the file layout), fill in the inputs below, and paste it as the first message. Start with one recognizable street and one playable task. It won't finish in one click. It will start in the right place.

Use the attached guide as a workflow for building an original browser-playable 3D game. Adapt the method to my location and idea; do not copy Tel Aviv-specific coordinates, names or geometry.

My inputs:
- Place and small starting area: [city, street or square; about 100 to 200 m]
- Player activity and objective: [what the player does and how they succeed]
- Visual direction: [stylized, low-poly, realistic proportions, etc.]
- Target devices: [desktop, mobile, or both]
- Available reference photos and maps: [links or files]
- Time and compute constraints: [optional]

First read the guide and inspect the available project and tools. Identify the essential missing inputs. Check whether you can run Blender, Node.js and a browser. State any unavailable tools instead of pretending to use them.

Create a short implementation brief: first playable loop, controls, success, failure, restart, one small geographic boundary and explicit exclusions.

Before modeling the location, collect dated references, mapped roads and building footprints. Record source URLs and IDs and separate measured facts, photo estimates and invented detail. Pick one shared origin, units, axis conversion and a locked comparison camera. Do not substitute a generic street.

Build a graybox to check scale and clearance. Then create original geometry using reproducible Blender Python scripts. Save the source, .blend, .glb, preview and source/assumption notes. Preserve named articulated parts and material roles needed for animation and customization. If multiple workers are available and appropriate, give each one bounded asset ownership and the same coordinate and export contract.

Load the exports in Three.js. Implement one complete playable task before expanding the map. Derive navigation and blockers from the same assembled world, and test the actual exports. Keep the environment recognizable from the player's camera, not just from an aerial showcase.

Capture the locked view and compare it with the dated reference. For an independent recognition test, show a fresh reviewer only the neutral render first; reveal the reference afterward. Fix the highest-impact mismatch, rebuild and repeat. Do not claim recognition based only on your own review.

Use batching, instancing and culling where appropriate. Measure the real browser build and report the device, viewport, method and remaining limits. Verify controls, collisions, completion, failure, restart and loading. If mobile is a target, distinguish responsive layout from physical-phone tests.

Keep a progress log with prompts, sources, before/after views, validation and next steps. Work locally and make each milestone reviewable. Do not publish or connect external services without authorization. Never invent successful tests, precise geography, playtest scores, development costs or elapsed time.

Start now with the tool check, the brief and the reference packet for the first section. Continue through the first runnable, reviewable milestone; do not stop after giving me a plan.

Gallery

Renders from the September 2026 builds. Labels say which parts are playable and which are scenic studies.

What it doesn't do yet

• It's not a 1:1 reconstruction. Footprints and roads are measured; a lot of heights and facade details are estimated from photos.
• The playable area is about 2 km with eight stops around Dizengoff, Frishman, the promenade and Opera Square. The other neighborhoods are scenic studies for now.
• No phone benchmark and no public link yet. The game is a local release candidate until it passes device tests.

Map data © OpenStreetMap contributors (ODbL) and Tel Aviv-Yafo open data. Blender 4.5, Three.js, Node.js 22, and an AI coding agent. Reference photos belong to their photographers and aren't redistributed here. Music: "Costa Cruising", a track of mine made with Suno. The game was called Tel Aviv Run while it was being built.