E1 Mono — cross-platform agent evaluation (expo → web, canary)
Does the cross-platform agent reliably build a mobile (Expo) app and a web app on one shared backend? Tested on canary across four build paths — a plain Expo parent, two forks that add a web app (one at feature parity with mobile, one an admin console), and one project built entirely from scratch — to decide whether the expo → web path is ready for a wider audience. Every number is queried per job and sourced (tap any i for the query + evidence); this report does not pre-judge a winner.
Verdict from functional + integrity: all 4 arms produced a working web app and a working Expo app on one shared backend. GA gated on hardening + deploy/native chain (see Open items).
Live UI testing (user-driven, guide mode) across the e-com and notes families: 31/31 + 28/29 = 59/60. The one non-pass is a blocked notes-admin edit control (minor). Fitness family not run.
Verified live on the from-scratch app: an order placed on web (#E0A587ED) appeared in the mobile app under the same account; web REACT_APP_BACKEND_URL = mobile EXPO_PUBLIC_BACKEND_URL. User-observed on the e-com fork family.
Reconstructed shipped source from each job's trajectory (
reconstruct.py), reviewed by ONE isolated reviewer per app on a fixed rubric (arch30 / reliability25 / tests20 / frontend25). Not a DB metric. Arm value = mean of the arm's apps. Evidence: per-app scores in the Code-quality table + Job details.The agent reliably builds a connected mobile app and website from one prompt. The generated code and the delivery steps need work before opening it to everyone.
Across all four build paths, a single prompt produced a working mobile app and a working website that share one backend: anything created on one (say, an order on the website) shows up on the other (the mobile app) for the same user, so the two are genuinely one product rather than two disconnected copies. The code runs and is consistent, but it is MVP-grade and carries the same security gaps in every build; and the steps that actually put an app in users' hands (deploy, native mobile builds, custom domain) were not exercised here. So this is a clear yes on the agent's core ability, and a not yet on general release — the remaining work is hardening and delivery, not build capability. Cost and speed differences between the paths are small and sit inside normal build-to-build variation.
Readiness gates
Each thing that has to be true before opening E1 Mono to a wider audience, with an honest status and what it would take to turn it green. Green = ready; amber = works but needs attention; red = blocks a public release; grey = not tested in this eval.
| Release gate | Status | What we found | To reach GO |
|---|---|---|---|
| Builds both apps on one backend | GO | All four build paths (Expo parent, both fork types, and from-scratch) produced a working mobile app and a working website backed by a single shared backend. | - |
| Apps stay in sync | GO | Data created on one surface appears on the other for the same user - so the website and the mobile app behave as one product, not two disconnected copies. Confirmed live on one product, observed on another. | Re-confirm on more products |
| Functional correctness | GO | 59 of 60 user-facing checks passed across the e-commerce and notes products, driven live in the UI (one minor blocked item). The fitness product was not run. | Run the fitness product checks |
| Code quality | CAUTION | 73/100 on average across all 10 apps (a tight 67-81 band). The code works and is consistent, but it is MVP-grade, not hardened. | Security + test hardening (below) |
| Security | NOT READY | Every build ships the same weaknesses: a secret key and a default admin password hard-coded into the source, an over-permissive access policy, and login tokens stored where a page script could read them. | Fix in the build recipe before any public exposure |
| Deployment | PIPELINE OK | All 10 apps deployed on canary (2026-07-21): every pipeline step passed - build, DB migrate, secrets, deploy, and internal health check - and each app is active. But canary deploys land on an internal cluster, so the public emergent.host URL returns "Deployment not found"; the pipeline is proven, public serving is not yet. | One prod deploy to confirm public serving |
| Native mobile builds (Android / iOS) | TRIGGERED | All 10 apps had Android + iOS builds triggered via the build API on 2026-07-21 (20/20 accepted, HTTP 200, build IDs returned) - no keystore dialog needed (EAS auto-manages credentials). Build success / artifacts are not yet verified: canary native-build status lives in the canary deployer (not reachable from here), and the prod deployer / emergent-dev EAS account do not hold them. The build pipeline itself is proven (other orgs' builds record success/failed). | Confirm artifacts per app via Get builds (UI) / EAS |
| Custom-domain linking | BLOCKED ON PROD | Tested on canary with a real domain (web-test.us): linking is attempted but the app returns "Deployment not found". Root cause (confirmed with QA lead): canary / internal-cluster apps serve from Kubernetes, whereas custom domains only resolve when the app is served through Cloudflare - i.e. an external / prod deploy. Not a mono defect; a canary-environment limitation. | Validate on one prod (Cloudflare-served) deploy |
| Reliability at scale | CAUTION | Every correctly-set-up build succeeded, but the sample is small (one per product per path) and there are known rough edges - see the Reliability notes. | More builds across more products |
Scoreboard — cost, speed and quality by build path
Every metric queried per job and averaged across each build path. Cells are conditionally formatted per row (greener = better on that metric's direction), the best value in each row is bold with a green ✓, and the tally counts, objectively, how many metrics each build path leads. Tap the i on any metric for its source and query.
| Metric | Expo parent3 jobs | Parity forkweb = mobile | Admin forkweb != mobile | From scratch1 job, AUTO |
|---|---|---|---|---|
| Code qualityiCode quality /100 Reconstructed shipped source from each job's trajectory ( reconstruct.py), reviewed by ONE isolated reviewer per app on a fixed rubric (arch30 / reliability25 / tests20 / frontend25). Not a DB metric. Arm value = mean of the arm's apps. Evidence: per-app scores in the Code-quality table + Job details.higher is better | 72.7 | 76.7 ✓ | 70.7 | 70.0 |
Build costiBuild cost (USD)credit_ledger.amount for the job's LLM_CALL rows (dollars, not ECU). SELECT sum(amount) FROM credit_ledger WHERE job_id=<id> AND type='LLM_CALL'. Arm = mean over jobs. Per-job values in Job details.lower is better | $5.25 ✓ | $6.34 | $9.90 | $5.99 |
ECU chargediECU chargedcredit_ledger.ecu (= usages.value_in_usd, a misnomer) - internal Emergent compute units, not USD. Dollars = amount / scaling_factor.lower is better | 29.1 ✓ | 35.8 | 53.4 | 32.9 |
Total tokensiTotal tokensusages prompt + completion (the *_new columns, not the deprecated total_tokens). Arm = mean, millions.lower is better | 4.06M | 4.96M | 8.34M | 2.90M ✓ |
| Active latencyiActive latency answer -> finish from traj_payload->>'timestamp' (NOT created_at, which is skewed on EPHs); sum of gaps < 300s so idle/HITL waits are excluded. Minutes, arm mean.lower is better | 29.9 | 25.0 | 37.3 | 22.2 ✓ |
Agent stepsiAgent stepscount(*) of trajectories rows for the job. Context metric - more steps is not inherently worse. contextlower is better | 55 | 68 | 97 | 41 |
| Parallel tool-callsiParallel tool-calls Share of steps whose traj_payload->>'function_name' = 'PARALLEL_TOOLS' (agent batched tool calls in one turn). Context metric. contexthigher is better | 23.2% | 12.9% | 15.3% | 26.8% |
| Best-in-class metricsrows led (of 7) | 2/7 | 1/7 | 0/7 | 4/7 |
Most best-in-class metrics: From scratch (Expo parent 2, Parity fork 1, Admin fork 0, From scratch 4). This is efficiency-weighted — 5 of 7 rows are cost/latency/token metrics that move together, so from-scratch leading is about efficiency, not a single-winner verdict; code-quality spread is a narrow 67–81. Admin-fork figures are n=1 per domain and the notes-admin build carried a mid-build re-steer that inflates its cost/steps (see Notes).
What you pay for what you get
Code quality against build cost, one point per build path. No arm dominates: quality sits in a narrow 70–77 band while cost ranges $5.3–$9.9, so the spread is mostly on the cost axis.
Every build scored, one reviewer each
Reconstructed source from each job's trajectory, reviewed in isolation on a fixed rubric (arch 30 / reliability 25 / tests 20 / frontend 25). Total column is conditionally formatted. Scores cluster in a tight 67–81 band and the weaknesses are the same everywhere, so they are recipe-level, not per-app.
| App | Domain | Arm | Total/100 | Arch/30 | Reliab/25 | Tests/20 | Frontend/25 |
|---|---|---|---|---|---|---|---|
| P1 | E-commerce | Parent | 75 | 26 | 15 | 13 | 21 |
| P2 | Notes | Parent | 74 | 23 | 16 | 14 | 21 |
| P3 | Fitness | Parent | 69 | 22 | 15 | 13 | 19 |
| F1a | E-commerce | Parity | 73 | 25 | 19 | 6 | 23 |
| F2a | Notes | Parity | 76 | 24 | 19 | 11 | 22 |
| F3a | Fitness | Parity | 81 | 26 | 20 | 13 | 22 |
| F1b | E-commerce | Admin | 67 | 22 | 17 | 7 | 21 |
| F2b | Notes | Admin | 71 | 23 | 16 | 13 | 19 |
| F3b | Fitness | Admin | 74 | 22 | 16 | 13 | 23 |
| SC | E-commerce | Scratch | 70 | 22 | 16 | 13 | 19 |
Systemic issues, found in every build
- CORS * with credentials on every backend.
- Committed / weak secrets — JWT secret plus seeded admin creds (admin1234, coach1234).
- JWT in localStorage on every web fork (XSS-exfiltratable).
- Zero frontend / mobile tests in any app — backends do have tests, which is why the Tests axis averages 58, not lower.
- Domain bugs: e-com price-trust / overselling; notes & fitness lose state on refresh.
By dimension, averaged across all 10
Proof the two apps are really one product
The whole point of a "mono" build is that the website and the mobile app are two faces of the same product on one backend — so anything a user does on one surface is instantly there on the other. If that were not true, you would just have two apps that happen to look alike. Here is the live proof, on the from-scratch app (user-observed on the e-commerce fork as well):
What to keep in mind reading this
- Sample size. n = 1 job per domain per fork type; per-build-path averages are directional, not statistically powered.
- Admin-fork cost is skewed. The notes-admin build (F2b) carried a mid-build re-steer — 124 steps, $14.65, 12.5M tokens — which alone lifts the admin-fork average well above the other two admin builds ($5.84, $9.23). Read admin cost as "wide," not "expensive."
- Fork scope default = parity. when the agent pauses to ask the user for scope is inconsistent: one build (F2a) finished as a parity app without asking, while the admin scope needs an explicit instruction. Not a defect, but worth knowing. Not a defect, but worth documenting.
- From-scratch is AUTO-only. 3 earlier scratch attempts were mis-provisioned web-only (my setup error: pinned model + wrong base image) and discarded. The analysed scratch job used AUTO mode. Process cost, not a finding.
- Functional coverage. e-com + notes families were driven live by the user (guide mode); the from-scratch app was Claude-driven live; the fitness family was not run (sheet ready).
- Code quality method. reconstructed-source review by an isolated reviewer, not a live pen-test; scores are directional but the systemic findings are consistent across all 10.
- Agent verification. forks + scratch confirmed in DS14 Cortex (cross_platform_fork_agent / cross_platform_main_agent + Gemini design + testing agent); parents confirmed via DS10 prompt_name.
- Deploy pipeline: verified. All 10 apps deployed on canary 2026-07-21, every step green (build→health check), apps active on the internal cluster. Public serving not confirmed (canary uses an internal cluster; emergent.host returns "Deployment not found") - needs one prod deploy. Android/iOS (EAS) native builds and custom-domain linking still pending.
Open items (not failures)
- Deploy pipeline done on canary (10/10). Do one prod deploy to confirm public serving, then Android/iOS (EAS) native builds (deploy precedes native so mobile binds to the deployed backend).
- Harden the systemic security/reliability issues (one recipe fix helps all builds) and add frontend tests.
- Reliability replicates — single run per domain today (n=1); add the fitness functional run.
- From-scratch is AUTO-only — document or fix the pinned-model web-only path.
- Notes admin: add the missing edit control (only open functional item, minor).
All 10 jobs — per-job metrics & identity
The full record behind every number on the Overview: one row per job, with cost, ECU, tokens, latency, steps, parallelism and code quality, plus the agent, model, environment and full job ID. The per-arm averages on the Overview are the means of these rows. Tap any i for the source and query.
Build telemetry, one row per job
Cost = credit_ledger.amount; tokens = usages (prompt+completion); latency = active minutes from traj_payload.timestamp; quality = isolated-reviewer /100.
| Job | Arm | Cost $iBuild cost (USD)credit_ledger.amount for the job's LLM_CALL rows (dollars, not ECU). SELECT sum(amount) FROM credit_ledger WHERE job_id=<id> AND type='LLM_CALL'. Arm = mean over jobs. Per-job values in Job details. | ECUiECU chargedcredit_ledger.ecu (= usages.value_in_usd, a misnomer) - internal Emergent compute units, not USD. Dollars = amount / scaling_factor. | TokensiTotal tokensusages prompt + completion (the *_new columns, not the deprecated total_tokens). Arm = mean, millions. | OutputiTotal tokensusages prompt + completion (the *_new columns, not the deprecated total_tokens). Arm = mean, millions. | Active miniActive latency answer -> finish from traj_payload->>'timestamp' (NOT created_at, which is skewed on EPHs); sum of gaps < 300s so idle/HITL waits are excluded. Minutes, arm mean. | StepsiAgent stepscount(*) of trajectories rows for the job. Context metric - more steps is not inherently worse. | Par %iParallel tool-calls Share of steps whose traj_payload->>'function_name' = 'PARALLEL_TOOLS' (agent batched tool calls in one turn). Context metric. | QualityiCode quality /100 Reconstructed shipped source from each job's trajectory ( reconstruct.py), reviewed by ONE isolated reviewer per app on a fixed rubric (arch30 / reliability25 / tests20 / frontend25). Not a DB metric. Arm value = mean of the arm's apps. Evidence: per-app scores in the Code-quality table + Job details. | Functional |
|---|---|---|---|---|---|---|---|---|---|---|
| P1 e-com | Parent | $4.77 | 26.5 | 3.68M | 76.4k | 26.2 | 48 | 25.0% | 75 | e-com family 31/31 |
| P2 notes | Parent | $6.24 | 34.6 | 4.94M | 88.2k | 36.3 | 71 | 14.1% | 74 | notes family 28/29 |
| P3 fitness | Parent | $4.74 | 26.3 | 3.56M | 74.3k | 27.2 | 46 | 30.4% | 69 | not run |
| F1a e-com parity | Parity | $6.02 | 35.0 | 4.54M | 67.3k | 19.2 | 70 | 2.9% | 73 | e-com family 31/31 |
| F2a notes parity | Parity | $6.13 | 32.9 | 5.01M | 72.6k | 26.2 | 64 | 18.8% | 76 | notes family 28/29 |
| F3a fitness parity | Parity | $6.87 | 39.4 | 5.34M | 84.2k | 29.5 | 71 | 16.9% | 81 | not run |
| F1b e-com admin | Admin | $5.84 | 32.7 | 4.77M | 66.7k | 18.8 | 68 | 13.2% | 67 | e-com family 31/31 |
| F2b notes admin | Admin | $14.65 | 76.2 | 12.48M | 154.5k | 54.3 | 124 | 8.1% | 71 | notes 28/29; 1 blocked |
| F3b fitness admin | Admin | $9.23 | 51.3 | 7.78M | 120.5k | 38.9 | 98 | 24.5% | 74 | not run |
| Scratch e-com | Scratch | $5.99 | 32.9 | 2.90M | 89.6k | 22.2 | 41 | 26.8% | 70 | live checks pass |
What ran, and where
Resolved agent + model per job (forks/scratch from DS14 Cortex; parents from DS10 prompt_name), environment, full job ID, and the single most-used tool.
| Job | Domain | Agent | Model | Env | Job ID (full) | Top tool |
|---|---|---|---|---|---|---|
| P1 e-com | E-commerce | expo_fullstack_v0 | opus-4-8 | canary | b48f8fc3-5fa1-405a-9f92-3a37cae6b013 | PARALLEL_TOOLS:12 |
| P2 notes | Notes | expo_fullstack_v0 | opus-4-8 | canary | dca219b5-0bae-4e91-b2d5-cebcf678c482 | PARALLEL_TOOLS:10 |
| P3 fitness | Fitness | expo_fullstack_v0 | opus-4-8 | canary | aa9c5a75-8dea-4651-9b7b-11f0fbe01001 | PARALLEL_TOOLS:14 |
| F1a e-com parity | E-commerce | cross_platform_fork_agent | opus-4-8 | canary | d276be34-c91d-4212-b025-4edfc15e6a72 | PARALLEL_TOOLS:2 |
| F2a notes parity | Notes | cross_platform_fork_agent | opus-4-8 | canary | fbaefb8d-e8f6-4b0c-af77-bacd686d0933 | PARALLEL_TOOLS:12 |
| F3a fitness parity | Fitness | cross_platform_fork_agent | opus-4-8 | canary | 14f743a9-4df0-483b-a2f7-3739b80d4361 | PARALLEL_TOOLS:12 |
| F1b e-com admin | E-commerce | cross_platform_fork_agent | opus-4-8 | canary | 92ce1886-d574-40bd-bb79-e0c1636fed2f | execute_bash:26 |
| F2b notes admin | Notes | cross_platform_fork_agent | opus-4-8 | canary | 9d2f6c27-6a27-42c4-9dad-d0728cbc63cd | execute_bash:44 |
| F3b fitness admin | Fitness | cross_platform_fork_agent | opus-4-8 | canary | fbd2bd01-a886-44dc-aa06-7e874d751ab4 | PARALLEL_TOOLS:24 |
| Scratch e-com | E-commerce | cross_platform_main_agent | opus-4-7 | canary | 15874a28-44ec-42c8-883e-f48d0437904a | PARALLEL_TOOLS:11 |
Per-app rubric scores
The four-axis breakdown behind each app's total (arch 30 / reliability 25 / tests 20 / frontend 25).
| App | Domain | Arm | Total/100 | Arch/30 | Reliab/25 | Tests/20 | Frontend/25 |
|---|---|---|---|---|---|---|---|
| P1 | E-commerce | Parent | 75 | 26 | 15 | 13 | 21 |
| P2 | Notes | Parent | 74 | 23 | 16 | 14 | 21 |
| P3 | Fitness | Parent | 69 | 22 | 15 | 13 | 19 |
| F1a | E-commerce | Parity | 73 | 25 | 19 | 6 | 23 |
| F2a | Notes | Parity | 76 | 24 | 19 | 11 | 22 |
| F3a | Fitness | Parity | 81 | 26 | 20 | 13 | 22 |
| F1b | E-commerce | Admin | 67 | 22 | 17 | 7 | 21 |
| F2b | Notes | Admin | 71 | 23 | 16 | 13 | 19 |
| F3b | Fitness | Admin | 74 | 22 | 16 | 13 | 23 |
| SC | E-commerce | Scratch | 70 | 22 | 16 | 13 | 19 |
Sources: Redash DS10 (jobs/trajectories), DS14 (Cortex agents), credit_ledger (cost/ECU), usages (tokens), traj_payload.timestamp (latency). Code quality: reconstructed source + one isolated reviewer per app. Functional: live UI, user-driven for e-com + notes, Claude-driven for from-scratch.