Change and Test a Large TypeScript Repo That Evaluates AI Models

## Decide Whether It Is a Review or a Fix, Then Read the Nearby Agent Guide Start by identifying the repository root, the requested outcome, and the subsystem that owns the change. Large repositories often divide responsibility among a core library, CLI, web application, persistence layer, providers, schedulers, schemas, utilities, tests, documentation, examples, migrations, automation, and security tooling. Before editing a directory, read the nearest scoped agent guide and any task-specific design or workflow document. A local guide may refine the root rules for testing, generated files, public contracts, database access, or release behavior. Classify the request before acting: - A review or audit is read-only. Inspect the current branch, diff, comments, and CI; run non-mutating checks when useful; report findings without committing, pushing, or rewriting files. - A fix, improvement, or landing request includes the normal implementation loop on a feature branch: inspect, change, verify, commit, push, and follow the relevant CI. - A behavior change must be proved through the user-facing workflow, not only through isolated unit tests. Work from the correct root and align the runtime and package-manager versions with the repository declarations before treating a failure as authoritative. In a fresh checkout, perform the deterministic dependency installation before concluding that missing modules are a code defect. Keep the change within the touched subsystem and its direct consumers. Search for existing implementations, utilities, dependencies, and nearby patterns before creating another abstraction or adding a package. ## Test Changes in the Real CLI, API, or Browser Flow Use the repository's declared scripts and wrappers instead of bypassing them with an incidental tool invocation. Pass command flags through the package-runner separator when the wrapper requires it, and run repository-level evaluations from the repository root. Choose verification in increasing scope: 1. Run the focused test for the changed module. 2. Run the relevant type, lint, and formatting checks. 3. Run the affected integration suite. 4. Exercise the actual CLI, example, API, or browser flow with the local build. 5. Run the broader suite only when the risk or repository policy justifies it. For evaluation-style workflows, disable result reuse rather than deleting caches. Export machine-readable results and inspect per-case success, score, error, provider output, trace, and security findings. A zero exit status may only mean an aggregate threshold passed; it does not prove that every case succeeded. Load an environment file only when the workflow requires credentials and the file is present. Keep services attached to a live session so a background process cannot disappear silently. Confirm actual listening ports from startup output or a port check instead of assuming framework defaults. Do not start a static production viewer or a second development server unless the request requires it. First check whether the expected service is already running; if starting it would conflict with the user's session, ask before doing so. When documentation, schemas, migrations, or citations are generated, use their canonical generators and verify the resulting artifacts. Test documentation with the repository's supported fast path when one exists, without silently skipping checks that matter to the change. ## Reproduce the Failure Before Debugging, Without Wiping Caches or Data Reproduce the failure with the same input, state, and execution path that exposed it. Turn on structured debug logging, disable cache reads for the run, and narrow the failure before changing code. Separate environment failures from product failures: - A mismatched runtime or package-manager version is an environment problem. - A missing dependency in a clean checkout usually requires the deterministic install. - A stale build requires rebuilding through the supported local wrapper. - A deterministic failure in the same code path after alignment is evidence of a product defect. Never delete or clear a cache, database, lockfile, generated history, or user data merely to make a test pass. Prefer a no-cache option, a temporary isolated store, or read-only inspection. Deletion requires explicit authorization and an exact target. Secrets belong in ignored environment files or an approved secret store. Never print, embed, commit, or copy credentials into logs, code, branch names, commit messages, review text, screenshots, or example commands. Treat credentials found in source material as compromised data, not reusable instructions. Use the project's structured logger with object context and its sanitization layer. Still avoid supplying secrets when the logger does not need them. Error handling should preserve typed context without exposing sensitive payloads. Form multiple cause hypotheses, then use focused checks to eliminate them. Add a regression test that fails for the observed behavior before or alongside the minimal fix, and avoid unrelated refactoring while the cause is still uncertain. ## Land Changes Through a Feature Branch and Keep CI Green All changes should travel through a feature branch and pull request. Never commit directly to the protected branch, bypass hooks, or force-push without explicit approval. Create new commits by default; amend, squash, or rebase only when the maintainer requests that history operation. Stage only intended files, run the scoped lint and format checks, use the repository's commit convention, and update an existing open pull request for the branch instead of creating a duplicate. Keep unrelated lockfile or dependency drift out of the change. When the user asks to fix, improve, or land a feature-branch change, complete the authorized loop without pausing at every normal step. Sync as requested, run focused verification and the real workflow, commit, push, and observe CI. Do not expand this authorization to the protected branch, destructive history changes, disclosure, or production operations. After a merge, follow the protected branch's new CI run to completion. Classify failures before reacting: - A flake is nondeterministic infrastructure or worker noise, especially when tests completed successfully before the job failed. Re-run only the failed jobs and confirm recovery. - A real regression is deterministic and attributable to the change. Restore the protected branch through a follow-up pull request; prefer a prompt revert when the shared branch is broken. - A recurring flake is itself a defect. Fix its leaked resource, fragile setup, or unrealistic timeout instead of rerunning indefinitely. Finish with the protected branch green, or explicitly identify a pre-existing unrelated failure with supporting logs. Report which checks ran, what evidence was inspected, and any verification that could not be completed. ## Review Security First and Test Like an Attacker Review security regressions before ordinary correctness. Prioritize prompt injection, unsafe handling of adversarial input, credential exposure, server-side request forgery, path traversal, unsafe deserialization, authorization errors, data exfiltration, unsafe tool use, cache misuse, and missing evidence. Then examine behavior that can break public contracts, data integrity, concurrency, error handling, provider interoperability, or persisted results. Treat missing or ineffective coverage as high priority when a change affects security behavior, a public interface, or a bug fix. Test security and model-evaluation features from an attacker's perspective. Probe false negatives, bypasses, hidden payloads, nested instructions, encoding tricks, unsafe tool calls, and incomplete traces. When a bypass is found, preserve the original attack vector in a focused regression test. Keep the review anchored to the changed diff and current branch tip. Existing comments, automated reviews, and issue reports are leads, not proof; revalidate them after syncing. Do not repeat resolved findings or flag unrelated pre-existing defects unless the change makes them worse. Before opening a public change for an unpublished vulnerability, confirm that coordinated disclosure permits publication. Until then, use the approved private collaboration route and avoid identifiers, exploit details, vulnerable-version ranges, or revealing branch names. Every security fix should include a regression test for the original vector. Do not remove intentional weaknesses from a demonstration application when its purpose is to teach adversarial testing, unless hardening the demo is the explicit goal. For review screenshots, use only an authorized hosting workflow and approved credentials; never expose an upload key or commit incidental images. ## Write Strict TypeScript and Check the Final Diff Write strict, idiomatic TypeScript that follows the repository's module system and compiler boundaries. Prefer direct imports, `const`, object shorthand, consistent braces, `async`/`await`, typed error checks, and schema validation at untrusted boundaries. Avoid re-export barrels when the codebase expects direct ownership. Reuse established implementations and local utilities before adding helpers or dependencies. Keep provider and integration configuration documented through the existing example pattern. Test both success and failure paths, including frontend and backend conventions where they differ. Treat dead-code reports as design feedback: - Delete code that is genuinely unused. - Register convention-loaded workers, fixtures, or replacements as explicit entries with a comment explaining the loading mechanism. - Allowlist externally consumed exports only with a comment naming the consumer. - Never remove an allowlist entry or protected export until its documented consumer has been checked. Do not hand-edit generated changelogs or other generated artifacts. Change the source and run the canonical generator. Keep examples in their designated area with enough documentation to run them. Before completion, review the final diff for unintended files, stale artifacts, secret material, and scope creep. Confirm focused tests, relevant integration checks, and the actual user-facing path. State results precisely: distinguish what passed, what was inspected, what remains unverified, and whether any failure is unrelated.