Build a Mobile UI Automation Tool and Fix Bugs in the Product, Not CI

## Put Each Change in the Module That Owns the Behavior Maintain a concise module map that identifies where code executes and who consumes its artifacts. Separate at least these responsibilities: - device-side drivers and their generated binaries; - host-side platform adapters and protocol clients; - command interpretation, retries, lifecycle control, and error classification; - shared command and data models; - command-line and agent-facing interfaces; - web drivers, wire-format definitions, utilities, and cross-module tests. Place a change where the behavior is owned. Device interaction and system-dialog handling belong in platform drivers or host adapters; retry and command sequencing belong in orchestration; shared request shapes belong in model or protocol modules. Do not solve an ownership problem in a distant module merely because it is easier to patch there. Keep the map as a single source of truth for workflows and specialist agents. Update it when module responsibilities change so downstream guidance does not drift. ## Pick the Cheapest Test Layer That Proves the Behavior Use a layered test strategy and default to the cheapest layer that can prove the behavior: 1. Unit tests for isolated classes, parsing, small utilities, and local decision logic. 2. Integration tests for command orchestration, cancellation, scripting engines, network boundaries, and multiple modules that can run without a device. 3. End-to-end tests for real drivers, device or browser behavior, operating-system integration, and complete user flows. 4. Agent-interface evaluations for model behavior and tool correctness when the product exposes an AI-facing protocol. Do not promote a test to a more expensive layer when fakes or in-process components can express the contract accurately. Conversely, do not replace device-level evidence with mocks when the risk is platform behavior. ## Keep Must-Pass and Must-Fail Test Suites Apart Separate end-to-end flows by expected outcome. A positive suite contains flows that must pass; any failure is a regression. A negative suite contains intentionally invalid assertions or commands and must fail in the expected way. The harness may invert success for the negative suite, so artifact interpretation must preserve the suite label. Never diagnose an expected negative result as a product regression, and never weaken or rewrite a negative flow merely to make it pass. When an automated diagnosis process consumes artifacts, restrict it to genuine positive-suite failures unless its contract explicitly supports negative-path analysis. Store artifacts under a predictable hierarchy containing application, platform, suite, and run identity. ## Reuse One Fixture App for End-to-End Tests on Every Platform Use one default fixture application to exercise the full command surface across supported platforms. Reuse the same flows for two purposes: regression smoke tests on current platform versions and compatibility validation on a newly introduced operating-system version. If behavior is framework-specific, add a dedicated fixture workspace rather than distorting the default application. Give each workspace its own flows and build artifact, and let manual dispatch narrow execution to one application or one flow. Keep the test driver parameterized by platform and explicit environment inputs. Validate dispatch inputs before provisioning expensive infrastructure: reject unsupported system versions, missing workspaces, and ambiguous flow selectors early. ## Fix Device Failures in the Driver, Not with CI Workarounds When an end-to-end flow fails because a platform driver cannot handle real device behavior, fix the driver, host adapter, or fixture application that users actually run. Do not hide the gap by adding emulator settings, shell commands, command-line tweaks, or preconfiguration to the CI workflow. Workflow changes are appropriate for workflow shape: matrices, retention, triggers, validated dispatch inputs, or narrowly documented infrastructure exceptions. They are not a substitute for product behavior. Use the same rule for new operating-system validation. A flow that passes only after CI-specific preparation still indicates a product compatibility defect unless that preparation is an explicit user prerequisite. ## Sort Errors by Whether to Retry and Always Clean Up Temp Files Use structured asynchronous execution with explicit dispatchers or executors. Reserve blocking bridges for application entry points. Classify exposed failures so orchestration can distinguish retryable conditions from terminal errors instead of retrying everything uniformly. Route temporary files and directories through a lifecycle-owned resource manager. Construct it near the operation that owns the resources, allocate through its API, and close it in a `finally` path so recursive cleanup occurs after success, failure, or cancellation. Do not bypass the resource manager with direct temporary-file calls. Unmanaged files accumulate in long-lived workers and make failures harder to reproduce. Test cancellation, timeout, supervisor behavior, and cleanup using real orchestration with deterministic fake drivers when no device is required. ## Test Agent Tools and AI Model Behavior Separately When the command-line application also exposes an agent-facing server, keep two distinct test families: - tool-functionality tests that verify requests, responses, device setup, and failure behavior; - model-behavior evaluations that measure whether an agent selects and uses tools correctly. Use declarative evaluation cases and platform-specific setup scripts so coverage is reproducible. Run fast tool tests for ordinary implementation changes; run slower model evaluations when prompts, tool descriptions, schemas, routing, or agent-visible behavior changes. Do not treat a model score as proof that the underlying tool works, or a functional tool test as proof that an agent will use it correctly. Both contracts need their own evidence. ## Rebuild Generated Binaries Instead of Editing Them by Hand Treat checked-in driver binaries, packaged runners, checksums, and other derived files as generator outputs. Never edit them manually. Run the owning build task or finalizer and commit the resulting artifacts together with the source change that produced them. Before completion: 1. Confirm the change is in the module that owns the behavior. 2. Run the lowest sufficient test layer, then any directly affected broader layer. 3. Interpret positive and negative suites according to their contracts. 4. Verify temporary resources close on success, failure, and cancellation. 5. Regenerate derived artifacts through the official build path. 6. Confirm CI changes describe infrastructure shape rather than masking product defects. Keep workflow documentation, fixture layout, and module ownership notes synchronized so later contributors and automated agents follow the same boundaries.