Change a Large Python Data System Without Breaking Lazy Execution

## Trace the Requested Behavior to the Layer That Owns It Before changing a large Python data or parallel-computing codebase, build a task-specific map instead of reading the repository indiscriminately. Identify the layers involved: 1. Public collections or user-facing interfaces. 2. Symbolic expressions or high-level graphs used for planning and optimization. 3. Materialized low-level tasks and dependency edges. 4. Scheduler or execution backends. 5. Configuration, dispatch, serialization, and utility layers. Trace the requested behavior from the public entry point to the smallest implementation layer that owns it. Read adjacent tests and configuration before editing. Use task-based navigation: - Scheduling behavior: public compute APIs, graph utilities, task representations, then scheduler adapters. - Graph optimization: high-level graph protocols, blockwise or expression layers, then materialization. - Array or dataframe features: collection API, operation implementation, dispatch, then focused tests. - Configuration changes: runtime accessors, default configuration, schema, and documentation. - Backend support: backend interface, entry-point registration, dispatch, and compatibility tests. Treat module names, line counts, and dependency versions in repository notes as navigation hints, not timeless facts. Confirm current source and manifests locally. Keep subsystem boundaries explicit. A change in a public collection may require matching metadata, graph, scheduler, test, and documentation updates, but do not edit unrelated layers. ## Use the Repo's Environments and Pick the Smallest Test Scope Use the environment manager and commands declared by the repository. Do not assume a global Python installation matches supported dependency combinations. Before implementation: 1. Inspect the project manifest and available environments. 2. Select the smallest environment that represents the affected feature. 3. Locate the nearest existing tests, fixtures, markers, and assertion helpers. 4. Record relevant optional dependencies, feature flags, and warning policies. Prefer repository-provided command wrappers for: - a single failing test; - the affected module's test file; - lint and static checks; - doctests for public examples; - minimum-dependency or optional-feature environments; - broader CI-equivalent checks only when the change warrants them. Respect strict test behavior such as warnings-as-errors, strict expected failures, timeouts, and environment-specific markers. Use shared equality helpers for lazy collections when they account for metadata, types, partitions, or numerical tolerance. For optional dependencies, use the project's import and skip helpers rather than ad hoc conditionals. Keep minimum and latest supported versions in mind when using new APIs. Do not install packages, change environments, or enable network-dependent tests unless the task authorizes it. Report a missing dependency as an environment limitation instead of silently changing the machine. ## Keep Execution Lazy: Never Call `compute()` While Building Graphs Preserve the central invariant of lazy graph systems: graph-construction code must describe computation without executing it. Never call eager materialization methods such as `compute()` or `persist()` while defining collection operations, graph layers, or symbolic expressions. Materialization belongs at an explicit user or execution boundary. During graph construction: - infer outputs from metadata whenever possible; - keep task dependencies explicit and deterministic; - preserve stable keys and tokenization behavior; - avoid hidden I/O or scheduler selection; - maintain separation between symbolic optimization and low-level execution. Follow repository conventions: - add complete type hints to new code; - keep compatibility imports and deferred annotations consistent; - use guarded type-only imports to avoid cycles; - organize standard-library, third-party, and local imports predictably; - use the documented docstring style and runnable examples for public APIs; - follow established naming for private helpers, tests, and dispatch functions. When adding a backend, expression, layer, or configuration option, update every required registration point and add a focused test that proves dispatch or planning reaches the new implementation. Feature flags that affect import-time registration or query planning must be set before the relevant module is first imported. Test both enabled and disabled behavior when both remain supported. Avoid broad refactors while repairing a focused defect. Preserve unrelated behavior and make the smallest change that explains the new test result. ## Widen Tests Step by Step and Never Release Without Permission Verify in increasing scope: 1. Reproduce the previous failure with a focused test. 2. Run the nearest test file or subsystem suite. 3. Run lint, type, or doctest checks directly affected by the edit. 4. Exercise relevant optional, minimum-dependency, or feature-flag variants. 5. Use the full suite only when the change crosses broad architectural boundaries. Check common failure modes: - import cycles and registration order; - feature flags changed after first import; - metadata that disagrees with computed results; - optional packages unavailable in some interpreters; - dependency pins drifting between related packages; - configuration defaults missing from the schema; - doctest examples that no longer run; - tests that pass only under one scheduler or backend. Before reporting completion, review the diff and state which checks actually ran, which passed, and which were not run. Do not infer success from code inspection alone. Release work is a separate authority boundary. Never create a release, publish packages, push tags, upload artifacts, or modify external release systems unless explicitly authorized. If release preparation is authorized, verify changelog entries, version and cross-package pins, packaging output, and the repository's documented sequence before any external state change. When asked to communicate with other people, present evidence and options without impersonating the user or inventing their decisions.