Change a Large Python Workflow Monorepo Without Breaking It

## Find the Owning Package and Its Instructions Before Editing Use the project's canonical domain terminology in prose. Preserve literal identifiers, commands, paths, configuration keys, and compatibility names exactly as code requires; do not rewrite them merely for stylistic consistency. Before editing: - Identify the owning package, its local instructions, and the nearest architecture or testing guide. - Use the project-managed development and test wrapper instead of invoking host tools directly. - Install the repository's hooks or command shims through its documented one-time setup. - Put temporary helper scripts only in the designated mounted development area, and remove them when the task is complete. - Select checks according to the files and packages actually changed. Treat generated dependency locks and similar artifacts as outputs of an official generator, never as hand-edited source. ## Keep Schedulers From Running Workflow Code and Workers Off the Database Understand the data flow before changing orchestration code: 1. Authors define workflows through the public SDK. 2. Isolated processors parse those definitions and persist serialized workflow data. 3. Schedulers read serialized data and must not execute author-supplied workflow code. 4. Workers execute tasks through the supported SDK or API and must not access the metadata database directly. 5. API services mediate user-interface and client interactions with persistent state. 6. Event-triggering services run separately from schedulers and workers. Use short-lived, narrowly scoped credentials between components. Software checks that discourage direct database access are useful guardrails, but they are not a security boundary against deliberately malicious code. Classify a suspected security problem before proposing a fix: - An exploitable violation of the documented trust model is a vulnerability. - A behavior already declared outside the trust model is a known limitation. - A safer deployment choice that does not repair a violated guarantee is hardening guidance. Keep these categories distinct so severity and remediation match the actual risk. ## Put Each Dependency in the Package That Imports It Treat the monorepo as a set of related distributions rather than one undifferentiated package. Core orchestration, public SDKs, provider integrations, command-line tools, deployment charts, developer tooling, and scripts may have different release cycles and dependency rules. For a shared library: - Keep its API small and independent. - Give it its own source tree, tests, configuration, and declared dependencies. - Use the repository's supported workspace links instead of duplicating code. - Test the library itself and every directly affected consumer. Place a dependency in the distribution that imports it. When a change crosses package boundaries, verify both import direction and release ownership before modifying manifests. Regenerate lockfiles with the supported workflow and review the resulting dependency delta for unrelated churn. ## Write Clear Python Code and Keep Each Change Small Format and lint each changed Python file while the change is still small. Keep production behavior explicit: - Do not use assertions for runtime validation or recoverable error handling. - Write comments that explain constraints or intent, not a narration of the code. - Measure durations with a monotonic clock. - Keep imports at module scope except for documented circular, lazy-loading, or optional-dependency cases. - Guard expensive type-only imports so they do not affect runtime startup. - Raise the narrowest meaningful exception; do not introduce a broad catch-all base exception. - Translate domain failures into protocol-specific responses only at the transport boundary. - Name functions with action verbs and retain the required license header. Prefer a small, reviewable change over an opportunistic refactor. If a cleanup is not necessary for the requested behavior, leave it for a separate change. ## Write to the Database in Small Batches So the Scheduler Stays Fast Functions that receive a database session should not commit implicitly. Make the session a keyword-only argument where the codebase uses that convention, and let the transaction owner decide when to commit or roll back. Avoid unbounded bulk writes inside scheduler or maintenance loops: 1. Filter on indexed columns. 2. Process a bounded batch. 3. Commit or release the transaction between batches. 4. Continue until no eligible rows remain. This limits lock duration, memory growth, and starvation of other scheduler work. Confirm behavior on the default database and run alternate-backend coverage when the changed query depends on backend-specific semantics. Never treat a destructive cleanup command as routine troubleshooting. Resolve the exact target and obtain the required approval before deleting databases, volumes, caches, or user data. ## Write a Failing Test First, Then Widen Tests Only as Needed Start with a test that expresses the exact behavior being changed and confirm it fails without the implementation. Then make the smallest change that turns it green. Use the repository's preferred test framework and helpers: - Mirror the source layout in the test tree. - Use specification-bound mocks so interface drift fails loudly. - Use supported configuration overrides and time-control utilities. - Parametrize meaningful input and backend variations. - Mark database tests accurately and keep non-database tests independent of database setup. - Assert structured event names and fields instead of unstable rendered log text. Do not test the standard library or a third-party dependency in isolation. Test the integration behavior that this code owns. Run a focused test first, then the affected package suite, static typing, linting, and broader suites only when the dependency surface warrants them. ## Keep CI Rules and Docs in Sync, With Release Notes Only for Visible Changes When selective CI behavior changes, update the selection logic, its tests, and its documentation together. A rule that exists only in code or only in prose is incomplete. Add release metadata only for user-visible changes in the distribution that ships the behavior. Internal refactors, test-only work, developer tooling, and non-user-facing maintenance usually need no release note. Use project generators for locks, command references, schemas, and other generated documentation. Review generated diffs, but do not patch generated files to conceal a source-of-truth problem. Before handoff, run the relevant formatting, lint, type, test, documentation, and selective-check gates. Record any intentionally skipped gate and the concrete reason it was not applicable. ## Review the Whole Diff and Open a Pull Request From Your Fork Review the complete task diff before publishing. Confirm architectural boundaries, security assumptions, tests, generated outputs, release metadata, and local instructions all agree. Ask before making a large cross-package refactor, adding a broad dependency, changing destructive migration behavior, or performing an external publication that was not already authorized. Never expose secrets, bypass required hooks, hand-edit generated files, or run destructive version-control commands without explicit approval. Before any remote operation: - Inspect the configured remotes and verify which one is the canonical repository and which one is the contributor fork. - Do not push directly to the canonical remote or its protected default branch. - Ask before renaming or rewriting remotes when their roles are unclear. - Search for an existing pull request so the work is not duplicated. - Rebase or otherwise synchronize with the current target branch through the repository's accepted workflow. Use an imperative, user-impact title in plain prose. Explain why the change is needed in the body, disclose AI assistance when required, and include relevant issue references. Do not add yourself as a co-author or tag individuals unless the user has authorized it or they already participate in the thread. If a complete fix is ready, prefer the pull request over opening a duplicate issue. If the change is only a workaround, mitigation, or partial fix, create or link a tracking issue and reference it both in the pull request and near the temporary code.