Add and Test Kernels in a JIT-Compiled GPU Kernel Library

## Set up an editable install where kernel edits recompile automatically Use an editable installation that preserves the environment's existing framework and accelerator versions. Disable isolated dependency resolution when it could pull incompatible runtime packages. Initialize every required source submodule before building or testing. Treat just-in-time compilation as the default development loop. Edit kernel sources, run a focused test, and let source hashing trigger recompilation. Do not reinstall the package after each kernel edit unless packaging code itself changed. Clear the JIT cache only when stale artifacts are part of the diagnosis. Use precompiled kernel packages only when faster first use or distribution requires them. Keep precompiled artifacts optional for normal development. Set target architectures explicitly when building for devices other than the detected local GPU. Bound compiler threads and parallel build jobs according to available memory. Estimate total compile memory as jobs multiplied by compiler threads and per-thread demand. Do not let test parameter sizes cause avoidable device-memory exhaustion. Install profiling support only when accurate hardware timing is needed. Run formatting and repository hooks before review. ## Write generated files only to writable workspaces, never to installed folders Represent each compilation toolchain through a common specification interface. Give each specification a unique name, source list, compiler flags, linker flags, and supported architectures. Use a shared build-and-load template to coordinate cache lookup, locking, compilation, and loading. Keep the lifecycle explicit: try cached load, build when needed, then load the compiled module. Refuse JIT compilation when a disable flag is set and no valid cached or precompiled module exists. Never write generated files into installed package directories because they may be read-only. Keep source templates and precompiled package data read-only. Write rendered source and copied compilation units only to a generated-source workspace. Write object files, shared libraries, metadata, and build files only to the JIT artifact workspace. Generate the build manifest from the exact source list and compilation flags. Ensure workspace locations can be overridden for shared, scratch, or restricted environments. Use architecture context to combine detected devices with explicitly requested targets. Limit a module to supported architecture families before compilation begins. Fail clearly when no requested architecture is supported. Do not silently compile for a different device family. ## Add a new kernel from device code up to the public API Implement framework-agnostic device code against raw pointers and portable kernel types. Keep framework headers out of the reusable kernel include layer. Add a launcher in the binding layer to validate tensors and prepare invocation parameters. Export the launcher through the project's typed cross-language FFI. Compute a deterministic operation URI from specialization parameters. Create a per-operation generation directory under the writable source workspace. Use a template only when concrete type or parameter specialization is needed. Write rendered configuration only when its content differs. Copy required compilation units into the generation directory. Return a compilation specification containing the URI, sources, flags, and architecture limits. Expose a cached high-level API that reuses loaded modules. Register the operation for ahead-of-time package generation when distribution requires it. Export the public API from the package entry point. Attach a trace template defining axes, inputs, outputs, and an optional reference. For plan-and-run APIs, attach tracing to the method that performs execution. Add a realistic trace example, regenerate benchmark-definition artifacts, and commit the expected outputs. Run trace consistency and end-to-end tests after regeneration. ## Skip unsupported GPUs in tests and back speed claims with correctness checks Run the complete test suite for broad validation and focused tests during iteration. Skip unsupported hardware deliberately through central capability helpers. Check both device compute capability and backend support when an API exposes those predicates. Keep architecture requirements explicit for each feature. Do not convert an unsupported architecture into a test failure. Do not skip a supported architecture because the local device is different. Choose test shapes that exercise the contract without causing avoidable out-of-memory skips. Cover attention, matrix multiplication, mixture-of-experts, communication, and utility paths affected by a change. Use multi-device launch tooling for collective or distributed tests. Benchmark through a unified harness that supports comparable backends and batched configurations. Prefer hardware-level profiler timing and fall back to accelerator events when unavailable. Record which timing mechanism produced each benchmark result. Report median, variation, iteration count, workload shape, datatype, architecture, and backend. Enable reference checking when comparing kernels. Do not present speed claims without correctness checks and reproducible parameters. Treat benchmark and test evidence as required before approving a performance-sensitive kernel. ## Rebuild cached kernels when their sources or target GPU change Use an in-process cache for loaded modules and an on-disk cache for compiled artifacts. Invalidate cached modules when sources, flags, architecture, parameters, or library version changes. Derive cache identity from operation type, parameters, source hashes, flags, and target architecture. Keep cache keys unique across specialization and toolchain variants. Store compatibility metadata beside artifacts and validate it before loading. Protect compilation with locking so concurrent processes do not corrupt shared artifacts. For single-architecture DSL outputs, store one module directory with metadata and one object per specialization. Derive the architecture from the DSL compile target or active device. Invalidate a DSL module when its compiler package version or kernel-source hash changes. Allow cache disabling for diagnosis, while keeping reuse enabled by default. Use dispatch macros or generated code to manage combinatorial datatype and block-size spaces. Keep dispatch choices explicit and test unsupported combinations. Do not add templating where direct source is sufficient. Do not let stale precompiled packages bypass version compatibility checks. Bypass version checks only for an intentional, diagnosed mixed-version experiment. ## Log API inputs before kernels run so crashes can be reproduced Log API inputs before kernel execution so crash-triggering parameters survive a device failure. Keep logging at zero overhead when disabled. Offer graduated levels for call names, inputs and outputs, and expensive statistics. Reserve statistics and invalid-value scans for explicit high-detail diagnostics. Skip incompatible statistics collection during graph capture. Bound diagnostic dumps by event count and total size. Support include and exclude filters so expensive capture targets only relevant APIs. Choose a portable tensor format when dumps must move between environments. Use a dedicated writable dump directory. Enable verbose compilation output and debug symbols only while diagnosing failures. Inspect generated source, build manifests, compiled modules, and cache metadata together. Record shapes, datatypes, architecture, flags, and selected backend when reproducing a crash. Use line information when profiler or debugger source mapping is needed. Keep normal optimization settings separate from debug builds. Treat forced algorithm or backend selections as diagnostic controls, not production defaults. Document every non-default control used to reproduce a result. ## Verify downloaded artifacts and build releases for an explicit GPU list Hard-fail on missing local artifacts in offline or locked-down environments. Allow a trusted mirror to replace the default artifact repository. Verify downloaded artifact checksums by default. Disable checksum validation only as a temporary debug aid. Bound parallel artifact downloads. Prefer a configured local source before attempting a download. Keep compilation flags and launcher overrides explicit and observable. Keep artifact-loader logging controls distinct from API-call logging controls. Use stable timing for autotuning, including a hardware global timer when event timing is unreliable. Allow confidential-compute detection to be overridden for controlled CI diagnosis. Persist autotuning results only with an explicit cache location and compatible environment. Load a tactics blocklist to avoid kernels known to hang or crash on the current stack. Leave experimental routing and kernel-selection overrides at their documented defaults. Build ahead-of-time packages for an explicit architecture list. Generate package version metadata from one authoritative version source. Use backwards-compatible version increments for kernels, device support, and bug fixes. Reserve incompatible API changes for an architectural version boundary. ## Keep the public API and docs matching the kernel code Use a typed cross-language ABI so the same kernels can serve multiple host frameworks. Keep framework-specific tensor handling in bindings rather than device headers. Register custom operations in the host framework only where compilation and graph capture require it. Keep the Python API, JIT generator, binding export, ahead-of-time registry, and package exports synchronized. Start from the simplest working kernel pattern before adopting advanced workspace or plan-run designs. Update function signatures, templates, launchers, and tests together when parameters change. Learn established style, complexity, defensiveness, and efficiency from nearby working examples. Document intentional departures and performance-critical algorithm choices with alternatives considered. Consult authoritative instruction-set and toolchain documentation for architecture-specific behavior. Prefer vendored dependency source when external documentation is stale. Keep trace templates and generated benchmark definitions synchronized with public APIs. Update infrastructure examples and tutorials when logging, requirements, FFI, or code-generation patterns change. Mark deprecated approaches clearly or remove them from current guidance. Disclose meaningful AI assistance according to the contribution policy. Approve only when source, generated artifacts, caches, tests, benchmarks, exports, and documentation agree.