You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We regularly observe the effects of changes to red-knot in the diagnostics emitted when running our (performance) benchmark against tomllib, and sometimes catch issues by seeing new false positives. But a) tomllib is one small codebase, not necessarily representative, and b) the ergonomics of catching these issues in the performance benchmark is poor.
We should select some larger real-world codebases and implement tooling to run red-knot on them and snapshot the diagnostics emitted, such that we can see (and easily update) this snapshot when we make changes to red-knot.
We should run this in CI to ensure our snapshot stays up to date.
(This is very similar to the ruff ecosystem check, and we can likely reuse some of that infrastructure?)
Bonus: if we implement #15696, we can also track changes to the prevalence of Todo types over time.
The text was updated successfully, but these errors were encountered:
Description
We regularly observe the effects of changes to red-knot in the diagnostics emitted when running our (performance) benchmark against tomllib, and sometimes catch issues by seeing new false positives. But a) tomllib is one small codebase, not necessarily representative, and b) the ergonomics of catching these issues in the performance benchmark is poor.
We should select some larger real-world codebases and implement tooling to run red-knot on them and snapshot the diagnostics emitted, such that we can see (and easily update) this snapshot when we make changes to red-knot.
We should run this in CI to ensure our snapshot stays up to date.
(This is very similar to the ruff ecosystem check, and we can likely reuse some of that infrastructure?)
Bonus: if we implement #15696, we can also track changes to the prevalence of Todo types over time.
The text was updated successfully, but these errors were encountered: