Feature request: Show Skipped Tests But Not Filtered Out Tests

[cwfitzgerald]

Motivation

wgpu uses nextest quite heavily. One particular quirk of the wgpu test suite is that we automatically run each one of our gpu-enabled tests on each gpu in the system. As we have tests for all sorts of features, not every gpu can run them. We currently make each test instance that cannot be run on that current system "no-op" itself. We use the test name to show why the test is disabled. Example of running a single test:

[INFO  xtask::test] Running cargo tests
warning: F:\Programming\wgpu\deno_webgpu\Cargo.toml: unused manifest key: target.cfg(not(target_arch = "wasm32")).dependencies.wgt.package
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.49s
    Starting 6 tests across 21 binaries (1511 skipped; run ID: b904e2de-06c3-49d6-9090-641819ae990e, nextest profile: default)
        PASS [   0.103s] wgpu-test::wgpu-test [Unsupported: Features] [Gl/NVIDIA GeForce RTX 4070/PCIe/SSE2/5] wgpu_test::binding_array::samplers::binding_array_samplers
        PASS [   0.648s] wgpu-test::wgpu-test [Executed] [Vulkan/AMD Radeon RX 7800 XT/1] wgpu_test::binding_array::samplers::binding_array_samplers
        PASS [   0.808s] wgpu-test::wgpu-test [Executed] [Vulkan/NVIDIA GeForce RTX 4070/0] wgpu_test::binding_array::samplers::binding_array_samplers                                                                                                                                                       
        PASS [   1.074s] wgpu-test::wgpu-test [Executed] [Dx12/Microsoft Basic Render Driver/4] wgpu_test::binding_array::samplers::binding_array_samplers                                                                                                                                                   
        PASS [   1.137s] wgpu-test::wgpu-test [Executed] [Dx12/NVIDIA GeForce RTX 4070/2] wgpu_test::binding_array::samplers::binding_array_samplers                                                                                                                                                         
        PASS [   1.189s] wgpu-test::wgpu-test [Executed] [Dx12/AMD Radeon RX 7800 XT/3] wgpu_test::binding_array::samplers::binding_array_samplers                                                                                                                                                           
------------
     Summary [   1.191s] 6 tests run: 6 passed, 1511 skipped

I would love for the "unsupported" test to show as something other than "PASS". I tried to do this by:

• Marking them as ignored in libtest-mock
• Having our xtask run with --status-level skip.

This worked great if I was running the whole suite. However, if I added any kind of filter like above, I would get 1500 lines of output instead of what I wanted, which was one.

Proposal

I would love some way to have a --status-level skip which includes ignored tests, but does not show filtered out tests.

Alternatives

I haven't really thought of any good ones. I would really love regex based coloring, or a more dataful contract between libtest-mock and nextest but those both sound much more complicated 😅

Additional context

No response