debugPrintfEXT - missing output for pass with previously bound but unused descriptor set

[flagrans-dev]

Environment:

• OS: Windows 11
• GPU and driver version: NVIDIA RTX 2070 Super 561.09
• SDK or header version if building from repo: 1.3.296
• Options enabled (synchronization, best practices, etc.): debugPrintfEXT

Describe the Issue

Compute dispatches that do not use any descriptor sets (e.g., because buffers are accessed with a buffer device address) do not output any messages from debugPrintfEXT if a descriptor set was bound for a previous compute pass.

// 1st compute pass: binds a descriptor set
vkCmdBindPipeline(cmd, pipeline0, VK_PIPELINE_BIND_POINT_COMPUTE);
vkCmdBindDescriptorSet(cmd, VK_PIPELINE_BIND_POINT_COMPUTE, ...);
vkCmdDispatch(cmd, ...); // debugPrintfEXT here works

// 2nd compute pass: does not require any descriptor sets (but previous set is still bound)
vkCmdBindPipeline(cmd, pipeline1, VK_PIPELINE_BIND_POINT_COMPUTE);
vkCmdDispatch(cmd, ...); // !!! debugPrintfEXT here does not output anything !!!

I tried the following things:

• Disable the extension and calls to debugPrintfEXT in the first compute shader: still no output
• Remove the first vkCmdDispatch: still no output
• Remove the vkCmdBindDescriptorSet: it works
• Swap the order of the two passes: debug outputs for both passes work
• Change the pipeline layout and shaders for the second pass to use the same descriptor set as the first pass: debug outputs for both passes work
• Change the pipeline layout and shaders for the second pass to use a different descriptor set and bind it with a second call of vkCmdBindDescriptorSet to the same slot: debug outputs for both passes work
• In all of the cases above: record a renderpass and graphics pipeline before the compute passes with the same or different descriptor set layouts: doesn't interfere with what happens to the compute passes

So to me it seems that the call to vkCmdBindDescriptorSet is the cause of the issue, as it binds the descriptor set to the compute bind point and there is no way to "unbind" it for the successive passes. If I understood the way debugPrintfEXT works correctly, the validation layer inserts another descriptor set for the message buffer. Maybe this actually unused descriptor set messes something up?
I did not have time to test whether the same issue occurs using successive render passes and graphics pipelines where some do not use any descriptor sets.

Expected behavior

debugPrintfEXT should also produce output for successive compute passes that do not use any descriptor sets.

Valid Usage ID
n/a

Additional context
n/a

[spencer-lunarg]

debugPrintfEXT should also produce output for successive compute passes that do not use any descriptor sets.

Agree... thanks for raising this, will take a look into later this week

[spencer-lunarg]

I wrote to reproduce, I can't

Are you using vkconfig? By chance does your second pass only print a few things while the first prints a lots... I could imagine that you might be getting warnings (and not seeing them if turned off) that you might be hitting the default buffer or VU limit

If you can try again, try with

set VK_LAYER_PRINTF_BUFFER_SIZE=65536

and/or

set VK_LAYER_DUPLICATE_MESSAGE_LIMIT=0

[flagrans-dev]

I tried what you suggested with vkconfig but it didn't change the result. The issue doesn't even depend on the dispatch of the first compute pipeline. Binding only the descriptor set is already enough to cause the second compute pipeline to not print anything for me.

I did some more testing on both Windows 11 and Linux; and with NVIDIA and AMD GPUs, and the problem seems to only occur with my NVIDIA GPU, on both operating systems. I also ran my program in renderdoc and, to my surprise, the debug message appears in the corresponding dispatch call. So my print seems to get lost somewhere in the validation layer output (either configured by the application, or by vkconfig).

To make sure the problem isn't caused by my application, I also ran your MultipleComputePasses test case and it passed. However, I think the test framework inserts a default descriptor set at pipeline layout creation, if no pipeline layout was specified.

So I modified the test case to actually use an empty pipeline layout:

    CreateComputePipelineHelper pipe2(*this);
    pipe2.cs_ = std::make_unique<VkShaderObj>(this, shader_source_2, VK_SHADER_STAGE_COMPUTE_BIT);
    const vkt::PipelineLayout pipeline_layout2(*m_device);  // <<< NEW: Create empty pipeline layout
    pipe2.cp_ci_.layout = pipeline_layout2.handle();        // <<< NEW: Set empty pipeline layout
    pipe2.CreateComputePipeline();

I reran the test and it still passed... However, when I run the test case with vkconfig opened (configured to the "Debug Printf Preset", but disabled the "Redirect Printf messages to stdout" to avoid interference with the test framework), the test case fails because it did not receive the output of the second dispatch. With "Redirect Printf messages to stdout" enabled I can also observe only the print of the first shader.

So in summary: for me, the issue only occurs with an NVIDIA GPU, and when using the validation layers to directly print the messages (either configured by my application, or by vkconfig). The issue does not occur in renderdoc and in the validation layer test framework (unless controlled by vkconfig). Do those maybe use a different way to access the debug prints, or enable some other validation features that solve the issue?

Can you reproduce my issue with the modification to the test case and with vkconfig running?

[spencer-lunarg]

@flagrans-dev thanks so much for helping triage this!

I don't have my NVIDIA machine on me this second, but will try all of these and try to reproduce!