I got this in the rpsc3 emulator, exact same problem as described here: https://github.com/RPCS3/rpcs3/issues/5408
I can also find people have the same issue in some other games:
https://www.reddit.com/r/Doom/comments/ ... t_running/
https://forum.zdoom.org/viewtopic.php?f=7&t=67974
I solved it by running UnregVulkanLayer.bat in Bandicam's folder, but that also means I can't capture those games.