This started happening a couple weeks ago and I have no idea why, it is present in all videos I record now regardless of if they are fullscreen or rectangular recording mode. I have found a solution at this point but it seems more like a bug to be honest
I have updated bandicam, I have tried all of the codecs, I have turned off enhanced capture mode, I am not adding a logo to the video, I am not using a webcam overlay, I have tried disabling mouse cursor and clicks, I have tried every single preset, I have tried mp4 and avi, I have tried every single video size option as well as quality. The result is always the same, this:
The solution was to right click on the system tray icon, go to dpi scaling and set it to 100%. It was being set to 125% by default, likely because my main 4K monitor is set to 125%. However this above error would happen no matter which monitor I was recording on and none of the other monitors had their scaling set to 125%
So there are 2 problems here, obviously the main bug is that the recording should not be affected by this DPI scaling as I assume that is only for the UI of the application itself right? Secondly, the DPI Scaling settings should be accessible from inside the app, not just by right clicking on the system tray icon as I only discovered them there by accident after spending 30 minutes turning every other feature on and off in the app in vain.
Hopefully this helps others with this problem