I have the primary sound device set to the default sound device (which are the speakers) or set to the speakers directly. The secondary sound device is disabled, and when I look back at the recorded video, I can hear the game sound, but also when I press the buttons on my Xbox controller. This implies that I recorded the microphone while I thought it was disabled. I couldn't find any other settings in Bandicam causing this.
The microphone option from the Xbox button menu is disabled too.