the thing you have to remember about YouTubeis that it will re-encode anything you upload. encoding something multiple times reduces the visual quality each and every time.
To that end, if you're going for 'quality' videos on YouTubeyour best bet is to give YouTubethe best quality video you can. Personally I re-encode all my bandicam footage in to 1080p 30fps 20MBps MP4... YouTube then re-encodes it as it sees fit, but as it's started with a better image the resulting YouTube output looks better.
So, if you're going for quality use RGB, or MPG1 @ 100% quality, and re-encode.
If you're going for the simplest process (capture with Bandicam -> upload to YouTube) then use something that is at least the same quality as YouTube.. I think it re-encodes 1080p @ 6MBps
hope that helps.