You’d like to think there was a system that helped good videos surface to the top and garbage go away. Despite the voting and comment functionality, popularity is no more democratic than it was in high school.
Proof: I have several videos where I’ve accidentally uploaded the same one twice. One will get great ratings, and the other won’t. One will get hundreds or thousands of views, and the other remains in the double digits. I am submitting to three sites regularly (Revver, YouTube and Google Video). Some of my videos die on one site, but gain popularity on another.
I think this has a lot to do with the luck of having someone with a big audience identify your content and share it with his/hers.