I don't think it matters that much; both of them can look fine when they are worn with an appropriate corresponding outfit, and usually look bad otherwise.

Also, I feel like I saw this image years and years ago.

Meet the superior Flat Cap masterace.