Let's think like this: would any season be better if it had 20+ episodes? I'm sure the general consensus would be less is better. Which I agree. S2 was the only truly without argument good season, and it had the most episodes but not 20.
To some, s1 is boring, s4 was trash, or the other way around. Personally I like the first two seasons the most but it is arguable on which were the best. Regardless on the number of episodes.