In my opinion, the size of your school does matter to a certain degree, but tradition trumps size every time.
Norton competes extremely well with the larger schools because of their tradition (I'm sure they have good coaches too). When kids grow up in that town they grow up knowing two things...... Football and Wrestling!
You can't reload year after year without having tradition.