It's EXTREMELY common for something fresh/new to happen, expectations are high, the site gets busy, lots of visitors, etc, etc. Then reality sits in, jeesh, I just spent a crap load of time on this thing and I could have made more money working at McDonalds. wait...that's
my story not Matts. (I hope!)
Slow growth is the key, fast growth, quick peaks, etc, can kill a site very quickly. Things get overwhelming, etc. I still think the whole competition thing is a winner, but just not so often, instead go for bigger and better prizes, etc, perhaps an annual big event, etc, etc.
My advice to Matt, slow down, take a deep breath, don't worry about what anyone says. Think long term, not so much short term. Think where you want the site to be in 3-5 years, not 3-6 months.