Game designing, how do you do it?

A few thoughts. Please note that this isn't a checklist starting with the first, just a few things that work for me. I don't have a process, and I don't sit down for an hour with the intention of having ideas.

Play lots of games. Every type, not just shmups. Note what you like and don't like about them.

Start with a design constraint, like we do for the speed coding competitions around here. It can be something you add to the game, or something you take away. This is something where playing those other games is useful, because if you want to complicate the game you have a wide variety of material to borrow from. I'm best at creative synthesis, so I'll ask something like "what would a shmup be like with an exploration element?" Or one with resource management... or a game where the player doesn't control a single ship... or add a physics engine... Find a combination that hasn't been used and try it out with a quick prototype. If you want to make a small or experimental game, you should wrap the core gameplay tightly around a single idea.

Keep a small notepad with you and write down ideas as you get them. I also have a depressingly-long text file on my desktop named "game ideas.txt" Depressing, because there are enough to work on for several years without interruption.

There are books on both the creative process and game design, but not many on creativity in game design (as the state of the industry clearly demonstrates...). You might look at A Theory of Fun in Game Design by Raph Koster. I've also heard good thing about A Whack On the Side of the Head by Roger Von Oech. I haven't actually read either.