This is now a problem, as traditional methods for extracting broken screws don't work well on the hard metal in drill bits.
This combined clearance hole and countersink proves just the thing for removing broken drill bits.
The broken bit was #8, so I got one of these counter sink/clearance hole drills in #10, took out the drill bit, put the outside part in a brace and cored out around the broken bit enough to get a needle nose pliars in.
I'd strongly suggest a brace instead of a power drill. High speed metal spinning in the vicinity of stuck hardened metal seems like a really bad plan.