Not always. If someone's in your house illegally, you've brandished your firearm, and they still charge you... well that's what we "on the job" call an "articulable fact" that this individual intends to do you harm, and is intent enough on doing it that he's charging someone threatening to respond with gunfire.
In that situation, whether he's armed or not, I'd dump a mag.
ETA: Of course, as a law enforcement officer, I have no duty to retreat. If you're not in a Castle Doctrine state, your mileage may vary.
Bookmarks