Face identification performance is very different for familiar and unfamiliar faces. Unfamiliar viewers typically perform poorly even in paired matching tasks, whereas familiar viewers perform with very high levels of accuracy. However, these standard results are based on studies in which the person being identified is cooperating with the effort to identify them. In the real world, people may have strong incentives to avoid identification by disguising themselves. Here we ask whether familiarity with a face can help the viewer see through a disguise. We distinguish between two different types of disguise—Evasion (trying not to look like oneself) and Impersonation (trying to look like a specific target person). To capture this distinction, we gave 26 volunteer models (i) a photo of themselves, and asked them to make themselves look unlike that reference photo for a subsequent shoot (Evasion condition), and (ii) a photo of someone else, and asked them to make themselves look like that person (Impersonation condition). We then constructed a face-matching task based on disguised and undisguised images of these models, and presented the resulting face pairs to 60 unfamiliar and 30 familiar viewers. Unfamiliar viewers made significantly more errors overall when matching disguised faces (m = 78% correct) compared with undisguised faces (m = 94% correct), with Evasion disguises proving especially challenging (m = 60% correct). Interestingly, performance for unfamiliar viewers was equivalent whether they were informed of the disguise manipulation or not. Familiar viewers were much more accurate for disguised faces (m = 93% correct) and also for undisguised faces (m =97% correct) but still committed errors in the Evasion condition (m = 86% correct). Our findings help to delineate strengths and limitations of familiar face recognition. They also have implications for face identification in security and forensic settings where disguises are commonly used.