There have been many studies dedicated to determining which part of the brain is in charge of face recognition, as it is an important evolutionary skill to be able to quickly recognize faces. In fact, if an infant is put in front of a paddle with scribbles and a paddle resembling an extremely basic face, they will automatically gravitate towards the face. At an early age, babies develop the ability to recognize their mothers scent and facial features, as their mother is typically the most important figure in their early lives.
Prosopagnosia, or face blindness, is an inability to recognize people's faces, no matter how often you interact with them. Instead, people with prosopagnosia rely on things like scent, hairstyle, or voice to distinguish people from each other. The experience of living with face blindness has been compared to looking at a picture like the one below, where faces are flipped upside down. Even though we should be able to immediately recognize these faces, it is extremely hard to do without turning your head in some way.
Through various studies, psychologists have determined that there is a section of the brain dedicated to face recognition, coined the fusiform face area, in the occipital lobe. Oftentimes, genetic prosopagnosia is reported alongside other afflictions such as the inability to recognize objects, or colorblindness.