The biologically active estrogen estradiol has important roles in adult brain physiology and sexual behavior. A single gene, Cyp19a1, encodes aromatase, the enzyme that catalyzes the conversion of testosterone to estradiol in the testis and brain of male mice. Estradiol formation was shown to regulate sexual activity in various species, but the relative contributions to sexual behavior of estrogen that arises in the brain versus from the gonads remained unclear. To determine the role of brain aromatase in regulating male sexual activity, we generated a brain-specific aromatase knockout (bArKO) mouse. A newly generated whole-body total aromatase knockout mouse of the same genetic background served as a positive control. Here we demonstrate that local aromatase expression and estrogen production in the brain is partially required for male sexual behavior and sex hormone homeostasis. Male bArKO mice exhibited decreased sexual activity in the presence of strikingly elevated circulating testosterone. In castrated adult bArKO mice, administration of testosterone only partially restored sexual behavior; full sexual behavior, however, was achieved only when both estradiol and testosterone were administered together. Thus, aromatase in the brain is, in part, necessary for testosterone-dependent male sexual activity. We also found that brain aromatase is required for negative feedback regulation of circulating testosterone of testicular origin. Our findings suggest testosterone activates male sexual behavior in part via conversion to estradiol in the brain. These studies provide foundational evidence that sexual behavior may be modified through inhibition or enhancement of brain aromatase enzyme activity and/or utilization of selective estrogen receptor modulators.