Infrequent ejaculation is unlikely to affect testosterone levels adversely. This is because research has revealed that ejaculation does not have much influence on testosterone levels. (It’s true that normal testosterone levels support sexual performance.)

Testosterone is generally slightly higher when abstaining from orgasm. And it does rise slightly during sexual activity—before dropping back down to normal.

See Orgasmic frequency and plasma testosterone levels in normal human males.

Testosterone also spikes and then drops back around day 7 after ejaculation, indicating that orgasm triggers a subtle hormonal cycle that lasts at least a week.

Also see: How does sex affect testosterone levels?