The lovers (1961)

From back cover Del Ray paperback March 1980: Escaping the religious tyranny of a 31st-century Earth by a fluke assignment to the planet of Ozagen, linguist Hal Yarrow found that the worst of Earth had followed him -- Pornsen, his personal Guardian Angel, vigilant for any evidence of sin or wrong thinking. Conditioned by a lifetime of submissions, Yarrow would have accepted Pornsen's constant spying as an unpleasant necessity and lost himself in the study of the language of Ozagen's intelligent dominant race, the Wogglebugs... but then he found Jeanette, a not-quite-human fugitive, hiding in ancient ruins built by humanoids long vanished from the planet. For a believer like Yarrow, unconsecrated contact with any female was forbidden -- and love for an alien was unthinkable. But to Yarrow, in every way that counted, Jeanette was warmly and bountifully human!