Georgia Gov. Brian Kemp and former Sen. David Perdue, R-Ga., squared off in a rancorous debate Sunday night as Perdue mounts a primary challenge against Kemp.