It's more correct to say that the gravitational force supplies a centripetal force, allowing the orbiting object to undergo circular motion.
(Note: (I'm not accusing you of this necessarily but it might help) it's a very common trap for students to think that when an object is undergoing...