How do i capture them? Also, how do i defeat enemies?

You nead trap for catch a wild pet, the '' catch a pet '' quest is an event, you don't nead anything for get this pet.

For defeat an ennemi you nead to put his heal to 0 % with the hability of your pets. The battle work by turn so you have to be carefull at what action you select (tho an lv 5 pet have only 2 option : a basic attack or mend).