I'm trying to put together a 5vis project, and do so in the least possible number of generations.

I remember hearing at some point that pets can be bred to their first cousins (who have the same grandparents, but different parents)

If that's true, a 3vis project could start like this:

GM1----GF1
XX--------Xo
Xo---------oo
Xo---------Xo

(1/8 chance success)

GM2----GF2
Xo--------oo
XX---------Xo
Xo---------Xo

(1/16 chance success for generating the opposite sex mate.)

Where the goal is to produce a 2vis 1carry from both pairings, then breed those until you get a 3vis.
If you can breed with first cousins, you can make the final product a mate by producing a copy of the parent generation; if you can't you need an additional set of essences.

As long as two pets do not have any of the same parents (and they aren't parent-child), they can mate. So pets can mate with their cousins, grandparents, aunts and uncles, or anybody that's not their parent or sibling/half sibling. Was that your question?