"'Excuse me, are you a Kohen?' 'No, I'm a Yisrael.'" (Glinert)

"The mother determines if the child is Jewish or not according to Halakha and then if the child is Jewish, the father determines if the child is Kohen, Levi or Yisrael." (source)