A thorough flea combing FOLLOWED by a bath with Dawn dish washing liquid to wash out the flea eggs.
Dawn kills fleas pretty well and should not harm the baby as long as you are careful to NOT get it in the baby's face.
I use it when I get babies, (even newborns), in with fleas.
~M