The only concern I've ever read about is possible poisoning. There really isn't much to a flower, though I suppose clover blossoms might have a drop or two of nectar in the base of each floret.
As an FYI, the New York ASPCA (with the really schmaltzy ads) runs a pet poisoning hotline.