Are fertilizers dangerous enough to harm/kill wildlife?

It is not so much the animals that will suffer from direct exposure; it is the plants. When fertilisers containing phosphate (and to a lesser extent nitrate) run off into rivers, the water plants and algae use them to start growing explosively fast. At night, when photosynthesis does not occur, they will use all the available oxygen in the water as they grow. This will kill off the fish and plants. These will be decomposed by bacteria that also use oxygen, meaning there will be even less available for life. In the end, if the pollution is serious enough, life in the whole stretch of water will die.