If a plant has toxic sap, I'd avoid using it for leaf litter. But, most hardwood tree leaves are fine. I usually put some less durable leaves at the base of my leaf litter, topped with more durable leaves like magnolia, live oak, or sea grape. Folks in the mountains use rhododendron leaves. Many members of the bay family (of which magnolia is one) make pretty good litter. The key, IMHO, is to have a 3-D leaf litter structure. You don't want a mat of flat wet rotting leaves. Leaves that curl and maintain their shape when wet are great for providing structure and voids in the leaf litter, which is good for the microfauna as well as small froglets such as pumilio.
Also, boiling will kill most insects and their larvae, bacteria, fungus, etc., but boiling cannot break down toxins like those found in pesticides, so it does matter where you collect the leaves.