Most insect- and other fauna found in mushrooms would be there because they're fungus-eaters, at least at certain stages, especially as larvae. Many known frog-food critters can be found in mushrooms: drosophila, springtails, mites, fungus gnats...'Shrooms are also home to many beetle larvae. Unless you can free the larvae from the fungal fruiting body, however, your amphibians will be mostly exposed to the adult insects that emerge. Most of these should be ok for frogs but certain beetles might be too chitinous.
For the best chances of a well-infested mushroom/fungus fruiting body look for the longest lasting ones. These tend to be those with pores for spore surfaces rather then gills. They are the polypores (shelf fungi, bracket fungi, etc.) and the boletes (shaped more like a traditional mushroom but with pores, not gills).
--Diane