I would not remove the infected leaves now. Powdery mildew is more unsightly than it is damaging. The plant needs the leaves for photosynthesis to build up the food reserves in the crown for next years flowers. As noted above, wait until late summer/early fall when the foliage begins to yellow and decline and then cut the plants back. This is also a good remedy for preventing botrytis. The mildew may or may not spread to the other peonies; they may have more resistance. In the future, you can also protect the foliage by using a preventative fungicide such as myclobutanil (Spectracide Immunox Multipurpose Fungicide) or potassium bicarbonate (Armicarb, Bonide Remedy).
