The only insecticide still registered for use for carpenter bees for homeowners is Carbaryl dust (sevin)
Recommendation is to dust holes with sevin dust, do not plug entrances now.
In the fall, fill the holes with wood putty, sand wood putty smooth. This will be a continuing problem w/ a cedar shake house. Paint or varnish is the only way to make them totally un-interested.