Hi Heather, we see Carpenter Bees quite a bit as they look for mates and places to nest. they arent going to come after you unless you disturb the nest and as you stated only the females can sting. males may "check you out and come near you but quickly fly away. if you are really concerned you may want to mow that area in the morning before they become active.
Being "Carpenter Bees" they burrow on wood to make nests, especially if the wood is bare, unpainted or weathered softwoods. Painted or pressure treated wood is much less susceptible to attack. Male bees show aggression to people around nests but they cannot sting. Females do however sting if they feel disturbed.
To manage Carpenter Bees, deter them by painting all exposed wood surfaces, stain or apply wood preservative. Keep garages and outdoor building closed when Carpenter Bees are actively looking for nesting sites.