They’re usually non-territorial. If they aren’t bothering anyone, why touch them?
Mud daubers make mud nests that kind of look like a series of closed-off tubes stacked on each other. They put caterpillars in the tubes, lay an egg, and close the tube, and the larva stays inside until it’s an adult. If the tube is open, a wasp came out.
They definitely aren’t territorial. They also spend very little time around the nest, the builder only comes back to lay more eggs and the adults vanish as soon as they come out. Unless you go up and grab one, they won’t sting you. You could safely stand by the nest and shout at it if you were so inclined.