The electrical signal in a wire travels at some fraction of the speed of light, depending on the properties of the wire and the insulation around the wire. For low-frequency signals such as audio, there's no practical latency difference between 20 feet of wire and 20 miles of wire.
A/D and D/A conversions cause latency - anywhere from .5ms to several ms, depending on sample rate and other factors.
Certain other mathematical operations done on the signal in the digital domain can cause latency depending on the complexity of the processing done. This can also be several milliseconds.
Digital packetization of audio (IP telephony, cell phones, etc.) can create latency as the system collects a certain number of samples before transmitting the packet. This can easily be 20ms in some systems. This is one of the hurdles preventing digital wireless mics from being mainstream.
The big one, though, is propagation of the sound through the air. In round numbers, it takes about 1ms for sound to travel 1 foot. Stand 10' from the speaker and you have 10ms of latency, not counting any latency before the speaker in the signal chain.
Your application determines the latency you can tolerate and what you can do about it. A band spread out on a large stage can have trouble staying together if they are hearing each other through the air. Monitor wedges, or better yet in-ear monitors tighten up the time reference by getting rid of the 40 or 50 milliseconds of slop due to the stage size.
It probably doesn't matter for speech, as the speaker is going to be more bothered by the bounce off the back wall of the room than any monitor timing issues.
As Mac said - don't worry about it - this is one of those problems that if you don't know if you have the problem, you don't have the problem.