I suppose you don't have to. ASA 1 manual doesn't mention terminators at all, and we use it also without any plugs (2x omni antennas + 2x boosters + ASA1 + 2x receivers powered from ASA).
To me it's an informed decision type of thing. The reason for terminators is to keep the unconnected port(s) from acting like an antenna. If sufficient signal is present it could come back into the preamp and hetrodyne (mix) with the wanted signals and produce sufficient harmonics to raise the noise floor or wipe the front end out on the RX's entirely so nothing would be received.
If you use in multiple locations I would highly recommend.
A terminator is simply a resistor that places a load on that port to shunt and received signals to ground.
If you happen to have an RF spectrum analyzer take a look at the noise floor without and without the terminators. I bet it moves at least 6dbm
Of course you should not be using a wireless path with only a 6db fade margin but that is a discussion for another day.