Another factor making true high speed rail more difficult in the US is sharing the right of way with freight trains. You can imagine the fubar ballet of trying to mix 200+ mph high speed passenger trains with 49 mph freights and slow switching movements, and the leapfrog movements that would need to be done to make it work, and the ripple effect that any movement glitch would cause within the network should it be attempted.
The European rail system is somewhat more conducive to higher passenger rail speeds; there is a relative lack of freight trains and local switching movements compared to the US. In Europe, something like 70-80% of the total freight tonnage is carried by trucks, and is about the inverse of what is carried by rail in the US. Again in Google maps satellite view, you will not see a lot of rail branches and siding tracks off to loading docks, grain elevators, coal mines, industrial sites, etc. in Europe as you will in the US.
Of course, the highest speed lines in Europe are reserved for passenger service only, or maybe express parcels. The proposed high speed service lines in Florida and California are being planned this way, but the majority of proposed "high speed" service in the US is really incremental higher speeds, mostly due to the 110 mph restriction in my earlier post as well as the inability to restrict the lines to passenger use only. It is wickedly expensive to build a new line or upgrade an existing line to true 200 mph+ service, but obviously some countries have decided to make that level of investment.