Oh, joy. Welcome to the caveats of the port loaded design..
In stuffing a port, you will definitely lose efficiency of output.. as well as change the response slope.. been there, done that. I found it's the least best way to solve the problem, and also, the force of the speaker tends to shread or alter the stuffing used (at least the materials I tried).
I have tried to deal with the Harmonics generated by the port in general, and have found several things that help a bit.
First, use the largest port diameter possible. Next, flairing both ends of the port seems to help as well, and last, use a shape other than cylindrical where possible.
No wonder the Shop Vac hose didn't work. Those ridges in the pipe really create a mess.. they "warble" at the right frequencies!
Best bet: Either seal the box.. and use some electronic boost.. or (hate to say it) scrap the design.. and start afresh. I've had to scrap many a design that SAID it would work in CAD.. but failed to deliver in real life.
Hmm.. last thought. You said the Freq's where the overtones were produced were "off" from what the software predicted? Are you sure the driver, port, and cabinet volume AS PRODUCED aren't slightly off from the calculated numbers - that would account for the radical difference in frequencies noted..
Regards,