Always sweep with, with thrashing oar
Our only goal will be the western shore
As noted by Doug Casey, two conditions must be present in order for completely open border policy to advance prosperity. There can be no welfare services made available to immigrants, lest a free rider problem develops in the country accepting immigrants. Those who are forced to pay for this free ride suffer.
The other condition, which can be seen as a general condition of the first, is that property must be privately owned. Otherwise, squatter camps are likely in 'public' areas.
Standard of living increases when labor is mobile. Prosperity increases from immigration are restrained, however, when property rights are not fully respected.