The Roman inability to close off the frontier

I consider myself to be somewhat of a fan of the Roman Empire, and a lot of things mystified me about it, but what I am considering right now is the ever present problem of the frontier.

It seems like the Romans could have been a lot more efficient with their frontiers. For example, and this may underrate the Picts and other Scottish tribes, but the need for Hadrian's wall and the presence there could have been somewhat mitigated had the Romans at least done some subjugation of the Highlands. 

A more famous example is, of course, the use of the Rhine river as a border versus the shorter Danube border which was very much within the means of the Roman Empire. 

Obviously, it is easy to look in hindsight at these things, but there have to be some factors which explain this. Did the Romans, in their time, understand the geographic necessities of their borders? 

It's an especially relevant issue since the guarding of the border became a huge problem in the latter days of the Empire, one which significantly undermanned the military against the threats from Parthia and the Sahara.