You are not missing anything, it tries to fit the entire contents. I generally work from the element side and use the find patch button to make it select the controller points that match the elements I have selected. You could submit a feature request and we can look into making it behave better.
Bookmarks