Sooner or later you have to look at your video and make choices. A term of the art is to "create video selects". And the organizational technique used in Premiere Pro is called a "stacked timeline" or a "pancake timeline"(with more details explained here). But to ease into these (and answer your question)...
You absolutely need to go through your footage to find the good stuff. Don't think of it as "editing out the unusable parts", but rather "find, mark, and label the good parts". Labeling is key because that's how you are going to find the good stuff later. Mark in and out points to quickly get rid of useless "flying to" and "flying from" parts of your footage.
Quoting from 1, "By stacking 2 timelines on top of each other…an editor can now access an entire reel of selects in chronological order (or whatever order desired) and instantly drop them into the active timeline below. To accomplish this layout…first open both timelines by clicking on them in the Project Panel. Drag the Selects timeline tab and hover over the top quarter of the Master timeline. Once it turns a purple color, release the tab and you’re in Pancake Land! By not having to bounce between tabbed timelines and using copy/paste…the process of analyzing and deciding which shot makes it into the cut is streamlined and intuitive. You can playback the Selects timeline in the program panel or hit the “match frame” keyboard shortcut and play it back in the source panel. Then just insert edit or drag-and-drop the shots you want into the Main timeline. Another very nice feature of this edit workflow is that the shots you pull from the Selects (top) timeline remain in their original position and are duplicated into the Main (bottom) timeline."
Another thing to keep in mind is that the construction and ordering of the selects timeline is up to you. You can order your footage chronologically, by subject, by altitude, whatever your labeling system and sorting system might be. Make metadata work for you!
At this point you probably have an idea of what your true "hero" footage is, and how much time you can fill with the top 95% of your material. (Or you have an idea how deeply you have to scrape the barrel to create a 2-3 minute video.) This will give you an idea of to find the right matching music--both thematically and in length. And it will give you an idea whether you are better off to start cutting the video to the audio, or vice-versa.
Usually the best results are those that make the best elements speak best, both by themselves and together in ensemble. If something sticks out like a sore thumb, take it out. If you realize there's a shot that would make the video complete and you don't have it, maybe (in your case) you can rig up the flycam and get it. Most video editors don't have that luxury because the footage is a given and they cannot just ask for more.
As your video converges with the audio, you will get a sense of just how important it is to align visual and audio "beats". Keep working until every visual moment is reinforced by the audio, and vice-versa.
