From the ffmpeg docs
ffmpeg -i LEFT -i RIGHT -filter_complex framepack=sbs OUTPUT
framepack only works with 2 inputs, but you could merge 3 with a more complex filtergraph.
sbs means side-by-side.
This doesn't correct for perspective or handle the overlap or anything else needed for it to look good, and the word "panorama" doesn't appear in the ffmpeg docs. If it's possible to do it properly with ffmpeg, it will require non-standard video filters. (And by properly, I mean without manually finding how many pixels to crop from the inputs before feeding them to framepack).
Just posting in case people were wondering if ffmpeg did this. Turns out the answer seems to be no. It does have filters called "perspective" and "lenscorrection", but nothing for smart-stitching panoramas that I can see.
After further questions from poor, I googled for ffmpeg panoramic, and found http://krpano.com/. It says it can do panoramic video, but I'm not exactly sure WHAT it can do with it. It doesn't list any video formats as input formats, but maybe it can work with image-per-frame PNGs or JPEGs or something. Ok, no nvm, http://comments.gmane.org/gmane.comp.misc.ptx/24263 says krpano can play back / "explore" panoramic video.
The other google hits were from people trying to make a panoramic single image from multiple frames of a panning shot.
Hmm, this looks interesting. Says it uses ffmpeg and pano tools.
Not sure it can actually stitch, though. Might just correct fisheye->panorama.
Maybe this one:
mentions http://www.video-stitch.com/. Commercial software with a free version that watermarks videos larger than 1024x768, but does have a download for Linux. (presumably a statically linked x86 32bit binary. It might use NVidia CUDA, at least on Windows.)