The BTS from The Matrix shows the early days of this technique:
The newest technology is Intel 360° Replay.
Its customized for Sports Arenas and other places where there's a high value in making instant (takes almost a minute) 360° stop motion replays.
In some setups a low number of low resolution Cameras are used, this is evident by a lack of zooming or 'fine circular panning' with more expensive setups using dozens of JAI 20MP Cameras (and a Semi-Trailer full of Server Racks hooked up with Fiber).
Baseball Game example:
Formerly it was called Recode's FreeD:
It's simply a matter of recording Video simultaneously from many angles (not necessarily lined up, but it makes the Math a bit easier and the result more universally useful (audience acceptance)) and then on one particular Frame you switch from one Camera to another.
You could have a curved track of Cameras that follows the action or evenly distributed Cameras around an Arena. They can be low resolution or extremely high resolution which allows interpolating between Cameras and zooming in without loss of resolution.
One of the most advanced uses of 360° Replay is this Video of the Super Bowl 50 in "EyeVision 360" (Motion Graphics overlaid with FreeD zooming and flying): https://vimeo.com/157809580 .
The Earliest Days ('6 Cameras in a Basement'): https://vimeo.com/162429040 .
How it works and Demo:
Simplified: https://vimeo.com/178228338 .
Technical: https://vimeo.com/155729581 .
In the future TV could be FreeD: https://vimeo.com/195225232 .
You would watch using VR Goggles. You could stand in the middle of a Football Field or fly through the air while watching the Game. You could run through the local park wearing VR Goggles and determine if YOU would have caught the ball that fumble fingers missed; you would be in the Matrix (oops, I mean the Game on VR TV).