What you're asking for is pretty common in TV circles (think CNN interviews with 3x people in different places, arguing with themselves and a host) - they use satellites and lots of expensive gear...
How tech savvy are the people on the other end? If you have staff who can assist, set up Wirecast on each computer and have each remote computer hooked up to a camera (for best quality, get a camera with HDMI output and a Blackmagic Intensity card) and just publish a simple RTMP stream from their computer directly.
Then, on your end, get a 4x computers to receive a stream each (or a computer with 4x monitor outputs), plug each computer into another computer, running Wirecast, with Blackmagic Decklink Quad card (with DVI/HDMI to SDI adaptors) and use Wirecast to receive each stream and live mix the streams as appropriate (like a sports broadcast for example) inside a single 720p or whatever frame.
You can then publish the single feed from Wirecast to any number of services (Livesteream for example) that won't list your video publicly, then set up a 2nd RTMP stream on a more robust connection, just for those in the chat can hear & see the other people they're talking with (using a 3rd party like Livestream adds a fair bit of latency, will make for a shitty conversation).
Does that make sense? I can draw a diagram if you need me to, hah.
This really depends on your budget and level of tech knowledge at each remote site. You could use Skype at full screen, instead of Wirecast, but it won't be as good quality (but fuckloads easier if you have nobody on the other end - just tell the people participating in the chat to load Skype, that's it, you do the rest)
An even easier option is just to make a Skype conference call, make it full screen, output it to two mirrored monitors - one for you to watch locally, one for output to second computer running Wirecast. Then just set up the Wirecast computer to broadcast to a stream location of your choice for everyone to view.
This is a bit long winded, so if you have any points you need clarified, ask :)