Both Pinnacle Studio 17 and Premiere Pro CC will suit the needs you listed, but only Premiere Pro accepts .psd files, Pinnacle Studio does not. Furthermore, Premiere Pro has an option to directly export frames to Photoshop, edit them there, and import them back to Premiere Pro. That sounds like a feature you will find useful.
The Avid Liquid product line was EOLed in 2010 and Windows XP SP2 is the latest Windows version supported.
At the Liquid forum, someone suggests a method to install Liquid 7.2 on Windows 7. See if you can use it for Windows 8.1
Here is a good guide to help you understand the compression.
Go Digital, scrap analog. You won't need a computer to handle the cameras or a video encoder.
Buy a cheap Axis camera and download Axis Companion software, then the camera just needs access to the internet and you can view it from anywhere in your smartphone.
You could also buy any other IP ...
Both options will achieve most of what you need (possibly not PSDs, but you can use transparent TGAs exported from Photoshop), however Premiere Pro is a far more powerful and professional option. If you don't like the idea of subscribing, I believe Premiere Pro CS6 is still available for one time purchase.
Alternately, Sony's Vegas Pro is also a highly ...