Cabac is lossless, but h264 is lossy. The part you are missing is that cabac is not THE compression algorithm. It is just the final step out of hundreds of steps in video compression. By the time you get to cabac, all the lossy steps have already been performed, and a final lossless step is added to squeeze a few more bits out.
Let the encoder set the level automatically. The encoder will have a default profile (High for x264) and higher the profile, better quality for a given bitrate or lower bitrate for a certain quality target. Unless you have compatibility concerns, default High is fine as well.
It is possible to use the select filter to select scenes for stabilisation and inclusion into the final result - see the example below. It seems that select is slow for skipping over unwanted material. This does not create any intermediate files except the *.trf file. In my example I am not using audio, which would have to be selected likewise.
Try adding the -pix_fmt yuv420p10le option (e.g. before -vf). Or only -pix_fmt + for using the same pixel format as the input video has.
If it doesn't help, you obviously have an inappropriate, albeit more common FFmpeg version — only for 8-bit colors. (Your original video uses 10-bit colors, and FFmpeg will try use the best 8-bit pixel format in ...
YouTube re-encodes whatever you give it. The process is lossy so there will be a quality reduction. All you can do is give it a very high quality:
ffmpeg -framerate 10 -pattern_type glob -i 'resized/*.JPG' -vf scale=1280:-2 -c:v libx264 -crf 17 out.mp4
Use -crf 17 or -crf 18 for higher quality.
Try without -pix_fmt yuv420p. I'm not sure what YouTube uses ...