In order to convert frame rates without impacting the speed of playback, you have to remove frames to decrease the frame rate or create new frames to speed it up. If the frame rate is an even multiple, then this process can be done losslessly for reduction in frame rate since you simply drop every unneeded frame.
If they are not multiples or you are increasing the frame rate, then it is necessary to interpolate new frames. Interpolation is a process by which the computer looks at the next and previous frames and invents one that looks like it would fit between them.
If you shoot at 60fps and drop it to 24fps, it is not an even multiple and frames have to be interpolated or taken at the wrong time in order to make up for the difference. This results in a lower quality of video than shooting in 24fps to start.