I know what is Alpha channel but when I rendered a motion logo using RGB and RGB + Alpha in After Effects, I didn't see any difference. So what's the basic difference between them and how to decide which should be used?
The alpha channel is a grayscale 2D image plane of the same size as the RGB image, and is used when overlaying (or blending) the image on top of another image. If the value of an alpha pixel is zero, then the corresponding RGB pixel is invisible when the image is overlaid. If the value is 255 (in an 8-bits per channel image), then the RGB pixel is fully opaque. For intermediate values, you get a composite result, depending on the blending mode used. When viewed in a video player, you won't notice a difference because there's no other video 'underneath'.
See Alpha compositing for more details.