You need the MTypeId of the override you want to create. Find typeIDs.py in your Maya installation folder (.../lib/python2.7/site-packages/maya/app/renderSetup/model/typeIDs.py
)
As you can see, the MTypeId of a shader override is 0x58000386.
Then, just create a layer and a collection, and call your createOverride method:
from maya.app import renderSetup
import maya.api.OpenMaya as OpenMaya
renderSetup.model.renderSetup.initialize()
rs = renderSetup.model.renderSetup.instance()
# Create layer
rsLayer = rs.createRenderLayer('MyFirstLayer')
# Create collection
rsColl = rsLayer.createCollection('MyFirstCollection')
# Create shading override
over_obj = rsColl.createOverride('MyFirstOverride', OpenMaya.MTypeId(0x58000386))
If you want to create another override, just find the according MTypeId.
Thanks to Fabrice that answered this question here: https://stackoverflow.com/questions/48305522/maya-python-how-to-create-shaderoverride