What does your temp gage say?? Exhaust manifold temp by feel is not all that accurate, remember there are really two sources of water running through them, bypass water and heated water from the block. When starting up you have mostly all by-pass water which is lake temp, then as the thermostat opens you switch to mostly heated water from the block which is 140-170 degrees plus the heat of exhaust inside the manifolds. Another variable is the manifold material, stainless, aluminium, or cast-iron, but the biggest variable is your skin tolerance.
Maintain a temp gage reading between 145 - 170 & your good to go.