You might take a look at your shower and outside shower faucets. Be sure they are turned off.
If, for example, Your outside shower faucets are turned on, that will allow cold water to feed back into the hot water side.
You may also have a bad check valve on the water heater.
Peace
Dave