I believe your thought is quite correct and the way you are using the car I would be surprised if you did not have sludge in the oil. This situation will be made worse in the cooler weather which would be confirmed by your car. If you cannot use the car for longer runs which would help, I would make the oil changes more frequent.