Steve Smith and Spencer Harbar (again - oh no I've become his groupie!) were discussing Best Practice around setting up Intranet, Extranet and Internet sites - far too detailed to relay here I'll stick a link up when the slide deck becomes available -
However, one area they discussed in detail was what should be on SSL and they recommended that if nothing else uses port 443 then Central Administration. I hadn't given it much thought but now that its been highlighted to me there is certainly one compelling reason for this. A rogue user could cause catastrophic damage to a farm if they got hold of the Central Admin password, so even in an intranet environment why would you have these credentials flying around unencrypted?