How to change hostname permanently without any issues?

So the title says it all.
I set up a hostname while installation which I don’t like now.
Is there any way to change my hostname permanently?

Once again this is far from a technical issue. Post in the correct forums please.

Also a 2 second Google search would have helped.