Making my website work over HTTPS – the proper way

While there are plenty of articles on how to move WordPress from http:// to https://, the documentation on how to set up a brand new installation with an SLL certificate is rather scarce. As a result, I omitted a step, or more precisely took the wrong shortcut. When I wanted to rectify my mistake, it was too late. Was it?