A step-by-step description of this process can also be found on the Rackspace Cloud Server Knowledge Base. In summary the process of securing the new Ubuntu server included the following main steps (I've included a few links for your reading, particularly for configuring ssh and the firewall);
- Give the root account a new password
- Create a new administrator account and give it the correct permissions
- Update Ubuntu
- Lock down SSH connectivity
If all is good it would also be a good time to backup the server so you have a secured image to restore from as you continue down setting up the server.