Also check the section on SSH (409-419) for adding extra security to SSH.
Limit Login
You have noticed that your server goes unused by official users between the hours of 12am and 5am. You have also noticed that during these hours are when the most breakin attempts occur.
Restrict remote login during these hours. Local login is alright (if there is an attempt, and you know of it, you will still want to be able to drive to the office and repair any damage that was caused).
Also, restrict is such that root cannot SSH into the machine directly. Make it so that users must first SSH as themselves and the su to root (this forces accounting to take place).
Finally, set SSH's timeout to a reasonable default to prevent interception and possible hijacking.