Automating Server Startup with HP iLO

We were recently in the position that we had to shut our servers down during building maintenance windows. This was due to excessive vibrations throughout the building structure, and we felt this would affect the disks in the servers. Shutting down the servers automatically was the easy part, starting them back up less so. We could have changed the BIOS settings to do this, but that would require downtime to make the change, and there was also no guarantee the BIOS would have a scheduled startup option. So, we turned to the HP iLO instead. I made these notes in the works wiki, so the language may not flow sensibly at times.

A variety of tools are used to automate server startup.
In order for it to be automated, it must be possible to log in to the iLO using SSH keys. This is key-based authentication instead of password based, so it retains security but allows non-interactive logins based on trust.

1. iLO User
To set up key based authentication, it is wise to set up a user with only the functions you require enabled. In this case, create a user called ilouser that only has privileges against the ‘Virtual Power and Reset’ option. Note down the password, but be aware that it is not required after this stage.

2. SSH Key
Once the user has been set up on the iLO, a key-pair with the same username has to be generated. At the command line on a Linux machine, generate the keypair with…
ssh-keygen -b 1024 -f ilouser -C ilouser

Do not enter a passphrase (this can be used to secure a key even more, but forces the authentication to become interactive.)
This creates two files.
KyleG@WMWS0001:/tmp> ls -l ilouser*
-rw——- 1 KyleG users 887 2008-12-09 11:21 ilouser
-rw-r–r– 1 KyleG users 217 2008-12-09 11:21 ilouser.pub
KyleG@WMWS0001:/tmp>

ilouser – this private key remains on the source machine
ilouser.pub – this is the public part that will get uploaded to the iLO

Do not change the permission modes on the files, as SSH checks them on use. It will not use files with loose permissions.

3. iLO SSH Key Authentication
Log in to the iLO again, and select the SSH Key Authorization menu. Browse to your ilouser.pub and upload it to the iLO. All being well, the iLO will accept it against the ilouser account (or an account with the same details as the comment field in the public key – ilouser.pub)

4. Testing Login
Test that the system can log in by specifying the location of the private key (the file without the .pub), the username to login with, and the hostname.
ssh -i /home/KyleG/ilouser/ilo-user ilouser@chsr0001-ilo.nes.scot.nhs.uk

All being well, you will be presented with the iLO SSH interface after a short delay. If you encounter an error involving dispatch_protocol_error, then your iLO firmware should be updated. It will still allow you to execute commands non-interactively though.

5. Running a command non-interactively
It sounds complex, but in reality use the above command and append the command you want to execute onto the end, enclosed in single quotes.
KyleG@WMWS0001:/tmp> ssh -i /tmp/ilouser ilouser@chsr0001-ilo.nes.scot.nhs.uk ‘power on’
dispatch_protocol_error: type 100 seq 8
power on

Server power already On

hpiLO> KyleG@WMWS0001:/tmp>

The dispatch_protocol_error is a bug in the iLO firmware on CHSR0001

6. Automating it
To automate it, use crontab on a different machine and use the above command. An example crontab line would be
30 2 * * * /usr/bin/ssh -i /home/KyleG/ilouser/ilouser ilouser@chsr0001-ilo.nes.scot.nhs.uk ‘power on’

This will execute the command every day at 0230. The command logs into the remote machine over SSH, and simply runs the ‘power on’ command.

4 comments

  • [...] Automating Server Startup with HP iLO [...]
  • Michael (837 days)
    I appreciate your efforts here however why not simply use wake on lan? I do this with many HP servers which perform nightly batch jobs. Rather than leave them running all day long doing nothing we configure them to shut down once they have completed. I have one server configured with schedule tasks to wake them up using a utility called mc-wol.exe. Works a treat, no issue with ssh or worrying about password authentication.
  • admin (837 days)
    Morning,

    At the time of writing, that *was* the WoL server :-) As everything on site had to be shut down, we had no way of bringing it back, and once it was back up it would pull everything else back online.

    In retrospect I should have investigated Subnet Directed Broadcasts to maybe trigger it from a remote network, but I’m comfy with iLO and SSH so just went down that route :-)

    As always, TIMTOWTDI :-)

    Cheers

    Kyle

  • [...] login to work. After much weeping, wailing, and gnashing of teeth. I finally discovered from here that the -f and -C options of the ssh-keygen command are required for iLo to accept the key. I had [...]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Go to Top