Test storage controller remotely over BMC/IPMI?
I'm working at home, remotely testing some HA cluster stuff at our 'datacenter', and** I suspect that the storage controller (LSI SAS1068) is bad** in one of the nodes.
I'm a lazy ass and don't want to go on-site**; is there any way to confirm my suspicions remotely?**
I have serial console access via IPMI (BMC, not DRAC; no graphical console), pxe boot works.
Some details:
We bought several refurbed/used Dell PE 1950s from vendor about a year ago. Nearly all of them have blown caps on the SAS controller, with the exception being the two nodes I'm testing right now. I've been getting weird kernel bugchecks from "node1" under heavy disk I/O with several distros, various kernel versions (prepackaged and manually built), which I've normally attributed to DLM/SCTP bugs. However, I get similar bugchecks using normal filesystems.
I'm going to zero out the drives on that node, then probably fill them with /dev/urandom and check md5sums a few times; but it'd be really nice to call up the vendor and say, "Yeah, this hardware is shitty, here's the Dell Diagnostic error code. Give me replacement parts(/servers)."
Posted Nov 16, 2011, 3:03 AM UTC
you can't see virtual disk status with lsi sas controllers (or at least on the several models I work with this is true) via ipmi.
You will need to:
(a) install the megacli tool (I would send you the link but lsi changes the link every 24 hours (at least it seams that way). They have it in zip + rpm form so shouldn't be a problem to get installed. Once there google for the megacli cheat sheet, and use the terrible commands to see the status of your drives (# of media errors, etc), and virtual disk health etc.
(b) you can install dell's omsa (which installs a rebranded megacli and several other tools). I am not a fan of omsa, it causes issues according to me, so I try to avoid it.
ipmitool can check the rest of your hardware (temp, voltage, fan rpms, etc) including and especially the system event log. (ipmitool sel list).
so if it was me, I would get megacli installed, and ipmitool working. you should not have to go to the datacenter.
good luck.
This controller is plain SAS; no raid, I've got raw access to the underlying disks. Disks look good, SMART checks are fine.
'ipmitool sel list' and 'ipmitool sensors' was exactly what I needed. Thanks.