MegaRAID drive inconsistent: consistency check stuck

Many server vendors ship their servers with branded MegaRAID RAID controllers. The relationship I have with the many flavours they come in is sometimes a bit poluted I dare to saay.

An excellent quick guide to the more modern StorCLI utility available to Linux, Windows, FreeBSD and others is Thomas Krenn’s Wiki page on StorCLI. More often than not I have to look up in the CLI reference made available by (today) Broadcom.

Now I’ve come across a slightly stubborn controller that reported a Virtual Drive not being consistent.

I’ve come across a system with an IBM-branded MegaRAID controller where one of the virtual disks (a RAID5) show an optimal state but not consistent.

# /opt/MegaRAID/storcli/storcli64 /c0 show
Generating detailed summary of the adapter, it may take a while to complete.

CLI Version = 007.0409.0000.0000 Nov 06, 2017
Operating system = Linux3.2.0-5-amd64
[...]

Product Name = ServeRAID M5110
[...]
FW Package Build = 23.34.0-0018
BIOS Version = 5.50.03.0_4.17.08.00_0x06110200
FW Version = 3.460.105-6456

[...]

VD LIST :
=======

---------------------------------------------------------------
DG/VD TYPE State Access Consist Cache Cac sCC Size Name
---------------------------------------------------------------
0/0 RAID1 Optl RW Yes RWBD - ON 278.464 GB
1/1 RAID5 Optl RW No RWBD - ON 3.635 TB
---------------------------------------------------------------

I’ve attempted to start a consistency check on that VD but the controller always reported it couldn’t:


root@ieu04-sr21:~# /opt/MegaRAID/storcli/storcli64 /c0 /v1 start cc
[...]
------------------------------------------------
VD Operation Status ErrCd ErrMsg
------------------------------------------------
1 CC Failed 255 Start CC not possible
------------------------------------------------

At first I suspected a running patrol read would block it. (storcli64 /c0 show pr) but the consistency check wouldn’t start afterwards either. In the end after trying to resume it it was finally OK:


# /opt/MegaRAID/storcli/storcli64 /c0 /v1 show cc
[...]
--------------------------------------------------
VD Operation Progress% Status Estimated Time Left
--------------------------------------------------
1 CC - Paused -

# /opt/MegaRAID/storcli/storcli64 /c0 /v1 resume cc
CLI Version = 007.0409.0000.0000 Nov 06, 2017
Operating system = Linux3.2.0-5-amd64
Controller = 0
Status = Success
Description = Resume CC Operation Success

Afterwards the consistency check quickly ended as done and the alert issued by the monitoring system got cleared. Nonetheless I manually started a consistency check in order to see if any issues would be found. I guess the consistency check was paused at some point in time due to a preceeding firmware updated but wasn’t restarted automatically afterwards.

January 17, 2018

Posted In: Uncategorized

Tags: , ,

Leave a Comment

Serial console server (2): The curse of the 4 ports

As said in the previous post, the “fancy” EXSYS 8 port card got happily recognized, but: It only showed 4 of the total of 8 Ports, and that was only when the 2 onboard serial controllers were disabled. *grrr*

After trying out also Debian’s backport kernel without success I checked the 8250’s driver code in Linux and only later I stumbled upon Kyle Anderson’s wiki: Debian (and basically most Linux distributions) are hard coded not to initialize more than 4 serial ports, we need to tell it to do so via a boot paramter. Honestly: Most modern computers don’t even have physical serial ports, maxing out at 2 if they are still present, so this setting makes sense for most computers. Kyle’s Wiki contains instructions to get it running with CentOS which is using GRUB, Debian has switched to GRUB2 with squeeze so his instructions needed slight changes since grub/bootloader doesn’t exist and /boot/grub/menu.cfg gets overwritten at kernel updates.

Edit and change: /etc/default/grub as follows

# squeeze default:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash"

# new:
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash 8250.nr_uarts=8"

Then issue ‘update-grub’ and reboot, then check if you additional ports get regoznized by issuing:

cat /proc/tty/drivers

For some reasons, I had even to up this to 16 because when I enabled the first 2 onboard tty’s the first serial port of the EXSYS Oxford-based were not ttyS2-tty9, but ttyS5 -ttyS12 maybe due to other controller using resources. Since playing with kernel boot parameters doesn’t belong to my daily business I was very happy to find this “sidenote” saving me a couple of hours learning deeper into driver options.

September 11, 2012

Posted In: Uncategorized

Tags: , , ,

Leave a Comment