Saturday, August 31, 2013

The LSI MegaRAID Chronicles

Having thousands of photos stored on my server, I decided that I needed a bit more speed in Lightroom, Photoshop, and Vegas.

Previously, I was running software RAID-5 using 6 Hitachi 4TB hard disks for 18TB of available capacity. Performance was quite decent for reads - 380-500 MB.s but quite poor for writes (as most software raid servers are) at 70 MB/sec using ATTO benchmarks. This made Lightroom editing of photos quite slow and painful.

I decided to try to speed up the system by using a Samsung 840 500GB SSD as my boot drive (running Windows x64 Ultimate), which was originally hooked up to a Marvell SATA port on my Asus P8Z68-V Pro motherboard. The Marvel controller was giving fairly poor performance - as I was seeing 330MB/sec reads and 220 MB/sec writes. When I moved the Samsung 840 SSD to Intel's SATA3/6 GBps ports, by bandwidth increased to 550MB/sec for reads and 340 MB/sec for writes (almost 50-100% improvement!

To increase the speed of the Intel software RAID-5, I decided to try a hardware raid card. After looking around, I found that LSI MegaRAID 9260-8i were highly rated but a bit expensive (around $470 on This was a bit too expensive for me, so I decided to see what's available on Ebay. A few people recommended looking for a used RAID card from an older server.

I found a seller listing a Dell OEM LSI 9260-8i for $250 (new). This sounded like a good deal, so I purchased the card. I received the card rather quickly but it took a long time to figure out how to get it to work. First, it would not boot from my motherboard's second PCI Express 2.0 x16 slot (the first being occupied by the Zotac GTX 570 Graphics card). Next, I tried the 3rd PCI Express 2.0 x16 slot and the card's LSI BIOS popped up. I was excited until I realized that this slot was only running at 1x speed (being shared with 2 other PCIe 1x slots, USB controller, and other devices).

I decided to update the firmware on the LSI. However, when running MegaCLI to burn the firmware, it complained about firmware corruption. What the heck! I found on a bulletin board a work-around. I had to install an intermediate firmware release first, and then I would install the latest release. It worked! But, what a pain! LSI does not do a good job testing their products.

The LSI's WebBios was impossible to use as my mouse control was unstable. However, I found that I could use MegaCLI commands to do the same thing. So, using quite cryptic commands, I initialized my array (taking over 10 hours).

I decided that if I could not get the LSI Raid card to work in a x16 slot, I would return it. But how? I decided to give one more try - I would move the Zotac GTX 570 to the 2nd PCIe 2.0 x16 slot and use the main x16 2.0 slot for LSI 9260-8i controller. it worked! This had to be one of the more painful RAID installations for me. I can't believe the steps I had to go through to get this to work!

After initializing the disk array, I tried formatting the volume but Windows x64 reported Format Failure each time. I tried quick format, normal format - all failed. I was getting quite frustrated!

I called LSI Technical Support to try to get some help, but hung up after waiting on hold for 30 minutes.

Finally, I realized that 4TB drives were not supported by "Basic" disks in Windows. I needed to have the disk array setup as a "Dynamic" volume and not a "Basic". Once I converted the array to Dynamic, the quick format worked beautifully.

Computer Configuration
Intel i7-2600K CPU at 3.4GHz
ASUS P8Z68-V Pro Motherboard with BIOS 3402
Crucial Ballistix 16GB (8GBx2) DDR3-1600
Samsung 840 SSD (500Gbyte) connected to SATA1 of Intel (6Gbps) <-boot drive
Seagate 1TB ST31999524AS hard disk connected to SATA2 of Intel (6Gbps)
Dell LSI MegaRaid 9260-8i card in PCIe 2.0 slot 1(running at 8x 2.0 mode)
6x Hitachi 4TB Hard Drives connected to the LSI MegaRaid
Zotac Amp! GeForce GTX 570 Video Card in PCIe 2.0 slot 2 (running at 8x 2.0 mode)
ASUS Xonar Essence STX in PCIe x1 slot
Blu-Ray Drive connected to SATA3 (1.5Gbps)
HP ZR30w 30" Monitor at 2560x1600
SIIG USB Keyboard
Logitech MX518 mouse
800W Power Supply
Antec Titan Server Case
Windows 7 x64 Ultimate

Exact Install Procedure of the LSI MegaRaid 9260-8i
1. Setup ASUS BIOS in preparation for LSI MegaRaid 9260-8i
2. Install LSI MegaRaid 9260-8i in PCIe_1 2.0 x16 slot (main slot)
3. Install GTX570 Graphics card in PCIe_2 2.0 x16 slot
4. Boot the system and log in as Administrator

  • With the LSI card present, you may no longer be able to change settings in your BIOS as LSI BIOS corrupts the behavior of USB mice.
  • You do not need to get into WebBIOS (unless you need to boot from the array). WebBIOS requires a PS/2 mouse, but you can do all the same functions in MegaCLI command line in Windows

5. Install the LSI MegaRaid's latest software driver
6. Install the firmware update using MegaCLI
C:\Users\Admin> megacli64 -adpfwflash -f mr2108.rom -a0
  • If the firmware succeeded, great, you're done. If you get a message about corruption, you have really old firmware and may need to first flash an intermediate firmware file 
  • Reboot the PC and verify the new version
  • Note, reboot is slow, don't give up hope
7. Create a RAID-5 array using either MegaRAID Storage Manager or MegaCLI

Let's assume we have a server with one MegaRAID SAS cards. The first one is already setup but we have just plugged a disk bays on the second card.

List physical disks on second card (only print enclosure and slots numbers):
C:\Users\Admin\CLI> megacli64 -PDlist -a0 
Enclosure Device ID: 0
Slot Number: 0
Enclosure Device ID: 0
Slot Number: 1
Enclosure Device ID: 0
Slot Number: 2
Enclosure Device ID: 0
Slot Number: 3
Enclosure Device ID: 0
Slot Number: 4
Enclosure Device ID: 0
Slot Number: 5
Enclosure Device ID: 0
Slot Number: 6
Now we have all enclosure and slot number. Let's create the new array:
C:\Users\Admin\CLI>  megacli64 -CfgLdAdd -r5 [0:0,0:1,0:2,0:3,0:4,0:5,0:6] -a0

Adapter 0: Created VD 0

Adapter 0: Configured the Adapter!!

Exit Code: 0x00

Read Cache, Write Cache, ReadAhead and Battery

A quick section about performance tunning....
Let's enable Read Cache, and always cache data:
C:\Users\Admin\CLI>  megacli64 -LDSetProp -Cached -LAll -aAll

Set Cache Policy to Cached on Adapter 0, VD 0 (target id: 0) success
Enable disks' cache:
C:\Users\Admin\CLI> megacli64 -LDSetProp EnDskCache -LAll -aAll

Set Disk Cache Policy to Enabled on Adapter 0, VD 0 (target id: 0) success

8. Once the array has initialized, you can format the disk and start using it:

  • Open Control Panel->Administrative Tools->Computer Management->Disk Management
  • Click on Disk Management and select the disk array
  • Right click on the disk and select Convert to Dynamic Disk 
  • Click on the a disk itself and select Format
  • Choose NTFS, 8192, Quick Format and press OK

The quick format is fast and you should be able to use the disk right away.

Thursday, August 29, 2013

Purchasing an Audi Q5 2.0T from Audi of Stevens Creek

Today, I made an appointment to purchase my leased 2011 Audi Q5 2.0T sports utility vehicle at Audi of Stevens Creek. As may 3-year lease was expiring, I decided that the value of my car was much greater than the lease purchase price.

When I arrived at 12:00pm for my appointment, I was planning to drop off the check, sign some paperwork and be on my way. Most of my other used vehicles were purchased this way (Mini Cooper S, Toyota Sienna XLE), and the whole process was quite fast at most dealerships.

However, it seemed that Audi of Stevens Creek had only one finance person, and apparently, he was working with another customer. No one else can print the forms. So I waited and waited.

I was getting ready to leave, when finally, another person showed up and printed the forms. Even though I was paying cash, I needed to go through a mandatory credit check! So, this normally 10 minute process of signing forms took over 90 minutes!

Even though I paid by check, Audi did not cash my check for about a week, delaying my DMV registration.

I finally received my title and registration about 3 months later.

Sunday, August 18, 2013

Watching Elysium

Today, we went to see "Elysium" at the local AMC Saratoga Movie Theater.

The movie had an interesting take of the future, where all the rich people live off earth, on the space station "Elysium" and all the poor wretched souls live on Earth, suffering from poor health care, crime, and unemployment.

The movie was enjoyable but not very deep, and did not develop characters very well. However, I still enjoyed it and it's worth watching at least once.

IMDb has an interesting write up.

Set in the year 2154, where the very wealthy live on a man-made space station while the rest of the population resides on a ruined Earth, a man takes on a mission that could bring equality to the polarized worlds.
In the year 2154 two classes of people exist: the very wealthy who live on a pristine man-made space station called Elysium, and the rest, who live on an overpopulated, ruined Earth. Secretary Rhodes, a government official, will stop at nothing to enforce anti-immigration laws and preserve the luxurious lifestyle of the citizens of Elysium. That doesn't stop the people of Earth from trying to get in, by any means they can. When unlucky Max is backed into a corner, he agrees to take on a daunting mission that if successful will not only save his life, but could bring equality to these polarized worlds.