Driver for Promise FastTrack TX8660, TX4660, TX8668

[joerg_ac]

Hi,

I am desperately trying to get a SATA/SAS Hostadapter Promise FastTrack TX8660 working with Ubuntu. I did try for two days now and I hope somebody here might have an Idea.

First of all the basics:

Mainboard is MSI H55-GD65
The bootdisk is attached to the SATA host integrated in the mainboard. I do not want to boot from the Promise host. It is supposed to be the based for a software raid.

3 disks are attached to the TX8660 and the adapter recognizes them. They are comfigured as JBOD.

Installation of Ubuntu desktop did go smoothly, but the 3 disks on the TX8660 are not available.

sudo lspci -vv

provides the following for this adapter

01:00.0 RAID bus controller: Promise Technology, Inc. Device 3781
Subsystem: Promise Technology, Inc. Device 3781
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 32 bytes
Interrupt: pin A routed to IRQ 10
Region 0: I/O ports at bc00 [size=128]
Region 2: I/O ports at b800 [size=256]
Region 3: Memory at fbcff000 (32-bit, non-prefetchable) [size=4K]
Region 4: Memory at fbcc0000 (32-bit, non-prefetchable) [size=128K]
Region 5: Memory at fbcfc000 (32-bit, non-prefetchable) [size=8K]
Expansion ROM at fbc80000 [disabled] [size=256K]
Capabilities: [50] Power Management version 2
Flags: PMEClk- DSI+ D1+ D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [70] Express (v1) Legacy Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <64ns, L1 <1us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr- TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L0s L1, Latency L0 <128ns, L1 <2us
ClockPM- Suprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
Capabilities: [94] SATA HBA <?>
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [140] Virtual Channel <?>
Capabilities: [160] Device Serial Number 01-00-00-00-02-00-00-00
Capabilities: [170] Power Budgeting <?>


lsmod provides the following

ls_iso8859_1 4633 1
nls_cp437 6351 1
vfat 10866 1
fat 55350 1 vfat
binfmt_misc 7960 1
fbcon 39270 71
tileblit 2487 1 fbcon
font 8053 1 fbcon
bitblit 5811 1 fbcon
softcursor 1565 1 bitblit
vga16fb 12757 0
vgastate 9857 1 vga16fb
snd_hda_codec_intelhdmi 13090 1
snd_hda_codec_realtek 279040 1
ohci1394 30260 0
usbhid 41084 0
snd_hda_intel 25677 2
i915 321160 3
snd_hda_codec 85759 3 snd_hda_codec_intelhdmi,snd_hda_codec_realtek,snd_ hda_intel
snd_hwdep 6924 1 snd_hda_codec
snd_pcm_oss 41394 0
snd_mixer_oss 16299 1 snd_pcm_oss
snd_pcm 87882 3 snd_hda_intel,snd_hda_codec,snd_pcm_oss
snd_seq_dummy 1782 0
snd_seq_oss 31219 0
snd_seq_midi 5829 0
snd_rawmidi 23420 1 snd_seq_midi
snd_seq_midi_event 7267 2 snd_seq_oss,snd_seq_midi
snd_seq 57481 6 snd_seq_dummy,snd_seq_oss,snd_seq_midi,snd_seq_mid i_event
snd_timer 23649 2 snd_pcm,snd_seq
snd_seq_device 6888 5 snd_seq_dummy,snd_seq_oss,snd_seq_midi,snd_rawmidi ,snd_seq
ppdev 6375 0
snd 71106 17 snd_hda_codec_intelhdmi,snd_hda_codec_realtek,snd_ hda_intel,snd_hda_codec,snd_hwdep,snd_pcm_oss,snd_ mixer_oss,snd_pcm,snd_seq_oss,snd_rawmidi,snd_seq, snd_timer,snd_seq_device
drm_kms_helper 30742 1 i915
parport_pc 29958 1
intel_agp 29095 2 i915
drm 199268 4 i915,drm_kms_helper
i2c_algo_bit 6024 1 i915
video 20623 1 i915
output 2503 1 video
soundcore 8052 1 snd
snd_page_alloc 8500 2 snd_hda_intel,snd_pcm
lp 9336 0
parport 37160 3 ppdev,parport_pc,lp
hid 83440 1 usbhid
usb_storage 49833 1
ahci 37870 2
ieee1394 94771 1 ohci1394
pata_jmicron 2747 0
r8169 39650 0
mii 5237 1 r8169

So I am sure that I am missing a driver for the Promise TX8660
Promise provides binary drivers for some older SUSE and RHEL on the drivers CD, so there must be some Linux support.

I did call the Promise support and they sent me the sources for compiling the driver myself. It is actually only partly open source. It contains an closed source object to be linked into the driver. it contains a makefile so I tried to compile it.

This is the /usr/src

joerg@marvin:/usr/src$ ll
total 20
drwxrwsr-x 5 root src 4096 2010-08-21 01:47 ./
drwxr-xr-x 10 root root 4096 2010-04-29 14:17 ../
lrwxrwxrwx 1 root src 31 2010-08-21 01:47 linux -> linux-headers-2.6.32-24-generic/
drwxr-xr-x 24 root root 4096 2010-08-21 00:11 linux-headers-2.6.32-24/
drwxr-xr-x 7 root root 4096 2010-08-21 11:38 linux-headers-2.6.32-24-generic/
drwxrwsrwx 6 root root 4096 2010-08-21 13:32 t8sas/

what promise did provide is in t8sas/
there is also linux linked to the installed kernel version.

in the t8sas solder I did a make clean. It looks like this:

joerg@marvin:/usr/src/t8sas$ sudo make clean
rm -rf *.o *.ko .*.cmd .?*.o.d .*tmp* linux/*.o linux/*.ko linux/.*.cmd linux/.?*.o.d linux/.*tmp* *.mod.c ftlib.obj *.symvers
rm ./partial/* -rf
ln -s ftlib/ftlib.obj.64 ftlib.obj
joerg@marvin:/usr/src/t8sas$ ll
total 44
drwxrwsrwx 5 root root 4096 2010-08-21 18:42 ./
drwxrwsr-x 5 root src 4096 2010-08-21 01:47 ../
drwx--S--- 2 root root 4096 2010-08-21 00:57 ftlib/
lrwxrwxrwx 1 root root 18 2010-08-21 18:42 ftlib.obj -> ftlib/ftlib.obj.64
drwxrwsrwx 2 root root 4096 2010-08-21 00:57 include/
drwxrwsrwx 2 root root 4096 2010-08-21 18:39 linux/
-rwxr-xr-x 1 root root 10532 2010-08-21 18:41 Makefile*
-rw-r--r-- 1 root root 31 2010-08-21 13:32 modules.order
-rwxr-xr-x 1 root root 57 2010-08-21 01:32 products.h*
-rwxr-xr-x 1 root root 2475 2010-08-21 01:32 README*

Now I did compile it with the following result

joerg@marvin:/usr/src/t8sas$ sudo make
make ARCH=x86_64 V=0 CC=cc LD=ld ARCH=x86_64 DRIVER_SRC_DIR=/usr/src/t8sas -C /usr/src/linux SUBDIRS=`pwd` modules
make[1]: Entering directory `/usr/src/linux-headers-2.6.32-24-generic'
CC [M] /usr/src/t8sas/linux/osd_main.o
CC [M] /usr/src/t8sas/linux/osd_cmpi.o
CC [M] /usr/src/t8sas/linux/osd_cmpm.o
CC [M] /usr/src/t8sas/linux/osd_ioctl.o
CC [M] /usr/src/t8sas/linux/osd_timer.o
CC [M] /usr/src/t8sas/linux/osd_wrap.o
LD [M] /usr/src/t8sas/t8sas.o
Building modules, stage 2.
MODPOST 1 modules
WARNING: could not find /usr/src/t8sas/.ftlib.obj.cmd for /usr/src/t8sas/ftlib.obj
WARNING: modpost: Found 1 section mismatch(es).
To see full details build your kernel with:
'make CONFIG_DEBUG_SECTION_MISMATCH=y'
CC /usr/src/t8sas/t8sas.mod.o
LD [M] /usr/src/t8sas/t8sas.ko
make[1]: Leaving directory `/usr/src/linux-headers-2.6.32-24-generic'

The folder content after the make is:

joerg@marvin:/usr/src/t8sas$ ll
total 876
drwxrwsrwx 6 root root 4096 2010-08-21 18:46 ./
drwxrwsr-x 5 root src 4096 2010-08-21 01:47 ../
drwx--S--- 2 root root 4096 2010-08-21 00:57 ftlib/
lrwxrwxrwx 1 root root 18 2010-08-21 18:46 ftlib.obj -> ftlib/ftlib.obj.64
drwxrwsrwx 2 root root 4096 2010-08-21 00:57 include/
drwxrwsrwx 2 root root 4096 2010-08-21 18:46 linux/
-rwxr-xr-x 1 root root 10532 2010-08-21 18:41 Makefile*
-rw-r--r-- 1 root root 31 2010-08-21 18:46 modules.order
-rw-r--r-- 1 root root 0 2010-08-21 18:46 Module.symvers
-rwxr-xr-x 1 root root 57 2010-08-21 01:32 products.h*
-rwxr-xr-x 1 root root 2475 2010-08-21 01:32 README*
-rw-r--r-- 1 root root 402156 2010-08-21 18:46 t8sas.ko
-rw-r--r-- 1 root root 209 2010-08-21 18:46 .t8sas.ko.cmd
-rw-r--r-- 1 root root 3388 2010-08-21 18:46 t8sas.mod.c
-rw-r--r-- 1 root root 7248 2010-08-21 18:46 t8sas.mod.o
-rw-r--r-- 1 root root 22291 2010-08-21 18:46 .t8sas.mod.o.cmd
-rw-r--r-- 1 root root 395808 2010-08-21 18:46 t8sas.o
-rw-r--r-- 1 root root 298 2010-08-21 18:46 .t8sas.o.cmd
drwxr-sr-x 2 root root 4096 2010-08-21 18:46 .tmp_versions/

There is now the t8sas.ko file, which is according to my understanding the driver's kernel module. We can have a closer look at it with modinfo:

joerg@marvin:/usr/src/t8sas$ modinfo t8sas.ko
filename: t8sas.ko
license: GPL
version: 1.2.0.58
description: Promise Technology FastTrak T8 Series Controllers
author: Promise Technology, inc. LTD
alias: pci:v0000105Ad00003781sv0000105Asd00003783bc*sc*i*
alias: pci:v0000105Ad00003781sv0000105Asd00003782bc*sc*i*
alias: pci:v0000105Ad00003781sv0000105Asd00003781bc*sc*i*
alias: pci:v0000105Ad00003781sv0000105Asd00003780bc*sc*i*
depends:
vermagic: 2.6.32.15+drm33.5 SMP mod_unload modversions

The README instruction said the driver can be loaded with

# modprobe -a sd_mod
# insmod t8.ko

I guess they meant t8sas.ko instad of t8.ko

so I did the following to try out the driver

# sudo modprobe -a sd_mod
# sudo insmod t8sas.ko

This made the system crash and freeze.

Please help!

I first of all wonder if the driver was correctly built, because there were warnings.

Then I would highly appreciate your opinion what I might try and where to look for a solution. I guess there are also no maintained driver packages for this hardware in some repositories. At least google does not find anything when searching for the product name or when searching for "t8sas"

Your help is very much appreciated. Sorry for the long post, I hope these details help more than they confuse. I thought Promise is not too exotic hardware and they claim to have Linux support, so I was a little surprised to have these problems to get it working and to find any info.

best regards, Joerg







...

[braky]

Hi there,
I am in the process of installing this card in Ubuntu 10.04 lts.

Can you send me the sourcecode, I need it also.
I'll try and build it.
Mail it to braky @ dommel.be

I think the problem is that you did a sudo make clean which erased the .cmd file that is needed for the compilation.

Regards,
Braky

[joerg_ac]

Hi Braky,

I did sent an email to you with what I got from promise.

I guess there are some errors in the README. For example t8sas.ko is generated rather than t8.ko and the module name should be t8sas rather than t3sas.

There is also no .cmd file in the provided archive. All cmd file were generated when doing the make.
I did also also edit my makefile (you have the original) in order to set VERBOSE to 1. it provides more detailed output.

I am also not sure if the ftlib.obj was really linked into the final driver. That may be the problem.

I am looking forward to read about your experience. It is good to have somebody to exchange ideas with when facing problems like this.

best regards and good luck, Joerg

[braky]

Compiling went ok, just the same warning as you had.

I can do an insmod of t8sas ONLY if there are no disks connected to the controller.
If I try to do it when there is a disk connected, I hear the disk doing something and the system just freezes and caps and scroll-lock start to blink.

Did the same on fedora 2.6.33.6-147.2.4.fc13.x86_64 : same effect.

Help

[joerg_ac]

Hi,

So it is at least not me being too stupid or something like that.

I have also no idea what I can try now. I think I will give the promise support another call tomorrow.

However, after I bought that card a colleague did already warn me that Promise products might be a problem together with Linux because they support it badly. I thought I would be safe, because they have some Linux distributions in their driver and compatibility list for this card.

If nobody else has an idea I might go for a different hostadapter card and call this a lesson learned.

Please, if somebody has an idea, please help.

best regards, Joerg

[braky]

If you do contact support, first do the following:

1. remove all disks from the controller
2. do insmod of t8sas
3. poweroff
4. attach the disks
5. boot
6. after the grub screen press ESC so that the graphical bootscreen is gone
7. wait for the kernel panic
8. take a photo of your screen (or write it down)
9. send this info to support, and maybe they'll have enough info to help

If they ask for setting acpi=off or acpi=3Doff ... it doesn't help.

To get your system back to normal, boot without disks attached to the controller, and remove the t8sas module.

Cheers

[joerg_ac]

hi,

thanks for the hint.

I tried, but after reboot the module seems to be gone. Does insmod only load it temporarily? I did

sudo insmod t8sas.ko

from /lib/src/ ... /driver/scsi where I have copied the file

also lsmod shows t8sas. after a reboot with disks attached the module is missing again.

What do I need to do in order to make it permanent or surviving a reboot?
I also have to refer to the actual file. I cannot just do "sudo insmod t8sas"

Did I miss a step?

best regards, Joerg

[braky]

Some progress made....

If you put the controller in SAS HBA MODE then it WORKS! (alt-F on boot)
Well it doesn't do a kernel panic and I can see the drive connected to the controller.
I'm doing some stability tests now to be sure it doesn't fail on me when there is heavy load on the disks.

Cheers

[joerg_ac]

That seems to be the trick, I can confirm that I see the disks now. (sudo fdisk -l)

to make it more clear for everybody, press ctrl-F at initialization of the adapter a t system startup before OS booting. This enters the card's BIOS configuration. Change the HBA mode to "SAS HBA". This disables the build-in RAID functions and presents the disks as JBOD. That is fully ok with me, because I anyway wanted to use software raid.

What tools do you use for verification of reliability and performance?

I still have problems to make the driver module load automatically. How will I do that? I guess I have to update some configuration file, or is there a tool/command for that?

Has anybody experience with module-assistant? That tool seems to be quite useful to automatically re-compile and install the driver in case there is a new kernel. Or can a .deb package be made from it? Or is there another tool for making the compilation of the driver automatic?

best regards, Joerg

[braky]

No specials tools for testing:
I'm first going to do some basics like formatting the drives, copying some 2TB of data around.
Some mdadm tests to see how it reacts with the controller on drive failures and such.
My current homeserver Fedora/Ubuntu has 6 satadisks connected on the mainbord (1bootdisk 160GB and 5 x 1TB in raid5 mdadm) + 1-port Sil3531 (esata) + 8-port TX8660 and 5 x 2TB WD 20EARS disks for testing...

Concerning the module, (in Fedora)

I copied the t8sas.ko to /lib/modules/$(uname -r)/kernel/drivers/scsi/
I copied the t8sas.ko to /lib/modules/$(uname -r)/updates/
#execute depmod
depmod $(uname -r) -v
#make a new initramfs that has the new module
mkinitrd /boot/initrd-$(uname -r).new.img $(uname -r)
#!!edit /boot/grub/grub.conf so that the new generated .img is used

I think in Ubuntu there is an easier way to do this, but it should work also ... (i hope)

Cheers,
braky