Results 1 to 5 of 5

Thread: 10.10 Server hung_task_timeout_secs?

  1. #1
    Join Date
    Jun 2006
    Location
    New York City
    Beans
    46
    Distro
    Ubuntu 10.10 Maverick Meerkat

    Unhappy 10.10 Server hung_task_timeout_secs?

    I'm testing a brand new installation of Ubuntu 10.10 Server. It works fine for a few hours, but then seems to get hung up. I can ping the machine, and port scans show that it is alive on ports 22 and 80, but attempts to connect via ssh or the web-front end just die.

    When I go to the physical machine, I see this:

    https://picasaweb.google.com/lh/photo/pZ3D38lMwUxl5XwR1THvquIwBdU7JO6QLuP4bOALNRE?feat=d irectlink


    [A bunch of kernel messages with tasks blocked for more than 120 seconds.]

    I am able to get a command prompt and enter my password, but then the machine just hangs; I never actually get into a shell.

    I found two related posts, but neither had a resolution:
    http://ubuntuforums.org/showthread.php?t=1306807
    https://lists.ubuntu.com/archives/ub...ly/001148.html

    A hard restart solves the problem temporarily, but then it comes back. Here is the most recent syslog:

    Code:
    Apr 14 19:38:48 METSBBDB01 kernel: imklog 4.2.0, log source = /proc/kmsg started.
    Apr 14 19:38:48 METSBBDB01 rsyslogd: [origin software="rsyslogd" swVersion="4.2.0" x-pid="554" x-info="http://www.rsyslog.com"] (re)start
    Apr 14 19:38:48 METSBBDB01 rsyslogd: rsyslogd's groupid changed to 103
    Apr 14 19:38:48 METSBBDB01 rsyslogd: rsyslogd's userid changed to 101
    Apr 14 19:38:48 METSBBDB01 rsyslogd-2039: Could no open output file '/dev/xconsole' [try http://www.rsyslog.com/e/2039 ]
    Apr 14 19:38:48 METSBBDB01 statd-pre-start: local-filesystems started
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Initializing cgroup subsys cpuset
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Initializing cgroup subsys cpu
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Linux version 2.6.35-28-server (buildd@allspice) (gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5) ) #49-Ubuntu SMP Tue Mar 1 14:55:37 UTC 2011 (Ubuntu 2.6.35-28.49-server 2.6.35.11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-2.6.35-28-server root=UUID=2a190a16-00bd-44b9-ac66-6c45480e4b04 ro quiet
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] BIOS-provided physical RAM map:
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 0000000000000000 - 000000000009f400 (usable)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 000000000009f400 - 00000000000a0000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 0000000000100000 - 00000000dfff3000 (usable)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 00000000dfff3000 - 00000000dfffb000 (ACPI data)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 00000000dfffb000 - 00000000e0000000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 00000000fec00000 - 00000000fed00000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 00000000ffc00000 - 0000000100000000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  BIOS-e820: 0000000100000000 - 000000021bfff000 (usable)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Notice: NX (Execute Disable) protection missing in CPU or disabled in BIOS!
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] DMI 2.3 present.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] No AGP bridge found
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] last_pfn = 0x21bfff max_arch_pfn = 0x400000000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] MTRR default type: write-back
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] MTRR fixed ranges enabled:
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   00000-9FFFF write-back
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   A0000-BFFFF uncachable
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   C0000-C7FFF write-protect
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   C8000-E7FFF uncachable
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   E8000-FFFFF write-protect
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] MTRR variable ranges enabled:
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   0 base 0E0000000 mask FE0000000 uncachable
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   1 disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   2 disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   3 disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   4 disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   5 disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   6 disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   7 disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] last_pfn = 0xdfff3 max_arch_pfn = 0x400000000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] e820 update range: 0000000000001000 - 0000000000010000 (usable) ==> (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Scanning 1 areas for low memory corruption
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] modified physical RAM map:
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 0000000000000000 - 0000000000010000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 0000000000010000 - 000000000009f400 (usable)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 000000000009f400 - 00000000000a0000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 00000000000f0000 - 0000000000100000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 0000000000100000 - 00000000dfff3000 (usable)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 00000000dfff3000 - 00000000dfffb000 (ACPI data)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 00000000dfffb000 - 00000000e0000000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 00000000fec00000 - 00000000fed00000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 00000000fee00000 - 00000000fee10000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 00000000ffc00000 - 0000000100000000 (reserved)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  modified: 0000000100000000 - 000000021bfff000 (usable)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] initial memory mapped : 0 - 20000000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] found SMP MP-table at [ffff8800000f4f80] f4f80
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] init_memory_mapping: 0000000000000000-00000000dfff3000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  0000000000 - 00dfe00000 page 2M
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  00dfe00000 - 00dfff3000 page 4k
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] kernel direct mapping tables up to dfff3000 @ 16000-1c000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] init_memory_mapping: 0000000100000000-000000021bfff000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  0100000000 - 021be00000 page 2M
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  021be00000 - 021bfff000 page 4k
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] kernel direct mapping tables up to 21bfff000 @ 1a000-25000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] RAMDISK: 37588000 - 37ff0000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: RSDP 00000000000f4f00 00024 (v02 HP    )
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: XSDT 00000000dfff3300 00044 (v01 HP     P51      00000002   Ò? 0000162E)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: FACP 00000000dfff3380 000F4 (v03 HP     P51      00000002   Ò? 0000162E)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI Warning: Invalid length for Pm1aControlBlock: 32, using default 16 (20100428/tbfadt-607)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: DSDT 00000000dfff3480 02DAB (v01 HP         DSDT 00000001 INTL 20030228)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: FACS 00000000dfff3100 00040
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: SPCR 00000000dfff3140 00050 (v01 HP     SPCRRBSU 00000001   Ò? 0000162E)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: MCFG 00000000dfff31c0 0003C (v01 HP     ProLiant 00000001      00000000)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: APIC 00000000dfff3200 000C2 (v01 HP     00000083 00000002      00000000)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] No NUMA configuration found
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Faking a node at 0000000000000000-000000021bfff000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Initmem setup node 0 0000000000000000-000000021bfff000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   NODE_DATA [0000000100000000 - 0000000100004fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]  [ffffea0000000000-ffffea00077fffff] PMD -> [ffff880100200000-ffff8801073fffff] on node 0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Zone PFN ranges:
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   DMA      0x00000010 -> 0x00001000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   DMA32    0x00001000 -> 0x00100000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   Normal   0x00100000 -> 0x0021bfff
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Movable zone start PFN for each node
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] early_node_map[3] active PFN ranges
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]     0: 0x00000010 -> 0x0000009f
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]     0: 0x00000100 -> 0x000dfff3
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]     0: 0x00100000 -> 0x0021bfff
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] On node 0 totalpages: 2080641
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   DMA zone: 56 pages used for memmap
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   DMA zone: 0 pages reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   DMA zone: 3927 pages, LIFO batch:0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   DMA32 zone: 14280 pages used for memmap
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   DMA32 zone: 899115 pages, LIFO batch:31
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   Normal zone: 15904 pages used for memmap
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   Normal zone: 1147359 pages, LIFO batch:31
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: PM-Timer IO Port: 0x908
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: Local APIC address 0xfee00000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IOAPIC (id[0x09] address[0xfec80000] gsi_base[24])
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] IOAPIC[1]: apic_id 9, version 32, address 0xfec80000, GSI 24-47
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IOAPIC (id[0x0a] address[0xfec80400] gsi_base[48])
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] IOAPIC[2]: apic_id 10, version 32, address 0xfec80400, GSI 48-71
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IOAPIC (id[0x0b] address[0xfec84000] gsi_base[72])
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] IOAPIC[3]: apic_id 11, version 32, address 0xfec84000, GSI 72-95
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IOAPIC (id[0x0c] address[0xfec84400] gsi_base[96])
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] IOAPIC[4]: apic_id 12, version 32, address 0xfec84400, GSI 96-119
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IRQ0 used by override.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IRQ2 used by override.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] ACPI: IRQ9 used by override.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Using ACPI (MADT) for SMP configuration information
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] SMP: Allowing 8 CPUs, 4 hotplug CPUs
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] nr_irqs_gsi: 136
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] early_res array is doubled to 64 at [20000 - 207ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000dfff3000 - 00000000dfffb000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000dfffb000 - 00000000e0000000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000e0000000 - 00000000fec00000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000fec00000 - 00000000fed00000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000fed00000 - 00000000fee00000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee10000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000fee10000 - 00000000ffc00000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PM: Registered nosave memory: 00000000ffc00000 - 0000000100000000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Allocating PCI resources starting at e0000000 (gap: e0000000:1ec00000)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Booting paravirtualized kernel on bare hardware
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] setup_percpu: NR_CPUS:256 nr_cpumask_bits:256 nr_cpu_ids:8 nr_node_ids:1
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PERCPU: Embedded 30 pages/cpu @ffff880001e00000 s92032 r8192 d22656 u262144
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] pcpu-alloc: s92032 r8192 d22656 u262144 alloc=1*2097152
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] pcpu-alloc: [0] 0 1 2 3 4 5 6 7 
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2050401
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Policy zone: Normal
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-2.6.35-28-server root=UUID=2a190a16-00bd-44b9-ac66-6c45480e4b04 ro quiet
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Checking aperture...
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] No AGP bridge found
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Calgary: detecting Calgary via BIOS EBDA area
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] early_res array is doubled to 128 at [20800 - 217ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Subtract (59 early reservations)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #1 [0001000000 - 0001d68254]   TEXT DATA BSS
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #2 [0037588000 - 0037ff0000]         RAMDISK
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #3 [0001d69000 - 0001d690d4]             BRK
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #4 [00000f4f90 - 0000100000]   BIOS reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #5 [00000f4f80 - 00000f4f90]    MP-table mpf
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #6 [000009f400 - 00000f280b]   BIOS reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #7 [00000f2a37 - 00000f4f80]   BIOS reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #8 [00000f280b - 00000f2a37]    MP-table mpc
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #9 [0000010000 - 0000012000]      TRAMPOLINE
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #10 [0000012000 - 0000016000]     ACPI WAKEUP
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #11 [0000016000 - 000001a000]         PGTABLE
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #12 [000001a000 - 0000020000]         PGTABLE
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #13 [0100000000 - 0100005000]       NODE_DATA
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #14 [0001d69100 - 0001d6a100]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #15 [0100005000 - 0100005600]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #16 [0100006000 - 0100007000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #17 [0100007000 - 0100008000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #18 [0100200000 - 0107400000]        MEMMAP 0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #19 [0001d6a100 - 0001d82100]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #20 [0001d82100 - 0001d9a100]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #21 [0001d9a100 - 0001db2100]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #22 [0001db3000 - 0001db4000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #23 [0001d68280 - 0001d683cf]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #24 [0001d68400 - 0001d68668]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #25 [0001d68680 - 0001d686e8]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #26 [0001d68700 - 0001d68768]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #27 [0001d68780 - 0001d687e8]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #28 [0001d68800 - 0001d68868]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #29 [0001d68880 - 0001d688e8]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #30 [0001d68900 - 0001d68968]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #31 [0001d68980 - 0001d689e8]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #32 [0001d68a00 - 0001d68a68]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #33 [0001d68a80 - 0001d68ae8]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #34 [0001d68b00 - 0001d68b68]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #35 [0001d68b80 - 0001d68ba0]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #36 [0001d68bc0 - 0001d68be0]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #37 [0001d68c00 - 0001d68c62]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #38 [0001d68c80 - 0001d68ce2]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #39 [0001e00000 - 0001e1e000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #40 [0001e40000 - 0001e5e000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #41 [0001e80000 - 0001e9e000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #42 [0001ec0000 - 0001ede000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #43 [0001f00000 - 0001f1e000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #44 [0001f40000 - 0001f5e000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #45 [0001f80000 - 0001f9e000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #46 [0001fc0000 - 0001fde000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #47 [0001d68d00 - 0001d68d08]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #48 [0001d68d40 - 0001d68d48]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #49 [0001d68d80 - 0001d68da0]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #50 [0001d68dc0 - 0001d68e00]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #51 [0001d68e00 - 0001d68f20]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #52 [0001d68f40 - 0001d68f88]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #53 [0001db2100 - 0001db2148]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #54 [0001db4000 - 0001dbc000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #55 [0001fde000 - 0005fde000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #56 [0001dbc000 - 0001ddc000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #57 [0005fde000 - 000601e000]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000]   #58 [0000021800 - 0000029800]         BOOTMEM
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Memory: 8114096k/8847356k available (5806k kernel code, 524792k absent, 208468k reserved, 5383k data, 836k init)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Hierarchical RCU implementation.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] 	RCU dyntick-idle grace-period acceleration is enabled.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] 	RCU-based detection of stalled CPUs is disabled.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] 	Verbose stalled-CPUs detection is disabled.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] NR_IRQS:16640 nr_irqs:2048
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Extended CMOS year: 2000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Console: colour VGA+ 80x25
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] console [tty0] enabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] allocated 83886080 bytes of page_cgroup
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Fast TSC calibration using PIT
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.000000] Detected 3200.264 MHz processor.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.020009] Calibrating delay loop (skipped), value calculated using timer frequency.. 6400.52 BogoMIPS (lpj=32002640)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.020015] pid_max: default: 32768 minimum: 301
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.020055] Security Framework initialized
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.020087] AppArmor: AppArmor initialized
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.020090] Yama: becoming mindful.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.022038] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.033795] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.036865] Mount-cache hash table entries: 256
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037114] Initializing cgroup subsys ns
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037121] Initializing cgroup subsys cpuacct
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037127] Initializing cgroup subsys memory
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037140] Initializing cgroup subsys devices
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037144] Initializing cgroup subsys freezer
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037148] Initializing cgroup subsys net_cls
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037199] CPU: Physical Processor ID: 0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037202] CPU: Processor Core ID: 0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037207] mce: CPU supports 4 MCE banks
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037222] CPU0: Thermal monitoring enabled (TM1)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037229] using mwait in idle threads.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037232] Performance Events: Netburst events, Netburst P4/Xeon PMU driver.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037241] ... version:                0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037244] ... bit width:              40
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037246] ... generic registers:      18
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037248] ... value mask:             000000ffffffffff
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037251] ... max period:             0000007fffffffff
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037253] ... fixed-purpose events:   0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.037255] ... event mask:             000000000003ffff
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.040403] ACPI: Core revision 20100428
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.050014] ftrace: converting mcount calls to 0f 1f 44 00 00
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.050023] ftrace: allocating 22987 entries in 91 pages
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.060094] Setting APIC routing to flat
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.060630] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.162412] CPU0: Intel(R) Xeon(TM) CPU 3.20GHz stepping 03
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.170000] Booting Node   0, Processors  #1 #2 #3
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.670012] Brought up 4 CPUs
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.670018] Total of 4 processors activated (25601.70 BogoMIPS).
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.671508] devtmpfs: initialized
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] regulator: core version 0.5
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] Time: 23:38:43  Date: 04/14/11
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] NET: Registered protocol family 16
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] ACPI: bus type pci registered
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] Trying to unpack rootfs image as initramfs...
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] PCI: Intel Corporation E7520 Memory Controller Hub with MMCONFIG support
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] PCI: not using MMCONFIG
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] PCI: Using configuration type 1 for base access
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] PCI: HP ProLiant DL380 detected, enabling pci=bfsort.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] mtrr: your CPUs had inconsistent fixed MTRR settings
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] mtrr: probably your BIOS does not setup all CPUs.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] mtrr: corrected configuration.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.673923] bio: create slab <bio-0> at 0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.680114] ACPI: EC: Look up EC in DSDT
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.682538] ACPI: Interpreter enabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.682542] ACPI: (supports S0 S4 S5)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.682567] ACPI: Using IOAPIC for interrupt routing
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.691651] ACPI: No dock devices found.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.691658] PCI: Ignoring host bridge windows from ACPI; if necessary, use "pci=use_crs" and report a bug
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.691790] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-7f])
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692010] pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7] (ignored)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692014] pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff] (ignored)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692018] pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff] (ignored)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692021] pci_root PNP0A03:00: host bridge window [mem 0xe0000000-0xfebfffff] (ignored)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692153] pci 0000:00:02.0: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692159] pci 0000:00:02.0: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692228] pci 0000:00:06.0: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692232] pci 0000:00:06.0: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692314] pci 0000:00:1d.0: reg 20: [io  0x3000-0x301f]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692377] pci 0000:00:1d.1: reg 20: [io  0x3020-0x303f]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692439] pci 0000:00:1d.2: reg 20: [io  0x3040-0x305f]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692500] pci 0000:00:1d.3: reg 20: [io  0x3060-0x307f]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692560] pci 0000:00:1d.7: reg 10: [mem 0xfbef0000-0xfbef03ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692623] pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692628] pci 0000:00:1d.7: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692739] pci 0000:00:1f.0: Force enabled HPET at 0xfed00000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692747] pci 0000:00:1f.0: quirk: [io  0x0900-0x097f] claimed by ICH4 ACPI/GPIO/TCO
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692751] pci 0000:00:1f.0: quirk: [io  0x0800-0x083f] claimed by ICH4 GPIO
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692779] pci 0000:00:1f.1: reg 10: [io  0x01f0-0x01f7]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692787] pci 0000:00:1f.1: reg 14: [io  0x03f4-0x03f7]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692794] pci 0000:00:1f.1: reg 18: [io  0x0170-0x0177]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692801] pci 0000:00:1f.1: reg 1c: [io  0x0374-0x0377]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692809] pci 0000:00:1f.1: reg 20: [io  0x0500-0x050f]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692816] pci 0000:00:1f.1: reg 24: [mem 0x00000000-0x000003ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692905] pci 0000:02:00.0: PXH quirk detected; SHPC device MSI disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692950] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692954] pci 0000:02:00.0: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.692990] pci 0000:02:00.2: PXH quirk detected; SHPC device MSI disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693034] pci 0000:02:00.2: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693039] pci 0000:02:00.2: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693056] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693068] pci 0000:00:02.0: PCI bridge to [bus 02-04]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693073] pci 0000:00:02.0:   bridge window [io  0x4000-0x4fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693078] pci 0000:00:02.0:   bridge window [mem 0xfde00000-0xfdffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693085] pci 0000:00:02.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693157] pci 0000:03:01.0: reg 10: [mem 0xfdef0000-0xfdefffff 64bit]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693204] pci 0000:03:01.0: PME# supported from D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693208] pci 0000:03:01.0: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693251] pci 0000:03:01.1: reg 10: [mem 0xfdee0000-0xfdeeffff 64bit]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693297] pci 0000:03:01.1: PME# supported from D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693302] pci 0000:03:01.1: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693352] pci 0000:02:00.0: PCI bridge to [bus 03-03]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693357] pci 0000:02:00.0:   bridge window [io  0xf000-0x0000] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693362] pci 0000:02:00.0:   bridge window [mem 0xfde00000-0xfdefffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693368] pci 0000:02:00.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693439] pci 0000:04:03.0: reg 10: [mem 0xfdff0000-0xfdff1fff 64bit]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693446] pci 0000:04:03.0: reg 18: [io  0x4000-0x40ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693457] pci 0000:04:03.0: reg 1c: [mem 0xfdf80000-0xfdfbffff 64bit]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693468] pci 0000:04:03.0: reg 30: [mem 0x00000000-0x0003ffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693496] pci 0000:04:03.0: supports D1
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693539] pci 0000:02:00.2: PCI bridge to [bus 04-04]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693543] pci 0000:02:00.2:   bridge window [io  0x4000-0x4fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693548] pci 0000:02:00.2:   bridge window [mem 0xfdf00000-0xfdffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693554] pci 0000:02:00.2:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693629] pci 0000:05:00.0: PXH quirk detected; SHPC device MSI disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693673] pci 0000:05:00.0: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693678] pci 0000:05:00.0: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693714] pci 0000:05:00.2: PXH quirk detected; SHPC device MSI disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693758] pci 0000:05:00.2: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693763] pci 0000:05:00.2: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693782] pci 0000:05:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693795] pci 0000:00:06.0: PCI bridge to [bus 05-0c]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693800] pci 0000:00:06.0:   bridge window [io  0xf000-0x0000] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693806] pci 0000:00:06.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693813] pci 0000:00:06.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693890] pci 0000:05:00.0: PCI bridge to [bus 06-09]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693895] pci 0000:05:00.0:   bridge window [io  0xf000-0x0000] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693899] pci 0000:05:00.0:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693906] pci 0000:05:00.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693976] pci 0000:05:00.2: PCI bridge to [bus 0a-0c]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693980] pci 0000:05:00.2:   bridge window [io  0xf000-0x0000] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693985] pci 0000:05:00.2:   bridge window [mem 0xfff00000-0x000fffff] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.693991] pci 0000:05:00.2:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694044] pci 0000:01:03.0: reg 10: [mem 0xfc000000-0xfcffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694052] pci 0000:01:03.0: reg 14: [io  0x2000-0x20ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694060] pci 0000:01:03.0: reg 18: [mem 0xfbff0000-0xfbff0fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694084] pci 0000:01:03.0: reg 30: [mem 0x00000000-0x0001ffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694107] pci 0000:01:03.0: supports D1 D2
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694140] pci 0000:01:04.0: reg 10: [io  0x1800-0x18ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694148] pci 0000:01:04.0: reg 14: [mem 0xfbfe0000-0xfbfe01ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694231] pci 0000:01:04.2: reg 10: [io  0x2400-0x24ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694240] pci 0000:01:04.2: reg 14: [mem 0xfbfd0000-0xfbfd07ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694247] pci 0000:01:04.2: reg 18: [mem 0xfbfc0000-0xfbfc1fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694255] pci 0000:01:04.2: reg 1c: [mem 0xfbf00000-0xfbf7ffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694275] pci 0000:01:04.2: reg 30: [mem 0x00000000-0x0000ffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694298] pci 0000:01:04.2: PME# supported from D0 D3hot D3cold
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694303] pci 0000:01:04.2: PME# disabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694355] pci 0000:00:1e.0: PCI bridge to [bus 01-01] (subtractive decode)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694360] pci 0000:00:1e.0:   bridge window [io  0x1000-0x2fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694366] pci 0000:00:1e.0:   bridge window [mem 0xfbf00000-0xfcffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694371] pci 0000:00:1e.0:   bridge window [mem 0xfff00000-0x000fffff pref] (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694375] pci 0000:00:1e.0:   bridge window [io  0x0000-0xffff] (subtractive decode)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694378] pci 0000:00:1e.0:   bridge window [mem 0x00000000-0xffffffffffffffff] (subtractive decode)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694400] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694525] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.IP2P._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694603] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PTA0._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694674] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PTA0.PCXA._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694744] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PTA0.PCXB._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694803] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PTC0._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.694875] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PTC0.PCXC._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.695034] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PTC0.PCXD._PRT]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.696962] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 7 10 11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697088] ACPI: PCI Interrupt Link [LNKB] (IRQs *5 7 10 11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697207] ACPI: PCI Interrupt Link [LNKC] (IRQs *5 7 10 11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697331] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 7 10 11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697450] ACPI: PCI Interrupt Link [LNKE] (IRQs 5 7 10 11) *0, disabled.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697576] ACPI: PCI Interrupt Link [LNKF] (IRQs *5 7 10 11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697698] ACPI: PCI Interrupt Link [LNKG] (IRQs *5 7 10 11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697816] ACPI: PCI Interrupt Link [LNKH] (IRQs *5 7 10 11)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.697881] HEST: Table is not found!
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698006] vgaarb: device added: PCI:0000:01:03.0,decodes=io+mem,owns=none,locks=none
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698011] vgaarb: loaded
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698215] SCSI subsystem initialized
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] libata version 3.00 loaded.
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] usbcore: registered new interface driver usbfs
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] usbcore: registered new interface driver hub
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] usbcore: registered new device driver usb
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] ACPI: WMI: Mapper loaded
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] PCI: Using ACPI for IRQ routing
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] PCI: pci_cache_line_size set to 64 bytes
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] reserve RAM buffer: 000000000009f400 - 000000000009ffff 
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] reserve RAM buffer: 00000000dfff3000 - 00000000dfffffff 
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] reserve RAM buffer: 000000021bfff000 - 000000021bffffff 
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] NetLabel: Initializing
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] NetLabel:  domain hash size = 128
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] NetLabel:  protocols = UNLABELED CIPSOv4
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] NetLabel:  unlabeled traffic allowed by default
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] hpet clockevent registered
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.698232] hpet0: 3 comparators, 64-bit 14.318180 MHz counter
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.720027] Switching to clocksource tsc
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.737702] AppArmor: AppArmor Filesystem Enabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.737735] pnp: PnP ACPI init
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.737793] ACPI: bus type pnp registered
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743780] pnp: PnP ACPI: found 9 devices
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743785] ACPI: ACPI bus type pnp unregistered
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743806] system 00:01: [io  0x0408-0x040f] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743811] system 00:01: [io  0x04d0-0x04d1] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743816] system 00:01: [io  0x0700-0x071f] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743821] system 00:01: [io  0x0800-0x083f] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743826] system 00:01: [io  0x0900-0x097f] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743831] system 00:01: [io  0x0c80-0x0c83] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743836] system 00:01: [io  0x0cd4-0x0cd7] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743841] system 00:01: [io  0x0f50-0x0f58] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743846] system 00:01: [io  0x02f8-0x02ff] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.743854] system 00:01: [mem 0xe0000000-0xefffffff] has been reserved
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751419] pci 0000:00:02.0: BAR 15: assigned [mem 0xf0000000-0xf00fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751427] pci 0000:00:1e.0: BAR 15: assigned [mem 0xf0100000-0xf01fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751433] pci 0000:00:1f.1: BAR 5: assigned [mem 0xf0200000-0xf02003ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751443] pci 0000:00:1f.1: BAR 5: set to [mem 0xf0200000-0xf02003ff] (PCI address [0xf0200000-0xf02003ff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751451] pci 0000:02:00.2: BAR 15: assigned [mem 0xf0000000-0xf00fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751457] pci 0000:02:00.0: PCI bridge to [bus 03-03]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751461] pci 0000:02:00.0:   bridge window [io  disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751468] pci 0000:02:00.0:   bridge window [mem 0xfde00000-0xfdefffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751473] pci 0000:02:00.0:   bridge window [mem pref disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751483] pci 0000:04:03.0: BAR 6: assigned [mem 0xf0000000-0xf003ffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751487] pci 0000:02:00.2: PCI bridge to [bus 04-04]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751492] pci 0000:02:00.2:   bridge window [io  0x4000-0x4fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751499] pci 0000:02:00.2:   bridge window [mem 0xfdf00000-0xfdffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751505] pci 0000:02:00.2:   bridge window [mem 0xf0000000-0xf00fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751513] pci 0000:00:02.0: PCI bridge to [bus 02-04]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751518] pci 0000:00:02.0:   bridge window [io  0x4000-0x4fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751525] pci 0000:00:02.0:   bridge window [mem 0xfde00000-0xfdffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751531] pci 0000:00:02.0:   bridge window [mem 0xf0000000-0xf00fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751539] pci 0000:05:00.0: PCI bridge to [bus 06-09]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751543] pci 0000:05:00.0:   bridge window [io  disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751549] pci 0000:05:00.0:   bridge window [mem disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751554] pci 0000:05:00.0:   bridge window [mem pref disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751563] pci 0000:05:00.2: PCI bridge to [bus 0a-0c]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751567] pci 0000:05:00.2:   bridge window [io  disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751573] pci 0000:05:00.2:   bridge window [mem disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751578] pci 0000:05:00.2:   bridge window [mem pref disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751586] pci 0000:00:06.0: PCI bridge to [bus 05-0c]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751589] pci 0000:00:06.0:   bridge window [io  disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751596] pci 0000:00:06.0:   bridge window [mem disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751601] pci 0000:00:06.0:   bridge window [mem pref disabled]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751611] pci 0000:01:03.0: BAR 6: assigned [mem 0xf0100000-0xf011ffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751618] pci 0000:01:04.2: BAR 6: assigned [mem 0xf0120000-0xf012ffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751623] pci 0000:00:1e.0: PCI bridge to [bus 01-01]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751629] pci 0000:00:1e.0:   bridge window [io  0x1000-0x2fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751637] pci 0000:00:1e.0:   bridge window [mem 0xfbf00000-0xfcffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751644] pci 0000:00:1e.0:   bridge window [mem 0xf0100000-0xf01fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751671]   alloc irq_desc for 16 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751676]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751694] pci 0000:00:02.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751707] pci 0000:00:02.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751724] pci 0000:02:00.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751741] pci 0000:02:00.2: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751755] pci 0000:00:06.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751761] pci 0000:00:06.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751775] pci 0000:05:00.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751788] pci 0000:05:00.2: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751798] pci 0000:00:1e.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751806] pci_bus 0000:00: resource 0 [io  0x0000-0xffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751810] pci_bus 0000:00: resource 1 [mem 0x00000000-0xffffffffffffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751815] pci_bus 0000:02: resource 0 [io  0x4000-0x4fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751819] pci_bus 0000:02: resource 1 [mem 0xfde00000-0xfdffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751823] pci_bus 0000:02: resource 2 [mem 0xf0000000-0xf00fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751828] pci_bus 0000:03: resource 1 [mem 0xfde00000-0xfdefffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751832] pci_bus 0000:04: resource 0 [io  0x4000-0x4fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751836] pci_bus 0000:04: resource 1 [mem 0xfdf00000-0xfdffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751841] pci_bus 0000:04: resource 2 [mem 0xf0000000-0xf00fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751845] pci_bus 0000:01: resource 0 [io  0x1000-0x2fff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751849] pci_bus 0000:01: resource 1 [mem 0xfbf00000-0xfcffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751854] pci_bus 0000:01: resource 2 [mem 0xf0100000-0xf01fffff pref]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751858] pci_bus 0000:01: resource 4 [io  0x0000-0xffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751862] pci_bus 0000:01: resource 5 [mem 0x00000000-0xffffffffffffffff]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.751932] NET: Registered protocol family 2
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.752642] IP route cache hash table entries: 262144 (order: 9, 2097152 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.756819] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.763729] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.764549] TCP: Hash tables configured (established 524288 bind 65536)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.764555] TCP reno registered
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.764600] UDP hash table entries: 4096 (order: 5, 131072 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.764767] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765117] NET: Registered protocol family 1
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765328] pci 0000:02:00.0: rerouting interrupts for [8086:0329]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765337] pci 0000:02:00.2: rerouting interrupts for [8086:032a]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765355] pci 0000:05:00.0: rerouting interrupts for [8086:0329]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765363] pci 0000:05:00.2: rerouting interrupts for [8086:032a]
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765370] PCI: CLS 64 bytes, default 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765374] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765380] Placing 64MB software IO TLB between ffff880001fde000 - ffff880005fde000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765384] software IO TLB at phys 0x1fde000 - 0x5fde000
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765518] platform rtc_cmos: registered platform RTC device (no PNP device found)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.765990] Scanning for low memory corruption every 60 seconds
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.766210] audit: initializing netlink socket (disabled)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.766228] type=2000 audit(1302824322.750:1): initialized
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.784333] HugeTLB registered 2 MB page size, pre-allocated 0 pages
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.786180] VFS: Disk quotas dquot_6.5.2
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.786255] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.787089] fuse init (API version 7.14)
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.787203] msgmni has been set to 15847
    Apr 14 19:38:48 METSBBDB01 kernel: [    0.998118] Freeing initrd memory: 10656k freed
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008088] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008094] io scheduler noop registered
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008098] io scheduler deadline registered (default)
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008152] io scheduler cfq registered
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008302] pcieport 0000:00:02.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008336]   alloc irq_desc for 136 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008339]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008369] pcieport 0000:00:02.0: irq 136 for MSI/MSI-X
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008435] pcieport 0000:00:06.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008464]   alloc irq_desc for 137 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008466]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008475] pcieport 0000:00:06.0: irq 137 for MSI/MSI-X
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008592] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008619] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008850] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.008859] ACPI: Power Button [PWRF]
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.011531] ACPI: acpi_idle registered with cpuidle
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.015748] thermal LNXTHERM:01: registered as thermal_zone0
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.015758] ACPI: Thermal Zone [THM0] (8 C)
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.015817] ERST: Table is not found!
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.016031] Linux agpgart interface v0.103
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.016058] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.016170] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.016242] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.016590] 00:07: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.018048] brd: module loaded
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.018702] loop: module loaded
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.018984] ata_piix 0000:00:1f.1: version 2.13
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.018994] ata_piix 0000:00:1f.1: enabling device (0005 -> 0007)
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019002] ata_piix 0000:00:1f.1: can't derive routing for PCI INT A
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019052] ata_piix 0000:00:1f.1: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019214] scsi0 : ata_piix
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019324] scsi1 : ata_piix
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019378] ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0x500 irq 14
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019382] ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0x508 irq 15
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019840] Fixed MDIO Bus: probed
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019882] PPP generic driver version 2.4.2
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019928] tun: Universal TUN/TAP device driver, 1.6
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.019930] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020050] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020075]   alloc irq_desc for 23 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020078]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020088] ehci_hcd 0000:00:1d.7: PCI INT D -> GSI 23 (level, low) -> IRQ 23
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020119] ehci_hcd 0000:00:1d.7: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020124] ehci_hcd 0000:00:1d.7: EHCI Host Controller
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020166] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.020205] ehci_hcd 0000:00:1d.7: debug port 1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.024094] ehci_hcd 0000:00:1d.7: cache line size of 64 is not supported
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.024117] ehci_hcd 0000:00:1d.7: irq 23, io mem 0xfbef0000
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.039818] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.039986] hub 1-0:1.0: USB hub found
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.039993] hub 1-0:1.0: 8 ports detected
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040102] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040120] uhci_hcd: USB Universal Host Controller Interface driver
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040186] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040194] uhci_hcd 0000:00:1d.0: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040198] uhci_hcd 0000:00:1d.0: UHCI Host Controller
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040240] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040273] uhci_hcd 0000:00:1d.0: irq 16, io base 0x00003000
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040433] hub 2-0:1.0: USB hub found
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040440] hub 2-0:1.0: 2 ports detected
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040514]   alloc irq_desc for 19 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040517]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040524] uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 19 (level, low) -> IRQ 19
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040531] uhci_hcd 0000:00:1d.1: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040534] uhci_hcd 0000:00:1d.1: UHCI Host Controller
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040579] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040615] uhci_hcd 0000:00:1d.1: irq 19, io base 0x00003020
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040756] hub 3-0:1.0: USB hub found
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040763] hub 3-0:1.0: 2 ports detected
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040834]   alloc irq_desc for 18 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040838]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040844] uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 18 (level, low) -> IRQ 18
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040851] uhci_hcd 0000:00:1d.2: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040854] uhci_hcd 0000:00:1d.2: UHCI Host Controller
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040894] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.040926] uhci_hcd 0000:00:1d.2: irq 18, io base 0x00003040
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041064] hub 4-0:1.0: USB hub found
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041070] hub 4-0:1.0: 2 ports detected
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041141] uhci_hcd 0000:00:1d.3: PCI INT A -> GSI 16 (level, low) -> IRQ 16
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041149] uhci_hcd 0000:00:1d.3: setting latency timer to 64
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041152] uhci_hcd 0000:00:1d.3: UHCI Host Controller
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041193] uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 5
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041216] uhci_hcd 0000:00:1d.3: irq 16, io base 0x00003060
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041351] hub 5-0:1.0: USB hub found
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041357] hub 5-0:1.0: 2 ports detected
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.041494] PNP: PS/2 Controller [PNP0303:KBD,PNP0f0e:PS2M] at 0x60,0x64 irq 1,12
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043130] serio: i8042 KBD port at 0x60,0x64 irq 1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043143] serio: i8042 AUX port at 0x60,0x64 irq 12
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043317] mice: PS/2 mouse device common for all mice
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043449] rtc_cmos rtc_cmos: RTC can wake from S4
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043506] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043530] rtc0: alarms up to one year, y3k, 114 bytes nvram, hpet irqs
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043687] device-mapper: uevent: version 1.0.3
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.043842] device-mapper: ioctl: 4.17.0-ioctl (2010-03-05) initialised: dm-devel@redhat.com
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.044076] device-mapper: multipath: version 1.1.1 loaded
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.044082] device-mapper: multipath round-robin: version 1.0.0 loaded
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.044550] cpuidle: using governor ladder
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.044555] cpuidle: using governor menu
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.044960] TCP cubic registered
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.045119] NET: Registered protocol family 10
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.045572] lo: Disabled Privacy Extensions
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.045838] NET: Registered protocol family 17
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.046015] PM: Resume from disk failed.
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.046033] registered taskstats version 1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.046337]   Magic number: 7:62:656
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.046431] rtc_cmos rtc_cmos: setting system clock to 2011-04-14 23:38:43 UTC (1302824323)
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.046435] BIOS EDD facility v0.16 2004-Jun-25, 0 devices found
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.046438] EDD information not available.
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.076043] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.190184] ata1.00: ATAPI: CD-224E-N, C.AA, max UDMA/33
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.230086] ata1.00: configured for UDMA/33
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.233453] scsi 0:0:0:0: CD-ROM            TEAC     CD-224E-N        C.AA PQ: 0 ANSI: 5
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.239575] sr0: scsi3-mmc drive: 24x/24x cd/rw xa/form2 cdda tray
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.239581] Uniform CD-ROM driver Revision: 3.20
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.239724] sr 0:0:0:0: Attached scsi CD-ROM sr0
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.239800] sr 0:0:0:0: Attached scsi generic sg0 type 5
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.239893] Freeing unused kernel memory: 836k freed
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.240340] Write protecting the kernel read-only data: 10240k
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.240499] Freeing unused kernel memory: 316k freed
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.240858] Freeing unused kernel memory: 1620k freed
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.265484] udev[101]: starting version 163
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.339983] tg3.c:v3.110 (April 9, 2010)
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.340017]   alloc irq_desc for 25 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.340021]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.340040] tg3 0000:03:01.0: PCI INT A -> GSI 25 (level, low) -> IRQ 25
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.340409] HP CISS Driver (v 3.6.20)
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.347694]   alloc irq_desc for 51 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.347701]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.347717] cciss 0000:04:03.0: PCI INT A -> GSI 51 (level, low) -> IRQ 51
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.358376] Floppy drive(s): fd0 is 1.44M
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.360208] cciss0: <0x46> at PCI 0000:04:03.0 IRQ 51 using DAC
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.363397]  cciss/c0d0: p1 p2 < p5 >
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.373488] FDC 0 is a National Semiconductor PC87306
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.407769] tg3 0000:03:01.0: eth0: Tigon3 [partno(N/A) rev 2100] (PCIX:133MHz:64-bit) MAC address 00:14:c2:c2:b0:e5
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.407775] tg3 0000:03:01.0: eth0: attached PHY is 5704 (10/100/1000Base-T Ethernet) (WireSpeed[1])
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.407779] tg3 0000:03:01.0: eth0: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[1] TSOcap[0]
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.407784] tg3 0000:03:01.0: eth0: dma_rwctrl[769f4000] dma_mask[64-bit]
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.407809]   alloc irq_desc for 26 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.407813]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.407823] tg3 0000:03:01.1: PCI INT B -> GSI 26 (level, low) -> IRQ 26
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.471074] tg3 0000:03:01.1: eth1: Tigon3 [partno(N/A) rev 2100] (PCIX:133MHz:64-bit) MAC address 00:14:c2:c2:b0:e4
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.471080] tg3 0000:03:01.1: eth1: attached PHY is 5704 (10/100/1000Base-T Ethernet) (WireSpeed[1])
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.471085] tg3 0000:03:01.1: eth1: RXcsums[1] LinkChgREG[0] MIirq[0] ASF[0] TSOcap[1]
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.471089] tg3 0000:03:01.1: eth1: dma_rwctrl[769f4000] dma_mask[64-bit]
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.549434] EXT4-fs (cciss!c0d0p1): INFO: recovery required on readonly filesystem
    Apr 14 19:38:48 METSBBDB01 kernel: [    1.549441] EXT4-fs (cciss!c0d0p1): write access will be enabled during recovery
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364535] EXT4-fs (cciss!c0d0p1): orphan cleanup on readonly fs
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364550] EXT4-fs (cciss!c0d0p1): ext4_orphan_cleanup: deleting unreferenced inode 14942218
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364595] EXT4-fs (cciss!c0d0p1): ext4_orphan_cleanup: deleting unreferenced inode 14942217
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364610] EXT4-fs (cciss!c0d0p1): ext4_orphan_cleanup: deleting unreferenced inode 14942216
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364625] EXT4-fs (cciss!c0d0p1): ext4_orphan_cleanup: deleting unreferenced inode 14942215
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364639] EXT4-fs (cciss!c0d0p1): ext4_orphan_cleanup: deleting unreferenced inode 14942212
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364651] EXT4-fs (cciss!c0d0p1): 5 orphan inodes deleted
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.364656] EXT4-fs (cciss!c0d0p1): recovery complete
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.365018] JBD: barrier-based sync failed on cciss!c0d0p1-8 - disabling barriers
    Apr 14 19:38:48 METSBBDB01 kernel: [    2.366269] EXT4-fs (cciss!c0d0p1): mounted filesystem with ordered data mode. Opts: (null)
    Apr 14 19:38:48 METSBBDB01 kernel: [    5.869294] Adding 11565052k swap on /dev/cciss/c0d0p5.  Priority:-1 extents:1 across:11565052k 
    Apr 14 19:38:48 METSBBDB01 kernel: [    5.870583] udev[301]: starting version 163
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.023395] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.023421] intel_rng: FWH not detected
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.048798] EDAC MC: Ver: 2.1.0 Mar  1 2011
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.067288] Contact your BIOS vendor to see if the E752x error registers can be safely un-hidden
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.076028] type=1400 audit(1302824328.521:2): apparmor="STATUS" operation="profile_load" name="/sbin/dhclient3" pid=440 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.076827] type=1400 audit(1302824328.521:3): apparmor="STATUS" operation="profile_load" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=440 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.077263] type=1400 audit(1302824328.521:4): apparmor="STATUS" operation="profile_load" name="/usr/lib/connman/scripts/dhclient-script" pid=440 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.090571] lp: driver loaded but no devices found
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.097443] EXT4-fs (cciss!c0d0p1): re-mounted. Opts: errors=remount-ro
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.133065]   alloc irq_desc for 22 on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.133073]   alloc kstat_irqs on node -1
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.133094] hpilo 0000:01:04.2: PCI INT B -> GSI 22 (level, low) -> IRQ 22
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.247904] type=1400 audit(1302824328.691:5): apparmor="STATUS" operation="profile_load" name="/usr/sbin/tcpdump" pid=565 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.248236] type=1400 audit(1302824328.691:6): apparmor="STATUS" operation="profile_load" name="/usr/sbin/mysqld" pid=564 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.249549] type=1400 audit(1302824328.691:7): apparmor="STATUS" operation="profile_replace" name="/sbin/dhclient3" pid=563 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.250405] type=1400 audit(1302824328.691:8): apparmor="STATUS" operation="profile_replace" name="/usr/lib/NetworkManager/nm-dhcp-client.action" pid=563 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.251094] type=1400 audit(1302824328.691:9): apparmor="STATUS" operation="profile_replace" name="/usr/lib/connman/scripts/dhclient-script" pid=563 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 sm-notify[553]: Version 1.2.2 starting
    Apr 14 19:38:48 METSBBDB01 init: Failed to open system console: Input/output error
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.341370] ADDRCONF(NETDEV_UP): eth0: link is not ready
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.345995] RPC: Registered udp transport module.
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.346001] RPC: Registered tcp transport module.
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.346005] RPC: Registered tcp NFSv4.1 backchannel transport module.
    Apr 14 19:38:48 METSBBDB01 cron[603]: (CRON) INFO (pidfile fd = 3)
    Apr 14 19:38:48 METSBBDB01 rpc.statd[633]: Version 1.2.2 starting
    Apr 14 19:38:48 METSBBDB01 rpc.statd[633]: Flags: 
    Apr 14 19:38:48 METSBBDB01 cron[651]: (CRON) STARTUP (fork ok)
    Apr 14 19:38:48 METSBBDB01 cron[651]: (CRON) INFO (Running @reboot jobs)
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.471515] type=1400 audit(1302824328.921:10): apparmor="STATUS" operation="profile_replace" name="/usr/sbin/mysqld" pid=677 comm="apparmor_parser"
    Apr 14 19:38:48 METSBBDB01 init: ssh main process (552) terminated with status 255
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.479331] Slow work thread pool: Starting up
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.479498] Slow work thread pool: Ready
    Apr 14 19:38:48 METSBBDB01 kernel: [    6.479688] FS-Cache: Loaded
    Apr 14 19:38:49 METSBBDB01 kernel: [    6.552382] JBD: barrier-based sync failed on cciss!c0d0p1-8 - disabling barriers
    Apr 14 19:38:49 METSBBDB01 kernel: [    6.564539] FS-Cache: Netfs 'nfs' registered for caching
    Apr 14 19:38:49 METSBBDB01 kernel: [    6.652807] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
    Apr 14 19:38:49 METSBBDB01 kernel: [    7.314234] input: ImPS/2 Generic Wheel Mouse as /devices/platform/i8042/serio1/input/input2
    Apr 14 19:38:49 METSBBDB01 /etc/mysql/debian-start[738]: Upgrading MySQL tables if necessary.
    Apr 14 19:38:50 METSBBDB01 /etc/mysql/debian-start[741]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
    Apr 14 19:38:50 METSBBDB01 /etc/mysql/debian-start[741]: Looking for 'mysql' as: /usr/bin/mysql
    Apr 14 19:38:50 METSBBDB01 /etc/mysql/debian-start[741]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
    Apr 14 19:38:50 METSBBDB01 /etc/mysql/debian-start[741]: This installation of MySQL is already upgraded to 5.1.49, use --force if you still need to run mysql_upgrade
    Apr 14 19:38:50 METSBBDB01 /etc/mysql/debian-start[749]: Checking for insecure root accounts.
    Apr 14 19:38:50 METSBBDB01 /etc/mysql/debian-start[753]: Triggering myisam-recover for all MyISAM tables
    Apr 14 19:38:51 METSBBDB01 kernel: [    8.726398] tg3 0000:03:01.0: eth0: Link is up at 1000 Mbps, full duplex
    Apr 14 19:38:51 METSBBDB01 kernel: [    8.726404] tg3 0000:03:01.0: eth0: Flow control is on for TX and on for RX
    Apr 14 19:38:51 METSBBDB01 kernel: [    8.727314] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
    Apr 14 19:38:51 METSBBDB01 exportfs[838]: /etc/exports [1]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/home".#012  Assuming default behaviour ('no_subtree_check').#012  NOTE: this default has changed since nfs-utils version 1.0.x
    Apr 14 19:38:51 METSBBDB01 exportfs[838]: /etc/exports [2]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/srv/nym".#012  Assuming default behaviour ('no_subtree_check').#012  NOTE: this default has changed since nfs-utils version 1.0.x
    Apr 14 19:38:51 METSBBDB01 kernel: [    9.294386] svc: failed to register lockdv1 RPC service (errno 97).
    Apr 14 19:38:51 METSBBDB01 kernel: [    9.295563] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
    Apr 14 19:38:51 METSBBDB01 kernel: [    9.295612] NFSD: starting 90-second grace period
    Apr 14 19:38:56 METSBBDB01 ntpdate[663]: step time server 91.189.94.4 offset 1.124892 sec
    Apr 14 19:39:02 METSBBDB01 kernel: [   19.060009] eth0: no IPv6 routers present
    Apr 14 19:39:03 METSBBDB01 CRON[919]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 20:09:01 METSBBDB01 CRON[958]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 20:17:01 METSBBDB01 CRON[975]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 14 20:39:01 METSBBDB01 CRON[1001]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 21:09:01 METSBBDB01 CRON[1040]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 21:17:01 METSBBDB01 CRON[1057]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 14 21:39:01 METSBBDB01 CRON[1083]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 22:09:01 METSBBDB01 CRON[1122]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 22:17:01 METSBBDB01 CRON[1139]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 14 22:39:01 METSBBDB01 CRON[1164]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 23:09:01 METSBBDB01 CRON[1203]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 14 23:17:01 METSBBDB01 CRON[1220]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 14 23:39:01 METSBBDB01 CRON[1246]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 00:09:01 METSBBDB01 CRON[1285]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 00:17:01 METSBBDB01 CRON[1302]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 15 00:39:01 METSBBDB01 CRON[1328]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 01:09:01 METSBBDB01 CRON[1367]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 01:17:01 METSBBDB01 CRON[1384]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 15 01:39:01 METSBBDB01 CRON[1410]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 02:09:01 METSBBDB01 CRON[1449]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 02:17:01 METSBBDB01 CRON[1466]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 15 02:39:01 METSBBDB01 CRON[1492]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 03:09:01 METSBBDB01 CRON[1531]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 03:17:01 METSBBDB01 CRON[1548]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 15 03:39:01 METSBBDB01 CRON[1574]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 04:09:01 METSBBDB01 CRON[1613]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 04:17:01 METSBBDB01 CRON[1630]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 15 04:39:01 METSBBDB01 CRON[1656]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 05:09:01 METSBBDB01 CRON[1695]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 05:17:01 METSBBDB01 CRON[1712]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 15 05:39:01 METSBBDB01 CRON[1738]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 06:09:01 METSBBDB01 CRON[1777]: (root) CMD (  [ -x /usr/lib/php5/maxlifetime ] && [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -n 200 -r -0 rm)
    Apr 15 06:17:01 METSBBDB01 CRON[1794]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
    Apr 15 06:25:01 METSBBDB01 CRON[1806]: (root) CMD (test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ))
    Does anyone have any ideas about what is going on? How can I diagnose the problem?

    Thanks.
    Attached Images Attached Images

  2. #2
    Join Date
    Jun 2006
    Location
    New York City
    Beans
    46
    Distro
    Ubuntu 10.10 Maverick Meerkat

    Re: 10.10 Server hung_task_timeout_secs?

    Any ideas?

    This is what happens when I log in via SSH:

    Code:
    bbaumer@METS-LT-100465-ubuntu:~$ ssh 10.128.5.60
    bbaumer@10.128.5.60's password: 
    Linux METSBBDB01 2.6.35-28-server #49-Ubuntu SMP Tue Mar 1 14:55:37 UTC 2011 x86_64 GNU/Linux
    Ubuntu 10.10
    
    Welcome to the Ubuntu Server!
     * Documentation:  http://www.ubuntu.com/server/doc
    
     System information disabled due to load higher than 1
    Last login: Mon Apr 25 09:32:18 2011 from 10.128.252.22
    Then it just hangs there. I never get to a prompt, so I can't do anything!

  3. #3
    Join Date
    Feb 2005
    Location
    Oregon
    Beans
    496
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: 10.10 Server hung_task_timeout_secs?

    I'm thinking it could be related to these, but that's only based on noticing "cciss" in the screenshot after seeing a similar bug report mention it:
    https://bugs.launchpad.net/ubuntu/+s...ux/+bug/666828
    https://bugzilla.redhat.com/show_bug.cgi?id=615543

    The blocked processes are probably calling sync() or fsync() to wait for writes to finish, and something is causing them to wait forever. I've seen it happen to dpkg on 10.04 when another process is writing heavily and there's a usb drive plugged in, but this could be another issue.

  4. #4
    Join Date
    Jun 2006
    Location
    New York City
    Beans
    46
    Distro
    Ubuntu 10.10 Maverick Meerkat

    Re: 10.10 Server hung_task_timeout_secs?

    Indeed, the Red Hat forum eventually contained the solution. The trick was upgrading the SmartArray firmware from version 2.36 to version 2.84.

    Thanks!

  5. #5
    Join Date
    Oct 2006
    Location
    Desert
    Beans
    75

    Re: 10.10 Server hung_task_timeout_secs?

    Anyone still having this issue? The above bug in the Redhat bugzilla is unaccessible. However, I do not have any sort of array, so a Smart array firmware update would be useless(Again no access to RH bugzilla).

    Running 2.6.38-11-generic "Natty"

    I have seen the is problem off-and-on since Hardy. It forces me to hard reset. Because of this, my max uptime is only 3 months. It is extremely sad that you have to PATCH THE KERNEL just to get some reliability. Does anyone know if this patch will make it into future ubuntu kernels?

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •