INFO: task * blocked for more than 120 seconds.

[aeiah]

i have it with 10.04 server, 2.6.32-21 running as a HyperV guest. ARGHH.

investigating now

[mglat]

I am experiencing the same problems with Ubuntu 10.04-server and kernel 2.6.32-25-server via MS Hyper-V. Any suggestions so far?

[andrex593]

I also have this problem, in Maverick with kernel 2.6.35.23.25 and Nvidia video driver, all up to date. It started a few weeks ago after a kernel upgrade IIRC. I've seen it get triggered twice when I ran relatively disk-intensive commands, like 'find -type d -print0 | xargs -0 chmod u+s' or 'cvs up' on a Drupal repository. This has the feel of a hardware problem, but SMART on my /home drive shows good health and no errors.

[0xCAFEFACE]

This has just happened to me also under Ubuntu server 10.04.1LTS running under Hyper-V.

Kernel: 2.6.32-26-generic-pae

It looks like it's related to the scheduling processes crashing.

kern.log :

Nov 30 06:51:27 host kernel: [681414.858508] Btrfs loaded
Dec  4 19:06:54 host kernel: [1071141.810134] BUG: scheduling while atomic: swapper/0/0x10000100
Dec  4 19:06:54 host kernel: [1071141.813736] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs lp fbcon tileblit font bitblit softcursor psmouse serio_raw parport i2c_piix4 vga16fb vgastate hv_netvsc(C) hv_blkvsc(C) hv_storvsc(C) floppy hv_vmbus(C)
Dec  4 19:06:54 host kernel: [1071141.813754] 
Dec  4 19:06:54 host kernel: [1071141.813757] Pid: 0, comm: swapper Tainted: G         C (2.6.32-24-generic-pae #43-Ubuntu) Virtual Machine
Dec  4 19:06:54 host kernel: [1071141.813759] EIP: 0060:[<c012ffea>] EFLAGS: 00000246 CPU: 0
Dec  4 19:06:54 host kernel: [1071141.813764] EIP is at native_safe_halt+0xa/0x10
Dec  4 19:06:54 host kernel: [1071141.813766] EAX: c0790000 EBX: c07e0a64 ECX: c1a04a60 EDX: 00000000
Dec  4 19:06:54 host kernel: [1071141.813767] ESI: 00000000 EDI: c079d000 EBP: c0791f7c ESP: c0791f7c
Dec  4 19:06:54 host kernel: [1071141.813769]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Dec  4 19:06:54 host kernel: [1071141.813770] CR0: 8005003b CR2: b77c46d0 CR3: 36534000 CR4: 000006f0
Dec  4 19:06:54 host kernel: [1071141.813774] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Dec  4 19:06:54 host kernel: [1071141.813775] DR6: ffff0ff0 DR7: 00000400
Dec  4 19:06:54 host kernel: [1071141.813777] Call Trace:
Dec  4 19:06:54 host kernel: [1071141.813782]  [<c0110c10>] default_idle+0x40/0x90
Dec  4 19:06:54 host kernel: [1071141.813785]  [<c0108634>] cpu_idle+0x94/0xd0
Dec  4 19:06:54 host kernel: [1071141.813789]  [<c059f368>] rest_init+0x58/0x60
Dec  4 19:06:54 host kernel: [1071141.813792]  [<c07e48fd>] start_kernel+0x351/0x357
Dec  4 19:06:54 host kernel: [1071141.813794]  [<c07e43d8>] ? unknown_bootoption+0x0/0x19e
Dec  4 19:06:54 host kernel: [1071141.813797]  [<c07e40bb>] i386_start_kernel+0xaa/0xb1
Dec  4 19:06:54 host kernel: [1071141.816182] BUG: scheduling while atomic: swapper/0/0x10000100
Dec  4 19:06:54 host kernel: [1071141.819838] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs lp fbcon tileblit font bitblit softcursor psmouse serio_raw parport i2c_piix4 vga16fb vgastate hv_netvsc(C) hv_blkvsc(C) hv_storvsc(C) floppy hv_vmbus(C)
Dec  4 19:06:54 host kernel: [1071141.819882] 
Dec  4 19:06:54 host kernel: [1071141.819885] Pid: 0, comm: swapper Tainted: G         C (2.6.32-24-generic-pae #43-Ubuntu) Virtual Machine
Dec  4 19:06:54 host kernel: [1071141.819887] EIP: 0060:[<c012ffea>] EFLAGS: 00000246 CPU: 0
Dec  4 19:06:54 host kernel: [1071141.819890] EIP is at native_safe_halt+0xa/0x10
Dec  4 19:06:54 host kernel: [1071141.819892] EAX: c0790000 EBX: c07e0a64 ECX: c1a04a60 EDX: 00000000
Dec  4 19:06:54 host kernel: [1071141.819909] ESI: 00000000 EDI: c079d000 EBP: c0791f7c ESP: c0791f7c
Dec  4 19:06:54 host kernel: [1071141.819911]  DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068
Dec  4 19:06:54 host kernel: [1071141.819913] CR0: 8005003b CR2: b77c46d0 CR3: 36a19000 CR4: 000006f0
Dec  4 19:06:54 host kernel: [1071141.819916] DR0: 00000000 DR1: 00000000 DR2: 00000000 DR3: 00000000
Dec  4 19:06:54 host kernel: [1071141.819917] DR6: ffff0ff0 DR7: 00000400
Dec  4 19:06:54 host kernel: [1071141.819918] Call Trace:
Dec  4 19:06:54 host kernel: [1071141.819922]  [<c0110c10>] default_idle+0x40/0x90
Dec  4 19:06:54 host kernel: [1071141.819925]  [<c0108634>] cpu_idle+0x94/0xd0
Dec  4 19:06:54 host kernel: [1071141.819927]  [<c059f368>] rest_init+0x58/0x60
Dec  4 19:06:54 host kernel: [1071141.819930]  [<c07e48fd>] start_kernel+0x351/0x357
Dec  4 19:06:54 host kernel: [1071141.819932]  [<c07e43d8>] ? unknown_bootoption+0x0/0x19e
Dec  4 19:06:54 host kernel: [1071141.819934]  [<c07e40bb>] i386_start_kernel+0xaa/0xb1

[bakinsoda]

I too have been having this issue since mid-late December. At first it happened about 1x a week, now it is happening daily. At first my hunch was that the disk is bad; one of my tables in mysql became corrupted after so many resets. Could this be it? Have you guys resolved this yet? What have you tried and what threads are you following?

Here is an excerpt from my kern.log:

Jan 18 22:48:10 vqnguyen-server2 kernel: [23507.540509] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:25 vqnguyen-server2 kernel: [23581.767565] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:37 vqnguyen-server2 kernel: [23593.861591] lo: Disabled Privacy Ext
ensions
Jan 19 02:32:23 vqnguyen-server2 kernel: [36960.552119] INFO: task mysqld:1938 b
locked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552224] "echo 0 > /proc/sys/kern
el/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552331] mysqld        D 0003df5a     0  1938      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552346]  e395ff38 00000086 fffb2858 0003df5a 00000000 c088b4c0 e3939c24 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552364]  d310e6d5 0000216f c088b4c0 c088b4c0 e3939c24 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552381]  d30ed65e 0000216f e3939980 e3939980 e395ff6c f65c6838 e395ff64 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552398] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552420]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552430]  [<c05b3b8d>] rwsem_down_write_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552440]  [<c05b3c26>] call_rwsem_down_write_failed+0x6/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552450]  [<c05b3144>] ? down_write+0x24/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552464]  [<c01f6166>] sys_mremap+0x26/0x60
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552475]  [<c01097ac>] syscall_call+0x7/0xb
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552491] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552577] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552683] mysqld        D 000acd4a     0  1997      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552696]  f6595f18 00000086 f771a878 000acd4a 00000000 c088b4c0 f6fc82a4 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552712]  2e669727 0000217a c088b4c0 c088b4c0 f6fc82a4 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552728]  2e6635af 0000217a f6fc8000 f6fc8000 f6595f4c f65c6838 f6595f44 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552744] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552755]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552766]  [<c01f0567>] ? handle_mm_fault+0x397/0x4a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552776]  [<c05b3bbd>] rwsem_down_read_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552787]  [<c05b3c17>] call_rwsem_down_read_failed+0x7/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552796]  [<c05b316c>] ? down_read+0x1c/0x20
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552806]  [<c05b64c5>] do_page_fault+0x395/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552815]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552823]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552240] INFO: task cron:3130 blocked for more than 120 seconds.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552449] cron          D 0001c590     0  3130    732 0x00000000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552464]  f563de1c 00000086 00000000 0001c590 00000000 c088b4c0 f5644264 c088b4c0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552482]  ff4b58f1 00002209 c088b4c0 c088b4c0 f5644264 c088b4c0 c088b4c0 c2c02000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552499]  ff3c2b24 00002209 f5643fc0 c1e094c0 f5643fc0 f563de68 f563de2c c05b207a
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552516] Call Trace:
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552540]  [<c05b207a>] io_schedule+0x3a/0x60
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552552]  [<c01d316d>] sync_page+0x3d/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552562]  [<c05b26c7>] __wait_on_bit_lock+0x47/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552573]  [<c01e3d3f>] ? vma_prio_tree_remove+0x7f/0xf0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552581]  [<c01d3130>] ? sync_page+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552590]  [<c01d30fe>] __lock_page+0x7e/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552601]  [<c016ff60>] ? wake_bit_function+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552612]  [<c01d52fa>] filemap_fault+0x39a/0x410
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552623]  [<c01eee7c>] __do_fault+0x4c/0x520
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552632]  [<c01f2820>] ? __vma_link_file+0x40/0x70
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552641]  [<c01f2ea4>] ? vma_link+0x84/0xa0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552651]  [<c01f0368>] handle_mm_fault+0x198/0x4a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552663]  [<c028f4e0>] ? ext4_file_mmap+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552674]  [<c01f4cbc>] ? do_mmap_pgoff+0x25c/0x300
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552688]  [<c05b623d>] do_page_fault+0x10d/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552699]  [<c01e7c1b>] ? sys_mmap_pgoff+0x12b/0x240
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552708]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552717]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556099] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556194] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556300] mysqld        D 0000c059     0  1997      1 0x00000000

[bakinsoda]

I too have been having this issue since mid-late December on my Ubuntu 10.04 server (2.6.32-27-generic-pae; P4 laptop). At first it happened about 1x a week, now it is happening daily. At first my hunch was that the disk is bad; one of my tables in mysql became corrupted after so many resets. Could this be it? Have you guys resolved this yet? What have you tried and what threads are you following?

Here is an excerpt from my kern.log:

Jan 18 22:48:10 vqnguyen-server2 kernel: [23507.540509] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:25 vqnguyen-server2 kernel: [23581.767565] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:37 vqnguyen-server2 kernel: [23593.861591] lo: Disabled Privacy Ext
ensions
Jan 19 02:32:23 vqnguyen-server2 kernel: [36960.552119] INFO: task mysqld:1938 b
locked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552224] "echo 0 > /proc/sys/kern
el/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552331] mysqld        D 0003df5a     0  1938      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552346]  e395ff38 00000086 fffb2858 0003df5a 00000000 c088b4c0 e3939c24 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552364]  d310e6d5 0000216f c088b4c0 c088b4c0 e3939c24 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552381]  d30ed65e 0000216f e3939980 e3939980 e395ff6c f65c6838 e395ff64 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552398] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552420]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552430]  [<c05b3b8d>] rwsem_down_write_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552440]  [<c05b3c26>] call_rwsem_down_write_failed+0x6/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552450]  [<c05b3144>] ? down_write+0x24/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552464]  [<c01f6166>] sys_mremap+0x26/0x60
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552475]  [<c01097ac>] syscall_call+0x7/0xb
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552491] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552577] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552683] mysqld        D 000acd4a     0  1997      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552696]  f6595f18 00000086 f771a878 000acd4a 00000000 c088b4c0 f6fc82a4 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552712]  2e669727 0000217a c088b4c0 c088b4c0 f6fc82a4 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552728]  2e6635af 0000217a f6fc8000 f6fc8000 f6595f4c f65c6838 f6595f44 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552744] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552755]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552766]  [<c01f0567>] ? handle_mm_fault+0x397/0x4a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552776]  [<c05b3bbd>] rwsem_down_read_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552787]  [<c05b3c17>] call_rwsem_down_read_failed+0x7/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552796]  [<c05b316c>] ? down_read+0x1c/0x20
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552806]  [<c05b64c5>] do_page_fault+0x395/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552815]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552823]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552240] INFO: task cron:3130 blocked for more than 120 seconds.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552449] cron          D 0001c590     0  3130    732 0x00000000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552464]  f563de1c 00000086 00000000 0001c590 00000000 c088b4c0 f5644264 c088b4c0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552482]  ff4b58f1 00002209 c088b4c0 c088b4c0 f5644264 c088b4c0 c088b4c0 c2c02000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552499]  ff3c2b24 00002209 f5643fc0 c1e094c0 f5643fc0 f563de68 f563de2c c05b207a
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552516] Call Trace:
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552540]  [<c05b207a>] io_schedule+0x3a/0x60
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552552]  [<c01d316d>] sync_page+0x3d/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552562]  [<c05b26c7>] __wait_on_bit_lock+0x47/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552573]  [<c01e3d3f>] ? vma_prio_tree_remove+0x7f/0xf0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552581]  [<c01d3130>] ? sync_page+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552590]  [<c01d30fe>] __lock_page+0x7e/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552601]  [<c016ff60>] ? wake_bit_function+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552612]  [<c01d52fa>] filemap_fault+0x39a/0x410
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552623]  [<c01eee7c>] __do_fault+0x4c/0x520
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552632]  [<c01f2820>] ? __vma_link_file+0x40/0x70
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552641]  [<c01f2ea4>] ? vma_link+0x84/0xa0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552651]  [<c01f0368>] handle_mm_fault+0x198/0x4a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552663]  [<c028f4e0>] ? ext4_file_mmap+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552674]  [<c01f4cbc>] ? do_mmap_pgoff+0x25c/0x300
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552688]  [<c05b623d>] do_page_fault+0x10d/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552699]  [<c01e7c1b>] ? sys_mmap_pgoff+0x12b/0x240
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552708]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552717]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556099] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556194] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556300] mysqld        D 0000c059     0  1997      1 0x00000000

[bakinsoda]

I too have been having this issue since mid-late December on my Ubuntu 10.04 server (2.6.32-27-generic-pae; P4 laptop). At first it happened about 1x a week, now it is happening daily. At first my hunch was that the disk is bad; one of my tables in mysql became corrupted after so many resets. Could this be it? Have you guys resolved this yet? What have you tried and what threads are you following?

Here is an excerpt from my kern.log:

Jan 18 22:48:10 vqnguyen-server2 kernel: [23507.540509] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:25 vqnguyen-server2 kernel: [23581.767565] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:37 vqnguyen-server2 kernel: [23593.861591] lo: Disabled Privacy Ext
ensions
Jan 19 02:32:23 vqnguyen-server2 kernel: [36960.552119] INFO: task mysqld:1938 b
locked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552224] "echo 0 > /proc/sys/kern
el/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552331] mysqld        D 0003df5a     0  1938      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552346]  e395ff38 00000086 fffb2858 0003df5a 00000000 c088b4c0 e3939c24 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552364]  d310e6d5 0000216f c088b4c0 c088b4c0 e3939c24 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552381]  d30ed65e 0000216f e3939980 e3939980 e395ff6c f65c6838 e395ff64 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552398] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552420]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552430]  [<c05b3b8d>] rwsem_down_write_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552440]  [<c05b3c26>] call_rwsem_down_write_failed+0x6/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552450]  [<c05b3144>] ? down_write+0x24/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552464]  [<c01f6166>] sys_mremap+0x26/0x60
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552475]  [<c01097ac>] syscall_call+0x7/0xb
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552491] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552577] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552683] mysqld        D 000acd4a     0  1997      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552696]  f6595f18 00000086 f771a878 000acd4a 00000000 c088b4c0 f6fc82a4 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552712]  2e669727 0000217a c088b4c0 c088b4c0 f6fc82a4 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552728]  2e6635af 0000217a f6fc8000 f6fc8000 f6595f4c f65c6838 f6595f44 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552744] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552755]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552766]  [<c01f0567>] ? handle_mm_fault+0x397/0x4a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552776]  [<c05b3bbd>] rwsem_down_read_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552787]  [<c05b3c17>] call_rwsem_down_read_failed+0x7/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552796]  [<c05b316c>] ? down_read+0x1c/0x20
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552806]  [<c05b64c5>] do_page_fault+0x395/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552815]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552823]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552240] INFO: task cron:3130 blocked for more than 120 seconds.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552449] cron          D 0001c590     0  3130    732 0x00000000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552464]  f563de1c 00000086 00000000 0001c590 00000000 c088b4c0 f5644264 c088b4c0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552482]  ff4b58f1 00002209 c088b4c0 c088b4c0 f5644264 c088b4c0 c088b4c0 c2c02000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552499]  ff3c2b24 00002209 f5643fc0 c1e094c0 f5643fc0 f563de68 f563de2c c05b207a
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552516] Call Trace:
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552540]  [<c05b207a>] io_schedule+0x3a/0x60
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552552]  [<c01d316d>] sync_page+0x3d/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552562]  [<c05b26c7>] __wait_on_bit_lock+0x47/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552573]  [<c01e3d3f>] ? vma_prio_tree_remove+0x7f/0xf0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552581]  [<c01d3130>] ? sync_page+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552590]  [<c01d30fe>] __lock_page+0x7e/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552601]  [<c016ff60>] ? wake_bit_function+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552612]  [<c01d52fa>] filemap_fault+0x39a/0x410
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552623]  [<c01eee7c>] __do_fault+0x4c/0x520
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552632]  [<c01f2820>] ? __vma_link_file+0x40/0x70
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552641]  [<c01f2ea4>] ? vma_link+0x84/0xa0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552651]  [<c01f0368>] handle_mm_fault+0x198/0x4a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552663]  [<c028f4e0>] ? ext4_file_mmap+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552674]  [<c01f4cbc>] ? do_mmap_pgoff+0x25c/0x300
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552688]  [<c05b623d>] do_page_fault+0x10d/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552699]  [<c01e7c1b>] ? sys_mmap_pgoff+0x12b/0x240
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552708]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552717]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556099] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556194] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556300] mysqld        D 0000c059     0  1997      1 0x00000000

[bakinsoda]

I too have been having this issue since mid-late December on my Ubuntu 10.04 server (2.6.32-27-generic-pae; P4 laptop). At first it happened about 1x a week, now it is happening daily. At first my hunch was that the disk is bad; one of my tables in mysql became corrupted after so many resets. Could this be it? Have you guys resolved this yet? What have you tried and what threads are you following?

Here is an excerpt from my kern.log:

Jan 18 22:48:10 vqnguyen-server2 kernel: [23507.540509] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:25 vqnguyen-server2 kernel: [23581.767565] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:37 vqnguyen-server2 kernel: [23593.861591] lo: Disabled Privacy Ext
ensions
Jan 19 02:32:23 vqnguyen-server2 kernel: [36960.552119] INFO: task mysqld:1938 b
locked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552224] "echo 0 > /proc/sys/kern
el/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552331] mysqld        D 0003df5a     0  1938      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552346]  e395ff38 00000086 fffb2858 0003df5a 00000000 c088b4c0 e3939c24 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552364]  d310e6d5 0000216f c088b4c0 c088b4c0 e3939c24 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552381]  d30ed65e 0000216f e3939980 e3939980 e395ff6c f65c6838 e395ff64 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552398] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552420]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552430]  [<c05b3b8d>] rwsem_down_write_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552440]  [<c05b3c26>] call_rwsem_down_write_failed+0x6/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552450]  [<c05b3144>] ? down_write+0x24/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552464]  [<c01f6166>] sys_mremap+0x26/0x60
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552475]  [<c01097ac>] syscall_call+0x7/0xb
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552491] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552577] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552683] mysqld        D 000acd4a     0  1997      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552696]  f6595f18 00000086 f771a878 000acd4a 00000000 c088b4c0 f6fc82a4 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552712]  2e669727 0000217a c088b4c0 c088b4c0 f6fc82a4 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552728]  2e6635af 0000217a f6fc8000 f6fc8000 f6595f4c f65c6838 f6595f44 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552744] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552755]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552766]  [<c01f0567>] ? handle_mm_fault+0x397/0x4a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552776]  [<c05b3bbd>] rwsem_down_read_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552787]  [<c05b3c17>] call_rwsem_down_read_failed+0x7/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552796]  [<c05b316c>] ? down_read+0x1c/0x20
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552806]  [<c05b64c5>] do_page_fault+0x395/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552815]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552823]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552240] INFO: task cron:3130 blocked for more than 120 seconds.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552449] cron          D 0001c590     0  3130    732 0x00000000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552464]  f563de1c 00000086 00000000 0001c590 00000000 c088b4c0 f5644264 c088b4c0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552482]  ff4b58f1 00002209 c088b4c0 c088b4c0 f5644264 c088b4c0 c088b4c0 c2c02000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552499]  ff3c2b24 00002209 f5643fc0 c1e094c0 f5643fc0 f563de68 f563de2c c05b207a
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552516] Call Trace:
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552540]  [<c05b207a>] io_schedule+0x3a/0x60
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552552]  [<c01d316d>] sync_page+0x3d/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552562]  [<c05b26c7>] __wait_on_bit_lock+0x47/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552573]  [<c01e3d3f>] ? vma_prio_tree_remove+0x7f/0xf0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552581]  [<c01d3130>] ? sync_page+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552590]  [<c01d30fe>] __lock_page+0x7e/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552601]  [<c016ff60>] ? wake_bit_function+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552612]  [<c01d52fa>] filemap_fault+0x39a/0x410
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552623]  [<c01eee7c>] __do_fault+0x4c/0x520
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552632]  [<c01f2820>] ? __vma_link_file+0x40/0x70
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552641]  [<c01f2ea4>] ? vma_link+0x84/0xa0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552651]  [<c01f0368>] handle_mm_fault+0x198/0x4a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552663]  [<c028f4e0>] ? ext4_file_mmap+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552674]  [<c01f4cbc>] ? do_mmap_pgoff+0x25c/0x300
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552688]  [<c05b623d>] do_page_fault+0x10d/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552699]  [<c01e7c1b>] ? sys_mmap_pgoff+0x12b/0x240
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552708]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552717]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556099] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556194] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556300] mysqld        D 0000c059     0  1997      1 0x00000000

[bakinsoda]

I too have been having this issue since mid-late December on my Ubuntu 10.04 server (2.6.32-27-generic-pae; P4 laptop). At first it happened about 1x a week, now it is happening daily. At first my hunch was that the disk is bad; one of my tables in mysql became corrupted after so many resets. Could this be it? Have you guys resolved this yet? What have you tried and what threads are you following?

Here is an excerpt from my kern.log:

Jan 18 22:48:10 vqnguyen-server2 kernel: [23507.540509] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:25 vqnguyen-server2 kernel: [23581.767565] lo: Disabled Privacy Ext
ensions
Jan 18 22:49:37 vqnguyen-server2 kernel: [23593.861591] lo: Disabled Privacy Ext
ensions
Jan 19 02:32:23 vqnguyen-server2 kernel: [36960.552119] INFO: task mysqld:1938 b
locked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552224] "echo 0 > /proc/sys/kern
el/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552331] mysqld        D 0003df5a     0  1938      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552346]  e395ff38 00000086 fffb2858 0003df5a 00000000 c088b4c0 e3939c24 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552364]  d310e6d5 0000216f c088b4c0 c088b4c0 e3939c24 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552381]  d30ed65e 0000216f e3939980 e3939980 e395ff6c f65c6838 e395ff64 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552398] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552420]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552430]  [<c05b3b8d>] rwsem_down_write_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552440]  [<c05b3c26>] call_rwsem_down_write_failed+0x6/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552450]  [<c05b3144>] ? down_write+0x24/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552464]  [<c01f6166>] sys_mremap+0x26/0x60
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552475]  [<c01097ac>] syscall_call+0x7/0xb
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552491] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552577] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552683] mysqld        D 000acd4a     0  1997      1 0x00000000
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552696]  f6595f18 00000086 f771a878 000acd4a 00000000 c088b4c0 f6fc82a4 c088b4c0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552712]  2e669727 0000217a c088b4c0 c088b4c0 f6fc82a4 c088b4c0 c088b4c0 f65c6800
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552728]  2e6635af 0000217a f6fc8000 f6fc8000 f6595f4c f65c6838 f6595f44 c05b3a45
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552744] Call Trace:
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552755]  [<c05b3a45>] rwsem_down_failed_common+0x75/0x1a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552766]  [<c01f0567>] ? handle_mm_fault+0x397/0x4a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552776]  [<c05b3bbd>] rwsem_down_read_failed+0x1d/0x30
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552787]  [<c05b3c17>] call_rwsem_down_read_failed+0x7/0x10
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552796]  [<c05b316c>] ? down_read+0x1c/0x20
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552806]  [<c05b64c5>] do_page_fault+0x395/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552815]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:33:58 vqnguyen-server2 kernel: [36960.552823]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552240] INFO: task cron:3130 blocked for more than 120 seconds.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552449] cron          D 0001c590     0  3130    732 0x00000000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552464]  f563de1c 00000086 00000000 0001c590 00000000 c088b4c0 f5644264 c088b4c0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552482]  ff4b58f1 00002209 c088b4c0 c088b4c0 f5644264 c088b4c0 c088b4c0 c2c02000
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552499]  ff3c2b24 00002209 f5643fc0 c1e094c0 f5643fc0 f563de68 f563de2c c05b207a
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552516] Call Trace:
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552540]  [<c05b207a>] io_schedule+0x3a/0x60
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552552]  [<c01d316d>] sync_page+0x3d/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552562]  [<c05b26c7>] __wait_on_bit_lock+0x47/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552573]  [<c01e3d3f>] ? vma_prio_tree_remove+0x7f/0xf0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552581]  [<c01d3130>] ? sync_page+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552590]  [<c01d30fe>] __lock_page+0x7e/0x90
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552601]  [<c016ff60>] ? wake_bit_function+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552612]  [<c01d52fa>] filemap_fault+0x39a/0x410
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552623]  [<c01eee7c>] __do_fault+0x4c/0x520
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552632]  [<c01f2820>] ? __vma_link_file+0x40/0x70
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552641]  [<c01f2ea4>] ? vma_link+0x84/0xa0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552651]  [<c01f0368>] handle_mm_fault+0x198/0x4a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552663]  [<c028f4e0>] ? ext4_file_mmap+0x0/0x50
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552674]  [<c01f4cbc>] ? do_mmap_pgoff+0x25c/0x300
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552688]  [<c05b623d>] do_page_fault+0x10d/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552699]  [<c01e7c1b>] ? sys_mmap_pgoff+0x12b/0x240
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552708]  [<c05b6130>] ? do_page_fault+0x0/0x3a0
Jan 19 02:42:23 vqnguyen-server2 kernel: [37560.552717]  [<c05b40f3>] error_code+0x73/0x80
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556099] INFO: task mysqld:1997 blocked for more than 120 seconds.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556194] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jan 19 05:16:24 vqnguyen-server2 kernel: [46800.556300] mysqld        D 0000c059     0  1997      1 0x00000000

[DimensionSlider]

Same problem since i`ve switched from Legacy Network Adapter to Network Adapter hv_netsvc. I`m running Hyper-V VPS on Ubuntu 11.10.

Happens every day.