There's some compatibility issue between Compiz and that option as far as I gathered from searching around.

I settled for the following:
- installed kicker-taskbar-compiz from universe
- removed usual taskbar aplet from panel
- added taskbar-compiz to panel
- opened /home/yourusername/.kde/share/config/ktaskbarrc and added the following:

[General]
ShowAllWindows=false

- Ctrl+Alt+Backspace to restart kde

It works with a small lag when switching desktops so for about half a second you'll see windows from previous desktop and then they switch for new ones.