Search:

Type: Posts; User: tkae-lp; Keyword(s):

Page 1 of 4 1 2 3 4

Search: Search took 0.16 seconds.

  1. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    No problem :) regardless of how much heavy lifting was done, I still appreciate you taking your free time to help me out :-) without people like you on forums, I'd be scratching my head and sobbing...
  2. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Right so hello again!

    My apologies for taking almost 2 months to reply this time, but I think we can probably all agree that I have given it more than enough time to test lol

    The good news is...
  3. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Thanks! Yeah it did a bit.... glad I didn't stick with rsync, it was still looking at what to do 5 hours in! I'm a ZFS send/recv convert lol

    Sorry I haven't replied sooner - the last week has been...
  4. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Finally.... restoring took quite a bit longer than the backup:

    Backup:


    11.4TiB 24:48:24 [ 134MiB/s]

    Restore:
  5. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    *facepalm* I didn't even think of du.. I had tunnel vision and was thinking there must be some ZFS command. I've had like an hours sleep in the last 36 hours... probably not the right time to be...
  6. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Ok, so this is after the send:


    ~ zfs list
    NAME USED AVAIL REFER MOUNTPOINT
    Tank 11.3T 9.03T 11.0T /mnt/Tank
    Tank-Backup ...
  7. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    :biggrin: Ok fab. You'll have to excuse the obviously daft questions - I have sleep issues and right now they are bad. A daft question is less daft if it saves a ton of data and 24 hours ;D
    ...
  8. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Ok, so what I am doing is correct?

    1. I snapped the entire tank using -r
    2. Then sent using -R

    I will mount the vols after completion and check. It certainly appears to be sending the lot....
  9. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Sorry my previous post probably made it look like I am doing both. I've ditched rsync because for whatever reason it was too slow.

    I'm just using send/recv now. By full snapshot you mean using the...
  10. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    I'm on it. It's taking a LONG time.... Long in the kind of way that makes me nervous. If anything else, this is a good time to be backing up!

    Edit: Quick question:


    logs ...
  11. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    It's looking like I might have the time I need to do the backup/restore tomorrow. Quick question before I do: Is it that I am literally just copying off then back onto the array or should I destroy...
  12. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    But on that logic (older disk has more written to it) My case is the opposite - the disk that's flagging on my end is newer.

    My output shows very little:



    NAME ...
  13. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    The drive in question (sdf) was only faulted for a few hours before I noticed and shutdown the server. I didn't have a drive to hand so I ordered one, then powered on when it arrived and resilvered....
  14. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Temps of array disks:



    sda1 =35c
    sdb1 = 36c
    sdc1 = 34c
    sdd1 = 32c
    sde1 = 31c
    sdf1 = 33c
  15. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Wait! I have something! Maybe...

    Here is the log: https://paste.ubuntu.com/p/ZW3PT9qdCN/

    I've monitored every connected disk.

    Disk key:


    nvme0n1p1 = zfs zraid2 cache
  16. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Will do. 1 year is very generous :D I was going to do a month! LOL

    Whatever this thing is, it must have ears and they must be burning... I've had perfect performance today! Skipping forwards and...
  17. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    I agree, it's very puzzling.

    I'll monitor over the next few days and give you some iostat logs!
  18. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    That's odd... not sure what to make of that! I have one of my NVMe drive rename itself on import, but it still works, it's just annoying.

    Today has been horrible for speed :sad: Can't even watch a...
  19. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Being an amateur tinkerer in many things, I often come across issues with whatever it is that I have decided to take my hand to. Usually, ~80% of them are solved with a quick search, ~15% with an...
  20. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    How odd! I did use btrfs for root a while back but am currently on ext4. However, your btrfs speeds are not what they should be even if they are faster. It's possible that this is simply random. I...
  21. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Wow!!....... that certainly looks like the behaviour I have been seeing. I've never seen you post speeds anywhere near that low. Whatever this thing is, I would say it's a fairly safe bet that it...
  22. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Ok, this is not gone... last couple of days I've had slow downs.

    Just now I see:


    READ: bw=2735MiB/s (2868MB/s), 2735MiB/s-2735MiB/s (2868MB/s-2868MB/s), io=10.0GiB (10.7GB), run=3744-3744msec...
  23. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    Very true! ;)

    No problems so far today:


    READ: bw=3561MiB/s (3733MB/s), 3561MiB/s-3561MiB/s (3733MB/s-3733MB/s), io=10.0GiB (10.7GB), run=2876-2876msec

    WRITE: bw=1782MiB/s (1869MB/s),...
  24. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    No problems today:


    READ: bw=4326MiB/s (4536MB/s), 4326MiB/s-4326MiB/s (4536MB/s-4536MB/s), io=10.0GiB (10.7GB), run=2367-2367msec



    WRITE: bw=1377MiB/s (1443MB/s), 1377MiB/s-1377MiB/s...
  25. Replies
    135
    Views
    136

    [SOLVED] Re: Seemingly sporadic slow ZFS IO since 22.04

    :-( I copied a file from one folder to another this evening (duplicate not move) and noticed it was slow... (20 secs for ~350MB)


    fio --name TEST --eta-newline=5s --filename=temp.file --rw=write...
Results 1 to 25 of 76
Page 1 of 4 1 2 3 4