Ubuntu 14 vs 20x File Copy Performance Slow

[MAFoElffen]

Dang. The different version of fio between the two are in a different output format from each other!!! Hmmm.

Just going off the times:
WRITE: maxt=148msec
WRITE: run=290-290msec
=================
14.04 was 19% faster on writes...

READ: maxt=155msec
READ: run=267-267msec
================
14.04 was 14% faster on Reads.

Dang.

[TheFu]

Mike, there's much more security in the kernel now as well as a number of those CPU security flaws that have been discovered and mitigated since 14.04. Some of the mitigations had 20-30% DBMS performance impacts.

[MAFoElffen]

+1 --- Thank you. Yes. Sorry/Got distracted looking at the numbers. LOL. That would make up for that. I did mention that it has a lot more going on with it.

I was going to say, even though it is somewhat slower. It really doesn't really matter. It would be insane to think that running 14.04 would be safe or a good idea. It has been EOL for about 5 years now.

I just helped a municipality to upgrade their closed, open-gaped embedded system from 14.04 to 18.04, where it dead-ended them, because the application they had on them only ran on 32bit. But that bought them some time, to get a 64bit application written.

There is nothing in 14.04.x that couldn't be migrated to something more current. And if you still can't get it all the way to 'current support', it shouldn't be on something that has connections to anything. Big Period.

There is not much you can do with VMware vSphere to optimize disk I/O under the covers. It takes over the very limited hardware it supports in it's own way (their way). And even though I used to beta test for VMware, and I have licenses for vSphere/vCenter from my past endeavors with them... BroadCom buying VMware and doing away with perpetual licensing, free versions of ESXi, and other things... Their market is in flux right now with a lot of unknowns. The initial direction they are heading is... I leave that unsaid right now. I may not be testing for them anymore, and saying my goodbyes. Their loss. But that is okay.

My first choice personally has always been with using KVM. You can do more with KVM. It is a lot more flexible, adaptable, and tunable. And it is opensource. It is the base for more of the Virtual Host market than anything else in the World. For very good reason. If you want more from what you are doing... And not end up in a dead-end with your hands-tied, I challenge you to try and learn KVM.

You are only looking at disk I/O. Deal with what is currently supported on current, modern hardware, on and for currently supported OS'es. If disk I/O is your only requirement (besides the system being secure and system running faster overall), then optimize your disk performance. As TheFu touched on, I use SSD's and PCIe M.2 NVME's in ZFS RAIDZ2 & RAIDZ3 with hardware SLOG and L2ARC caches optimized to the arrays. Even being a COW filesystem, I don't think you will find anything faster than that for Disk I/O performance. I can lose 2 disks, keep live, and not notice the performance hit unless I keep looking at my logs and alerts. It's honestly that fast.

You mentioned performance in VM's. ZFS RAIDZ is what I put my VM Storage pools onto. I optimize the block and record sizes to run VMs. They run faster in virtual, than most machines on metal. The performance is wild. Of course there is more 'to that', in the tuning and optimization. It is not just throw a switch and go. There are things that have to work together for that to happen.

[sthoreson]

Well... I increased all the dirty values and confirm the issue is resolved.
Part of the issue was that the use of the time in a loop was not correctly showing direct copy results after any change was made to the kernel parameters.
Here is the actual results now and a simple file copy is lightning fast again using a simple time cp command (without the loop):

Server2 Ubunt23: time cp ubuntu-20.04.6-live-server-amd64.iso test.iso

real    0m2.542s
user    0m0.024s
sys     0m2.512s

I apologize to you guys for wasting so much of your time and is my fault for not remembering the basics of what a loop does that attributed to the skewed time results from earlier in this thread.
Again, I thank you for assisting me with resolving this issue.

[MAFoElffen]

No problem at all. It keeps things entertaining! LOL I need a distraction from time-to-time.

Before you go and duck out:
So it still begs for me to ask: Why 14.04? What is your use case? Are your plans to moving something from 14.04 to 23.10?

Then why to an interim version of Server (23.10), and not to an LTS version (22.04 -> 24.04)?

Please feed my curiosity...

I'm on the Server team, so I have to ask these questions for my own "keep-in-touch" reasons to help push directions.

[sthoreson]

Here is the final resolution notes:
vi /etc/sysctl.conf
Add or Modify:
vm.dirty_background_ratio = 10
vm.dirty_ratio = 20
vm.dirty_expire_centisecs = 3000
vm.dirty_writeback_centisecs = 500
vm.dirty_background_bytes = 67108864
vm.dirty_bytes = 536870912


sysctl -p

[TheFu]

Please ----

"Thread Tools" button ---> "SOLVED"

[sthoreson]

The long story short:
Initially a server was required years ago: Installed Ubuntu 14.04
The application to which it was create for required Ubuntu 20x
We installed a new server with Ubuntu 20x and the file copies were slow.
For this test, I installed the latest Ubuntu 23 on a clean server in hopes to find the issue.

Basically, the new Ubuntu 20x server was supposed to be replace the existing one.

Thanks again for your help and patience.

[MAFoElffen]

Glad you found your answer. Thank you for sharing the use case. 20.04 goes EOL in one year. I would really look at migrating to 22.04 and get ahead of that. Especially for a production server. Most of my pushes for security are still currently for 22.04.3.

As TheFu mentioned, please select the "Thread Tools" link on the upper right of the first page of this thread, then select "Solved". That will help other Users to find what worked for you to help with their similar issues.