Page 2 of 2 FirstFirst 12
Results 11 to 15 of 15

Thread: Improve the cope/paster data capability in Lubuntu?

  1. #11
    Join Date
    Jun 2021
    Beans
    12

    Re: Improve the cope/paster data capability in Lubuntu?

    Quote Originally Posted by monkeybrain20122 View Post
    So I copied one column with 1048576 rows form my test csv in Cacl and paste it to the text editor just as an experiment. I don't experience the problem you described. Copy was instantaneous with no lockup at all , paste took longer and text editor turned grey for a little while indicating high usage of memory but that lasted for only a few seconds and there was no systrm lockup even when the text editor window was grey, I could still switch to Firefox. I have only 6 G of ram on this machine running Ubuntu 20.04 (with unity desktop, haven't tested it on gnome) My column consists of random gibberish (long strings with both alphabets and numerals)
    So I am confused by the computer performance which is not high enough as expected. At least it is a more efficient by using gnumeric to handle the current job.
    The software in long-time working occupies a heavy load in RAM : https://imgur.com/a/qTrn2oG

  2. #12
    Join Date
    Jun 2021
    Beans
    12

    Re: Improve the cope/paster data capability in Lubuntu?

    Quote Originally Posted by TheFu View Post
    IDK, but does copying just the values, not the formatting make it work faster?
    Also, are non-ASCII characters used? Just another point of information - if characters are unicode, they could be 4x larger.

    For anything larger than 500 rows, I'd look for a batch solution and avoid the GUI completely.
    Yes. I had an attempt to visualize the giant data by matplotlib coding.

  3. #13
    Join Date
    May 2010
    Beans
    1,457

    Re: Improve the cope/paster data capability in Lubuntu?

    Could use a clipboard manager like Parcellite and so on

  4. #14
    Join Date
    Mar 2010
    Location
    Squidbilly-Land
    Beans
    Hidden!
    Distro
    Ubuntu

    Re: Improve the cope/paster data capability in Lubuntu?

    For huge datasets, avoid the GUI completely.
    Scripts can pull data from spreadsheets as CSV. Then that CSV can be fed into Qtiplot on the command line. According to the manual,
    The name can also refer to an ASCII file:


    qtiplot ASCII_file_name

    In this latter case, a new "untitled" project will be created, containing a table or matrix with the ASCII data from the file. The file is read and interpreted using the current settings from the Import -> Import ASCII... command dialog.
    That would bypass the clipboard and any limitations it may have. I don't have a clipboard. Prefer to use the X-buffer instead, but I don't know if libreOffice or Gnumeric will work with that. Could quickly try a "Copy ..." --> values only test. I suspect the extra formatting is getting in the way.

    As for tools that can read spreadsheets - every scripting language like perl, python, ruby, has a module for that. Avoiding the GUI should make it 10x faster, at least. It might be instantaneous.

  5. #15
    Join Date
    Jun 2021
    Beans
    12

    Re: Improve the cope/paster data capability in Lubuntu?

    Quote Originally Posted by TheFu View Post
    For huge datasets, avoid the GUI completely.
    Scripts can pull data from spreadsheets as CSV. Then that CSV can be fed into Qtiplot on the command line. According to the manual,


    That would bypass the clipboard and any limitations it may have. I don't have a clipboard. Prefer to use the X-buffer instead, but I don't know if libreOffice or Gnumeric will work with that. Could quickly try a "Copy ..." --> values only test. I suspect the extra formatting is getting in the way.

    As for tools that can read spreadsheets - every scripting language like perl, python, ruby, has a module for that. Avoiding the GUI should make it 10x faster, at least. It might be instantaneous.
    Yes. This is a good way to import huge datasets by qtiplot. It works! Because of the operation habit, I mostly import the data by GUI and copy/paste operation. 'import ASCII' is very good function, avoiding system stuck during copy/paste. The command 'qtiplot xxx.txt' also works, it will import the single file and plot automatically. It is suitable to deal with just one file instead of a project.

Page 2 of 2 FirstFirst 12

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •