Results 1 to 10 of 10

Thread: How to force all cores to work

  1. #1
    Join Date
    Nov 2010
    Beans
    12

    How to force all cores to work

    Hi,

    When I work with very big files, all simple commands takes a long time and by default my machine seems to work with only one core (Intel Corel i5 480 2.67Ghz) so I have 2 questions:

    1. How to force my machine to work with all cores when I use commads or scripts?

    2. Is there a problem if I force all cores?

    Thanks.

  2. #2
    Join Date
    Feb 2009
    Location
    Belgrade, Serbia
    Beans
    70
    Distro
    Ubuntu 10.04 Lucid Lynx

    Re: How to force all cores to work

    Are you sure that all of your cores are detected? That's not an uncommon problem.

    Cheers,
    George

  3. #3
    Join Date
    Nov 2010
    Beans
    12

    Re: How to force all cores to work

    Quote Originally Posted by Tiganjero View Post
    Are you sure that all of your cores are detected? That's not an uncommon problem.

    Cheers,
    George
    If all of my cores are detected? sorry but I don't know, when I run a command working with this hard file, I saw for example, core1: 90%, core2-4: ~20-30%, I guess If my machine work with all its cores close to 100%, the work will end faster.

  4. #4
    Join Date
    Nov 2007
    Beans
    450

    Re: How to force all cores to work

    That depends on if the programs you use are programmed for parallel processing. Some are, but that depends on what you're doing. Having multiple cores allows a process to run full speed on a core or two while the system still remains responsive.

    It might be possible to use more cores but you'd have to tell us what you're trying to do.

  5. #5
    Join Date
    Feb 2012
    Location
    el_GR
    Beans
    Hidden!
    Distro
    Ubuntu 14.04 Trusty Tahr

    Re: How to force all cores to work

    Quote Originally Posted by whitethorn View Post
    that depends on if the programs you use are programmed for parallel processing.
    +1

  6. #6
    Join Date
    Nov 2010
    Location
    India
    Beans
    4,839
    Distro
    Ubuntu Development Release

    Re: How to force all cores to work

    Hi actually you can force applications as up to what limit they have to use by using cpulimit.

    but if much applications are running then more CPU will be used . less Apps Less CPU . whats the use of forcing the CPU to high use even you're not running in that range .


    If you wanna work with all your CPU , play a big sized game . that will take i hope .

    all the best .
    Google First your issue , if no solution then ask.

    రాజ శేఖర్ రెడ్డి

  7. #7
    Join Date
    Oct 2006
    Location
    New York
    Beans
    1,118
    Distro
    Xubuntu 12.10 Quantal Quetzal

    Re: How to force all cores to work

    This is actually a more complicated question for larger files than it is for many small files. For many small files my general solution is to use xargs or parallel to farm out the smaller jobs.

    For example, say I want to find all .gz files that contain the word "ubuntuforums".

    The slow way (this runs zgrep on each .gz file found):
    Code:
    find . -name "*.gz" -exec zgrep -l "ubuntuforums" {} \;
    But since I don't care about the order of the returned list I can use xargs to do things in your case about twice as fast:
    Code:
    find . -name "*.gz" | xargs -P 2 -n 10 zgrep -l "ubuntuforums"
    This splits the list of .gz files into chunks of 10 and starts two different instances of zgrep passing 10 files to each, when an instance of zgrep finishes it starts a new one. The OS will take care of putting one on each core.

    And finally you can use parallel to accomplish the same:
    Code:
    parallel zgrep -l "ubuntuforums" -- `find . -name "*.gz"`
    Parallel is cool in that it determines how many jobs to start. It processes each argument individually (at least by default).

    If you're running a script on a large file you could plausibly split the file in two and run the command on each half and then combine the results. But that sounds like it's going to involve trading your old problem (slow) for a new one (how to combine results from the two halves of the file run separately). If you're thinking of something like image processing, the software has to be written in such a way as to take advantage of multiple cores, there's no way to split processes across multiple cores. So short of getting your hands dirty (as in absolutely filthy, disassembling and reassembling the program) there's no easy way.

    And the simple answer to your second question, there's nothing wrong with making all your cores to work. Your fan might spin up pretty high, but it shouldn't be an issue.
    xubuntu minimal, extensive experience, lshw: http://goo.gl/qCCtn
    blog: http://goo.gl/yLg78
    Linux viruses: http://goo.gl/6OCKA

  8. #8
    Join Date
    Nov 2010
    Beans
    12

    Re: How to force all cores to work

    Well, for example my file has 20 GB, is only text very simple but very hard, so when I use very simple command (grep, cat, wc) the work take a long time. My machine have 4 cores, but the command use only one (seems) and I want to work with 3 cores.

  9. #9
    Join Date
    Jan 2011
    Beans
    1,151

    Re: How to force all cores to work

    sounds like you need a better harddrive
    (\ /)
    (O.o)
    (> <)
    This is Bunny. Copy Bunny into your signature to help him on his way to world domination.

  10. #10
    Join Date
    Feb 2012
    Beans
    29

    Re: How to force all cores to work

    Quote Originally Posted by Trotel View Post
    Well, for example my file has 20 GB, is only text very simple but very hard, so when I use very simple command (grep, cat, wc) the work take a long time. My machine have 4 cores, but the command use only one (seems) and I want to work with 3 cores.
    Thats not related to the CPU.

    Its called having an high IOWAIT. Check something like iotop to see if your maxing your HD throughput.
    If you are, consider getting 10K or SSD drives. with SATA/SAS.

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •