Page 3 of 3 FirstFirst 123
Results 21 to 27 of 27

Thread: C programming files

  1. #21
    Join Date
    Nov 2012
    Beans
    43

    Re: C programming files

    Yes, it works now but I didn't use the free() function. I need to free the buffer but I'm not sure at which part of the code to do it? The very end, before return?

  2. #22
    Join Date
    Jun 2007
    Location
    Maryland, US
    Beans
    6,288
    Distro
    Kubuntu

    Re: C programming files

    Quote Originally Posted by binaryperson View Post
    Yes, it works now but I didn't use the free() function. I need to free the buffer but I'm not sure at which part of the code to do it? The very end, before return?
    Where you currently have it, just before the return statement, is fine.

    Btw, I wanted to add that allocating a large amount of space is probably not the best approach. You should consider reading from the source file in chunks of 1024 bytes (or something reasonable) until you fulfill the amount of bytes you need to write to the destination file.

    You would want to avoid scenarios in which a large file (say 4 GB) is split into 1 part!
    Last edited by dwhitney67; January 4th, 2013 at 02:42 PM.

  3. #23
    Join Date
    Nov 2012
    Beans
    43

    Re: C programming files

    How difficult would it be to modify this code to have each new part created in a separate thread? So, 3 parts, 3 threads that run at the same time. Would it take a lot of modifying? Would writing each file in a separate thread be equivalent to or even better than the chunking you were talking about?
    Last edited by binaryperson; January 4th, 2013 at 03:13 PM.

  4. #24
    Join Date
    Jun 2007
    Location
    Maryland, US
    Beans
    6,288
    Distro
    Kubuntu

    Re: C programming files

    Quote Originally Posted by binaryperson View Post
    How difficult would it be to modify this code to have each new part created in a separate thread? So, 3 parts, 3 threads that run at the same time. Would it take a lot of modifying? Would writing each file in a separate thread be equivalent to or even better than the chunking you were talking about?
    I could be wrong, but I do not think that accessing a file descriptor is thread-safe. If I am correct, then you would need to open the file in each thread which might negate the time savings of just using a single thread. Within each thread you would need to use lseek() to adjust the file descriptor to the appropriate location within the file to start reading. The start location would need to be passed to the thread.

    I won't design the app for you, but in essence you would need to let the thread know of the source file name, the start position for reading, and the part-number to create the output file. A data structure would be handy to store all this data.

    As for writing, this is a non-issue because you would be creating distinct files.

  5. #25
    Join Date
    Feb 2009
    Beans
    1,469

    Re: C programming files

    Quote Originally Posted by binaryperson View Post
    How difficult would it be to modify this code to have each new part created in a separate thread? So, 3 parts, 3 threads that run at the same time. Would it take a lot of modifying? Would writing each file in a separate thread be equivalent to or even better than the chunking you were talking about?
    Splitting into threads is not going to speed up disk access. You would be much more likely to hurt your performance than help it.

    You should know what they say about premature optimization...

  6. #26
    Join Date
    Nov 2012
    Beans
    43

    Re: C programming files

    How can I write a program that joins different files into one? Each file part is named file.extension.part0,file.extension.part1 and so on. One requirement is that you can't tell the program how many file parts there are. You just tell it the original name of the file: file.extension. Somehow it must realize how many parts there are and put them together.
    Last edited by binaryperson; January 7th, 2013 at 04:46 AM.

  7. #27
    Join Date
    Aug 2011
    Location
    47°9′S 126°43W
    Beans
    2,172
    Distro
    Ubuntu 16.04 Xenial Xerus

    Re: C programming files

    Quote Originally Posted by binaryperson View Post
    How can I write a program that joins different files into one? Each file part is named file.extension.part0,file.extension.part1 and so on. One requirement is that you can't tell the program how many file parts there are. You just tell it the original name of the file: file.extension. Somehow it must realize how many parts there are and put them together.
    [code]
    If the parts files are sortable by name (ie; you have *part01-*part09 if there is a part10):
    Code:
    cat file.extension.part*  > file.extension
    (bash guarantees the "file.extension.part*" will be replaced by an alphabetically sorted list)

Page 3 of 3 FirstFirst 123

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •