Memory leaks

[worseisworser]

It's kinda hard to tell what's leaking because you do not show the entire code.

Perhaps you can deal with this yourself by using Valgrind?

[dwhitney67]

Forking a process is expensive, considering that it is a full-blown process versus that of a light-weight process (ie. thread).

I will get back to you with a full/working solution to what I was proposing earlier. It should be scalable to using any number of threads.

P.S. When I think about it more, the solution I proposed earlier probably was similar to a design I developed which would allow one or more clients (literally, users) to connect to, and interact with, a shell running under VxWorks. The VxWorks OS was being used to support the run-time environment for processes that were used to manage a satellite. So needless to say, the client and the server were far apart!

[dumbsnake]

Using fork is probably not any more expensive, in fact, it may be less expensive, however, you will lose the ability to share memory (and a bunch of other stuff too). You also can NOT mix fork() with the creation of new threads without doing some very complicated locking and other stuff.

As far as your memory leak, you're running Linux, so take advantage of all the awesome that is provided. In particular, use valgrind. There is a package in Ubuntu for it, so just install that. Then, from the command line, run your program as follows:

$ valgrind --leak-check=yes ./your_program

This will print out information about where the memory is being allocated. You may have some other problems that it detects as well, but heck, that is why valgrind is awesome!

Also, you might want to google a bit more if you need additional info on valgrind. One last thing, make sure you are compiling your program with symbolic debugging information. If you are using gcc, you'll want to add the -g option. Good luck!

[hosseinyounesi]

Thanks all of you,

1) I decided to test my program with fork too. here it is:

void SigCatcher(int n)
{
	//Avoiding Zombie
	wait3(NULL,WNOHANG,NULL);
}

while(true)
{
	Accepter = accept( ListenSocket, NULL, NULL );
	cout << "Accepted" << endl;
	if( Accepter==-1 ) {
		sprintf(logstr,"%d",errno);
		LOG_ERROR(logstr,"couldn't accept");
		continue;
	}
	pid_t pID = fork();
	if (pID == 0)// child
	{
		ProcessFrok((void*)Accepter);
		exit(0);
	}
	else if (pID < 0)// failed to fork
	{
		cerr << "Failed to fork" << endl;
		LOG_ERROR("FATAL","Failed to fork.");
	}
}

void * ProcessFrok(void * arg)
{
        SEND & RECV
        PARSING PACKET
        FREE MEMORIES
	SHUTDOWN & CLOSE ACCEPTED SOCKET
}

I executed my program, it's working now for 3 hours. I'll report any possible problem. But is there any limitation for number of forking? For example parent can just 2000 childes?!

2) You are not talking about pthreads? It seems that multithreading in linux is not like windows?! I write the same program in linux with windows threads and it's working fine without any leakage! So what's wrong with pthreads or linux?

3) I'm using Valgrind and result will be reported too.

Thanks b4

[hosseinyounesi]

Hi again,

Forking example after a while is just printing "Accepted" again and again! But no child process is created and nothing is being logged!
Now I'm really confused

I have to mention that i used this for listening:

listen(ListenSocket, SOMAXCONN)

How about using libHoard? Anyone used it before?

[worseisworser]

I can help you with pthread, but it is impossible without some code...

[dwhitney67]

I will get back to you with a full/working solution to what I was proposing earlier. It should be scalable to using any number of threads.

I've attached a tar-ball with the working framework for implementing a thread-pool. It is very basic, and it does not include any of the networking stuff that hosseinyounesi was working on. The addition of the networking stuff has been left as an exercise.

hosseinyounesi, feel free to play with this code, and add the necessary hooks to have the child and parent thread terminate gracefully; of just rely on ctrl-c as I did.

Also test using valgrind; these are the results I got:

...
==9199== LEAK SUMMARY:
==9199==    definitely lost: 0 bytes in 0 blocks
==9199==    indirectly lost: 0 bytes in 0 blocks
==9199==      possibly lost: 864 bytes in 6 blocks
==9199==    still reachable: 696 bytes in 8 blocks
==9199==         suppressed: 0 bytes in 0 blocks
...

[hosseinyounesi]

I sent the code in message number 7. Please let me know what exactly should i send.

[worseisworser]

Something that actually compiles and runs on a computer and showing/illustrating the problem would be helpful; there's a million different things not shown here that might leak.

http://www.catb.org/esr/faqs/smart-questions.html#code

The most effective way to be precise about a code problem is to provide a minimal bug-demonstrating test case. What's a minimal test case? It's an illustration of the problem; just enough code to exhibit the undesirable behavior and no more. How do you make a minimal test case? If you know what line or section of code is producing the problematic behavior, make a copy of it and add just enough supporting code to produce a complete example (i.e. enough that the source is acceptable to the compiler/interpreter/whatever application processes it). If you can't narrow it down to a particular section, make a copy of the source and start removing chunks that don't affect the problematic behavior. The smaller your minimal test case is, the better (see the section called “Volume is not precision”).

Generating a really small minimal test case will not always be possible, but trying to is good discipline. It may help you learn what you need to solve the problem on your own — and even when it doesn't, hackers like to see that you have tried. It will make them more cooperative.

People expect the same when posting bug-reports too by the way.

[hosseinyounesi]

Here is the code:

main:

mutex = (pthread_mutex_t*) malloc( MAX_THREADS*sizeof(pthread_mutex_t) );
thrd = (pthread_t*) malloc( MAX_THREADS*sizeof(pthread_t) );
/*ThreadInfo is the ret value of accept that is set as a global array for threads*/
ti = (ThreadInfo*) malloc( MAX_THREADS*sizeof(ThreadInfo) );
//initialize
for(int i=0;i<MAX_THREADS;i++)
{
	pthread_mutex_init(&mutex[i],NULL);
}
//block all threads
for(int i=0;i<MAX_THREADS;i++)
{
        pthread_mutex_lock(&mutex[i]);
}
//pool creation
for(int i=0;i<MAX_THREADS;i++)
{
	pthread_create(&thrd[i],NULL,ProcessThread,(void*)i);
	//all threads are available
	availableThreads.push_back(i);
}

while(true)
{
	Accepter = accept( ListenSocket, NULL, NULL );
	cout << "Accepted" << endl;
	if( Accepter==-1 ) {
		sprintf(logstr,"%d",errno);
		LOG_ERROR(logstr,"couldn't accept");
		continue;
	}
	if(availableThreads.size()!=0)
	{
		pthread_mutex_lock(&data_mutex);

		int index=availableThreads.front();
		availableThreads.pop_front();
		ti[index].Accepter=Accepter;

		pthread_mutex_unlock(&data_mutex);

		pthread_mutex_unlock(&mutex[index]);
	}
}

thread function:

void * ProcessThread(void * arg)
{

while(true)
{
pthread_mutex_lock(&mutex[i]);

pthread_mutex_lock(&data_mutex);
Accepter=ti[i].Accepter;
availableThreads.remove(i);
pthread_mutex_unlock(&data_mutex);

char* packet=NULL;
char* Resp=NULL;
char* respons=NULL;
char **test=NULL;
char logstr[100];
XMLNode Response,xmlpacket;
unsigned long size=0;
int i=(int)arg;
int Accepter=0;
char tmp;
struct timeval tv;

{
	tv.tv_sec = TIMEOUT;
	tv.tv_usec = 0;
	if( setsockopt(Accepter,SOL_SOCKET,SO_RCVTIMEO,(char*)&tv,sizeof(tv))==-1 )
	{
		sprintf(logstr,"%d",errno);
		LOG_ERROR(logstr,"Error in socket settings");
		goto Thread_Cleanup;
	}
}

size=0;
if(recv(Accepter,(char*)&size,sizeof(size),0)==-1) {//recieves the incoming packet's size!
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"Error in recv1");
	goto Thread_Cleanup;
}

if(send(Accepter,(char*)&tmp,1,0)==-1) {//sends ack
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"Error in send1");
	goto Thread_Cleanup;
}

packet=new char[size];
if(recv(Accepter,packet,size,0)==-1) {//recieves the packet
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"Error in recv2");
	goto Thread_Cleanup;
}
packet[size]=0;

XMLResults Xres;
if(strlen(packet)!=0)//packet is recieved as a string, so it should be parsed to xml format
	xmlpacket=XMLNode::parseString(packet,NULL,&Xres);


if(Xres.error == eXMLErrorNone)
	ProcessPacket(xmlpacket,Response);
else
{
	LOG_ERROR("XML_ERROR:",XMLNode::getError(Xres.error));
	goto Thread_Cleanup;
}

Resp = Response.createXMLString(0);
size = strlen(Resp)+1;
respons = new char[size];
strcpy(respons,Resp);

size=strlen(respons)+1;
if(send(Accepter,(char*)&size,sizeof(size),0)==-1) {//sends the size of response
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"Error in send2");
	goto Thread_Cleanup;
}

if(recv(Accepter,(char*)&tmp,1,0)==-1) {//recieves ack
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"Error in recv3");
	goto Thread_Cleanup;
}

if(send(Accepter,respons,size,0)==-1) {//sends the response
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"Error in send2");
	goto Thread_Cleanup;
}

Thread_Cleanup:
if(Resp)
	free(Resp);
if(packet)
	delete[] packet;
if(respons)
	delete[] respons;

usleep(500*1000);

if( shutdown(Accepter,SHUT_RDWR)==-1 ) {
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"couldn't shutdown socket");
}
if( close(Accepter)==-1) {
	sprintf(logstr,"%d",errno);
	LOG_ERROR(logstr,"couldn't close socket");
}


pthread_detach(pthread_self())

pthread_mutex_lock(&data_mutex);
availableThreads.push_back(i);
pthread_mutex_unlock(&data_mutex);

}
}

So this is my implementation of thread pool and socket programming. I'm using libhoard and XMLParser