One of the decisions you need to take when you write a multi tasking application is whether to use processes (using fork system call) or threads (using posix library)

The main benefit of using multiple threads is the memory that is shared between them but this feature can also make problems while one thread for example smashes another thread buffer.

It is hard to debug such cases because we see garbage data but we can’t figure out which thread did that

One way to solve the problem is to add a guard page with no permission between the buffers. While using posix threads in Linux it is done automatically with the guard size attribute that is one page by default

Example:

#include<stdio.h>
#include<unistd.h>
#include<pthread.h>
#include <sys/mman.h>


void *threadfn(void *p)
{
	int val=100;
	printf("thread val address=%lx\n",&val);
	sleep(100);
	return NULL;
}


int main()
{
	int val=100;
	printf("main val address=%lx\n",&val);
	pthread_t t1,t2,t3;
	pthread_create(&t1,NULL,threadfn,NULL);
	pthread_create(&t2,NULL,threadfn,NULL);
	pthread_create(&t3,NULL,threadfn,NULL);
	sleep(100);
	pthread_exit(NULL);
	return 0;
}




/* Output:
main val address=7ffe2438515c
thread val address=7fdba3989f44
thread val address=7fdba3a8af44
thread val address=7fdba4274f44
*/

We create 3 threads and print the address of local variable (located in the stack)

Now lets see the process memory maps using cat /proc/[pid]/maps

You can see that linux thread library added one page (4k) between the threads stacks to prevent one from crapping the other. You can change the guard size by using pthread_attr_t parameter while creating a new thread and with the function  pthread_attr_setguardsize

But what about memory that is not located in the stack like dynamic allocation or buffers in BSS/DATA sections? here you can use mprotect to create that guard area

mprotect can be used to change MMU permissions on any mapped memory that belongs to the process for example:

mprotect(buffer,0x1000,PROT_READ);

here we set the buffer address page to read only, any try to write will toggle a SIGSEGV signal (segmentation fault).

One point to notice is that the buffer must be aligned to a page boundary so we can’t allocate it using malloc. we can use memalign or aligned_alloc but the best way in this case is to call mmap directly and not use any heap. mmap returns page(s) and if we use it with MAP_ANONYMOUS we get memory from the kernel

p1=mmap(0,0x20000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0);

mprotect(p1+page_size,page_size,PROT_NONE);
mprotect(p1+0x1f000,0x1000,PROT_NONE);

pthread_attr_setstack(&attr,p1+0x1000,0x1d000);
	
pthread_create(&t1,&attr,threadfn1,NULL);

In this code we first allocate 128kb (0x20000) , then we use mprotect to set the first and the last page with no permissions and make this memory region the thread stack.

if the thread overflow the buffer , the process gets a SIGSEGV and if you write a signal handler for that signal , It will run in the problematic thread context so we can catch it

Full example:

#include<stdio.h>
#include<unistd.h>
#include<pthread.h>
#include<unistd.h>
#include <sys/mman.h>

void *threadfn1(void *p1)
{
	int num;
	int val=90;
/*                       uncomment this to see the thread fault
	int *p=&val;
	printf("val addr=%lx\n",&val);
	for(num=0;num<0x500;num++){
		*p=20;
		p++;
	}
*/
	while(1){
		puts("thread1");
		sleep(10);
	}
	return NULL;
}

void *threadfn2(void *p)
{
	while(1){
		puts("thread2");
		sleep(10);
	}
	return NULL;
}


int main()
{
	void *p1,*p2;
	pthread_t t1,t2,t3;
	int page_size=sysconf(_SC_PAGESIZE);
	printf("page size=%x\n",page_size);
	pthread_attr_t attr;
	
	struct sched_param param;
	  
	pthread_attr_init(&attr);
	p1=mmap(0,0x20000,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0);

	printf("mmap ret=%lx\n",p1); // the return address is page aligned

	mprotect(p1+page_size,page_size,PROT_NONE);
	mprotect(p1+0x1f000,0x1000,PROT_NONE);

	p2=malloc(0x20000);
	printf("malloc ret=%lx\n",p2); // malloc return address is not page aligned

	pthread_attr_setstack(&attr,p1+0x1000,0x1d000);
	
	pthread_create(&t1,&attr,threadfn1,NULL);

	pthread_attr_setstack(&attr,p2,0x20000);
		
	pthread_create(&t2,&attr,threadfn2,NULL);
	sleep(100);
	puts("end test");
	return 0;
}

The result is the same but you can use this method for any buffer , not only in the stack