Using mprotect system call to debug memory problems

One of the decisions you need to take when you write a multi tasking application is whether to use processes (using fork system call) or threads (using posix library)

The main benefit of using multiple threads is the memory that is shared between them but this feature can also make problems while one thread for example smashes another thread buffer.

It is hard to debug such cases because we see garbage data but we can’t figure out which thread did that

One way to solve the problem is to add a guard page with no permission between the buffers. While using posix threads in Linux it is done automatically with the guard size attribute that is one page by default

Example:

We create 3 threads and print the address of local variable (located in the stack)

Now lets see the process memory maps using cat /proc/[pid]/maps

You can see that linux thread library added one page (4k) between the threads stacks to prevent one from crapping the other. You can change the guard size by using pthread_attr_t parameter while creating a new thread and with the function  pthread_attr_setguardsize

But what about memory that is not located in the stack like dynamic allocation or buffers in BSS/DATA sections? here you can use mprotect to create that guard area

mprotect can be used to change MMU permissions on any mapped memory that belongs to the process for example:

here we set the buffer address page to read only, any try to write will toggle a SIGSEGV signal (segmentation fault).

One point to notice is that the buffer must be aligned to a page boundary so we can’t allocate it using malloc. we can use memalign or aligned_alloc but the best way in this case is to call mmap directly and not use any heap. mmap returns page(s) and if we use it with MAP_ANONYMOUS we get memory from the kernel

In this code we first allocate 128kb (0x20000) , then we use mprotect to set the first and the last page with no permissions and make this memory region the thread stack.

if the thread overflow the buffer , the process gets a SIGSEGV and if you write a signal handler for that signal , It will run in the problematic thread context so we can catch it

Full example:

The result is the same but you can use this method for any buffer , not only in the stack

 

 

 

Tagged ,

2 thoughts on “Using mprotect system call to debug memory problems

  1. Hi Liran
    Thanks for your articles, I find them very useful, keep uploading useful/interesting stuff.
    In case I am not wrong please see my following remarks

    Should change the image from:
    1K – 120K – 1K
    to
    4k -120k – 4k

    also, I think you should change the code:
    from:
    mprotect(p1+page_size,page_size,PROT_NONE);
    to:
    mprotect(p1,page_size,PROT_NONE);

    and
    pthread_attr_setstack(&attr,p1+0x1000,0x1d000);
    to
    pthread_attr_setstack(&attr,p1+0x1000,0x1e000); // 0x1e000 == 120k

    I hope I am not misleading you, so please verify above.

    1. you are right in 2/3 remarks
      The second fix is not correct because the stack is going from upper addresses down
      Thanks

Leave a Reply

Your email address will not be published. Required fields are marked *