Linux – fork system call and its pitfalls

To create a child process in Linux/Unix , you can use clone(2) or fork(2). We use clone(2) is to implement multithreading and namespaces. We use fork(2) to create a “real” (child) process, i.e. separate address space and resources

In this post i will cover fork, its patterns and its pitfalls

Lets start with a simple example:

If you run this code you will see on the output:

You see ‘bye’ twice because fork returns twice – when we call fork it will create a child process and returns twice – one for the parent and one for the child. On the parent process , fork returns the child process id and on the child process it returns 0

Call once, returns twice – WTF???

While running a complex algorithm , you want to use more than one processor but you need to write your code using parallel patterns. fork helps you to implement a simple pattern – fork – join

For example we have a big array (in a shared memory) and we want to calculate something in each array element, with fork its easy:

As you can see, before the fork we have one process only, we create one child , the child do his job and exit (returning value to the parent) and the parent do his job and wait for the child to finish so after the if statement its again one process only

exit closes the process and send the result to the parent. The parent knows why its child terminated (normal or by signal) and the return value (or signal number) from the status parameter on wait system call (see man pages for details and macros)

What about the memory?

If we declare an array and change it after fork we can see that it is duplicated:

The child is waiting 3 seconds to make sure the parent changed the value and print the array element. the result is arr[1]=2 because each process has its own memory space. It looks like the kernel duplicate the memory on fork.

CoW (copy on write)

To save time on fork, the kernel only duplicate its mapping. For example we have 100 pages mapping for the array , the kernel will duplicate the TLB entries and change all the mapping to read only. If both processes only read, they work with shared memory but when one process try to write, it creates a page fault, the kernel trap handler copy the page and change the permission to read-write returning the program counter to the previous statement to try again.

If we measure the time of the first assignment , we will get a longer time:

To test this using Ubuntu 64 bit we add a simple assembly function

Output:

As you can see from the output , the first time we write take 3 times longer as a result of a page fault

Fail to fork?

fork fails in some situations: if we reached the maximum user processes allowed (see ulimit -a) , out of memory, no MMU and more (see man page)

It also fails if the parent process consume more than 50% of the system memory. Let take an example:

The program declares a 200MB array and clear it using memset to make it map to physical memory. As we saw, on fork the memory is not duplicate but the system check if we have the required free memory in case the child will write. If the system doesn’t have 200MB free, fork will fail

The above behaviour can create a strange bug in the following situation:

  • The parent fill a 200MB array and fork
  • On fork, the system has 250MB free – fork returns successfully
  • As a result of Copy on write mechanism , the memory is not consumed
  • Another process consume 200MB
  • The child process access the array elements and while trying to handle the page faults the kernel crashed

Lets test it on Qemu image with 512MB:

 

Code Example:

Run this program and after fork:

We can see the only 200MB consumed. Now run a simple program to consume more memory:

and check again:

Now the child write to 4MB every 30 seconds and make a copy of the pages , when it will reach the system limit (100MB only) we will see a kernel oops:

We can see the oops generated on a page fault (do_page_fault)

To avoid such cases we need to pre fault the array (read and write its content at least one element per page) immediately after fork

Files are not duplicated on fork

It is important to understand that file descriptor object are not duplicated on fork. In this way we can share resources between the child and the parent. All the anonymous objects (pipes, shared memory, etc.) can only be shared using the file descriptor. One way (the easy way) is declaring the resource before forking and the other way is sending the file descriptor using unix domain socket. If we open a regular file the child and the parent are using the same kernel object i.e. position, flags, permission and more are shared:

Example:

We open a file, read 29 chars on the parent and 29 chars on the child, the parent update the position so the child is reading where the parent ended:

Output:

 

Using fork to create tasks

Because fork is weird , it is useful to be familiar with its patterns. If we want to create tasks and we have a shared code (like RTOS) but we want to create the tasks as seperate processes (not threads) so if one will fail, it will not crash all the others. We create the tasks on a loop and then the main process will wait for child exit and re spawn it.

If you run the above program, you will see 6 processes running. The main process waiting for a child to exit (by signal or normal exit) and recreate it. If you send a signal to one of the children , you will see it re born.

Parent-child with different code

Sometimes we need 2 processes with parent and child relation (to send signals or share resource) but they have completely different code. In a wireless router for example we have a routing control process and a web server. We want the web server process to send a signal to the router control every time the configuration changes. We can use fork with execve to implement it:

For example: 2 processes using pipe for communication:

Parent app:

Compile it and call the executable parent_app

Child app:

compile it and call it child_app

Now write the code to connect them:

compile and run it (placing the previous executables in the same directory)

We create a pipe, on the parent we close the read fd and move the write fd as a parameter to the main , on the child we do the opposite

Note that in this design if we want to change the communication type to use unix domain socket (or any other object base on file descriptor) we only need to change and compile the last program

You have other important notes about fork , add it as a comment (automatically approved)

 

 

 

Tagged

Leave a Reply

Your email address will not be published. Required fields are marked *