Confession of an amateur process manager

We would like to be witty here, but right now this topic we’re tackling is a real Wild West. Vast with many edge cases that would bleed you out every corner. But it’s that aspect that we love to be at Linux all the time.

It should be noted that ps and jobs are some of most talked tools when Linux process management is discussed. Please don’t worry that they are neglected in this talk, we will talk about them in later articles.

The shell is truly a theater if you look at it in some specific ways. One can see actors are processes, able to work in front of audiences and rehearse the show behind the scene. A process can work in the foreground if it is called explicitly by the user. Here, we are running evince to open a PDF file in the shell and while the program is running, we try to input some legitimate Linux commands.

As expected, the commands only run when evince exits.

To avoid above situation, we could put the running evince to background by starting the program, its arguments and the & symbol.

See the number enclosed by a pair of square bracket? That’s the hint to put the actor in background to the audience. That can be realized by using the command fg and the argument of said number

The kernel assign a unique number for every spawned process, called process ID or PID. If a process is put into background, i.e. not not blocking the current environtment then our shell would conveniently display its PID next to job number which is placed in a pair of square bracket (we’ll get to it later). For example, we generate a trivial process that do nothing but sleep for 10 second. The PID is shown as you can see below

It should be noted that the kernel does not possess any kind of system call to initiate a new process to run a particular program, like htop. Instead, it works like this: an existing process clone itself and that cloned run specified program. The existing process, from the POV of the clone, is now the parent and assigned a particular value called parent’s PID or PPID.

For example, the current shell has the PID of 8664 and is of zsh shell. Firing up a new sh shell and the parent’s PID from the new shell will be 8664, i.e. the zsh shell is now the parent of sh shell.

So far, the story has been pretty fast-paced. So we would like to try another approach, a slower one to inspect the life and death of a process. As said before, a process must clone itself to run a program. It achieves that by executing a fork system call, the forked process is now a child compared to the original and possess different PID. The child process would then run an exec routine to run a program as requested. When the program is terminated or reaches its lifetime naturally, the child process calls a routine named exit to notify the kernel that it is ready to die. An integer, called exit code, is supplied to mark reason it exited. But that’s not all, a child process will also need its parental approval with the latter doing a wait system call (the wait annotation in below diagram is not the system call in discussed). Sometimes, a child process could die off without telling its parent its last breath, the exit code and become a zombie process.

A machine is swarmed with processes at any moment and depends on user’s need, some processes might gravely consume for CPU computing resources reserved for others. To solve this, Linux put into concrete a marvelous idea: niceness. Each process is assigned to an integer value, called niceness, which representing a “nice” process is the one that doesn’t contend for CPU resources very highly. And thus, a process with high niceness is lower in priority queue for the CPU compared to one having low niceness.

Modern Linux distributions are shipped with highly optimized schedulers, ensuring manual setup of processes with niceness is kept at minimal. However, when situations call for, one can modify a process’s niceness with the command renice during the former’s lifetime or at the beginning with nice. It should be noted that there are versions of nice built into shells (not Bash fortunately) and thus, to every sysadmin’s sake, one should use the absolute path to nice when issue the command.

Here are we lower priority for our familiar PDF reading process then compared to the process in its normal form. We end the block with the low-prioritized process raised its niceness back to normal thanks to renice.

Signals are entities that can be used to convey message to a running process. The real meaning of sent messages are varied, potentially go from simply communication to wishes for receivers to be frozen, stopped and killed.

When a signal is received by a process, if the process’s creator had expected such signal and design functionalities to work with it then the handler is called with contextual data with the signal. Else, the kernel will take over and handle signals on behalf of the process.

A process can either ignore or block a sent signal to prevent arrival of it. Ignoring signal works as intended; signal is sent but swiftly removed and the process keeps doing what it is doing. Blockade of signal means that the process’s handler doesn’t have to readied for work unless the signal is unblocked. Moreover, handlers doesn’t have to repetitively work with waves of identical blocked signals, only once at most.

There are a few signals that anyone mingling with operations should know as much as their inner palms.

Anyone could talk reasons into troublesome process, using kill command and signals. For example, tell a PID of 123 to die

In case one want to send some stronger message, substitution p in kill -p with proper signal in integer form would work. For example, tell a PID of 123 to QUIT

Runaway processes refer to ones sucking up significantly more resources, be it CPU time, disk, bandwidth, then their normal selves would do. The possibilities for such phenomenon are almost next to endless: process that refused to exit and attempt to retry only to fail every time, process that is inefficient due to its implementation, process that fail to control its spawned processes, ..etc.

However, one might mistake between an overloaded system and a runaway one. It is best to verify the type of process before termination. User processes like a web server, database or search engine are always candidates for being heavily loaded.

Using ps or top to figure out top processes in terms of CPU time usage and load average in specified intervals, any of them being more than 1 might indicate a lurking problem. High load average could stem from CPU utilization or high I/O load; to verify either of them, it is best to check CPU utilization with top and I/O usage with iostat

For potential of processes using excessive memory, one could check it using top and observe the VIRT column, representing the amount of virtual memory allocated by each process. To be even more specific, once top is entered, manage displayed fields to enable DATA column, a column designed to show amount of memory in each process’s data and stack. The advice is to look for growth over time and absolute size.

This is barely the tip of a giant iceberg known as “process management” in the world of Linux. We hope our share of research into this domain is sufficient to entertain you readers a few minutes. Stay tune for more!

21/04/2020

Đỗ Trọng Hải

My greatest thank to the authors of Unix and Linux Administration Handbooks for the granted knowledge and the succinct layout.

Related posts:

Dukun Pel. DUKUN PELET AMPUH RADEN JANTAN SAKTI SALAH SATU DUKUN TERSAKTI DI INDONESIA | UNTUK KONSULTASI LANGSUNG DENGAN RADEN JANTAN SAKTI ….

A marketing plan is a detailed list of actions that outlines and explains what exactly needs to be done across all relevant marketing channels to achieve your global marketing goals. A marketing plan…

As a young founder and CEO, most things I do at the moment are a ‘first time’, I’m learning on the job, making mistakes and learning a lot. I’m going to start recapping my months on here. One day I…