Application profiling

For now this only works on T3-higgs, where we have interactive access.

Running your job in background, fetch PID

Please make sure that you either start your job in background :

$my-usual-script.sh &

Example :

[root@t3-higgs ~]# sleep 60 &
[2] 27309
[1]   Done                    sleep 60

PID is 27309

Or

$my-usual-script.sh # Press ctrl+z
$bg

Example :

[root@t3-higgs ~]# sleep 60

[1]+  Stopped                 sleep 60
[root@t3-higgs ~]# bg
[1]+ sleep 60 &
[root@t3-higgs ~]# ps aux | grep sleep
root     27204  0.0  0.0  58920   528 pts/7    S    11:45   0:00 sleep 60
root     27206  0.0  0.0  61168   712 pts/7    S+   11:45   0:00 grep sleep

# PID is 27204

On the next step, we will need the PID of your job's process. So make sure you get it here.

Running the profile script :

With the PID in hands, run the script already in the system's $PATH , passing as first argument the PID of your job :

[root@t3-higgs ~]# mem-profile.sh 27342

You don't need to do anything else as this small script already puts itself in background. It will terminate itself once your job stops (PID is no more).

Looking at memory profile

The script will create a log file that is updated live as it runs, so you can do anything with it at the end, or watch it live as it runs :

[root@t3-higgs ~]# tail -n200 /tmp/profile-root-27342.log -f
Started at : Mon Dec  1 11:47:48 PST 2014
Printing RSS at the interval of 3 seconds
528
528
528
528
528
528
528
528
528
528
528
528
528
528
528
528
528
528
Finished at : Mon Dec  1 11:48:43 PST 2014

Note that the filename format is /tmp/profile-$USER-$PID.log

For the curious, the output is pretty much the 6th column of the "ps aux" command :

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0  10372   668 ?        Ss   Nov21   0:01 init [3] 

-- Main.samir - 2014-12-01