Command-line Data analysis tools for Linux

Command-line Data analysis tools for Linux by Chi-Hang Lam

Posted below are PERL scripts I have written to perform simple file-based data analysis and plotting. To get help, run the command without any parameter.

Data file format:

Numerical data files must be in M x N rectangular form, i.e M lines each with N numbers separated by spaces/tabs. Multiple rectangular blocks are allowed and each should be separated by a blank line. For example, data.txt has 14 blocks of 19 x 3 numbers.

e.g.

    data.txt

Split a data file with multiple blocks separated by blank lines into multiple files: split

e.g.

    split data.txt

    split -d data.txt        (split file into a new folder)

Gnuplot command-line frontend in PERL: gp

Most functions of gnuplot is available via a single line of command.

e.g.

    gp data.txt        (plot data file using columns 1 and 2 )

    gp s data.txt l        (split file and plot in semi-log scale)

Averaging and coarsening data: av

e.g.

    split data.txt; av data.txt.s*; gp data.txt.s1        (split file, average over resulting files, and plot result)

    av -c2 data.txt; gp data.txt data.txt.av l        (average over every 2 rows, and plot both original and coarsened file)

Mathematical calculations: cal

e.g.

    cal -e 's2=s2+s3' data.txt        (add column 3 to column 2)