Linux at work

Posted on |

bash environment

/etc/profile ->  hold shell environment & startup settings 
~/.bashrc  -> system wide definitions for shell funcs & alias 
~/.bash_profile -> user environment individually

ususally during coapplication configuration on Mac or Linux, mostly use ~/.bashrc to add project related environment variables.

regular expressions

  • the$ :matches the line ending with “the”
  • ^the :matches the line starting with “the”
  • [^g]abc : search “abc” but not starting with “g”
  • [^abc…z][0-9] : search numbers but not with alphabet in front
  • ^[a-z] : search the line starting with low alphabet
  • ^[^a-z]: search the line starting without low alphabet
  • ^$ : match white space
  • [xyz]: general match anyone from “xyz”
  • [!xyz]: general match in opposite, neither from “xyz”

shell expressions

  • command & : to run command in bg
  • $! : current PID
  • $# : number of arguments
  • $1 : the first paramter
  • $* : wildcard for all variables

Linux commands

  • ldd: check runtime lib
  • unix2dos / dos2unix: change file format between Windows and Linux
  • ps : print running processes in current terminal
  • ipcs : check share memory portion in current system
  • ipcrm -M ipc_key : remove the shared memory portion id by ipc_key
  • top: print virtual,resident,shared memory percentage
  • basename: e.g. basename /path/to/file -> file
  • wc -l .file: print total lines of the file
  • df: check the usage of disk
  • find . -type f -print0 | xargs -0 grep -l “xx”
  • find D:\ | grep xml
  • less .file : read the last few lines of file
  • ln item link : soft link
  • ln -s item link : hard link
  • type command : check the type of command
  • cat f1 f2 > merged
  • sort
  • uniq
  • comm f1 f2 : the common part of f1 and f2
  • diff f1 f2 : the difference of f1 and f2
  • 2 >& 1: redirect std output(fd=2) to std error(fd=1)
  • nm : check libs used in current executable(T defined, U undefined)

gdb

MPI gdb: (each MPI thread will have an independent terminal window)
mpirun -np #cpus xterm -e gdb ./exe

set breakpoint in different src:
b sth.cxx:20

print an array:
p (int[length]* a)

add input file:
set args -j input_file

load src after triggering GDB:
gdb
file exe
set args -j input_file

books about Linux

bash guide for beginners
tao of regular expression
Linux programmer’s manual
Linux system administrators guide
C expert programming
what every programmer should know about CPU caches

resources in multi-core optimization

understand CPU utilization & optimization
Intel: optimization applications for NUMA
Intel guide for developing multithreaded applications
MPI parallel programming in Python
considerations in software design for multi-core, multiprocessor architectures
how to optimize GEMM
optimize for Intel AVX using MKL with DGEMM
GEMM: from pure C to SSE optimized micro kernel
a practical guide to SSE SIMDD with C++
multi-core design
Unix and pthread programming

resources in applications

introduction to post processing finite element results with AVS
CAE Linux: FEA inter-operability
open sourcing a Python project the right way
PPSS
pyNastran
hdfviewer
valgrind

postscript

starting from 2016, I had went through each hot topic nowadays, even did some study in DL, AV, CV etc. every time the passion burst out and I promised to e.g. study a framework, or contribute an open project. in reality, the passion dies away soon. It’s like a new and very attracting concept bump out in the market, but no business model can handle it, then it dies out.

downside of this learning pattern is that the fundmental is ignored. e.g. I can’t success in code interview, few experience in basic algorithms. kind of person always vision the big, but don’t realize how to reach there. it’s kind of wasting finally, just want to be focus at this moment.