Command-Line on Raveen Kumar

rsync is weird, does not sync "tags" directory

Fri, 24 Feb 2023 00:00:00 +0000

For whatever reason rsync decided not to sync tags directory since it is ignored in CVS mode. Took a lot time to figure out the problem. Finally this command works correctly.

--include used to sync everything, despite ofthe cvs option, -C. But removed in this command.

	rsync -av --delete --checksum --copy-links -rPuzhi ./public/ root@raveenkumar.xyz:/var/www/raveenkumar/

Programming

Sun, 19 Feb 2023 00:00:00 +0000

AWK

Sat, 18 Feb 2023 00:00:00 +0000

Example 1

awk -F ',' '{print $NF}' {{filename}}

To print only rows with columns matching a specific string

awk '$3 == "hello"' file

This will match with, 1 2 hello something but not with, 1 2 world something

Example 2: Awking and cutting some stuff

Lets assume that you want to print the 1st and 3rd field in one line.

echo "A:B:C:D:E:F:G" | awk -F ':' 'ORS=" " {print $1} {print $3}'

Here ORS stands for, The output record separator, by default its a newline.

Cat

Sat, 18 Feb 2023 00:00:00 +0000

Join multimedia files

cat movie.mpeg.0* > movie.mpeg
cat *mp3 > combined.mp3

cp

Sat, 18 Feb 2023 00:00:00 +0000

Dereference / follow and copy symbolic links

cp -L source dest

COPY ONLY FILE THAT DO NOT EXIST IN THE DESTINATION DIRECTORY - UPDATE

cp -u *.html destination

Cut

Sat, 18 Feb 2023 00:00:00 +0000

Cut from 25th column

https://superuser.com/questions/316785/is-there-a-diff-utility-that-allows-you-to-exclude-columns

cut -c25- file.txt

Cut fields

echo "1:2:3" | cut -d ":" -f 1

1

du

Sat, 18 Feb 2023 00:00:00 +0000

Find size of all files and directories under current directory and sort it based on size

du -sh * | sort -h

Locate

Sat, 18 Feb 2023 00:00:00 +0000

Description

Locate is a alternative to find for files. Locate uses updatedb command to update its database. Note that we must run the updatedb command everytime there is file system change.

Example

Create a database

updatedb

Locate files

locate filename

Create a database locally

updatedb_path="$HOME/.local/locate_db"
updatedb_home="$updatedb_path/home.db"
updatedb -l 0 -o "$updatedb_home" -U ~/

Locate files for locally created database

locate -d "$updatedb_home"

pandoc

Sat, 18 Feb 2023 00:00:00 +0000

https://pandoc.org/MANUAL.html

pandoc -o output.html input.txt

Powershell notes

Sat, 18 Feb 2023 00:00:00 +0000

Winget

To install any program using powershell in similar way to linux use, winget

Example:

winget install pandoc

Sed

Sat, 18 Feb 2023 00:00:00 +0000

To replace text

sed 's/text/replacement_text/g' <file_name>

To replace text and write file

sed -i 's/text/replacement_text/g' <file_name> ** Append to beginning of a file sed -i ‘1i text’ file.txt * back

Find and replace a specific line from all files in current directory matching a specific extension

Here, all .md files have

<a id="org55550fd"></a>

Line in it. The following command deletes those lines from all the files.

find . -iname "*.md" -exec sed -i '/<a id=*/d' {} \;

TMUX

Sat, 18 Feb 2023 00:00:00 +0000

The greatest samurai to ever exist?

TMUX plugins

GitHub - tmux-plugins/tpm: Tmux Plugin Manager

GitHub - tmux-plugins/tmux-urlview: Quickly open any url on your terminal window!

GitHub - tmux-plugins/tmux-resurrect: Persists tmux environment across system restarts.

To move a pane from another window to current window

join-pane -s target-window:target-pane

To use 256 colours in Tmux

256 colours in TMUX

You should set the default terminal in .tmux.conf to be screen-256color,

set -g default-terminal "screen-256color"

bc

Fri, 17 Feb 2023 00:00:00 +0000

bc as a hex calculator

obase=16
ibase=16
9+1

bc fractional

scale=10
1/3

Evaluating expressions from commandline

bc <<< "ibase=10; obase=16; $(grep cru_spare -r | awk -F "," '{print $NF}')"

Diff

Fri, 17 Feb 2023 00:00:00 +0000

To diff 2 files

diff file1 file2

To diff 2 files and view side by side

diff -y file1 file2

Example pstree command

Fri, 17 Feb 2023 00:00:00 +0000

pstree -u username -hp

ImageMagick

Fri, 17 Feb 2023 00:00:00 +0000

Adding blur effect to images convert input.jpg -blur 0x8 output.jpg

Convert a series of images to gifs convert -delay 100 -loop 0 image* animation.gif

Less

Fri, 17 Feb 2023 00:00:00 +0000

Usage

less filename

Chop long lines with `less`

less -S filename Note that -S at the end doesnot work

Linux notes general

Fri, 17 Feb 2023 00:00:00 +0000

Commands

cat - contatenate
tac - concatenate in reverse

`column` command

The column utility formats its input into multiple columns.

printf "a:b:c\n1::3\n" | column -t -s ':'

a	b	c
1	3

Note that this output is different from one that is printed in terminal. Strange… Below is the terminal output.

a	b	c
1		3

`uuidgen` universal unique id generator

Generate unique ID’s

`script` Records terminal

man script
man scriptreplay

Example

script --timing=file.tm script.out
# Write some scripts
# Exit with `exit` or `^d`
scriptreplay --timing file.tm --typescript script.out

`reset` Recover a borked terminal

Count number of files in a directory

ls -1 | wc -l

Building from source

./configure
./configure --prefix=$path

Redirections

2>&1
2>&1 > /dev/null
>
>/dev/null
>>

Keymaps

Ctrl-c
Ctrl-d
Ctrl-l

Readline

C-x C-e to edit current commant in default text editor
or v in vi mode

`ldd` Print shared libraries

This command can be used to find all the shared library or libraries of a program

Lockfile

Fri, 17 Feb 2023 00:00:00 +0000

`lockfile`

Installation

sudo apt install procmail

Description

man lockfile

Suppose you want to make sure that access to the file “important” is serialised, i.e., no more than one program or shell script should be allowed to access it. For simplicity’s sake, let’s suppose that it is a shell script. In this case you could solve it like this:

...
lockfile important.lock
...
access_"important"_to_your_hearts_content
...
rm -f important.lock
...

Now if all the scripts that access “important” follow this guideline, you will be assured that at most one script will be executing between the lockfile and the rm commands.

paste

Fri, 17 Feb 2023 00:00:00 +0000

print 2 files side by side

paste file1 file2

-d {{delimiter}}
-d''

Ranger file manager

Fri, 17 Feb 2023 00:00:00 +0000

Managing files from commandline might be cumbersome time to time. To make the job easy there are quit a lot of terminal curses based file managers available like, ranger which uses Vi keybindings.

Installation

pip install ranger-fm

usage

ranger

keybindings

Official pages

https://ranger.github.io/documentation.html https://github.com/ranger/ranger

Tabbed mode

Ctrl-n
<Tab> key to switch between tabs

Show hidden files

Ctrl-h
zh

To cd into current curectory after exit

https://superuser.com/questions/1043806/how-to-exit-the-ranger-file-explorer-back-to-command-prompt-but-keep-the-current

alias ranger='ranger --choosedir=$HOME/.rangerdir; LASTDIR=`cat $HOME/.rangerdir`; cd "$LASTDIR"'

Open Multiple files in vim from ranger

Select all the files with ranger

rsync

Fri, 17 Feb 2023 00:00:00 +0000

Usage

rsync -P source dest

This is the command that I used for syncing this website

# For whatever reason rsync excludes tags directory, so --include used
rsync -av --delete --checksum --copy-links -rPuzhi ./public/ root@raveenkumar.xyz:/var/www/raveenkumar/

Example - Directory mirroring

rsync -PCahzvu --delete $ORIG_PATH/$ORIG_DIR $MIRROR_PATH/.

-P -C: Skip revision control files
-a: Archive 
-h: Human readable
-z: zip
-v: Verbose
-u: Update
--delete: Delete directories and files which  is present in mirror directory but not in original directory.

To exclude

--exclude=”\*/file.txt”

https://superuser.com/questions/555799/how-to-setup-rsync-without-password-with-ssh-on-unix-linux

seq

Fri, 17 Feb 2023 00:00:00 +0000

http://www.compciv.org/topics/bash/variables-and-substitution/

seq 1 5
echo $(seq 1 5)
# Or, to create 5 new directories:
# mkdir $(seq 1 5)

Slurm Job manager

Fri, 17 Feb 2023 00:00:00 +0000

To allocate 4 CPU cores and 8 GB of RAM for an emacs process using the srun command, you can use the following command:

srun --cpus-per-task=4 --mem=8192 emacs

The --cpus-per-task option specifies the number of CPU cores to allocate, and the --mem option specifies the amount of memory in megabytes.

To pass the -nw argument to emacs when using the srun command, you can simply include it after the emacs command in the srun command line. For example:

ssh

Fri, 17 Feb 2023 00:00:00 +0000

Cd to a particular dir after ssh

$ ssh -t ostechnix@192.168.225.52 'cd /home/ostechnix/dir1 ; bash'

To automatically install a ssh public key on a remote machine

ssh-copy-id -i ~/.ssh/id_rsa.pub $USER@$HOSTNAME

To manually install a ssh public key on a remote machine

cat ~/.ssh/id_rsa.pub; # copy the contents of the file; paste into ssh/authorized_keys file;
chmod 600 ~/.ssh/authorized_keys

Unix find

Fri, 17 Feb 2023 00:00:00 +0000

To exclude files or perform reverse match

find . -not -iname "file"

To perform regex matching

find . -regex ".*file"

To find and print only symbolic links in a directory and its subdirectories

find . -type l -print

Zip

Fri, 17 Feb 2023 00:00:00 +0000

Basic usage

zip -r my_folder.zip my_folder

For max compression

zip -9 -r my_folder.zip my_folder

If you want to split the zip file into multiple parts of a certain size, you can use the -s option. For example, to split the zip file into 1 GB parts, you would type:

zip -s 1g -r my_folder.zip my_folder

This will create files named my_folder.zip, my_folder.z01, my_folder.z02, etc. To unzip them, you need to use the unzip command with the first file only 2:

Command-line ChatGPT

Wed, 15 Feb 2023 00:00:00 +0000

Here I have written a python script that will take user input string and ask OpenAI text generator to generate text.

https://raw.githubusercontent.com/raveensrk/dotfiles-main/main/stow/.scripts/chatgpt.py

I should probably try writing a elisp code for this.

How to merge pdf file in command-line

Wed, 19 Oct 2022 00:00:00 +0000

pdfunite pdf1.pdf pdf2.pdf out.pdf

pdfunite ~/my_repos/raveenkumar.xyz/Blog/linux_and_programming/vimrefcard.pdf ~/my_repos/raveenkumar.xyz/Blog/linux_and_programming/vimrefcard.pdf 2022-10-19-out.pdf

How to create qrcodes in command-line

Fri, 30 Sep 2022 00:00:00 +0000

Install qrencode

sudo dnf install qrencode

Run the following command.

qrencode -o "test.png" "Hello World!"
# xdg-open test.png

qrencode -o "test2.png" "This is \
        a\
        multiline\
        qr-code\
      with a lot of text arsithraishtraihtsirts\
    rastarstnorastirahtensrahtierahts\
  arsnthierashtiersnhtearhtsierhsat\
ariesthresnth"
# xdg-open test2.png

How to download large files with onedrive on linux

Thu, 29 Sep 2022 00:00:00 +0000

I wanted to download about 25 GB size files on linux from onedrive but it kept hanging and failing after 50 MB sometimes it can download about 2.5 GB but still fails in-between. Tried different browsers but still it keeps failing.

These are the following steps to use linux client to download large files,

sudo dnf install onedrive

Instructions to use

https://github.com/abraunegg/onedrive/blob/master/docs/USAGE.md

https://abraunegg.github.io/

This works well. But still slow.

How to find out what display server I am using

Thu, 15 Sep 2022 00:00:00 +0000

https://unix.stackexchange.com/questions/236498/how-to-get-information-about-which-display-server-is-running

ps -e | grep tty

Start your commands with a comma

Tue, 13 Sep 2022 00:00:00 +0000

https://rhodesmill.org/brandon/2009/commands-with-comma/

Command-Line on Raveen Kumar

rsync is weird, does not sync "tags" directory

Programming

AWK

Example 1

Example 2: Awking and cutting some stuff

Cat

Join multimedia files

cp

Dereference / follow and copy symbolic links

COPY ONLY FILE THAT DO NOT EXIST IN THE DESTINATION DIRECTORY - UPDATE

Cut

Cut from 25th column

Cut fields

du

Find size of all files and directories under current directory and sort it based on size

Locate

Description

Example

Create a database

Locate files

Create a database locally

Locate files for locally created database

pandoc

Powershell notes

Winget

Sed

To replace text

To replace text and write file

Find and replace a specific line from all files in current directory matching a specific extension

TMUX

TMUX plugins

To move a pane from another window to current window

To use 256 colours in Tmux

bc

bc as a hex calculator

bc fractional

Evaluating expressions from commandline

Diff

To diff 2 files

To diff 2 files and view side by side

Example pstree command

ImageMagick

Less

Usage

Chop long lines with less

Linux notes general

Commands

column command

uuidgen universal unique id generator

script Records terminal

Example

reset Recover a borked terminal

Count number of files in a directory

Building from source

Redirections

Keymaps

Readline

ldd Print shared libraries

Lockfile

lockfile

Installation

Description

paste

print 2 files side by side

Ranger file manager

Installation

usage

keybindings

Official pages

Tabbed mode

To cd into current curectory after exit

Open Multiple files in vim from ranger

rsync

Usage

This is the command that I used for syncing this website

Example - Directory mirroring

To exclude

seq

Slurm Job manager

Chop long lines with `less`

`column` command

`uuidgen` universal unique id generator

`script` Records terminal

`reset` Recover a borked terminal

`ldd` Print shared libraries

`lockfile`