Tag: bash

Bash script to clean movies name

22 June 201722 June 2017 Fabien bash, Linux 3 Comments

The issue when you download movies online, is that most of the time the file name is going to be complete trash , filled with useless info like the team that made the rip or the website from where the movie was downloaded.
This is very annoying when you sort by name and the website made the idiotic decision to put their name in front of the actual film name.

for example here a typical movie name
[ Torrent9.info ] Dont.Think.Twice.2016.FRENCH.WEBRip.XviD-NEWCiNE.avi
the actual useful data that i wish to keep is Dont.Think.Twice.2016.avi

for this purpose i’ve developped a small bash script that will list all the files in a chosen directory then , if necessary , will rename the file.

#!/bin/bash
#the location of the folder full of movies to rename.
for file in /media/freebox/*
do
if [[ -f $file ]]; then
#that line will delete all the text after the date but keep the file extension.
nfile=$(echo $file | sed "s/\(20[0-1][0-8]\).*\(mkv\|avi\|mp4\)/\1.\2/")

#this line is going to try to delete a space a the start of the filename if it exist
nfile=$(echo $nfile | sed  "s/\/ /\//")

#this line is going to delete double dots in the filename and replace them with simple dots
nfile=$(echo $nfile | sed -e "s/\.\./\./g")

#that line is going to delete everything between [] including the [] 
nfile=$(echo $nfile | sed -e 's/\[[^][]*\]//g')


#after all this work on the filename
#you can now check if the any changes have been made , 

if [[ "$file" == "$nfile" ]]
then
   #if no change display this
   echo "Not touched : $file"
else
   #if change detected , rename the file
   echo ______
   echo Renamed :
   echo "Original name : $file"
   echo "Renamed to  : $nfile"
   echo ______
#doing the actual renaming
mv "$file" "$nfile"
fi
fi
done

this is not a very efficient script
but it’s fulfill it’s purpose and very rarely encounter a filename that he is unable to clean.

BASH:check if a number is even or odd

19 June 201719 June 2017 Fabien Linux 5 Comments

In a bash script you might need to check if a var is even or odd ,

in my case im going to use a if to make that check

#!/bin/bash
for i in $(seq 10);do
    if (( $i % 2 )); then
        echo $i is odd
    else
        echo $i is even
    fi
done

You can use this simple code to make that check and display the result.

the result will be :


1 is odd
2 is even
3 is odd
4 is even
5 is odd
6 is even
7 is odd
8 is even
9 is odd
10 is even

BASH : Make a SQL query from bash

14 June 201714 June 2017 Fabien SQL Leave a comment

MySQL is a very convenient way to store data .
To use stored MySQL data in a BASH script you can use this simple command

echo "SELECT * FROM table WHERE column=xyz | MYSQL_PWD=sqlpassword mysql database -u root -s

this will display the result in you terminal without ASCII decorations.
if you need to use that data for something , you can store it in a var

var=$(echo "SELECT * FROM table WHERE column=xyz | MYSQL_PWD=sqlpassword mysql database -u root -s)

BASH : Comparison of values

30 May 201730 May 2017 Fabien Linux Leave a comment

In a if test, you often have to compare the value of two variable
of the value a variable against a fixed value

is equal to
if [ "$a" -eq "$b" ]
if [ "$a" -eq "35" ]

not equal to
if [ "$a" -ne "$b" ]

is greater than
if ["$a" -gt "$b" ]

is greater or equal than
if [ "$a" -ge "$b" ]

is lower than
if [ "$a" -lt "$b" ]

is lower or equal than
if [ "$a" -le "$b" ]

BASH : IP validation

30 May 201730 May 2017 Fabien Linux Leave a comment

if you have a list of ip to validate ,
you can use this little bash script ,
this will open your IP list ,
then test if all the 4 numbers that make a standard IP.are beeten 0 and 255

if the ip is valid then the script will return valid , else it will return : not valid

for IP_ADDRESS in `cat ip-list.txt` ; do
#this var is a test that will see if the number is beetween 1 and 254
test='([1-9]?[0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])'
if [[ $IP_ADDRESS =~ ^$test\.$test\.$test\.$test$ ]]; then
echo "valid:     "$IP_ADDRESS
else
echo "not valid: "$IP_ADDRESS
exit
fi
done

after execution , you will get the following result :

valid:     127.0.0.1
valid:     8.8.8.8
valid:     82.48.250.6
valid:     230.230.230.230
not valid: 255.255.2301
valid:     8.8.4.4
not valid: 80.46.70.22.5
valid:     255.255.255.0
valid:     0.0.0.0

Use AWK to display x-th and y-th lines after a regex match

28 May 2017 Fabien Linux Leave a comment

In a script you might be interested to keep 2 or more lines after a regex match ,
the line that interest you are not always going to be right after the match ,
the first one might be X lines after the match and the second one Y lines afters the match

<item desc>
<name=Hammer>
<brand=none>
<weight=700>
<price=12> 
<location=A5H3>
<stock=56>

in this example , i’m interested in keeping the lines with the price and the number in stock
i’m going to use awk to search for item desc , and display the 4th line after the match, and 6th line.

awk '/item desc/{nr[NR+4]; nr[NR+6]}; NR in nr' filename

Bash : Delete some characters from a file with TR

23 May 201728 May 2017 Fabien Linux Leave a comment

in this little example of usage of the command tr
we want to clean a file so the values contained in it can be used
Example File

<10>
<23>
<34>
<78>
<44>
<12>
<40>
<81>
<30>
<11>
<14>
<09>
<02>
<66>

In my example i wish to only keep the value and deleting the character < and >

 cat file| tr -d '<>'

the tr command used with the -d (delete) option will allow you to delete the list of characters contained between the quotes.

You will obtain the following result

BASH : Delete the last X chars from a string

25 April 20177 May 2017 Fabien Linux Leave a comment

sed delete last x chars

When you scrap data in HTML , it’s common , depending on the quality of you regex to end up with a string with some useless character at the end (Or the Beginning),

If you string looks like that :

My useful data</a> <

You can use the command head to drop the X last character of your string ,
So in my Case , to Keep only the string My useful data and drop the </a> <

I’m going to pipe my string to

head -c -6

This will delete the last 6 chars of my string , giving the data that i need later in my script

but in some implementation of the head command this will not work ,

you can use an alternative with sed

sed 's/.\{6\}$//'

This will delete the last 4 characters of your line , or alternatively

sed 's/......$//'

Linux : Sort a list of IP address

19 April 20177 May 2017 Fabien Linux Leave a comment

like we saw previously in that article , it’s possible to ping an entire /24 subnet with a little convenient bash one liner.
You will get a result like this.

64 bytes from 192.168.1.21: icmp_seq=1 ttl=64 time=0.163 ms
64 bytes from 192.168.1.31: icmp_seq=1 ttl=64 time=0.360 ms
64 bytes from 192.168.1.8: icmp_seq=1 ttl=128 time=66.8 ms
64 bytes from 192.168.1.32: icmp_seq=1 ttl=64 time=25.1 ms
64 bytes from 192.168.1.42: icmp_seq=1 ttl=255 time=1.93 ms
64 bytes from 192.168.1.46: icmp_seq=1 ttl=255 time=60.8 ms
64 bytes from 192.168.1.41: icmp_seq=1 ttl=255 time=67.0 ms
64 bytes from 192.168.1.254: icmp_seq=1 ttl=64 time=0.344 ms

The Problem is , the IP adresses are going to be displayed in the order they answer the ping , and some devices are going to answer faster than other , making a list that is not in order

the sort command in her default setting will sort using the first changing character , which is going to give me a list like that : 21, 254, 31, 32, 41, 42, 46, 8

If you have installed on you host a recent version of the sort command , you will be able to use sort -V
but on older and some embedded devices the -V option is not available ,

You can then launch the sort command with these options :

sort -n -t . -k 4,4

the result will be :

64 bytes from 192.168.1.8: icmp_seq=1 ttl=128 time=0.978 ms
64 bytes from 192.168.1.21: icmp_seq=1 ttl=64 time=0.246 ms
64 bytes from 192.168.1.31: icmp_seq=1 ttl=64 time=0.383 ms
64 bytes from 192.168.1.41: icmp_seq=1 ttl=255 time=76.5 ms
64 bytes from 192.168.1.42: icmp_seq=1 ttl=255 time=2.47 ms
64 bytes from 192.168.1.46: icmp_seq=1 ttl=255 time=65.4 ms
64 bytes from 192.168.1.254: icmp_seq=1 ttl=64 time=0.290 ms

if you need to sort a /16 list of ip address :

you can just add -k 3,3 to the command :

sort -n -t .-k 3,3 -k 4,4

Bash : Reuse the last argument

19 April 20177 May 2017 Fabien Linux Leave a comment

Let’s say you are waiting for a host to come back online . to ssh into it
you launch a little

ping 192.168.1.33

To see when the host return online ,

when the host finally ping , you can ctrl+c to kill the ping , and launch

ssh !$

this will have the effect of launching ssh with the previous argument of the last command , in this case 192.168.1.33