Category: Linux

Use AWK to display x-th and y-th lines after a regex match

In a script you might be interested to keep 2 or more lines after a regex match ,
the line that interest you are not always going to be right after the match ,
the first one might be X lines after the match and the second one Y lines afters the match

in this example , i’m interested in keeping the lines with the price and the number in stock
i’m going to use awk to search for item desc , and display the 4th line after the match, and 6th line.

Bash : Delete some characters from a file with TR

in this little example of usage of the command tr
we want to clean a file so the values contained in it can be used
Example File

In my example i wish to only keep the value and deleting the character < and >

the tr command used with the -d (delete) option will allow you to delete the list of characters contained between the quotes.

You will obtain the following result

UNPROTECTED PRIVATE KEY FILE fix

If you try to connect using a key system on ssh instead of a classic password login, you might encounter this error message.

this message tells you that because other users can read the content of the key , shh refuse to use it to try authentification on you host ssh.

to fix that problem you should change the permissions on you file so that only you can read it

the 3 numbers in the chmod command are like this
the First is for you
the second is for your group .
and the last is for other users.

If you want to limit the access of others to you ssh key
you could do a

after this command , you can relaunch you ssh command , and you should connect without providing a password.

Re-apply .bashrc

After you made some changes on your .bashrc file (like creating some new aliases) You will notice that the changes are not applied upon the saving of the .bashrc file

After having saved the file , first insure you are in your home folder, by typing cd .

and to reapply your .bashrc config file , launch a :

after this you can test you alias right away , it should work

 

BASH : Delete the last X chars from a string

sed delete last x chars

When you scrap data in HTML , it’s common , depending on the quality of you regex to end up with a string with some useless character at the end (Or the Beginning),

If you string looks like that :

My useful data</a> <

You can use the command head to drop the X last character of your string ,
So in my Case , to Keep only the string My useful data and drop the </a> <

I’m going to pipe my string to

This will delete the last 6 chars of my string , giving the data that i need later in my script

but in some implementation of the head command this will not work ,

you can use an alternative with sed

This will delete the last 4 characters of your line ,  or alternatively

 

Linux : Sort a list of IP address

like we saw previously in that article , it’s possible to ping an entire /24 subnet with a little convenient bash one liner.
You will get a result like this.

The Problem is , the IP adresses are going to be displayed in the order they answer the ping , and some devices are going to answer faster than other , making a list that is not in order

the sort command in her default setting will sort using the first changing character , which is going to give me a list like that : 21, 254, 31, 32, 41, 42, 46, 8

If you have installed on you host a recent version of the sort command , you will be able to use sort -V
but on older and some embedded devices the -V option is not available ,

You can then launch the sort command with these options :

the result will be :

if you need to sort a /16 list of ip address :

you can just add -k 3,3 to the command : 

Display your current external IP address with WGET

To know your ip address , you can use the command ifconfig
You will know the ip address for each of your network interfaces.

But if you are not directly connected to the internet and use a Gateway that’s nating the internet traffic for you , ifconfig is not going to be helpful to know the ip that connect you to the internet.

with this command the current external ip address associated with you default gateway will display

bash : Put the source of a webpage in a variable

When you do some bash scripting , this is often useful to get data from  webserver because they are a very simple way to exchange data from computer from computer.
to put the data that’s available on a webserver into a $var you just have to use the command :

this will fetch the data from http://server.com/file.htm in a non verbose way and put it in variable named var .

you can now do whatever you want with that $var , like greping it for example.

 

Extract values separated by character ( : , . – )

In a bash script if you find yourself with values presented that way

you might want to keep only the second or third of this values that are separated by the : character

like always , in bash there is a lot of ways to accomplish that , but i like to use a one that use the AWK utility because it’s very easy to read and understand.

my list of value is contained in a var name $var let’s extract the third value which is 18.5

Will return the value 18.5

but i can do some math operation in the utility, like :

this will return 326.5 the result of 345 – 18.5

awk is a fast and powerful text editor , there is a lot things you can do with it.