LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-23-2011, 02:12 PM   #1
mandyapenguin
Member
 
Registered: Nov 2011
Location: India
Distribution: RedHat, Cent OS, Fedora, Debian, Ubuntu
Posts: 106

Rep: Reputation: Disabled
Command to merge linewise from 2 files


Hi..Everybody,
I am newbie in shell script, I want to know when the user password will expire. for that I made a small shell script like below
Quote:
#!/bin/bash
users=$(grep /home /etc/passwd | grep -i "sh" | cut -d: -f 1 >> users.txt)
expire_date=$(grep /home /etc/passwd | grep -i "sh" | cut -d: -f 1 | while read user; do chage -l $user | sed -n '/Last/,+2p' | grep -v Last | grep -v inactive | cut -d: -f 2 >> expire.txt; done)
echo $users $expire_date
It will create users.txt and expire.txt files and format will be like below
Quote:
cat users.txt
user1
user2
user3
cat expire.txt
Dec 23, 2011
Oct 25, 2012
May 31, 2011
Now I need a command in the same shell script which will create a newfile and arrange the format like below
Quote:
user1 Dec 23, 2011
user2 Oct 25, 2012
user3 May 31, 2011
So please help me how it can be done?

Last edited by mandyapenguin; 11-23-2011 at 02:15 PM. Reason: correction
 
Old 11-23-2011, 04:06 PM   #2
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
First things first. We have GOT to fix up what you already have.

Code:
users=$(grep /home /etc/passwd | grep -i "sh" | cut -d: -f 1 >> users.txt)
There are three main problems here.

1) It unnecessarily uses multiple tools to do what one instance of the proper tool can do. awk is designed precisely for doing this kind of job.

2) You're sending all the output of the commands into users.txt, which means that the users variable gets nothing.

3) Even if we fix #2, multiple outputs should not be stored in a single variable. This is a job for an array.


Code:
users=( $( awk -F':' '/\/home.*sh/ { print $1 }' /etc/passwd ) )
There are probably also better ways to get a list of user names, but we won't worry about that now. Don't worry about exporting it to a text file yet either. We'll take care of that later.

Code:
expire_date=$(grep /home /etc/passwd | grep -i "sh" | cut -d: -f 1 | while read user; do chage -l $user | sed -n '/Last/,+2p' | grep -v Last | grep -v inactive | cut -d: -f 2 >> expire.txt; done)
Holy freaking cow. Congratulations. This has got to be the most convoluted single command I've ever seen. Why do you go to the trouble to use sed to grab three lines, only to use grep to remove two of them again? All you really want is the "Password expires" line, don't you? So why not just grab that line directly?

Anyway, if you have the user list properly stored in an array, then it becomes trivial to get the date you want.

Code:
for i in "${!users[@]}"; do

     expire_date[i]=$( chage -l "${users[i]}" | grep '^Password expires' )
     expire_date[i]=${expire_date[i]##*:}

done
I could've used awk again instead of grep, but I decided to go with parameter substitution instead. No need for overkill.

Now you have matching array indexes for name and date, and can easily echo them to text files individually or as a whole.

Code:
for i in "${!users[@]}" ; do

	echo "${users[i]}" >> users.txt
	echo "${expire_date[i]}" >>expire.txt
	echo "${users[i]} ${expire_date[i]}" >>list.txt

done
This can even be done in the same loop that sets the date array, of course.

Check out the BashGuide for a good intro to scripting. You'll want to read the section on arrays in particular:

http://mywiki.wooledge.org/BashGuide

Last edited by David the H.; 11-23-2011 at 04:11 PM. Reason: fixed minor coding mistake
 
1 members found this post helpful.
Old 11-23-2011, 04:07 PM   #3
Juako
Member
 
Registered: Mar 2010
Posts: 202

Rep: Reputation: 84
You could do both in one pass,

Code:
while read user; do echo "user: $user, expires: $(chage -l $user | sed -rn '2 s/^.*: (.*)$/\1/p')"; done< <(sed -rn 's/^([^:]*):.*$/\1/gp' /etc/passwd)
The command starts at the end:
Code:
< <(sed -rn 's/^([^:]*):.*$/\1/gp' /etc/passwd)
Filters the names and sends the output to the while ... done loop.

Each while...loop round reads one name, and echoes the name followed by a command substitution where 1) chage -l is called and 2) sed filters the last part after the : on the second line (the "expires" line).

Aka: no need for intermediate files.

Last edited by Juako; 11-23-2011 at 04:09 PM.
 
1 members found this post helpful.
Old 11-23-2011, 10:10 PM   #4
mandyapenguin
Member
 
Registered: Nov 2011
Location: India
Distribution: RedHat, Cent OS, Fedora, Debian, Ubuntu
Posts: 106

Original Poster
Rep: Reputation: Disabled
know the user's expiration date

Quote:
#!/bin/bash
users=( $( awk -F':' '/\/home.*sh/ { print $1 }' /etc/passwd ) )
for i in "${!users[@]}"; do
expire_date[i]=$( chage -l "${users[i]}" | grep '^Password expires' )
expire_date[i]=${expire_date[i]##*:}
done
for i in "${!users[@]}" ; do
echo "${users[i]}" >> users.txt
echo "${expire_date[i]}" >> expire.txt
echo "${users[i]} ${expire_date[i]}" >> list.txt
done
Excellent, Thank you very much master. You saved me my lot of time in struggling to make the format in new file that I had requested like below.
Quote:
cat list.txt
user1 Dec 23, 2011
user2 Oct 25, 2012
user3 May 31, 2011
Now I need to segregate only those users which are already expired and going to expire which user expire date is less than 15 days from today.
Hope you definitely help me in this.
once again thank you very much master.

Last edited by mandyapenguin; 11-23-2011 at 10:12 PM. Reason: Heading correction
 
Old 11-24-2011, 01:55 PM   #5
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Glad to help out.

Please use [code][/code] tags around your code, to preserve formatting and to improve readability. Do NOT use quote tags, which don't protect whitespace.

Your new requirement is a bit more of a challenge, but not too hard, as long as you have the gnu version of date (standard on almost all Linux distros*). With it, you can convert the dates to epoch format (seconds passed since Jan 1, 1970). This allows you to do simple numeric comparisons between different times.

Code:
now=$( date +%s )
soon=$( date -d "15 days " +%s )

for i in "${!expire_date[@]}" ; do

     userdate=$( date -d "${expire_date[i]}" +%s 2>/dev/null )


     if [[ ${expire_date[i]} == "never" ]]; then

          echo "Password for ${user[i]} is set to never expire"

     elif (( userdate < now )); then

          echo "Password for ${user[i]} has expired"

     elif (( userdate <= soon )); then

          diff=$(( ( userdate - now ) / 60 / 60 / 24 ))
          echo "Password for ${user[i]} will expire in $diff days"

     else

          echo "Password for ${user[i]} will not expire soon"

     fi

done
Note that date's -d option can handle many "natural English" style entries, but it's not perfect. It's possible that there could be input strings that it can't understand. My current username is set to "never", for example, which is why I had to set date to discard errors (with 2>/dev/null), and included a specific test for it. If there are other possible values, make sure the script can handle them as well.

By the way, to convert an epoch number back to another format, you have to prefix it with "@"

Code:
datestring='1322159922'
date -d "@$datestring" "+%b %d, %Y"

*The date implementations on other unixes generally don't have -d, so if you aren't on Linux you'll probably have to find some other tool for converting arbitrary dates, such as perl.
 
1 members found this post helpful.
Old 11-25-2011, 02:23 AM   #6
salemhouda
LQ Newbie
 
Registered: Nov 2011
Posts: 8

Rep: Reputation: Disabled
Try this command
Code:
$paste users.txt expire.txt > merged.txt
 
Old 11-26-2011, 10:59 PM   #7
mandyapenguin
Member
 
Registered: Nov 2011
Location: India
Distribution: RedHat, Cent OS, Fedora, Debian, Ubuntu
Posts: 106

Original Poster
Rep: Reputation: Disabled
Code:
while read user; do echo "user: $user, expires: $(chage -l $user | sed -rn '2 s/^.*: (.*)$/\1/p')"; done< <(sed -rn 's/^([^:]*):.*$/\1/gp' /etc/passwd)
Super, Excellent master.

Code:
#!/bin/bash
while read user; do echo "user: $user, expires: $(chage -l $user | sed -rn '2 s/^.*: (.*)$/\1/p')"; done< <(sed -rn 's/^([^:]*):.*$/\1/gp' /etc/passwd) | grep -v never > list.txt
This is what I really expected.

Code:
bash pe
echo $?
0
cat list.txt
user: kumar, expires: Dec 23, 2011
user: roopa, expires: Oct 25, 2012
user: user1, expires: May 31, 2011
user: user2, expires: Dec 24, 2011
Code:
chage -l roopa | sed -rn '2 s/^.*: (.*)$/\1/p'
Oct 25, 2012
Code:
sed -rn 's/^([^:]*):.*$/\1/gp' /etc/passwd
root
daemon
bin
sys
sync and so on
Could you please explain for these two
sed -rn '2 s/^.*: (.*)$/\1/p'
sed -rn 's/^([^:]*):.*$/\1/gp' /etc/passwd

Or could you please send me some url links by which I can be also like you in shell scripts.

Last edited by mandyapenguin; 11-27-2011 at 12:08 AM. Reason: correction
 
Old 11-26-2011, 11:50 PM   #8
mandyapenguin
Member
 
Registered: Nov 2011
Location: India
Distribution: RedHat, Cent OS, Fedora, Debian, Ubuntu
Posts: 106

Original Poster
Rep: Reputation: Disabled
get mail about password expiration

H..David master.
Sorry for the late response. I could not see your kind reply since I was at out of station.
Please excuse me. I am spoiling your valuable time with this thread. You are really genius.
Code:
bash pass_expire
Password for kumar will not expire soon
Password for roopa has expired
Password for user1 will not expire soon
Password for user2 will expire in 3 days
Code:
date
Sun Nov 27 10:21:56 IST 2011
Code:
chage -l user2
Last password change					: Nov 01, 2011
Password expires					: Dec 01, 2011
Password inactive					: never
Account expires						: never
Minimum number of days between password change		: 0
Maximum number of days between password change		: 30
Number of days of warning before password expires	: 7
The last one thing I would like to expect from you. Is it possible to send mails once in a day via crontab to related people to change their password whose account is going to expire within 15 days with this below message
Code:
Hi.(related username)
The password for the user account "related_account_name_which_is_going_to_expire" is going to expire. Please change the password before it get expired. To change the passowrd follow the below steps
1) ssh username@IP_address
2) passwd  (give current passowrd and update with new one. Don't use previous and simple password
3) chage -l $USER (note down the next expiration date and change it before it get expired using same above steps
4) exit
Sometimes one mail ID will be having many accounts, for example
kumar=kumar@domain.com
roopa=roopa@domain.com
user1=user@domain.com
user2=user@domain.com
Now the user1 and user2 is having the same mail ID i.e user@domain.com. Now the user@domain.com should get mail to change only for user2 account. This is my last question in this thread. I hope you definitely help me in this.

Last edited by mandyapenguin; 11-27-2011 at 12:17 AM. Reason: heading correction
 
Old 11-27-2011, 06:33 AM   #9
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
This may be the first time I've ever been called a "genius". But everything posted so far is well-understood by anyone with a reasonable amount of scripting experience. You'll get there too, eventually.

sed, grep and many other commands use regular expressions to parse out sections of text. There are many good tutorials available both online and in printed form. Just find one that looks good to you and study it until you at least understand the basic techniques, such as the lines juako gave. You'll be glad you did.

As for mailing out the results, it's certainly not hard to do. But I'd probably set it up as a separate mail script that launches the first, or at least reads the output file from it, and sends the messages out based on that. Then just set cron to launch that script once per day.

I'm not too familiar with cli mail programs though, so you'll have to figure out the exact commands yourself. Post what you come up with here and we'll be glad to look it over and give advice.
 
1 members found this post helpful.
Old 11-27-2011, 09:14 AM   #10
mandyapenguin
Member
 
Registered: Nov 2011
Location: India
Distribution: RedHat, Cent OS, Fedora, Debian, Ubuntu
Posts: 106

Original Poster
Rep: Reputation: Disabled
Hi..David master, Thanks for your kind reply.
Before adding body message.
Code:
bash pass_expire 
Password for roopa will expire in 11 days.
Password for user2 will expire in 1 days.
Password for user3 has expired
Password for user4 has expired
Please find the attachment. I added body message. Now the helpdesk members can get 4 mails i.e 2 about going to expire and other 2 about expired. Meanwhile the related accounts are not able to get mails. Please help me to send mails to both i.e helpdesk and the related accounts. And if I add "chage -l $USER" in the body message it showing the currently logged in user in the mails instead of "$USER". I also tried chage -l "$USER", but still it showing currently logged in user in the related field. So at last I put "chage -l username" only in the body message.
Attached Files
File Type: txt pass_expire.txt (1.5 KB, 11 views)

Last edited by mandyapenguin; 11-27-2011 at 09:31 AM.
 
Old 11-29-2011, 02:18 PM   #11
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
Well, as I said, I don't know much about how mail works, so I don't really know what you need to do to get it to send the right things to the right people.

I'm also not sure what you mean by sending messages to both "helpdesk and related". It's not clear to me who is "helpdesk" and who is "related". Could you elaborate please?

The $USER variable is preset by the shell to the name of the user who owns the shell. In this case it's the name of the user running the script, and the value is expanded before the message is sent. You need to use the ${users[i]} array value for the name you want to supply to the message. But then again you should know that, as you're already doing it on the first line.


In any case I think the script would be cleaner, and the text more easily modifiable, if you used a variable or function to supply the default message. A function would probably be the most flexible:

Code:
printmess(){
	#$1 is expired/expsoon, $2 is the username

	case $1 in
		expired) echo "Password for $2 has expired." ;;
		expsoon) echo "Password for $2 will expire in $diff days." ;;
	esac
	echo "To change the password follow the below steps"
	echo "   1) ssh username@IP_address"
	echo "   2) passwd  (type current password and update with new one, Don't use previous and simple password)"
	echo "   3) chage -l username (note down the next expiration date and change it before it get expired using the same above steps."
	echo "   4) exit"
	echo "Regards"
	echo "root@domain"
}


for i in "${!expire_date[@]}" ; do

	userdate=$( date -d "${expire_date[i]}" +%s 2>/dev/null )
	if (( userdate < now )); then
		printmess expired "${users[i]}" | mail -s "password expire" "$admins" "$users"
	elif (( userdate <= soon )); then
		diff=$(( ( userdate - now ) / 60 / 60 / 24 ))
		printmess expsoon "${users[i]}" | mail -s "password expire" "$admins" "$users"
	fi
done
 
1 members found this post helpful.
Old 12-01-2011, 10:16 AM   #12
mandyapenguin
Member
 
Registered: Nov 2011
Location: India
Distribution: RedHat, Cent OS, Fedora, Debian, Ubuntu
Posts: 106

Original Poster
Rep: Reputation: Disabled
Hi.. David master, Thanks for your kind reply.
You are helping me a lot. Thank you very much.
Please excuse me, if you could not understand from my explanation.
You are already near to that what I am expecting. I don't want you to confuse with helpdesk members, and you just leave it.
Here I will tell clearly.
The related users are nothing but the mail IDs for the user accounts from the passwd file, and those mail IDs for the user accounts as below.
For example
Code:
roopa=roopa@domain.com
kumar=kumar@domain.com
Please see these below
Code:
Password for roopa will expire in 2 days.
Password for kumar has expired.
Now mail ID called "roopa@domain.com" should get mail only about roopa account and other mail ID i.e "kumar@domain.com" account should get mail about kumar account.
Now if I add as only "mail -s "password expire" "$users" the related mail IDs are not getting mail. Some mail ID will get mails only then, when system user account and webmail account both are same and only if I add as
Code:
mail -s "password expire" "${users[i]}@domainname
but some user accounts will be having their project name also like kumar_proj_name at that time they will not get mails since the mail ID itself does not exist.
I can also get mail if I add like below
admins=mymailID@domain.com
"mail -s "password expire" "$admins" "${users[i]}@domainname"
Please find the attachment, I have removed also "$admins" because again I don't want you to get confuse with $admins and $users. It is enough, if the related mail IDs are getting mails about related accounts.
I hope, now you are not confused and help me in this.
Attached Files
File Type: txt pass_expire.txt (1.3 KB, 12 views)

Last edited by mandyapenguin; 12-01-2011 at 01:06 PM. Reason: correction
 
Old 12-01-2011, 09:16 PM   #13
David the H.
Bash Guru
 
Registered: Jun 2004
Location: Osaka, Japan
Distribution: Debian sid + kde 3.5 & 4.4
Posts: 6,823

Rep: Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957Reputation: 1957
I'm still not completely sure I follow you.

I think I see your main issue now though. If "users" is defined as an array, then "$users" is the same as "${users[0]}". So you're sending everything to the same address. If you want the full array list, you need to use "${users[@]}". Unless of course you want to send it to that user only, in which case use "${users[i]}".

For the second part, are you saying that some users have to have their messages sent to different addresses from the default?

If so, we'd have to use some kind of a test to compare the username to a preset list, and send the message to a different address if there's a match. Assuming you're using bash 4+, an associative array would probably be the best option.

Code:
#define a list of exceptions to the default mail addresses
declare -A xpns
xpns=(
	[roopa]='roopa@domain.com'
	[kumar]='kumar@domain.com'
)
It might be a good idea instead to save the list to a separate "config" file and source it, so you don't have to edit the script itself (You can also store other default values in it, like "IP").

Code:
### external file (exceptions.txt) ###
xpns[roopa]='roopa@domain.com'
xpns[kumar]='kumar@domain.com'

### script ###
#define a list of exceptions to the default mail address.
#import values from the file "exceptions.txt"
declare -A xpns
if [[ -r exceptions.txt ]];
	. exceptions.txt
else
	echo "Warning: can't find or read the exceptions file!" >&2
fi
Then in the loop, you can easily test to see if the current name is in the exception list.

Code:
if [[ "${!xpns[*]}" =~ "${users[i]}" ]]; then
	address="${xpns[${users[i]}]}"
else
	address="${users[i]}"
fi

printmess expired "${users[i]}" | mail -s "password expire" "$address"
(This is assuming again that I understand the mail command correctly. The last argument is supposed to be either the username or full address to send to, correct?)

It's even possible to shorten the full if test to this:

Code:
address="${xpns[${users[i]}]:-${users[i]}}"
The last argument means it will use the name from the exception list if it exists, and defaults to to the username if it doesn't (it's the ${var:-default} substitution pattern).
 
1 members found this post helpful.
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Split and merge files from the command line LXer Syndicated Linux News 1 10-10-2012 03:43 AM
[SOLVED] awk command to merge two files silkysue Linux - Newbie 7 01-27-2011 11:14 AM
awk command to merge columns from two separate files into single file? johnpaulodonnell Linux - Newbie 4 01-23-2007 11:10 AM
Is there a command to merge two files as two columns of one file? davee Linux - General 2 07-19-2005 11:52 AM


All times are GMT -5. The time now is 12:35 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration