LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-30-2013, 01:48 AM   #1
rushikeshgaradade
LQ Newbie
 
Registered: Mar 2013
Posts: 7

Rep: Reputation: Disabled
Regarding .sh and variable declaration


I have one directories.sh file which contains a variable as below:

DFS_NAME_DIR="/grid/hadoop/hdfs/namenode /grid1/hadoop/hdfs/namenode"

and many more similar varibles they have declared. and it is said to execute it

i have executed it with
#bash dierctories.sh

but i didnt find any affect of it


and also it is said

mkdir -p $DFS_NAME_DIR

but this command showing error:

mkdir: missing operand
Try `mkdir --help' for more information.



Does this mean that directories.sh file is not executed. or is there any other alternative for "DFS_NAME_DIR"

Please help me.........
 
Old 03-30-2013, 01:55 AM   #2
rushikesh24
LQ Newbie
 
Registered: Mar 2013
Posts: 11

Rep: Reputation: Disabled
ya i found this step in HDP installation... i am also facing the same problem... i have also tried the mkdir by placing the whole path instead of variable $DFS_NAME_DIR ut it doesnt work..
 
Old 03-30-2013, 02:03 AM   #3
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
The string contains two directories. Is that what they want? The perhaps should have named the variable DFS_NAME_DIRS instead to avoid confusion.

Anyway, you need to source the file instead:
. ./directories.sh

Then try "echo DFS_NAME_DIRS" to verify that the variable is set.
 
Old 03-30-2013, 02:21 AM   #4
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Run your script with following debugging mode:
Code:
~$ bash -xv directories.sh
OR
~$ sh -xv directories.sh
OR, add following line just below #!/bin/sh in your script and then execute it:
Code:
#!/bin/sh
set -xv
......
......
It will then show you step by step execution of the script.

Also in following line, remove -p flag. (it means that dir. does not exist):
Code:
mkdir $DFS_NAME_DIR
 
Old 03-30-2013, 02:24 AM   #5
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
You don't want to remove -p since the parent directories do not exist.
 
Old 04-01-2013, 04:19 AM   #6
rushikesh24
LQ Newbie
 
Registered: Mar 2013
Posts: 11

Rep: Reputation: Disabled
thnks shivaa... i have done it as u mentioned... some of the folders are created but some are not get created...how? Is there any condition for that...
 
Old 04-01-2013, 04:27 AM   #7
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Quote:
Originally Posted by rushikesh24 View Post
... i have done it as u mentioned... some of the folders are created but some are not get created...how? Is there any condition for that...
Share what you got when you run script with set -xv? Or, post what's in your script.
 
Old 04-01-2013, 04:29 AM   #8
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,

if you want help you need to post exactly what is happening. Saying:

Quote:
some of the folders are created but some are not get created...how?
doesn't provide enough information. You're making people guess. Unless you post exactly what the problem is, it is unlikely anyone can help you.

Eg. exactly what is this script? How are you running it? What output is there? What do you expect to happen?

Putting the relevant part of output of your terminal session into the forum in [code] tags is usually a good start.

HTH,

Evo2.
 
Old 04-01-2013, 04:56 AM   #9
rushikesh24
LQ Newbie
 
Registered: Mar 2013
Posts: 11

Rep: Reputation: Disabled
i have a directories.sh file which contains a variable as below:

DFS_NAME_DIR="/grid/hadoop/hdfs/namenode /grid1/hadoop/hdfs/namenode"

HADOOP_CONF_DIR="/etc/hadoop/conf"

HDFS_PID_DIR="/var/run/hadoop/hdfs"

and many more variables they hav declared. I have run this directories.sh file and "/grid/hadoop/hdfs/namenode" these folders get created automatically however the folders "/var/run/hadoop/hdfs" are not get created. what will be the cause for it?
 
Old 04-01-2013, 05:05 AM   #10
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,

Quote:
Originally Posted by rushikesh24 View Post
however the folders "/var/run/hadoop/hdfs" are not get created. what will be the cause for it?
Without seeing the script or its output we can only guess. Here goes:

1. There is no line in the script like "mkdir $HDFS_PID_DIR"
2. The script exits before calling "mkdir $HDFS_PID_DIR"
3. The user running the script does not have permission to write to /var/run/hadoop/hdfs
4. The directory /var/run/hadoop does not exist and/or mkdir is called without -p
5. etc...
6. etc...
7. Ad infinitum

Evo2.
 
Old 04-01-2013, 05:40 AM   #11
turboscrew
Member
 
Registered: Apr 2009
Location: Nokia (town), Finland
Distribution: Sabayon, Debian, Puppy (Racy), Slackware
Posts: 548

Rep: Reputation: 45
Also, when you run a script, a new shell is started to run it.
You need to "source" the scipt to run it with current shell.
Also the new variables are not passed to the new shell unless you "export" them.
 
Old 04-02-2013, 05:21 AM   #12
rushikesh24
LQ Newbie
 
Registered: Mar 2013
Posts: 11

Rep: Reputation: Disabled
thanks evo2

1.My script doesnt contain command "mkdir".
2.In the script there is line like DFS_NAME_DIR="/grid/hadoop/hdfs/namenode /grid1/hadoop/hdfs/namenode". After running the script, i am supposed to run a command in a terminal as
# mkdir -p $DFS_NAME_DIR
after running above command oit displays following:
mkdir: missing operand
Try `mkdir --help' for more information.
3.after running above command successfully i am supposed to run next command
# chmod -R 755 $DFS_NAME_DIR

I also want to ask that after assigning a variable in a script and after running that script, can we get that variable value by using command # echo $variable_name. I have tried it but i was unable to get the value by echo command, does it mean that variable doesnt holding the assigned value permanantly?
 
Old 04-02-2013, 05:28 AM   #13
shivaa
Senior Member
 
Registered: Jul 2012
Location: Grenoble, Fr.
Distribution: Sun Solaris, RHEL, Ubuntu, Debian 6.0
Posts: 1,800
Blog Entries: 4

Rep: Reputation: 286Reputation: 286Reputation: 286
Quote:
1.My script doesnt contain command "mkdir".
Then it will not create a directory. You should add it, as:
Code:
mkdir $DIRNAME
Quote:
2.In the script there is line like DFS_NAME_DIR="/grid/hadoop/hdfs/namenode /grid1/hadoop/hdfs/namenode". After running the script, i am supposed to run a command in a terminal as
# mkdir -p $DFS_NAME_DIR
after running above command oit displays following:
mkdir: missing operand
Try `mkdir --help' for more information.
Simply running mkdir will not create a directory, but you need to provide directory name as argument to mkdir command, as said above.

Quote:
3.after running above command successfully i am supposed to run next command
# chmod -R 755 $DFS_NAME_DIR
I also want to ask that after assigning a variable in a script and after running that script, can we get that variable value by using command # echo $variable_name. I have tried it but i was unable to get the value by echo command, does it mean that variable doesnt holding the assigned value permanantly?
Yes, you can display any variable name using echo, as:
Code:
echo $VARIABLE_NAME
If it doesn't show anything, then it means there's nothing assigned to variable VARIABLE_NAME.

Note: Without checking full script, it's not easy to suggest anything. Also as said earlier, invoke your script with set -xv and post script's output here.

Last edited by shivaa; 04-02-2013 at 05:32 AM.
 
Old 04-02-2013, 05:41 AM   #14
rushikesh24
LQ Newbie
 
Registered: Mar 2013
Posts: 11

Rep: Reputation: Disabled
MY directories.sh file is as follow: after running this script i am supposed to run rest of the commands like
# mkdir -p $DFS_NAME_DIR
# chmod -R 755 $DFS_NAME_DIR

Pls hav a look in following script:

#!/bin/sh

#
# Directories Script
#
# 1. To use this script, you must edit the TODO variables below for your environment.
#
# 2. Warning: Leave the other parameters as the default values. Changing these default values will require you to
# change values in other configuration files.
#

#
# Hadoop Service - HDFS
#

# Space separated list of directories where NameNode will store file system image.
DFS_NAME_DIR="/grid/hadoop/hdfs/namenode /grid1/hadoop/hdfs/namenode"

# Space separated list of directories where DataNodes will store the blocks.
DFS_DATA_DIR="/grid/hadoop/hdfs/datanode /grid1/hadoop/hdfs/datanode /grid2/hadoop/hdfs/datanode"

# Space separated list of directories where SecondaryNameNode will store checkpoint image.
FS_CHECKPOINT_DIR="/grid/hadoop/hdfs/secondarynamenode /grid1/hadoop/hdfs/secondarynamenode /grid2/hadoop/hdfs/secondarynamenode"

# Directory to store the Hadoop configuration files.
HADOOP_CONF_DIR="/etc/hadoop/conf"

# Directory to store the HDFS logs.
HDFS_LOG_DIR="/var/log/hadoop/hdfs"

# Directory to store the HDFS process ID.
HDFS_PID_DIR="/var/run/hadoop/hdfs"

#
# Hadoop Service - MapReduce
#

# Space separated list of directories where MapReduce will store temporary data.
MAPREDUCE_LOCAL_DIR="/grid/hadoop/mapred /grid1/hadoop/mapred /grid2/hadoop/mapred"

# Directory to store the MapReduce logs.
MAPRED_LOG_DIR="/var/log/hadoop/mapred"

# Directory to store the MapReduce process ID.
MAPRED_PID_DIR="/var/run/hadoop/mapred"

#
# Hadoop Service - Hive
#

# Directory to store the Hive configuration files.
HIVE_CONF_DIR="/etc/hive/conf"

# Directory to store the Hive logs.
HIVE_LOG_DIR="/var/log/hive"

# Directory to store the Hive process ID.
HIVE_PID_DIR="/var/run/hive"

#
# Hadoop Service - Templeton
#

# Directory to store the Templeton configuration files.
WEBHCAT_CONF_DIR="/usr/lib/hcatalog/conf"

# Directory to store the Templeton logs.
WEBHCAT_LOG_DIR="/var/log/webhcat"

# Directory to store the Templeton process ID.
WEBHCAT_PID_DIR="/var/run/webhcat"

#
# Hadoop Service - HBase
#

# Directory to store the HBase configuration files.
HBASE_CONF_DIR="/etc/hbase/conf"

# Directory to store the HBase logs.
HBASE_LOG_DIR="/var/log/hbase"

# Directory to store the HBase process ID.
HBASE_PID_DIR="/var/run/hbase"

#
# Hadoop Service - ZooKeeper
#

# Directory where ZooKeeper will store data.
ZOOKEEPER_DATA_DIR="/grid1/hadoop/zookeeper/data"

# Directory to store the ZooKeeper configuration files.
ZOOKEEPER_CONF_DIR="/etc/zookeeper/conf"

# Directory to store the ZooKeeper logs.
ZOOKEEPER_LOG_DIR="/var/log/zookeeper"

# Directory to store the ZooKeeper process ID.
ZOOKEEPER_PID_DIR="/var/run/zookeeper"

#
# Pig
#

# Directory to store the Pig configuration files.
PIG_CONF_DIR="/etc/pig/conf"

#
# Hadoop Service - Oozie
#

# Directory to store the Oozie configuration files.
OOZIE_CONF_DIR="/etc/oozie/conf"

# Directory to store the Oozie data.
OOZIE_DATA="/var/db/oozie"

# Directory to store the Oozie logs.
OOZIE_LOG_DIR="/var/log/oozie"

# Directory to store the Oozie process ID.
OOZIE_PID_DIR="/var/run/oozie"

# Directory to store the Oozie temporary files.
OOZIE_TMP_DIR="/var/tmp/oozie"

#
# Hadoop Service - Sqoop
#
SQOOP_CONF_DIR="/etc/sqoop/conf"
 
Old 04-02-2013, 07:20 AM   #15
evo2
LQ Guru
 
Registered: Jan 2009
Location: Japan
Distribution: Mostly Debian and Scientific Linux
Posts: 5,753

Rep: Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288Reputation: 1288
Hi,

that script, as posted, does nothing but set some variables that will go out of scope as soon as it exits. Ie it does absolutely nothing. Is that really the whole script?

Evo2.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] C++ multiple variable declaration error, even after I set #ifndef preprocessor chinho Programming 4 01-18-2011 04:37 AM
declaration a variable in one file and initializing in another C program jamesbon Programming 5 11-25-2010 07:06 AM
Variable Declaration in shell scripts... vinaytp Linux - Newbie 4 02-04-2010 10:39 AM
Need to extract name of C variable from C declaration. judgex Programming 3 09-22-2007 12:46 PM
Using Flex and Bison to parse the variable declaration section in Pascal unreal07 Programming 1 03-10-2007 08:25 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:49 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration