Create call tree for shell scripts

bkelly · 05-15-2018, 10:53 AM

Please remember that this is the newbie forum.
My first question was declared not clear so an edit is merited. And edit again with this comment: I still may not have described this well so I will show my work so far.
Note to readers: I have made major edits to this question so answers that look inappropriate were appropriate when I started this.

There is a directory called MCS with multiple sub directories. Within those subdirectories are multiple script files. My understanding is that each of those scripts end with the phrase .sh.

I need to discover which scripts are called by other scripts. Presume that t1.sh is called by t2.sh. The first tentative goal is to end up with a file that contains something like:

t1.sh is called by
t2.sh
t6.sh

t2.sh is called by
t7.sh

In my bash script I have

Quote:

find ./ -type f $ -name "*sh" $ > list1.txt

That produces a nice list of all the files with the full pathname. I found a discussion on baseline. Next I need (think I need?) a list of the file names without the paths so I try this:

Quote:

find ./ -type f $ -name "*sh" $ | baseline list2.txt

But the baseline does not appear to accept a pipe input because the list2.txt file is twice as long and contains the full path.

So, to the best of my ability: How do I pipe a list of file names with full path into baseline and get the file name out?
Keeping the .sh is preferred but I will take what I can get.

AwesomeMachine · 05-15-2018, 11:14 AM

You'll need explain more what you're doing. "There's a bunch of scripts that call each other" just doesn't mean much.

BW-userx · 05-15-2018, 11:24 AM

so you got a bunch of scripts and within them scripts they call another script. Now you're hoping there is an easy way to run a program that will figure out what scripts call which scripts and put them in a list of some sort.

Quote:

Now I need to parse out the files names like: t2.sh and make a list of those, then I can run a grep to find all the scripts then call t2.sh. What utility is best for that operation?

you'd have to not use a wildcard but the actual pattern you're looking for to get all of the t2.sh but they cannot be in the same directory, nor would I have named them all t2.sh. I do not use .sh

I do not think there is an easy way.

Code:

find /path -type f -iname "*.sh" > firstList

then look at your firstList then decided how to proceed. to look inside of the files.
there is more than one way to do this.

Code:

grep -rwn /path or /file -e "pattern"

shows all matches to the pattern. It looks like you already know how to redirect stdout to a file.

rtmistler · 05-15-2018, 11:29 AM

AwesomeMachine (Edit add: and also BW-userx) are correct, you'll have to describe your intentions more clearly. However I wonder that once you do, you'll no longer have a question.

Never tried exactly this, but find can -exec what it finds. I use the -exec option a ton, but usually to mv, cp, or rm or something, not directly execute a found file, but .... why not? Do a quick test.

As far as filtering what you want and don't want, find will also do that by way of filename search string criteria and/or exclude filters (which I rarely use, but they are supported).

The next problem becomes if any of these scripts require passing arguments.

And then my whole background thought of, "I would not search my system for .sh files and then run them all!" Or at the very least I'd do a dry run and verify the list of what I found matched what I intended for the result.

Additional edit:
What BW-userx is talking about is something I've done and why I've learned to use the -exec option of find 'ere long ago:

Code:

find . -name "pattern" -exec grep <string> {} /dev/null \;

That will search within the files found for the string specified.

bkelly · 05-15-2018, 04:06 PM

Milestones are rather slow for me. This script

Quote:

find /home/users/MCS -type f $ -name "*.sh" $ > list1.txt
#
cat list1.txt | while read LINE;
do
basename "${LINE}";
done

Gets all the files names, puts them into list1.txt
It then shows all the files names without the paths in the terminal window.
I have not been able to send that output of just the file names to file list2.txt so I can then work that list.

Quote:

basename "${LINE}" > list2.txt;

does not work. Suggestions as to where to look for this answer please.

BW-userx · 05-15-2018, 04:27 PM

here try this you'll have to add in what your looking for

Code:

#!/bin/bash

working_dir="/media/userx1/userx/Dropbox/linux/scripts"

find "$working_dir"  -type f  > list1.txt
#
cat list1.txt | while read LINE;
do
  if [[ $(grep -rwn "$LINE" -e "move_to") ]] ; then
  {
    echo "${LINE##*/}" >> text2
  }
  fi
done

it skips a step and finds what you're looking for using the first file, if present within the script then it gives you the name to that file (script) that contains the thing your looking for within it. But it now does not tell you where it is at, only the name of the script. if you want path to and file name so you can go get it.

Code:

echo "$LINE" >> text2

restrictions to using grep ')' or '(' within the scripts text, I did not work that out.

or if you just want to consolidate the matches in one directory.

Code:

#!/bin/bash

working_dir="/home/userx/Dropbox/linux/how-tos"
holding_dir="/home/userx/testing_dir"

mkdir -pv "$holding_dir"

find "$working_dir"  -type f  > list1.txt
#
cat list1.txt | while read LINE;
do
  if [[ $(grep -rwn "$LINE" -e "quality") ]] ; then
  {
   #change to mv if you want to move it
    cp -v "$LINE" "$holding_dir"
    #echo "${LINE##*/}" >> text2
  }
  fi
done

how it works

Code:

$ ./findstuff
mkdir: created directory '/home/userx/testing_dir'
'/home/userx/Dropbox/linux/how-tos/FFmpeg quality' -> '/home/userx/testing_dir/FFmpeg quality'

I used a unique word which is within that file and only in that file. That is why you only see one file being copied over.

BW-userx · 05-15-2018, 05:53 PM

Here I reread your Conditions, and rewrote this script, it may or may not work, depending on how you actually have you scripts set up. it might look complicated but its not.

Code:

 

#!/bin/bash

#set -xv

working_dir="/home/userx/testing_dir"
holding_dir="/home/userx/testing_dir2"

mkdir -pv "$holding_dir"
search_word="file" # <-- search word in file
count=0 # <-- attached number to like name 

# ending with
# file0 ... 

#gets the list of (needed shell scripts) files within a directory
#to look inside of
find "$working_dir"  -type f  > list1.txt
#gets the total number of files (scripts) within that
#list that will be looked at
d=$(cat list1.txt | wc -l)

# two loops to use as an iterator to search each corresponding script for the
# script name within the file that it's defining name uses
#a number to distinguish them apart
#t1.sh is called by
#t2.sh
#t6.sh
#t2.sh is called by
#t7.sh

# looks for file0 through every file in the list
# then increments to file1 and looks through every file
#within the list then increments to file2 and looks though
#every file within that list for matching file1 , file2 ,... etc
while [[ $g -lt $d ]] ;
do     
  cat list1.txt | while read LINE;
  do
    if [[ $(grep -rwn "$LINE" -e "$search_word""$count" ) ]] ; then
    {
   #change to mv if you want to move it
    mv -v "$LINE" "$holding_dir"
    #echo "${LINE##*/}" >> text2
    }
    fi
    done
     ((count++))
     ((g++))
  echo $search_word$count
done

you'll have to put more conditions in it if you need to curtail it more.