LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 07-01-2014, 05:50 PM   #16
mbrauni
LQ Newbie
 
Registered: Jul 2014
Posts: 1

Rep: Reputation: Disabled

Hi,

today I found your nice script and tried to use it on my machine, but it does not work for me and I don't understand why. When I run the script, tcpdump gets started, but after about a minute, it get stopped by the script. This is the output from the main.log file:

Code:
[2014-07-02 00:47:08] Starting tcpdump manager script...
[2014-07-02 00:47:08] Starting tcpdump...
[2014-07-02 00:47:08] Output File: /home/backup/tcpdump/capture-2014-07-02.0.log
[2014-07-02 00:48:08] A new day has come.
[2014-07-02 00:48:08] Restarting tcpdump...
[2014-07-02 00:48:08] Stopping tcpdump...
[2014-07-02 00:48:08] Starting tcpdump...
[2014-07-02 00:48:08] Output File: /home/backup/tcpdump/capture-2014-07-02.1.log
[2014-07-02 00:48:09] Stopping tcpdump...
[2014-07-02 00:48:09] Ending tcpdump manager script.
In my log directory I just have the capture-2014-07-02.0.log file.

Can you please help me with this issue?
 
Old 07-02-2014, 02:05 PM   #17
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
@mbrauni Hi. On what system do you use this? It could be a tcpdump problem or perhaps something related to date command especially if you're on mac. find may act differently as well.

P.S. I just found out: The newer version has a typo:
Code:
CURRENTDATE=CURRENTDATE=...

Last edited by konsolebox; 07-02-2014 at 02:08 PM.
 
Old 08-23-2014, 02:13 AM   #18
anandpu70
LQ Newbie
 
Registered: Jan 2010
Posts: 4

Rep: Reputation: 0
And this is yet another version for it. This one's more flexible and lighter performance-wise.

Hi Consolebox,

I have gone through your script and it has written some time back. I have requirement of running tcpdump 24 hours captured into the same file for 24 hours packets. Next days packets it has to be captured in nex date's file. Acutally its working but problem is tcpdump is for some reason stopping after 2 hours. Do you know why. If you could help it will be great help.

I am running following version Linux.

Linux 2.6.18-274.el5 #1 SMP Fri Jul 8 17:36:59 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

[bash-3.2-32.el5]


Thanks,
Anand.
 
Old 06-12-2015, 12:52 PM   #19
keith.evans
LQ Newbie
 
Registered: Jun 2009
Posts: 4

Rep: Reputation: 0
Run for 60 seconds and quit?

First off konsolebox, this script is really great. Thank you.

I've learned a bunch just from working with and getting to understand it. I had an issue with temporary blindness finding clearly stated variables... Found what was needed just after posting (TCPDUMPCHECKINTERVALS=180).

Last edited by keith.evans; 06-12-2015 at 02:05 PM. Reason: Cancel question, resolved issue.
 
Old 06-13-2015, 12:00 AM   #20
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,248
Blog Entries: 8

Rep: Reputation: 235Reputation: 235Reputation: 235
@mbrauni, @anandpu70, @keith.evans

I reformatted the code to make it more readable. I also found some issues:

(1) CURRENT_DATE should have been not set as a local variable. What happens is the date value set to it is lost after start_tcpdump() exits which makes [[ ${NEW_DATE} != "${CURRENT_DATE}" ]] always valid.

(2) stop_tcpdump() should have not been setting QUIT to true. It makes the script end quickly when stop_tcpdump() or restart_tcpdump() is called.

I now have the script placed in Github. Please check it for all updates.

https://github.com/konsolebox/script...dump-master.sh
 
Old 06-18-2015, 01:24 PM   #21
keith.evans
LQ Newbie
 
Registered: Jun 2009
Posts: 4

Rep: Reputation: 0
Thumbs up Great newer version, thanks and some updates/enhancements

Thank you again konsolebox for this excellent script. I had some requirements that I needed to satisfy since I don't have root on production and not managing that environment wanted to ensure I didn't blow it up as I can't monitor the progress and the systems are very high traffic. Below I list the changes/updates made and provide the updated code (using your newest version).

Primary additions: updated logging for no-out when cron, min and max number capture files max disk space for capture file, added make dir function for non-root access to capture files after, capture file extension configurable, and prompt for pre-set protos with default selection.


Code:
#!/bin/bash
# ----------------------------------------------------------------------

# tcpdump-master
#
# The script is a tcpdump service starter and manager.  It can also
# automatically delete files older than C days, and reduce the size of
# the main log file if it's already larger than N bytes.
#
# The script was originally a solution for this thread in LQ:
# https://www.linuxquestions.org/questions/linux-networking-3/rotating-capture-files-using-tcpdump-800385/
#
# Note: It is recommended to use Bash version 4.3 or newer to prevent
# crashes related to race conditions in catching signals and handling
# traps.
#
# Disclaimer: This tool comes with no warranty.
#
# Author: konsolebox
# Copyright Free / Public Domain
# June 13, 2015
#
# Modification Log
#    date      ver     author             description
# ---------- ------- ----------- --------------------------------------
# 06/15/2015  v.0.2 kevans       Addded updated logging for no-out when
#                                cron, min and max number capture files
#                                max disk space for capture file, added
#                                make dir function for non-root access
#                                to capture files after, capture file
#                                extension, and prompt for proto with
#                                default proto selection.
# ----------------------------------------------------------------------

if [ -z "${BASH_VERSION}" ]; then
        echo "You need Bash to run this script."
        exit 1
fi

shopt -s extglob

# Settings

LOG_DIR='/var/log/tcpdump'
NOTROOT_DEFAULT_GROUP="users"
case "$-" in
*i*) INTERACTIVE=0 ;;
*) INTERACTIVE=-1 ;;
esac

MAIN_LOG_FILE='main.log'
MAIN_LOG_FILE_MAX_SIZE=$(( 20 * 1024 * 1024 ))  ## in bytes. File is reduced when this size is reached.
MAIN_LOG_FILE_ALLOWANCE=$(( 1 * 1024 * 1024 ))  ## in bytes. This is the extra space given when file is reduced.
MAIN_LOG_CHECK_INTERVALS=300 ## seconds. Recommended: >= 300

TCP_DUMP='/usr/sbin/tcpdump'
HELP_MESSAGE="Usage: `basename $0` options (-p|-proto [ssh|http|ftp|all]) -h for help"
TCPDUMP_PROTO="(tcp port 444 or tcp port 443)" # default
while getopts ":p:proto:h" opt; do
  case ${opt} in
    p|proto)
      case `echo ${OPTARG} | tr '[:upper:]' '[:lower:]'` in
         ssh|scp)
           TCPDUMP_PROTO="(tcp port 22)"
           ;;
         http|https|ssl)
           TCPDUMP_PROTO="(tcp port 444 or tcp port 443)"
           ;;
         ftp|ftps)
           TCPDUMP_PROTO="(tcp port 21)"
           ;;
         h|help)
           if [ $INTERACTIVE ]; then log $HELP_MESSAGE; fi
           ;;
         all|*)
           TCPDUMP_PROTO="(tcp port 444 or tcp port 443 or tcp port 21 or tcp port 22)"
           ;;
      esac
      ;;
   h|H|help|HELP)
      if [ $INTERACTIVE ]; then log $HELP_MESSAGE; fi
      ;;
  esac
done
echo $TCPDUMP_PROTO
TCP_DUMPARGS="(-C 1 -ieth0 -s168 -Z $USER '${TCPDUMP_PROTO}')" ## customize arguments here e.g. (-C 1 "another with spaces")
TCP_DUMP_CAPTURE_FILE_PREFIX='capture-'
TCP_DUMP_CAPTURE_FILE_SUFFIX=''
TCP_DUMP_CAPTURE_FILE_EXT='.log'
TCP_DUMP_CAPTURE_FILES_MIN=20
TCP_DUMP_CAPTURE_FILES_MAX=100
TCP_DUMP_CAPTURE_FILE_MAX_STORAGE=1048576 # bytes (1 MB: 1048576, 1 GB: 1073741824)
TCP_DUMP_CHECK_INTERVALS=180 ## seconds run time before kill (1 day: 86400, 1 hour: 3600)

OLD=14  ## days
DD_BLOCKSIZE=512  ## bytes
TEMP_DIR='/var/tmp'

# other runtime variables

CURRENT_DATE=''
TCPDUMP_PID=0
QUIT=false

# functions

function makedir {
        # make sure we have access after logs and captures created
        mkdir -p $1
        if [[ ${EUID} -ne 0 ]]; then
                chgrp ${NOTROOT_DEFAULT_GROUP} ${1}
		chmod 777 ${1}
                chmod g+s ${1}
        fi
}

function log {
        echo "[$(date '+F %T')] $1" >> "${MAIN_LOG_FILE}"
        if [ ${INTERACTIVE} ]; then echo "$1"; fi
}

function check_tcpdump {
        [[ ${TCPDUMP_PID} -ne 0 ]] && [[ -e /proc/${TCPDUMP_PID} ]] && kill -s 0 "${TCPDUMP_PID}" 2>/dev/null
}

function start_tcpdump {
        log "Starting tcpdump."

        CURRENT_DATE=$(date +%F)
        local BASENAME="${TCP_DUMP_CAPTURE_FILE_PREFIX}${CURRENT_DATE}${TCP_DUMP_CAPTURE_FILE_SUFFIX}"
        local EXISTING_FILES=()

        {
                if [[ BASH_VERSINFO -ge 4 ]]; then
                        readarray -t EXISTING_FILES
                else
                        local -i I=0
                        while read LINE; do
                                EXISTING_FILES[I++]=${}LINE}
                        done
                fi
        } < <(compgen -G "${LOG_DIR}/${BASENAME}.+([[:digit:]])${TCP_DUMP_CAPTURE_FILE_EXT}*([[:digit:]])")

        local NEXT_SESSION=0

        if [[ ${#EXISTING_FILES[@]} -gt 0 ]]; then
                local SESSION_NUMBER
                for FILE in "${EXISTING_FILES[@]}"; do
                        SESSION_NUMBER=${FILE%${TCP_DUMP_CAPTURE_FILE_EXT}*}
                        SESSION_NUMBER=${SESSION_NUMBER##*.}
                        [[ ${SESSION_NUMBER} == +([[:digit:]]) && SESSION_NUMBER -ge NEXT_SESSION ]] && NEXT_SESSION=$(( SESSION_NUMBER + 1 ))
                done
        fi

        local OUTPUT_FILE=${LOG_DIR}/${BASENAME}.${NEXT_SESSION}${TCP_DUMP_CAPTURE_FILE_EXT}

        log "Will run with these args: ${TCP_DUMP} ${TCPDUMP_ARGS[@]} -w ${OUTPUT_FILE}" 
        if [[ "$(id -u)" != "0" && ${INTERACTIVE} ]]; then
                sudo "${TCP_DUMP}" "${TCPDUMP_ARGS}" -w "${OUTPUT_FILE}" &
        else
                "${TCP_DUMP}" "${TCPDUMP_ARGS}" -w "${OUTPUT_FILE}" &
        fi

        if [[ $? -ne 0 ]]; then
                TCPDUMP_PID=0
                return 1
        fi

        TCPDUMP_PID=$!
        disown "${TCPDUMP_PID}"
        log "PID of tcpdump: ${TCPDUMP_PID}"
        check_tcpdump
}

function start_tcpdump_loop {
        until start_tcpdump; do
                log "Error: Failed to start tcpdump.  Waiting for 20 seconds before next attempt."
                read -t 20

                if [[ ${QUIT} == true ]]; then
                        log "Ending tcpdump manager script."
                        exit
                fi
        done
}

function stop_tcpdump {
        log "Stopping tcpdump."
        kill "${TCPDUMP_PID}"
        sleep 1
        check_tcpdump && kill -s 9 "${TCPDUMP_PID}"
        TCPDUMP_PID=0
}

function restart_tcpdump {
        log "Restarting tcpdump."
        check_tcpdump && stop_tcpdump
        start_tcpdump_loop
}

function signal_caught_callback {
        local SIGNAL=$1
        log "Caught signal ${SIGNAL}."
        QUIT=true
}

function main {
        local CAPTURE_FILE_PATTERN FILE NEW_DATE SIZE TEMP_FILE I

        CAPTURE_FILE_PATTERN="${TCP_DUMP_CAPTURE_FILE_PREFIX}[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]${TCP_DUMP_CAPTURE_FILE_SUFFIX}${TCP_DUMP_CAPTURE_FILE_EXT}*"
        [[ ${MAIN_LOG_FILE} != */* ]] && MAINLOGFILE=${LOG_DIR}/${MAINLOGFILE}

        makedir ${LOG_DIR}
        pushd ${LOG_DIR}

        log "----------------------------------------"
        log "Starting up."
        log "PID: $$"

        [[ ${MAIN_LOG_FILE_MAX_SIZE} == +([[:digit:]]) && MAIN_LOG_FILE_MAX_SIZE -gt DD_BLOCK_SIZE ]] || {
                echo "MAIN_LOG_FILE_MAX_SIZE is not valid: ${MAIN_LOG_FILE_MAX_SIZE}"
                return 1
        }

        [[ ${MAIN_LOG_FILE_ALLOWANCE} == +([[:digit:]]) && MAIN_LOG_FILE_ALLOWANCE -gt DD_BLOCK_SIZE && MAIN_LOG_FILE_ALLOWANCE -lt MAIN_LOG_FILE_MAX_SIZE ]] || {
                echo "MAIN_LOG_FILE_ALLOWANCE is not valid: ${MAIN_LOG_FILE_ALLOWANCE}"
                return 1
        }

        for S in SIGQUIT SIGINT SIGKILL SIGTERM; do
                eval "
                        function catch_${S} { signal_caught_callback ${S}; }
                        trap catch_${S} ${S}
                "
        done

        start_tcpdump_loop

        for (( I = 1;; I = (I + 1) % 10000 )); do
                read -t 1  ## we have to separate this from the next statement to ensure proper handling of signals

                [[ ${QUIT} = true ]] && break

                if (( (I % TCP_DUMP_CHECK_INTERVALS) == 0 )); then
                        NEW_DATE=$(exec date +%F)

                        if [[ ! ${NEW_DATE} = "${CURRENT_DATE}" ]]; then
                                log "A new day has come."

                                # cleanup based on packet capture age
                                if read FILE; then
                                        log "Deleting ${OLD}-days old files."

                                        while
                                                log "Deleting ${FILE}."
                                                rm -f "${FILE}"
                                                read FILE
                                        do
                                                continue
                                        done
                                fi < <(exec find "${LOG_DIR}" -name "${CAPTURE_FILE_PATTERN}" -daystart -ctime "+${OLD}")   # or -mtime?

                                # cleanup based on packet capture max size and min count
                                while [[ `du -bc ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1 | cut -f1` -gt ${TCP_DUMP_CAPTURE_FILE_MAX_STORAGE} ]]
                                do
									if [[ `ls -1 ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | wc -l` -gt ${TCP_DUMP_CAPTURE_FILES_MIN} ]]; then
										log "Deleting files exceeding max storage (${TCP_DUMP_CAPTURE_FILE_MAX_STORAGE} bytes) and min count (${TCP_DUMP_CAPTURE_FILES_MIN})..."
										log " - Current storage used: `du -bc ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1` bytes"
										CNT=`ls -1 ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | wc -l`
										log " - Current file count: ${CNT}"
															FILE=`ls -t ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1`
															log " - Deleting ${FILE}..."
															rm -f ${FILE}
									fi
                                done

                                while [[ `ls -1 ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | wc -l` -gt ${TCP_DUMP_CAPTURE_FILES_MAX} ]]
                                do
                                        log "Deleting files exceeding max allowed (${TCP_DUMP_CAPTURE_FILES_MAX})..."
                                        FILE=`ls -t ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1`
                                        log "Deleting ${FILE}..."
                                        rm -f ${FILE}
                                done

                                restart_tcpdump
                        fi
                fi

                if (( (I % MAIN_LOG_CHECK_INTERVALS) == 0 )); then
                        SIZE=$(stat --printf=%s "${MAIN_LOG_FILE}")

                        if [[ ${SIZE} == +([[:digit:]]) && SIZE -gt MAIN_LOG_FILE_MAX_SIZE ]]; then
                                log "Reducing log data in ${MAIN_LOG_FILE}."
                                TEMP_FILE=${TEMP_DIR}/tcpdump-${RANDOM}.tmp
                                SKIP=$(( (SIZE - (MAIN_LOG_FILE_MAX_SIZE - MAIN_LOG_FILE_ALLOWANCE)) / DD_BLOCK_SIZE ))

                                if
                                        dd "bs=${DD_BLOCK_SIZE}" "skip=${SKIP}" "if=${MAIN_LOG_FILE}" "of=${TEMP_FILE}" \
                                        && cat "${TEMP_FILE}" > "${MAIN_LOG_FILE}" \
                                        && rm -f "${TEMP_FILE}"
                                then
                                        log "Done."
                                else
                                        log "Failed.  Something went wrong."
                                fi
                        fi
                fi
        done

        log "Shutting down."
        check_tcpdump && stop_tcpdump
        log "----------------------------------------"
}

# Start.

main

Last edited by keith.evans; 06-18-2015 at 01:30 PM.
 
  


Reply

Tags
log, name, rotate, tcpdump



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Help with tcpdump to capture traffic. abefroman Linux - Networking 4 04-04-2008 03:08 AM
tcpdump does not capture all packets logicalfuzz Linux - Networking 1 03-19-2007 12:47 PM
Using Tcpdump and Tethereal to capture packets shanu_technical Linux - Networking 3 06-14-2006 08:54 AM
not capture payload with tcpdump? hedpe Linux - Networking 6 02-07-2006 02:23 PM
retransmiting tcpdump capture file? JWT2 Linux - Networking 9 10-09-2005 08:27 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 05:05 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration