Linux - NetworkingThis forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
today I found your nice script and tried to use it on my machine, but it does not work for me and I don't understand why. When I run the script, tcpdump gets started, but after about a minute, it get stopped by the script. This is the output from the main.log file:
@mbrauni Hi. On what system do you use this? It could be a tcpdump problem or perhaps something related to date command especially if you're on mac. find may act differently as well.
P.S. I just found out: The newer version has a typo:
Code:
CURRENTDATE=CURRENTDATE=...
Last edited by konsolebox; 07-02-2014 at 02:08 PM.
And this is yet another version for it. This one's more flexible and lighter performance-wise.
Hi Consolebox,
I have gone through your script and it has written some time back. I have requirement of running tcpdump 24 hours captured into the same file for 24 hours packets. Next days packets it has to be captured in nex date's file. Acutally its working but problem is tcpdump is for some reason stopping after 2 hours. Do you know why. If you could help it will be great help.
First off konsolebox, this script is really great. Thank you.
I've learned a bunch just from working with and getting to understand it. I had an issue with temporary blindness finding clearly stated variables... Found what was needed just after posting (TCPDUMPCHECKINTERVALS=180).
Last edited by keith.evans; 06-12-2015 at 02:05 PM.
Reason: Cancel question, resolved issue.
I reformatted the code to make it more readable. I also found some issues:
(1) CURRENT_DATE should have been not set as a local variable. What happens is the date value set to it is lost after start_tcpdump() exits which makes [[ ${NEW_DATE} != "${CURRENT_DATE}" ]] always valid.
(2) stop_tcpdump() should have not been setting QUIT to true. It makes the script end quickly when stop_tcpdump() or restart_tcpdump() is called.
I now have the script placed in Github. Please check it for all updates.
Great newer version, thanks and some updates/enhancements
Thank you again konsolebox for this excellent script. I had some requirements that I needed to satisfy since I don't have root on production and not managing that environment wanted to ensure I didn't blow it up as I can't monitor the progress and the systems are very high traffic. Below I list the changes/updates made and provide the updated code (using your newest version).
Primary additions: updated logging for no-out when cron, min and max number capture files max disk space for capture file, added make dir function for non-root access to capture files after, capture file extension configurable, and prompt for pre-set protos with default selection.
Code:
#!/bin/bash
# ----------------------------------------------------------------------
# tcpdump-master
#
# The script is a tcpdump service starter and manager. It can also
# automatically delete files older than C days, and reduce the size of
# the main log file if it's already larger than N bytes.
#
# The script was originally a solution for this thread in LQ:
# https://www.linuxquestions.org/questions/linux-networking-3/rotating-capture-files-using-tcpdump-800385/
#
# Note: It is recommended to use Bash version 4.3 or newer to prevent
# crashes related to race conditions in catching signals and handling
# traps.
#
# Disclaimer: This tool comes with no warranty.
#
# Author: konsolebox
# Copyright Free / Public Domain
# June 13, 2015
#
# Modification Log
# date ver author description
# ---------- ------- ----------- --------------------------------------
# 06/15/2015 v.0.2 kevans Addded updated logging for no-out when
# cron, min and max number capture files
# max disk space for capture file, added
# make dir function for non-root access
# to capture files after, capture file
# extension, and prompt for proto with
# default proto selection.
# ----------------------------------------------------------------------
if [ -z "${BASH_VERSION}" ]; then
echo "You need Bash to run this script."
exit 1
fi
shopt -s extglob
# Settings
LOG_DIR='/var/log/tcpdump'
NOTROOT_DEFAULT_GROUP="users"
case "$-" in
*i*) INTERACTIVE=0 ;;
*) INTERACTIVE=-1 ;;
esac
MAIN_LOG_FILE='main.log'
MAIN_LOG_FILE_MAX_SIZE=$(( 20 * 1024 * 1024 )) ## in bytes. File is reduced when this size is reached.
MAIN_LOG_FILE_ALLOWANCE=$(( 1 * 1024 * 1024 )) ## in bytes. This is the extra space given when file is reduced.
MAIN_LOG_CHECK_INTERVALS=300 ## seconds. Recommended: >= 300
TCP_DUMP='/usr/sbin/tcpdump'
HELP_MESSAGE="Usage: `basename $0` options (-p|-proto [ssh|http|ftp|all]) -h for help"
TCPDUMP_PROTO="(tcp port 444 or tcp port 443)" # default
while getopts ":p:proto:h" opt; do
case ${opt} in
p|proto)
case `echo ${OPTARG} | tr '[:upper:]' '[:lower:]'` in
ssh|scp)
TCPDUMP_PROTO="(tcp port 22)"
;;
http|https|ssl)
TCPDUMP_PROTO="(tcp port 444 or tcp port 443)"
;;
ftp|ftps)
TCPDUMP_PROTO="(tcp port 21)"
;;
h|help)
if [ $INTERACTIVE ]; then log $HELP_MESSAGE; fi
;;
all|*)
TCPDUMP_PROTO="(tcp port 444 or tcp port 443 or tcp port 21 or tcp port 22)"
;;
esac
;;
h|H|help|HELP)
if [ $INTERACTIVE ]; then log $HELP_MESSAGE; fi
;;
esac
done
echo $TCPDUMP_PROTO
TCP_DUMPARGS="(-C 1 -ieth0 -s168 -Z $USER '${TCPDUMP_PROTO}')" ## customize arguments here e.g. (-C 1 "another with spaces")
TCP_DUMP_CAPTURE_FILE_PREFIX='capture-'
TCP_DUMP_CAPTURE_FILE_SUFFIX=''
TCP_DUMP_CAPTURE_FILE_EXT='.log'
TCP_DUMP_CAPTURE_FILES_MIN=20
TCP_DUMP_CAPTURE_FILES_MAX=100
TCP_DUMP_CAPTURE_FILE_MAX_STORAGE=1048576 # bytes (1 MB: 1048576, 1 GB: 1073741824)
TCP_DUMP_CHECK_INTERVALS=180 ## seconds run time before kill (1 day: 86400, 1 hour: 3600)
OLD=14 ## days
DD_BLOCKSIZE=512 ## bytes
TEMP_DIR='/var/tmp'
# other runtime variables
CURRENT_DATE=''
TCPDUMP_PID=0
QUIT=false
# functions
function makedir {
# make sure we have access after logs and captures created
mkdir -p $1
if [[ ${EUID} -ne 0 ]]; then
chgrp ${NOTROOT_DEFAULT_GROUP} ${1}
chmod 777 ${1}
chmod g+s ${1}
fi
}
function log {
echo "[$(date '+F %T')] $1" >> "${MAIN_LOG_FILE}"
if [ ${INTERACTIVE} ]; then echo "$1"; fi
}
function check_tcpdump {
[[ ${TCPDUMP_PID} -ne 0 ]] && [[ -e /proc/${TCPDUMP_PID} ]] && kill -s 0 "${TCPDUMP_PID}" 2>/dev/null
}
function start_tcpdump {
log "Starting tcpdump."
CURRENT_DATE=$(date +%F)
local BASENAME="${TCP_DUMP_CAPTURE_FILE_PREFIX}${CURRENT_DATE}${TCP_DUMP_CAPTURE_FILE_SUFFIX}"
local EXISTING_FILES=()
{
if [[ BASH_VERSINFO -ge 4 ]]; then
readarray -t EXISTING_FILES
else
local -i I=0
while read LINE; do
EXISTING_FILES[I++]=${}LINE}
done
fi
} < <(compgen -G "${LOG_DIR}/${BASENAME}.+([[:digit:]])${TCP_DUMP_CAPTURE_FILE_EXT}*([[:digit:]])")
local NEXT_SESSION=0
if [[ ${#EXISTING_FILES[@]} -gt 0 ]]; then
local SESSION_NUMBER
for FILE in "${EXISTING_FILES[@]}"; do
SESSION_NUMBER=${FILE%${TCP_DUMP_CAPTURE_FILE_EXT}*}
SESSION_NUMBER=${SESSION_NUMBER##*.}
[[ ${SESSION_NUMBER} == +([[:digit:]]) && SESSION_NUMBER -ge NEXT_SESSION ]] && NEXT_SESSION=$(( SESSION_NUMBER + 1 ))
done
fi
local OUTPUT_FILE=${LOG_DIR}/${BASENAME}.${NEXT_SESSION}${TCP_DUMP_CAPTURE_FILE_EXT}
log "Will run with these args: ${TCP_DUMP} ${TCPDUMP_ARGS[@]} -w ${OUTPUT_FILE}"
if [[ "$(id -u)" != "0" && ${INTERACTIVE} ]]; then
sudo "${TCP_DUMP}" "${TCPDUMP_ARGS}" -w "${OUTPUT_FILE}" &
else
"${TCP_DUMP}" "${TCPDUMP_ARGS}" -w "${OUTPUT_FILE}" &
fi
if [[ $? -ne 0 ]]; then
TCPDUMP_PID=0
return 1
fi
TCPDUMP_PID=$!
disown "${TCPDUMP_PID}"
log "PID of tcpdump: ${TCPDUMP_PID}"
check_tcpdump
}
function start_tcpdump_loop {
until start_tcpdump; do
log "Error: Failed to start tcpdump. Waiting for 20 seconds before next attempt."
read -t 20
if [[ ${QUIT} == true ]]; then
log "Ending tcpdump manager script."
exit
fi
done
}
function stop_tcpdump {
log "Stopping tcpdump."
kill "${TCPDUMP_PID}"
sleep 1
check_tcpdump && kill -s 9 "${TCPDUMP_PID}"
TCPDUMP_PID=0
}
function restart_tcpdump {
log "Restarting tcpdump."
check_tcpdump && stop_tcpdump
start_tcpdump_loop
}
function signal_caught_callback {
local SIGNAL=$1
log "Caught signal ${SIGNAL}."
QUIT=true
}
function main {
local CAPTURE_FILE_PATTERN FILE NEW_DATE SIZE TEMP_FILE I
CAPTURE_FILE_PATTERN="${TCP_DUMP_CAPTURE_FILE_PREFIX}[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]${TCP_DUMP_CAPTURE_FILE_SUFFIX}${TCP_DUMP_CAPTURE_FILE_EXT}*"
[[ ${MAIN_LOG_FILE} != */* ]] && MAINLOGFILE=${LOG_DIR}/${MAINLOGFILE}
makedir ${LOG_DIR}
pushd ${LOG_DIR}
log "----------------------------------------"
log "Starting up."
log "PID: $$"
[[ ${MAIN_LOG_FILE_MAX_SIZE} == +([[:digit:]]) && MAIN_LOG_FILE_MAX_SIZE -gt DD_BLOCK_SIZE ]] || {
echo "MAIN_LOG_FILE_MAX_SIZE is not valid: ${MAIN_LOG_FILE_MAX_SIZE}"
return 1
}
[[ ${MAIN_LOG_FILE_ALLOWANCE} == +([[:digit:]]) && MAIN_LOG_FILE_ALLOWANCE -gt DD_BLOCK_SIZE && MAIN_LOG_FILE_ALLOWANCE -lt MAIN_LOG_FILE_MAX_SIZE ]] || {
echo "MAIN_LOG_FILE_ALLOWANCE is not valid: ${MAIN_LOG_FILE_ALLOWANCE}"
return 1
}
for S in SIGQUIT SIGINT SIGKILL SIGTERM; do
eval "
function catch_${S} { signal_caught_callback ${S}; }
trap catch_${S} ${S}
"
done
start_tcpdump_loop
for (( I = 1;; I = (I + 1) % 10000 )); do
read -t 1 ## we have to separate this from the next statement to ensure proper handling of signals
[[ ${QUIT} = true ]] && break
if (( (I % TCP_DUMP_CHECK_INTERVALS) == 0 )); then
NEW_DATE=$(exec date +%F)
if [[ ! ${NEW_DATE} = "${CURRENT_DATE}" ]]; then
log "A new day has come."
# cleanup based on packet capture age
if read FILE; then
log "Deleting ${OLD}-days old files."
while
log "Deleting ${FILE}."
rm -f "${FILE}"
read FILE
do
continue
done
fi < <(exec find "${LOG_DIR}" -name "${CAPTURE_FILE_PATTERN}" -daystart -ctime "+${OLD}") # or -mtime?
# cleanup based on packet capture max size and min count
while [[ `du -bc ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1 | cut -f1` -gt ${TCP_DUMP_CAPTURE_FILE_MAX_STORAGE} ]]
do
if [[ `ls -1 ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | wc -l` -gt ${TCP_DUMP_CAPTURE_FILES_MIN} ]]; then
log "Deleting files exceeding max storage (${TCP_DUMP_CAPTURE_FILE_MAX_STORAGE} bytes) and min count (${TCP_DUMP_CAPTURE_FILES_MIN})..."
log " - Current storage used: `du -bc ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1` bytes"
CNT=`ls -1 ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | wc -l`
log " - Current file count: ${CNT}"
FILE=`ls -t ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1`
log " - Deleting ${FILE}..."
rm -f ${FILE}
fi
done
while [[ `ls -1 ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | wc -l` -gt ${TCP_DUMP_CAPTURE_FILES_MAX} ]]
do
log "Deleting files exceeding max allowed (${TCP_DUMP_CAPTURE_FILES_MAX})..."
FILE=`ls -t ${LOG_DIR}/${TCP_DUMP_CAPTURE_FILE_PREFIX}* | tail -1`
log "Deleting ${FILE}..."
rm -f ${FILE}
done
restart_tcpdump
fi
fi
if (( (I % MAIN_LOG_CHECK_INTERVALS) == 0 )); then
SIZE=$(stat --printf=%s "${MAIN_LOG_FILE}")
if [[ ${SIZE} == +([[:digit:]]) && SIZE -gt MAIN_LOG_FILE_MAX_SIZE ]]; then
log "Reducing log data in ${MAIN_LOG_FILE}."
TEMP_FILE=${TEMP_DIR}/tcpdump-${RANDOM}.tmp
SKIP=$(( (SIZE - (MAIN_LOG_FILE_MAX_SIZE - MAIN_LOG_FILE_ALLOWANCE)) / DD_BLOCK_SIZE ))
if
dd "bs=${DD_BLOCK_SIZE}" "skip=${SKIP}" "if=${MAIN_LOG_FILE}" "of=${TEMP_FILE}" \
&& cat "${TEMP_FILE}" > "${MAIN_LOG_FILE}" \
&& rm -f "${TEMP_FILE}"
then
log "Done."
else
log "Failed. Something went wrong."
fi
fi
fi
done
log "Shutting down."
check_tcpdump && stop_tcpdump
log "----------------------------------------"
}
# Start.
main
Last edited by keith.evans; 06-18-2015 at 01:30 PM.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.