LinuxQuestions.org - Extracting strings

- Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)

- - Extracting strings (https://www.linuxquestions.org/questions/linux-newbie-8/extracting-strings-682148/)

Extracting strings

Hello friends,

I captured and saved to a file in plain txt from wireshark, i am trying to use awk or any command to extract the time, source ip address and destination address, i tried using grep it gives me all the line containg ip address, pls can anybody help me on this

this is how the information looks when you open the file:

No. Time Source Destination Protocol Info
1617 14.30 10.49.48.95 64.191.203.30 HTTP POST /login HTTP/1.1 (application/x-www-form-urlencoded)

tt1ect

This looks like a simple task for AWK. Assuming that the format is always the same, you simply have to extract by field, using " " (space) as the delimiter.

Example---this print the 2nd, 3rd and 4th field, with tabs:

awk '{print $2"\t"$3"\t"$4}' filename

Really good AWK tutorial (and much more) here:

http://www.grymoire.com/Unix/Awk.html

Moin,

you don't need grep in this case. A simple way to get the fields 2 - 4 is cut:

Code:

jan@jack:~/tmp> echo '1617 14.30 10.49.48.95 64.191.203.30 HTTP POST /login HTTP/1.1 (application/x-www-form-urlencoded)' | cut -f2-4 -d' '

14.30 10.49.48.95 64.191.203.30

Another possibility is awk:

Code:

jan@jack:~/tmp> echo '1617 14.30 10.49.48.95 64.191.203.30 HTTP POST /login HTTP/1.1 (application/x-www-form-urlencoded)' | awk ' { print $2, $3, $4 } '

14.30 10.49.48.95 64.191.203.30

Or sed:

Code:

jan@jack:~/tmp> echo '1617 14.30 10.49.48.95 64.191.203.30 HTTP POST /login HTTP/1.1 (application/x-www-form-urlencoded)' | sed 's/^[0-9]* \([^ ]*\) \([^ ]*\) \([^ ]*\) .*/\1 \2 \3/'

14.30 10.49.48.95 64.191.203.30

I would prefer cut for the given input format - it's the simpliest tool.

Jan

Please don't post the same thing twice----(perhaps it was an error)
I'll ask that the two be merged since both have replies

Moin,

Quote:

Originally Posted by pixellany (Post 3336174)

Please don't post the same thing twice----(perhaps it was an error)
I'll ask that the two be merged since both have replies

sorry - it really was an error - think I accidently double clicked the submit button.

Jan

To be honest, I'd just save the data from Wireshark/tcpdump as a binary file and then write a C(++) program using libpcap to get the info you want.

Quote:

Originally Posted by Nylex (Post 3337355)

I'd just save the data from Wireshark/tcpdump as a binary file and then write a C(++) program using libpcap to get the info you want.

Nice answer! Could you please post some code then?

Quote:

Originally Posted by unSpawn (Post 3337365)

Nice answer! Could you please post some code then?

Will do so when I've got a spare minute :).

Quote:

Originally Posted by jan61 (Post 3337335)

Moin,

sorry - it really was an error - think I accidently double clicked the submit button.

Jan

My comment was not directed at you!!

OP posted this twice and each one had responses, so the threads were merged.

When I was doing my MSc project, I had to analyse tcpdump data and initially tried string parsing. It was very messy, so I just went and learned to use libpcap. Makes life easier!

As requested, here's a C++ program making use of libpcap to get source and destination addresses and ports and timestamps:

Code:

#include <pcap.h>

#include <netinet/ip.h>

#include <netinet/in.h>

#include <netinet/ether.h>

#include <netinet/tcp.h>

#include <arpa/inet.h>

#include <iostream>

using namespace std;



void packet_handler(u_char *args, const struct pcap_pkthdr *packet_header, const u_char *packet);



int main(int argc, const char *argv[])

{

  const char *file = "tcpdump_file";

  char error_buffer[PCAP_ERRBUF_SIZE];

  pcap_t *handle = pcap_open_offline(file, error_buffer);



  pcap_loop(handle, -1, packet_handler, NULL); // Loop until we reach the end of the file

  return 0;

}



void packet_handler(u_char *args, const struct pcap_pkthdr *packet_header,

                    const u_char *packet)

{



  // The IP packet is the payload of the Ethernet frame, so we need to skip the

  // Ethernet frame's header to get to it.

  // The TCP packet is the payload of the IP packet, so we do a similar thing.

  struct ip *ip_packet = (struct ip*)(packet + sizeof(struct ether_header));

  struct tcphdr  *tcp_packet = (struct tcphdr*)(packet + sizeof(struct ether_header)

                                                + sizeof(struct ip));



  cout << "Source address: " << inet_ntoa(ip_packet->ip_src) << endl;

  cout << "Destination address: " << inet_ntoa(ip_packet->ip_dst) << endl;



  cout << "Source port: " << ntohs(tcp_packet->source) << endl;

  cout << "Destination port: " << ntohs(tcp_packet->dest) << endl;

 

  struct timeval ts = (*packet_header).ts;

  cout << "Seconds: " << ts.tv_sec << " Microseconds: " << ts.tv_usec << endl;

}

I suggest reading the pcap.h man page ("man pcap") and possibly man pages for other headers too.

Thanks for the -lpcap example.