[SOLVED] sorting

bosong · 03-28-2016, 03:34 AM

I need to sort a massive file, for username and command used, is there any simple way?

syg00 · 03-28-2016, 03:38 AM

"man sort" would be a good start.

bosong · 03-28-2016, 03:52 AM

how do i sort out and display only the 3rd column?? is it possible?

syg00 · 03-28-2016, 04:12 AM

Read the doco, do some testing, see what happens. That's how you learn.
Use a subset of the data - 10-20 records should do.

sundialsvcs · 03-28-2016, 08:46 AM

Believe it or not, the sort command can do things like that.

I don't know what you consider "massive." (What's the order of magnitude? MIllions of records? Tens of millions? Hundreds? Only the latter category might be considered truly "massive," and only in the sense that it will take a little while longer.)

dijetlo · 03-28-2016, 09:24 AM

Check the size of /tmp since when your massive file gets sorted, that's where the scratch files get created.

BW-userx · 03-28-2016, 09:27 AM

massive: exceptionally large

exceptionally: to a greater degree than normal; unusually
large: of considerable or relatively great size, extent, or capacity

by his usage of the term massive file would then indicate this is is abnormally or unusually large.

bosong · 03-28-2016, 09:13 PM

The server I am doing has maybe about 1 million records per day, i want to sort out the username along side with the command use and also count for each command, is there anyway i can do it???

bosong · 03-28-2016, 09:58 PM

There are about 100 users and thousands of different commands
Can anyone help me?
Thanks in advance

tshikose · 03-29-2016, 03:31 AM

Hi,

You have been giving good advices to start with the sort command.
Apparently, you are asking for more, without sharing here what you have tried so far.

Remember that while people are keen to help, they do not want to do your tasks or your homework.
Read the man page of sort, and test something.

pan64 · 03-29-2016, 05:36 AM

Yes, you have got help already: see man sort, and if you need examples just look for tutorials/examples (of sort) on the web. You will find everything you need. In case you want to fine-tune a solution just show us what you have already picked and what is your problem with that. Since you gave no usable info we can hardly give you usable answer (by the way the answer is yes, there is a simple way)
https://is.gd/8WT0AM

rtmistler · 03-29-2016, 07:20 AM

@OP Welcome to LQ. Please review that link.

Recommend you attempt the sort command and then report how that attempt occurred.

Another thought is that in addition to sorting, you can also reduce this massive data set by using grep to only locate the records which matter out of that very large file.

When reporting results, it is best to place them within [code] tags.

jpollard · 03-29-2016, 07:31 AM

If you are using a system with /tmp as tmpfs, you will likely need to use the
-T option to use an alternate location for your scratch files while sorting.

There is a hint that using an environment variable TMPDIR may redirect it also.

bosong · 03-30-2016, 04:35 AM

Sorry for the late reply.
Thanks all for help
I have solve the problem through using sort + awk + uniq

jpollard · 03-30-2016, 07:10 AM

Those are frequently the solution. Sometimes the uniq is extra as sort has a unique option.