LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
Search this Thread
Old 08-29-2010, 03:04 PM   #1
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Rep: Reputation: 86
printable file catalog or index for (external) USB and flash drives


Can anyone point me to already built and mostly working utilities that will catalog the contents of (external) USB and flash media in a way that they might be searched?

With cheap (under $100 US) terabyte external drives, it is too easy to get another drive and fill it. I'm looking for utilities or applications that will help me know what I have, cull the duplicates, and avoid the need to spin a drive just to see if it holds what I seek.

I know that I could:
  1. connect a drive
  2. find ... >driveID_contents.txt
  3. grep .... driveID_contents.txt | less
Maybe I'm overthinking this, but it seems that they ought to be an existing application or utility for this sort of data management.

Years ago there were utilities that would read the contents of diskette media and create a printable "index" or "catalog" page. The pages were conveniently sized to match the diskette so that one could store the page in the diskette sleeve for future reference.

Later, the pages got replaced with applications that stored a disketter ID along with each file name. One could then search for a name, or pattern, and discover which diskette held the file(s) of interest. Today we don't have diskette sleeves, but we do have cases for our external USB drives and wallets for our flash media. A printed index would be nice to have.

Grateful in advance,
~~~ 0;-Dan
 
Old 08-30-2010, 11:43 AM   #2
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,033

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
The CLI should have all the tools you need.

Update the locate db and then
locate /media/disk/* > BigDisk_1Index
Will give you a list of all the files on the drive, you can then cut sort and grep this file (and the files for your other disks) to your heart's content.

To have some real fun, you could install mysql apache and php and set up a little database. Enter a filename, and have it tell you all of the disks it is on. It would be a good introductory project, with a definable endpoint.

Have fun.
 
Old 08-30-2010, 03:31 PM   #3
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Original Poster
Rep: Reputation: 86
I'm hoping that someone has already done the heavy lifting on the database application.

I found gnome-catalog in the apt repositories. It looks like what I want, but I'm having troubles making it dance as expected.
  • over here I have some top-level folder, file-system, USB or flash media that I want to add to the database
  • I launch gnome-catalog
  • I want a "new catalog" for all my "external drives" -- flash or USB
  • I don't fine new or import or similar that works as expected.
 
Old 08-30-2010, 04:56 PM   #4
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 3,369
Blog Entries: 1

Rep: Reputation: 1088Reputation: 1088Reputation: 1088Reputation: 1088Reputation: 1088Reputation: 1088Reputation: 1088Reputation: 1088
do a find, sort it, remove duplicates, only show certain columns, then import into excel or something.

having a list of contents of a 5.25 disk in the sleeve was easy because it couldnt hold more than a handful of files.

if you grab a list of a external hdd or flash drive you could easily have 100 pages. I just did a find on my 250g external and got 1281 pages. i dont see how having that could be useful unless it were in a searchable file.
 
Old 08-31-2010, 07:50 AM   #5
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,033

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
If you want something pre-built, maybe you should take a look at
Quote:
gwhere - Removable media catalog manager

GWhere allows to manage a database of your CDs and other removable media (hard disks, floppy drive, Zip drive, CD-ROM, etc...). With GWhere it's easy to browse your CDs or to make a quick search without needing to insert your CDs

Homepage: http://www.gwhere.org/
 
Old 09-02-2010, 10:32 PM   #6
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Original Poster
Rep: Reputation: 86
Quote:
Originally Posted by tredegar View Post
If you want something pre-built, maybe you should take a look at gwhere
I found gwhere and tried it. It seems to want CSV or similar data as catalog to display. How do I:
  • point the tool at a folder tree or file system
  • scan for all files and folders found there
  • create a catalog of what it finds
Then I could view and slice and dice the reports of what is there.

Otherwise is there some standard way to create the CSV files that
gwhere wants to process?

Thanks,
~~~ 0;-Dan
 
Old 09-03-2010, 11:44 AM   #7
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,033

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
I haven't used gwhere, but they have a website with some documentation.

I just played with gwhere :
I plugged in a USB disk.
Opened gwhere

File - New -Catalog - test
Click Management Tab
New Disk, Select media. Select something sensible
Archive # 1
Name test1
Category None
Enter disk Name: Freecom400GB

Click Add Disk.

I can hear the disk being searched and, presumably indexed.
This may take some time it seems.
I expect you'll get the hang of it if you just experiment a bit.
 
1 members found this post helpful.
Old 09-03-2010, 08:06 PM   #8
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Original Poster
Rep: Reputation: 86
That was so obvious as to be nuts that I didn't get that to work without your comments.

I followed your advice and things worked just fine ...
... well almost. Is there some special action so that I don't need to restart gwhere so that I picks up the most recently connected or removed media? If I was working with /media/SomeFiles then dismounted and connected /media/OtherFiles, I had to save->close gwhere. When I restarted gwhere, it would see whatever I had mounted since the previous
program start.

Thanks,
~~~ 0;-Dan
 
Old 09-05-2010, 05:05 PM   #9
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Original Poster
Rep: Reputation: 86
... Follow-up

Now that I have a gwhere catalog, does anyone have any suggestions for how I might filter for duplicates and such?

I've looked at the exported CSV file, but the records are all sorts of various formats ... or am I missing something?

Merci d'avance
~~~ 0;-Dan
 
Old 09-05-2010, 05:22 PM   #10
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,033

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
Clicky - clicky, and experiment a bit more?

Otherwise, find gwhere's configuration files and hack them with a text editor. Or the command line ( grep cut & sort come to mind. )

CSV files can be imported into OpenOffice, then sorted, filtered and re-exported.

Web search engines will help you.

Let us know how you get on.
 
Old 09-06-2010, 12:53 PM   #11
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Original Poster
Rep: Reputation: 86
Quote:
Originally Posted by tredegar View Post
Clicky - clicky, and experiment a bit more?
...
I'm very surprised that there is no [relatively] mature application to address these issues?!

Like diskettes of years past, we now accumulate external drives that are USB, Firewire, e-SATA, or flash. Unlike diskettes, these new drives hold hundreds or thousands of files. It is so easy (and inexpensive) to grab a drive a copy files that it is trivial to accumulate duplicates or more instances of the same files. All of this makes a catalog very valuable AND makes duplicate identification all the more so.

Merci d'avance,
~~~ 0;-Dan
 
Old 09-06-2010, 01:29 PM   #12
tredegar
LQ 5k Club
 
Registered: May 2003
Location: London, UK
Distribution: Debian "Jessie"
Posts: 6,033

Rep: Reputation: 371Reputation: 371Reputation: 371Reputation: 371
Quote:
I'm very surprised that there is no [relatively] mature application to address these issues?!
It is possible that this (making a catalogue with find /media/Disk1 * > ~/Indices/Disk1_index ) and then searching for a filename in those catalogues with grep filename ~/Indices/* is so trivial from the command line, that no one has bothered to develop anything more complicated than gwhere (which seems to work, though I only used it the once).

Duplicates are another matter altogether: Same filenames does not necessarily mean that the contents of those files are the same. Conversely, different filenames does not necessarily mean that the contents of the files are different.

There are several *nix utilities for dealing with this sort of problem, fdupes comes to mind, but there are others.
See here
 
Old 09-06-2010, 03:00 PM   #13
knudfl
LQ 5k Club
 
Registered: Jan 2008
Location: Copenhagen, DK
Distribution: pclos2014.08, Slack14.1 DebWheezy, +50+ other Linux OS, for test only.
Posts: 14,125

Rep: Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639Reputation: 2639
No application could compete with the method " find for indexing,
grep for search " described by @tredegar, posts # 12 :
Speed and simplicity.

I have used that for six years. It's now a 70 MB text archive.
Easy to search for old info / files. ( Backups on DVD, cd etc.)
..

Last edited by knudfl; 09-06-2010 at 05:02 PM.
 
Old 09-06-2010, 09:17 PM   #14
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Original Poster
Rep: Reputation: 86
Quote:
Originally Posted by knudfl View Post
No application could compete with the method
"find for indexing, grep for search"

described by @tredegar, posts # 12 :Speed and simplicity.
...
I could grep a text file with my financial transactions, too, but I think that a PIM does a better job.

The only reliable duplicate finder I know about involves the md5sum and the ls -l details and diff details about each file. Even that is only high 90's percent and not 100%.

Merci d'avance,
~~~ 0;-Dan
 
Old 09-30-2010, 04:14 PM   #15
SaintDanBert
Senior Member
 
Registered: Jan 2009
Location: Austin, TX
Distribution: Mint-15 with Cinnamon & KDE
Posts: 1,367
Blog Entries: 3

Original Poster
Rep: Reputation: 86
Now that I have a catalog of drive contents, can anyone recommend some way to print some sort of drive table-of-contents page to put into each drive's protective pouch?
I'd like to have something roughly 3in by 5in (about the size of the drive). When I'm looking for something, I could start with the TOC page before I connect the drive.

If you only have one or two such drives, try and see works. If you have dozens (yeah) then some other process makes sense ... at least to me.

~~~ 0;-Dan
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Which file sys structure is best re formatting an external USB flash drive JuliaHenson Linux - General 4 10-16-2009 05:08 PM
Is there a DVD that can read usb flash drives/external HDD? swalker23 General 2 11-21-2008 11:58 PM
Catalog index files removable media John A. Scrotum Linux - Software 1 07-19-2008 04:20 PM
Mounting External DVDs, Flash Drives, etc. depam Linux - Software 4 02-09-2006 04:07 PM
udev, USB Flash Drives, and usb-mount talkingwires Debian 13 11-07-2004 01:15 PM


All times are GMT -5. The time now is 03:10 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration