Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Introduction to Linux - A Hands on Guide
This guide was created as an overview of the Linux Operating System, geared toward new users as an exploration tour and getting started guide, with exercises at the end of each chapter.
For more advanced trainees it can be a desktop reference, and a collection of the base knowledge needed to proceed with system and network administration. This book contains many real life examples derived from the author's experience as a Linux system and network administrator, trainer and consultant. They hope these examples will help you to get a better understanding of the Linux system and that you feel encouraged to try out things on your own.
Click Here to receive this Complete Guide absolutely free.
I've been looking for a while, and I cannot find an app, or system plugin, or desktop setting, which will allow me to take a set of files, and tag them according to my wishes. Let me add a set of filenames to this application, tag each as "bill," "receipt," "medical," "auto," etc., and I will be happy.
So far, my solution is referencer, but my use case is very much not the goal of the project. Referencer is a way to organize bibliographical documents in a research project, not a general purpose organizer for personal scanned documents.
How do other people do this? How do you organize the scanned bills and other stuff you get in the mail? How do you take a stack of files a categorize them with useful tags?
It's a bit of a do-it-yourself system, which could be good or bad depending on your persuasion. I could swear I've seen a pre-packaged, tag-based document management system somewhere though. I'll probably do a little more searching just in case.
--post edit: I looked a little more closely at the description, and I can't guarantee that DocMGR allows for tag-based organization. There is reference to "keywords" in some of the documentation, but no clear indication that "keywords" and "tags" are equivalent.
What desktop (name and version) are you using and what distro (name and version)?
ubuntu at work, mint at home, and debian on the server. But it shouldn't matter. If the solution is so coupled to the desktop, it isn't a solution to this problem. Referencer does fit the puzzle, only it is not designed for my use case. I'm trying to make sure there isn't an obvious solution, before I start dedicating development time to referencer to mold it to my needs.
It's a bit of a do-it-yourself system, which could be good or bad depending on your persuasion.
It's is often good, but like most of us, my time is valuable. I stopped reading the link after it mentioned "open office base." OObase is interesting, but I'm not going to develop a personal microsoft-access-style solution around OO just because I can. It seems that this problem should be more common. I get paper in the mail, I scan it into a computer, and I want to organize it. How do I do this?
The last two options I'll throw at you are from this Open Source Document Management blog post. The link to KnowledgeTree in the article is dead. But the link to jLibrary is still good. Scanning over the description, it looks like jLibrary supports customizable meta information for files.
I did find (what I assume is) the open source Knowledge Tree document management system on Sourceforge.
I have been working on a general file tagging program for a little while now. It's called TMSU and works by providing a tool with which you can tag your files. It also then lets you mount a tag based view of your files so that you can use tags to access your files from any other program.
(It's GPL3 and tested only on Linux at present, though it should theoretically work on BSD too. I've had a report it's not yet working on OSX. Windows port planned but not started yet.)
---------- Post added 06-07-12 at 11:07 AM ----------
Add one more to the list =). Sorry for the necro, but I'm wondering if this is still an issue and whether the following seems like a good solution for it: (I can't post links yet, but search for 'arch linux hitagiFS' for the thread) (disclaimer: this is my project). In short, it provides a tag-based general file organization system based on hard links.
The reason for this shameless plug is that I'm looking to see if there is a need for the project I'm working on. If no one needs it, then I won't be as motivated getting it into shape. If there IS a need for it, I'll be glad to work on it, and hopefully others can get some use out of it.
I've looked at tagsistant, but it wasn't for me. Plus, it looks abandoned now. I haven't seen TMSU before, but it looks similar. The difference is that mine (hitagiFS) is much more transparent, relying solely on file system hard links and symlinks. There's no database or portability issues. If you use it for a while and decide that you don't like it, you can ditch the program, but the directory structure is left the same.
The reason I'm putting this here is (again) I would like to see if there's still interest in yet another (although quite different) tag-based file organization system. If not, I won't work as hard on it. As much as I like contributing, I'm not going to put in hours if no one'll appreciate it =).
Sorry if I was wrong. I think the last time I checked, the last news item was from a while back, and the documentation and everything seemed incomplete/old. But I seem to be wrong.
I think your tagsistant is cool, and it was one of the options I considered for my needs, but the documentation is a little messy. Could you explain the internals of tagsistant a little more? As I'm working on my program (now called "dantalian" because the first name was ill-chosen), I think there may be some duplication of effort here.
Are you storing the files in MySQL, or just the paths to the files? How do you handle filenames (I heard something about unique names for the entire database, but that may be wrong/outdated)?
P.S. I just read the 0.6 howto and I must say it looks a lot better than when I looked before, I am considering dropping my own project with mixed feelings, but I still think there's a fundamental difference in our approaches to the problem. For example, the deduplication: What if I want two separate identical files (because I want to edit one of them in the future)? It seems like you can't do that with tagsistant.
I'm also interested in your backend, and performance with huge numbers of tags/files (100,000 tags with as many files each, for example), as a programmers asking another for advice.
@darkfeline: I noticed right now that your post dates to January 2013 and I'm not sure I published news about 0.6 before February, so it's perfectly possible that you perceived Tagsistant as an abandoned project. The good news it's: it isn't.
There are still minor issues with Tagsistant 0.6 I'm fixing in the SVN repository, publishing a new release candidate every week or two. Yesterday I've changed a callback used to retrieve an integer from an SQL query because it used arbitrary chosen libDBI functions to get the number, while now the callback checks the return type and uses the proper libDBI function to fetch it.
I'm not storing the file content in MySQL, just the name of the file. The file is actually stored in the archive/ directory inside the repository. I plan to organize archive/ in subfolders based on the inode of the objects stored to avoid hogging the archive/ directory and slowing down its browsing.
The issue about unique names in the filesystem is related to Tagsistant 0.2 (another reason for dropping it in favour of Tagsistant 0.6).
You are right about deduplication: if two files with the same content (same MD5 hash) are created, the second gets deleted and its tag-set gets transferred to the first copy. So it's not possible to edit just one copy: both are altered because they're one.
This is also a bit rough because can confuse the user: if two identical files named A.jpg and B.jpg are respectively copied inside tag1/ and tag2/ directories, after deduplication the file in tag2/ is called A.jpg too! But I'll address this in a future release.
About performance: this is hitting a nerve. The biggest load in every query is due to:
parse the query in tokens
build the tag tree (a tree representing the tags involved and the unions made with +/)
reasoning the tag tree (that's finding related tags)
build the corresponding SQL query (just for readdir)
The second and third steps are the most SQL intensive. So I decided to create a query cache which follows the previous list just the first time, duplicates cached queries from second time on, and deletes cache entries as soon as a new relation involves at least one of the tags included in the entry.
This is giving a 5-10x improvement! But I still don't have metrics about 100K tags, so I can't really answer to your question.
Last edited by Tx0; 04-14-2013 at 06:15 AM.
Reason: improve the language
I haven't seen TMSU before, but it looks similar. The difference is that mine (hitagiFS) is much more transparent, relying solely on file system hard links and symlinks. There's no database or portability issues. If you use it for a while and decide that you don't like it, you can ditch the program, but the directory structure is left the same.
TMSU does not work this way. It does not store any files (only metadata) in its database and does not alter the original filesystem in any way, so you can likewise ditch TMSU and start up exactly where you started (only with a marginally shorter life). I think I'll throw up a comparison table on the TMSU wiki so that prospective users can pick the tool most appropriate for them.