Hello,
I have about 10tb clonezilla images in archives. I would like to make any order with that.
I would like to find any folder/file deduplicating solution which can allow me to set the lowest level as a driectory level, not at file level.

So,
I have about 10 image directories in each directory listed below (in /root/1st and /mnt/usbdrive). In each image directory few files can be repeated between specific images. I do not want to touch this.
When i run rmlint on that - rmlint will scan all files and removes all repeatable files from each directory. That is what I want to avoid.

But I would like to analyze whole diretiories - and compare at this level.
AND - If the another path will contain SAME directory with all it's contains - then will remove whole directory.

I understand that it looks like without sense, but for my problem is the only one answer

Example directory structure

And I would like to automatically (by any script, binary, etc) mark for deletion as duplicated content only:
/mnt/usbdrive/dir1
/mnt/usbdrive/dir2
/mnt/usbdrive/dir3

BUT not mark files inside the directories as duplicated.
So, after that we will have one occurence each of image dir. Any image directories can have have some small files duplicated between the image dirs, but that is not a problem for me.

How can I achieve that ?
regards

since you have a special comparison criteria you need to implement your own solution.
First you need to write a compare tool to know if A and B (two dirs) are equal to each other.
Next you need to walk thru your folders. I think you can find solutions for this on the net for a lot of different languages. You need to skip files, which is most probably something you need to add, but looks not so hard.
Finally you can construct a report and if you find it acceptable you can remove the unwanted directories.

I would start with a definitely smaller structure, because you can test it much faster.

From the other hand you can run an ls -lR <dir> (or similar) and work on that text file which can be fast enough.