LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > General
User Name
Password
General This forum is for non-technical general discussion which can include both Linux and non-Linux topics. Have fun!

Notices


Reply
  Search this Thread
Old 05-02-2020, 02:46 AM   #1
qrange
Senior Member
 
Registered: Jul 2006
Location: Belgrade, Yugoslavia
Distribution: Debian stable/testing, amd64
Posts: 1,004

Rep: Reputation: 47
bittorrent protocol


it happens sometimes that torrent contains one big file (like movie), and several unimportant small files (like info, screenshots..).
the unimportant files get lost over time, so when someone wants to 'revive' torrent, it gets stuck at 99%.

I guess the new seed could make fake small files with hash collisions.
or, the protocol itself could be modified so that in these cases, people can ask and download incomplete chunks somehow?
 
Old 05-02-2020, 01:48 PM   #2
Lucko666
Member
 
Registered: Apr 2017
Location: Olympia WA
Distribution: Mint, Raspbian, Debian, elementaryOS, LinuxLite, Puppy, Manjaro, Armbian
Posts: 49

Rep: Reputation: Disabled
When using a GUI torrent client like QBitorrent, you get to choose which files to download, so if the availability for an unimportant one shows as less than 100% you can choose not to download... But if you are looking at the seeding end, not sure how to avoid this issue.
 
Old 05-02-2020, 02:17 PM   #3
qrange
Senior Member
 
Registered: Jul 2006
Location: Belgrade, Yugoslavia
Distribution: Debian stable/testing, amd64
Posts: 1,004

Original Poster
Rep: Reputation: 47
problem is, those tiny trash files are usually smaller than chunk. chunks ignore file boundaries.
one could create zero filled files for padding, though. they wouldn't increase total size much.

Last edited by qrange; 05-02-2020 at 02:18 PM.
 
Old 05-02-2020, 02:18 PM   #4
ehartman
Senior Member
 
Registered: Jul 2007
Location: Delft, The Netherlands
Distribution: Slackware
Posts: 1,585

Rep: Reputation: 818Reputation: 818Reputation: 818Reputation: 818Reputation: 818Reputation: 818Reputation: 818
Quote:
Originally Posted by Lucko666 View Post
But if you are looking at the seeding end, not sure how to avoid this issue.
Simple: don't change the directory the torrent contents came IN at all, as long as you have any intention of RE-seeding them (which in my case normally are subdirectories of a "tmp" dir). Just copy the files you want out of them to their final destination.
Like this (this is for the recent CentOS 7.8 iso's):
Code:
drwxr-xr-x  2 hartman beheer        200 2020-04-30 15:23:58 CentOS-7-x86_64-DVD-2003
-rw-r--r--  1 hartman beheer     182948 2020-04-24 13:38:00 CentOS-7-x86_64-DVD-2003.torrent
drwxr-xr-x  2 hartman beheer        208 2020-04-30 15:23:58 CentOS-7-x86_64-Everything-2003
-rw-r--r--  1 hartman beheer     392490 2020-04-24 13:39:00 CentOS-7-x86_64-Everything-2003.torrent
drwxr-xr-x  2 hartman beheer        208 2020-04-30 15:23:58 CentOS-7-x86_64-LiveGNOME-2003
-rw-r--r--  1 hartman beheer      59445 2020-04-24 13:39:00 CentOS-7-x86_64-LiveGNOME-2003.torrent
drwxr-xr-x  2 hartman beheer        200 2020-04-30 15:23:58 CentOS-7-x86_64-LiveKDE-2003
-rw-r--r--  1 hartman beheer      79079 2020-04-24 13:39:00 CentOS-7-x86_64-LiveKDE-2003.torrent
drwxr-xr-x  2 hartman beheer        200 2020-04-30 15:23:58 CentOS-7-x86_64-Minimal-2003
-rw-r--r--  1 hartman beheer      41959 2020-04-24 13:39:00 CentOS-7-x86_64-Minimal-2003.torrent
The iso's themselves are in the subdirs, with some extra files like
Code:
-rw-r--r-- 1 hartman beheer       2495 2020-04-30 15:23:58 0_README.txt
-rw-r--r-- 1 hartman beheer        598 2020-04-30 15:23:58 sha256sum.txt
-rw-r--r-- 1 hartman beheer       1458 2020-04-30 15:23:58 sha256sum.txt.asc
As long as I'm seeding (probably till the end of this month) those torrent files and dirs will stay UNtouched.

PS: IF you by accident removed some of those files, your bittorrent client should first try to RE-download them again before going into the seeding state.
 
Old 05-02-2020, 02:28 PM   #5
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 14,169
Blog Entries: 9

Rep: Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920
The OP seems to be talking about other people polluting the torrents they downloaded, and uploaders uploading superfluous OS-specific files. I have seen this over the years. Thumbs.db files that don't match up with the originally uploaded one anymore etc.
Then you're stuck at 99% even though the file you're interested in is perfectly fine.
Since the bittorrent protocol treats all files in a torrent equally I don't see what you could do about that.
But it's not a big problem - you can still download & share the important data.

TBH, this hasn't happened to me in a long time. Maybe I've learned to choose clean torrents.
 
Old 05-02-2020, 05:25 PM   #6
qrange
Senior Member
 
Registered: Jul 2006
Location: Belgrade, Yugoslavia
Distribution: Debian stable/testing, amd64
Posts: 1,004

Original Poster
Rep: Reputation: 47
yes, 'Thumbs.db' is smaller than chunk size, so you can't download the important file beginning or ending, even though the partial seeder has it.

edit:
they should have allowed variable chunk size option, so that file boundaries could be 'respected'.

Last edited by qrange; 05-03-2020 at 12:36 AM. Reason: update
 
Old 05-06-2020, 12:57 AM   #7
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 14,169
Blog Entries: 9

Rep: Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920Reputation: 3920
Quote:
Originally Posted by qrange View Post
they should have allowed variable chunk size option, so that file boundaries could be 'respected'.
By "they" I assume you mean the developers of the bittorrent protocol.
The way I see it, the idea is that a torrent is an absolutely fixed bunch of data that can be shared with others.
It needs to be the same everywhere, on every peer's machine, otherwise the sharing doesn't work.
If people upload stuff that does not adhere to that idea (e.g. it contains files that are periodically being changed by their OS) then they did it wrong.
So, if I understand you correctly, you would like to work around that problem by changing the bittorrent protocol?
 
Old 05-07-2020, 12:25 AM   #8
qrange
Senior Member
 
Registered: Jul 2006
Location: Belgrade, Yugoslavia
Distribution: Debian stable/testing, amd64
Posts: 1,004

Original Poster
Rep: Reputation: 47
well updating it.
it could be avoided if uploaders added zero padded files.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Message error: "No modules named BitTorrent.platform" Ijust can't run Bittorrent everal Slackware 2 09-26-2010 09:32 PM
What's the best P2P client that uses the BitTorrent protocol royeo Linux - Newbie 7 12-29-2008 09:58 PM
Where to turn SSHv1 protocol and SSHv2 protocol on and off Minnie Nguyen Linux - Enterprise 3 07-05-2006 02:12 PM
is there anyone using bittorrent bittorrent-shadowsclient-5.8.3 moonloader Linux - Software 0 01-12-2004 05:35 AM
Unsupported protocol 'Compression Control Protocol' (0x80fd) received RKris Linux - Software 0 08-21-2002 08:24 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > General

All times are GMT -5. The time now is 12:08 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration