torrent files that only load correctly in a few clients

urx585

In the last month or so, we've been getting some .torrent files that don't work in ruTorrent or Transmission or several other clients, but do work in some other clients like Tixati. Here are some examples:

https://www.gaytor.rent/details.php?id=a9d6281ce6fb1b97e36d8b0cc7c8502145ad456132029da0

https://www.gaytor.rent/details.php?id=6708aa31a547ef24e36d8b0cc7c85021f9a46f9d71a92173

https://www.gaytor.rent/details.php?id=ffbb1a27c53ddd6ae36d8b0cc7c850212b4abc5a373bf860

https://www.gaytor.rent/details.php?id=2e033de8619efa31e36d8b0cc7c8502127a1783d445c6349

This is not the same as the gaytor.rent truncated download bug (which was reported fixed). This bug happens the same, whether you download from gaytorrent.ru or gaytor.rent. The .torrent file is the correct length, but only loads properly in a few clients.

This might be a site-specific bug. I've never seen this happen on any other torrent site.

@raphjd or any other admin - could you please ask somebody to look into this? Maybe it can be fixed with a change or upgrade to the .torrent processing code during the upload process? Thanks!!!

I also sent a query to the Helpdesk.

If this keeps happening, I'll try to post more examples here...

john32123666

all of this torrent has padding files [created by bittorrent/qbitorrent]], so it has something to do with it ... Either file has been configured correctly from posting or not ... That is why it is creating issues ...

lololulu19

@urx585 said in torrent files that only load correctly in a few clients:

In the last month or so, we've bee

All four worked fine for me.. I use Qbittorrent

Utorrent aka Bittorrent have bugs which will never be fixed because the programmers are gone. The only updates being done are to things not connected with the functionality of torrenting.

urx585

Thanks for the info @lololulu19 and @john32123666 that's very helpful.

Here is another torrent that works with Tixati and qbittorrent, but doesn't load in ruTorrent and Transmission:

https://www.gaytorrent.ru/details.php?id=a918ac96bb2367e4e36d8b0cc7c85021226b58414ad7c0bf

This one also has padding files.

urx585

Here are two more that load correctly in Tixati, qbittorrent, and uTorrent 2.2.1, but don't load correctly in ruTorrent, Transmission, some earlier versions of uTorrent, and other clients:

https://www.gaytorrent.ru/details.php?id=fa91ed68c168997de36d8b0cc7c85021629758187e2f9192

https://www.gaytorrent.ru/details.php?id=8fe43a6c31f4883ee36d8b0cc7c85021cb7784d5f5ba22e1

Both of these have padding files in a .pad directory.

There are lots of torrents on this site with padding files that load fine in all clients. I think this bug is specific to a recent update of bittorrent/qbittorrent. It started happening until a few months ago. Prior to that, I never had any trouble loading any .torrent file in ruTorrent.

urx585

Here is another that doesn't load correctly in ruTorrent, Transmission, and some others:

https://www.gaytorrent.ru/details.php?id=b96845db072700efe36d8b0cc7c85021438ab60de8b82e6d

The Uploader was kind enough to reply and let us know that the original .torrent file was created with QBittorrent 4.4.2 (the latest version).

Because this particular problem started happening about two months ago, I'm guessing the latest 4.4.2 version changes how torrents are created, in a way that is incompatible with some other clients.

Mods @raphjd or @DandyLion - can you please ask if it's possible to fix this problem during the .torrent upload process, by removing or altering the new thing that's causing problems?

I've also communicated with @MrMazda via the Helpdesk.

Thank you!!!

kahrun

@urx585 said in torrent files that only load correctly in a few clients:

https://www.gaytorrent.ru/details.php?id=fa91ed68c168997de36d8b0cc7c85021629758187e2f9192

urx585

Here is another that doesn't load in all torrent clients:

https://www.gaytor.rent/details.php?id=523d76ef473b7357e36d8b0cc7c85021494b1f155e4f23a2

urx585

Here's another that doesn't load in some clients:

https://www.gaytorrent.ru/details.php?id=81f383cfd172816fe36d8b0cc7c85021fe6fa0f9067eb7f4

But this one, with the same .pad files and from the same uploader, did load correctly in ruTorrent:

https://www.gaytorrent.ru/details.php?id=84dbfeeba59b9ef2e36d8b0cc7c8502144a4fda46cd45dd3

No idea what the difference is, except the second is smaller than the first.

And here's another larger torrent that doesn't load in some clients, also with .pad files:

https://www.gaytorrent.ru/details.php?id=ee8de7a2fd80e9e0e36d8b0cc7c850212568fe7288f45e39

urx585

Here's another: https://www.gaytor.rent/details.php?id=6a809e77600f674be36d8b0cc7c8502181602e1508fbea21

@raphjd or @DandyLion or @MrMazda -- would it be possible to post a site-wide notice recommending that uploaders turn off padding? This torrent has > 33 GB of padding files with meaningless random data. Some downloaders will not realize that they have to unselect all the padding files, so they will waste quite a bit of their ratio on useless files. And most Uploaders don't even realize that their torrent clients are adding pad files... Thanks!

WikiDude

@urx585

I am not the staffer you asked for, but I want to make a few things clear:

Padding is not useless - it is used in "collections" (torrents with multiple files) to aid users who want to download only PORTIONS of the torrent.
Read more here: https://stackoverflow.com/questions/31925507/torrent-padding-files#
Padding (in clients that recognize it for what it is) does not inflate download size. Clients that properly recognize the padding won't actually download the "pad" data. ALSO: when done right, the PAD data is all-zeros... so that when compressed, it has almost no content at all.
When uploaders supply a torrent to the site, the site DOES validate the torrent - so what you're asking might be possible programmatically, but with so many other things being worked on right now, it might not make it to the "high-priority" list.

So, what's a boy to do?
Well, one of the torrent clients that DOES handle padding properly is qBitTorrent. Also, there is absolutely no reason you can't have TWO torrent clients installed on your system... when your older-but-favored client chokes on a torrent due to padding, use the qBitTorrent client for that torrent, then go back to your normal life.

Here's a how-to for the qBitTorrent client - for Windows, Mac, and Linux: https://community.gaytorrent.ru/topic/53086/qbittorrent-gt-ru-guide

urx585

@WikiDude thanks for looking at this, I really appreciate it!

I am responding to your most recent post in that torrent's comments, quoted here:

Thanks for your comments:

this isn't a qBitTorrent issue, it is a libtorrent issue (libtorrent is the "driver" software that qBitTorrent uses - but it is from a different development team. Kind of like differentiating between the Windows OS, and the qBitTorrent app.)
I hadn't thought of it, but yes - a non-padded torrent WOULD appear significantly smaller... however, as I noted in my post above: those "padding" files are created as "sparse" files with all zero data. ACTUAL XFERED DATA is practically nil! Depending on the filesystem you download into, they may also store as practically zero-length files (in storage, not in the OS listing of the file size).
exactly how is requiring uploaders to use a specific encoding client any different from requiring downloaders to use a different torrent client? Keep in mind that there is nothing preventing you from using multiple torrent clients on your system... I would presume that you have more than 1 browser on your system?
Neither this site, nor any torrent site that I am aware of, actually CREATES the .torrent data! Uploaders (users) do that!
That you haven't run into this issue on another site is simply a matter of chance. To wit: How many "collections" are you downloading from other sites? (For my own personal use, this site has an enormously larger percentage of "collection" torrents - the ONLY torrents this would apply to, as padding data exists solely to "space out" data in the stream so that individual files always start on a "block boundary"). I'd say that, other than torrents where there is a separate .srt file, or those damnable uploader text files, the VAST majority of my downloads from other locations are single files...

#2 is unfortunately incorrect for most torrent clients and filesystems. When I download a 32MB file of all zero bytes from my seedbox, it takes exactly as long as downloading a 32MB file with non-zero bytes.

Same for ratio: when I download a torrent with lots of padding in Tixati (my alternate client when rutorrent doesn't work) all of the padding bytes count against my ratio. Not that I care, I have a 2 petabyte buffer, but some people have to worry about their ratio.

For instance, when I deleted the .pad directory for the ASG torrent, I got about 34 GB of disk space back. File with all zero bytes are only more efficient if you're using compression. If you're not using compression, then on the disk, and through the network, there is no difference between files with just zeroes and files with both ones and zeroes.

qbittorrent may never actually download the padding files at all. I believe the default behavior is to completely hide everything having to do with padding. That's why so many Uploaders don't know their torrent client is adding pad files, I usually have to ask them to click on [See full list] before they believe that such a thing exists. So if you're using qbittorrent, the pad files may indeed appear to take no time to download, because they're not being downloaded. However most torrent clients don't have this feature, so they treat the pad files just like any other pic or vid file.

#3 I don't want to require anyone to do anything, but this error was introduced recently, and it would be very nice if we could "patch" it by removing whatever extension field or metadata that's causing this error in so many other torrent clients.

I've read a lot of comments from frustrated users on the torrents with this error, and it appears most people only use one torrent client, so when they get an error, they think there's something wrong with the torrent file, they don't think there's something wrong with their torrrent client, and they don't try a different one.

BTW, it's possible this error was fixed by an even newer version of qbittorrent/libtorrent, because I've recently seen a few large torrents with lots of padding files that load correctly in rutorrent. So maybe the solution is to ask all the libtorrent users to do one more upgrade. I don't know for sure, because I can't figure out what's causing the error. However when I look at the headers in these .torrent files it seems like there's some entries I haven't seen before.

#4 I download a large number of multi-file torrents from other sites. You're right, no other site has anywhere near the content that we have here, but I get probably a third of my torrents from other sites, and many of them have those pesky .srt files and thumbnail pics etc. This particular error (fails to load in rutorrent, Transmission, and others) started a few months ago. On GT, I run into this error maybe once per day (I don't post every link here, and I didn't start this thread until recently.) On the other sites, I've never had this happen, not once. Just based on the numbers, I don't think it's coincidence.

Yes, Uploaders create the initial .torrent file, but this site's processing engine modifies them too. When I upload a new torrent, the .torrent file I download from the post-upload page is not the same as the .torrent file I initially created with my torrent client.

Maybe the other sites are using some logic that strips non-standard extensions and metadata from .torrent file headers.

Again, really appreciate you taking the time to discuss this. If there's a solution, it may apply more broadly, and shield us from the next bug that the programmers will inevitably introduce...

bi4smooth

@urx585 said in torrent files that only load correctly in a few clients:

I work in the IT industry, so this is kind-of "up my alley".

I hadn't thought of it, but yes - a non-padded torrent WOULD appear significantly smaller... however, as I noted in my post above: those "padding" files are created as "sparse" files with all zero data. ACTUAL XFERED DATA is practically nil! Depending on the filesystem you download into, they may also store as practically zero-length files (in storage, not in the OS listing of the file size).
#2 is unfortunately incorrect for most torrent clients and filesystems. When I download a 32MB file of all zero bytes from my seedbox, it takes exactly as long as downloading a 32MB file with non-zero bytes.

I wouldn't be so sure. Firstly, compression on networks isn't like compression on old-style modems: it's enabled , generally, by default.

Secondly, the way "sparse" files are designed to work is to be "invisible" to the end user. The OS reports the "size" of the file as the "advertised" size. There is another field that is illustrative in Windows: "size on disk".

That said, many NAS devices do not support "sparse files" and if your external drives are using an older FAT32 or older NTFS filesystem, they won't support sparse files at all. But that is not the fault of the torrent uploader.

Same for ratio: when I download a torrent with lots of padding in Tixati (my alternate client when rutorrent doesn't work) all of the padding bytes count against my ratio. Not that I care, I have a 2 petabyte buffer, but some people have to worry about their ratio.

Tixati is new to me... however, it tries to be ultra-lightweight, so I suspect the logic to process sparse files (or padding files) may be absent.

For instance, when I deleted the .pad directory for the ASG torrent, I got about 34 GB of disk space back. File with all zero bytes are only more efficient if you're using compression. If you're not using compression, then on the disk, and through the network, there is no difference between files with just zeroes and files with both ones and zeroes.

That may or may not be true - that depends on your OS and filesystem. (My Macs support sparse files, as does my Linux VM where I do most of my torrent traffic. My Windows desktop does not - at least not on the NAS device where I store my porn...

qbittorrent may never actually download the padding files at all. I believe the default behavior is to completely hide everything having to do with padding. That's why so many Uploaders don't know their torrent client is adding pad files, I usually have to ask them to click on [See full list] before they believe that such a thing exists. So if you're using qbittorrent, the pad files may indeed appear to take no time to download, because they're not being downloaded. However most torrent clients don't have this feature, so they treat the pad files just like any other pic or vid file.

Again, this isn't a qBitTorrent thing, it's a libtorrent thing. Deluge, qBitTorrent, and about a dozen smaller, less popular, torrent clients all use the same core library.

#3 I don't want to require anyone to do anything, but this error was introduced recently, and it would be very nice if we could "patch" it by removing whatever extension field or metadata that's causing this error in so many other torrent clients.

I will again use the analogy of web browsers. There are still some sites out there that require IE to operate properly (pity, now that IE is officially D-E-A-D... RIP). We've all adjusted to websites being a little different on different browsers, why is it so hard to apply this to torrents?

Having said that, I do agree that a recent update to libtorrent appears to have broken backward compatibility with some other torrent clients - and that definitely should be addressed. (And as you point out below, maybe it already has...). My point is that this isn't unprecedented territory in the history of the Internet - indeed, it's actually been commonplace!

I've read a lot of comments from frustrated users on the torrents with this error, and it appears most people only use one torrent client, so when they get an error, they think there's something wrong with the torrent file, they don't think there's something wrong with their torrrent client, and they don't try a different one.

Maybe they'll learn something here, as a result of our discussion.

BTW, it's possible this error was fixed by an even newer version of qbittorrent/libtorrent, because I've recently seen a few large torrents with lots of padding files that load correctly in rutorrent. So maybe the solution is to ask all the libtorrent users to do one more upgrade. I don't know for sure, because I can't figure out what's causing the error. However when I look at the headers in these .torrent files it seems like there's some entries I haven't seen before.

#4 I download a large number of multi-file torrents from other sites. You're right, no other site has anywhere near the content that we have here, but I get probably a third of my torrents from other sites, and many of them have those pesky .srt files and thumbnail pics etc. This particular error (fails to load in rutorrent, Transmission, and others) started a few months ago. On GT, I run into this error maybe once per day (I don't post every link here, and I didn't start this thread until recently.) On the other sites, I've never had this happen, not once. Just based on the numbers, I don't think it's coincidence.

Yes, Uploaders create the initial .torrent file, but this site's processing engine modifies them too. When I upload a new torrent, the .torrent file I download from the post-upload page is not the same as the .torrent file I initially created with my torrent client.

You confuse the .torrent encoding (the blocks and checksums) with the torrent metadata - which includes the tracker URLs... thoe tracker URLs are unique for a private tracker like ours - it's how the tracker knows you're you, and who to "apply" upload and download credits to... That's a FAR CRY from editing the contents of the torrent stream data itself!

Maybe the other sites are using some logic that strips non-standard extensions and metadata from .torrent file headers.

There are other sites that edit the metadata, but without the source data, it'd be a tall, TALL order to rebuild the stream cheksum data.... just saying...

Again, really appreciate you taking the time to discuss this. If there's a solution, it may apply more broadly, and shield us from the next bug that the programmers will inevitably introduce...

I am personally hopeful that libtorrent will see the error of their ways (wrt backward compatibility) and release a patch soon. In the meantime, just as our site is best viewed on Chrome, maybe some of these torrents are best managed by Deluge or qBitTorrent? Just an idea...

Peace be with you...

WikiDude

Can I assume this topic is done then? (Sorry, I was away from this topic for a bit - distracted by other business)

urx585

@WikiDude - "distracted by other business" - me too, sorry for the delay.

I want to separate my dislike of padding files from this particular problem.

As you mentioned, other sites edit the .torrent file headers / metadata by which I mean the human-readable ASCII text stuff at the start of the .torrent file. Of course it is mathematically impossible to edit the CRC data without access to the original files.

I believe this particular problem is in the metadata, not the CRC data, which means the site processing code could fix it, at least theoretically.

Here's a really interesting data point: someone recently uploaded 3 HYB collections, all with padding files. The first has this particular problem, it doesn't load in rutorrent etc:

https://www.gaytorrent.ru/details.php?id=fc1302d52e36a1f4e36d8b0cc7c850219bb656a051b3bd24

But the second and third load correctly!

https://www.gaytorrent.ru/details.php?id=daf5924632c7b6eae36d8b0cc7c850216126d5762d5a277f

https://www.gaytorrent.ru/details.php?id=21c9b85cbd858c96e36d8b0cc7c85021a59b36317074b75d

Unfortunately I can't see any major differences in the header / metadata. All three have this odd "root32" keyword followed by non-ASCII text, which I've never seen before in any other .torrent file.

Maybe the other sites are automatically stripping out any keyword they don't recognize, like "root32", and that kept this particular problem from happening to them.

Anyway, thanks again for all the help and support, it's always nice to think through these issues with another pro

WikiDude

@urx585

Since the point of the padding files is to "shift" the start of each separate file to the beginning of a new "block", it most certainly DOES affect the CRC encoding: the contents of the blocks - even the NUMBER of blocks - is altered.

Let's use a simple example - say I have 3 files, each exactly 1kb in length, and let's suppose our .torrent file has been coded to use 4kb blocks.

without padding, the FIRST 4kb block would contain ALL THREE FILES - so, no matter what you "selected", you would have to download all 3 files. 1 block total though (and it would be a "short block": only 3kb of the max 4kb block size).
WITH padding, you would have to download 3 separate 4kb blocks ... 12kb in total, for data that is truly only 3kb in size! (Actually, you would download 3 blocks, but the last one would NOT have any padding - it would be a "short block", with only the 1kb of data - that last block, and ONLY the last block, is allowed to be (expected to be) "short"... so, in FACT, you would download 9kb of data (4k+4k+1k)).

Thus, using padding is a decent idea when you have a torrent with lots of MOVIES (large files), but it is much less so when you have a collection of short (picture) files. In fact, a collection of 1000's of pictures would be absurdly large if used with padding...

Just some thoughts....

urx585

@WikiDude completely agree with your last post.

Some good news! This torrent has the problem with loading, despite the fact the Uploader is using the latest version of qbittorrent:

https://www.gaytor.rent/details.php?id=a63f1c805bc2cb03c5b667a78b72ba10eaa36d4243ea72b9

But @duncalem posted a solution in the comment section:

"When using qbittorrent, under tools ---> torrent creator, set torrent format to V1. That got rid of the padding for me and I upload big chunky torrents. Nobody has had an issue."

So it sounds like this problem is NOT fixed in the latest libtorrent release... but if everyone sets their torrent type to V1, this problem will not occur again.

Would it be possible to add this to the site-wide banner? "If you're using qbittorrent, under tools ---> torrent creator, please set torrent format to V1, otherwise many users will not be able to download your torrent."

Thanks again!

Douseikekkon

@urx585
I have the same issue (using Transmission on mac).
Here is an example of a torrent which is declared as “invalid” by Transmission, and has no .pad file inside:

https://www.gaytor.rent/details.php?id=60032beb6c08a7e113fdca579e25a0ff996dd7cc60dbd838&hit=1