@urx585 said in torrent files that only load correctly in a few clients:
I work in the IT industry, so this is kind-of "up my alley".
- I hadn't thought of it, but yes - a non-padded torrent WOULD appear significantly smaller... however, as I noted in my post above: those "padding" files are created as "sparse" files with all zero data. ACTUAL XFERED DATA is practically nil! Depending on the filesystem you download into, they may also store as practically zero-length files (in storage, not in the OS listing of the file size).
#2 is unfortunately incorrect for most torrent clients and filesystems. When I download a 32MB file of all zero bytes from my seedbox, it takes exactly as long as downloading a 32MB file with non-zero bytes.
I wouldn't be so sure. Firstly, compression on networks isn't like compression on old-style modems: it's enabled , generally, by default.
Secondly, the way "sparse" files are designed to work is to be "invisible" to the end user. The OS reports the "size" of the file as the "advertised" size. There is another field that is illustrative in Windows: "size on disk".
That said, many NAS devices do not support "sparse files" and if your external drives are using an older FAT32 or older NTFS filesystem, they won't support sparse files at all. But that is not the fault of the torrent uploader.
Same for ratio: when I download a torrent with lots of padding in Tixati (my alternate client when rutorrent doesn't work) all of the padding bytes count against my ratio. Not that I care, I have a 2 petabyte buffer, but some people have to worry about their ratio.
Tixati is new to me... however, it tries to be ultra-lightweight, so I suspect the logic to process sparse files (or padding files) may be absent.
For instance, when I deleted the .pad directory for the ASG torrent, I got about 34 GB of disk space back. File with all zero bytes are only more efficient if you're using compression. If you're not using compression, then on the disk, and through the network, there is no difference between files with just zeroes and files with both ones and zeroes.
That may or may not be true - that depends on your OS and filesystem. (My Macs support sparse files, as does my Linux VM where I do most of my torrent traffic. My Windows desktop does not - at least not on the NAS device where I store my porn...
qbittorrent may never actually download the padding files at all. I believe the default behavior is to completely hide everything having to do with padding. That's why so many Uploaders don't know their torrent client is adding pad files, I usually have to ask them to click on [See full list] before they believe that such a thing exists. So if you're using qbittorrent, the pad files may indeed appear to take no time to download, because they're not being downloaded. However most torrent clients don't have this feature, so they treat the pad files just like any other pic or vid file.
Again, this isn't a qBitTorrent thing, it's a libtorrent thing. Deluge, qBitTorrent, and about a dozen smaller, less popular, torrent clients all use the same core library.
#3 I don't want to require anyone to do anything, but this error was introduced recently, and it would be very nice if we could "patch" it by removing whatever extension field or metadata that's causing this error in so many other torrent clients.
I will again use the analogy of web browsers. There are still some sites out there that require IE to operate properly (pity, now that IE is officially D-E-A-D... RIP). We've all adjusted to websites being a little different on different browsers, why is it so hard to apply this to torrents?
Having said that, I do agree that a recent update to libtorrent appears to have broken backward compatibility with some other torrent clients - and that definitely should be addressed. (And as you point out below, maybe it already has...). My point is that this isn't unprecedented territory in the history of the Internet - indeed, it's actually been commonplace!
I've read a lot of comments from frustrated users on the torrents with this error, and it appears most people only use one torrent client, so when they get an error, they think there's something wrong with the torrent file, they don't think there's something wrong with their torrrent client, and they don't try a different one.
Maybe they'll learn something here, as a result of our discussion.
BTW, it's possible this error was fixed by an even newer version of qbittorrent/libtorrent, because I've recently seen a few large torrents with lots of padding files that load correctly in rutorrent. So maybe the solution is to ask all the libtorrent users to do one more upgrade. I don't know for sure, because I can't figure out what's causing the error. However when I look at the headers in these .torrent files it seems like there's some entries I haven't seen before.
#4 I download a large number of multi-file torrents from other sites. You're right, no other site has anywhere near the content that we have here, but I get probably a third of my torrents from other sites, and many of them have those pesky .srt files and thumbnail pics etc. This particular error (fails to load in rutorrent, Transmission, and others) started a few months ago. On GT, I run into this error maybe once per day (I don't post every link here, and I didn't start this thread until recently.) On the other sites, I've never had this happen, not once. Just based on the numbers, I don't think it's coincidence.
Yes, Uploaders create the initial .torrent file, but this site's processing engine modifies them too. When I upload a new torrent, the .torrent file I download from the post-upload page is not the same as the .torrent file I initially created with my torrent client.
You confuse the .torrent encoding (the blocks and checksums) with the torrent metadata - which includes the tracker URLs... thoe tracker URLs are unique for a private tracker like ours - it's how the tracker knows you're you, and who to "apply" upload and download credits to... That's a FAR CRY from editing the contents of the torrent stream data itself!
Maybe the other sites are using some logic that strips non-standard extensions and metadata from .torrent file headers.
There are other sites that edit the metadata, but without the source data, it'd be a tall, TALL order to rebuild the stream cheksum data.... just saying...
Again, really appreciate you taking the time to discuss this. If there's a solution, it may apply more broadly, and shield us from the next bug that the programmers will inevitably introduce...
I am personally hopeful that libtorrent will see the error of their ways (wrt backward compatibility) and release a patch soon. In the meantime, just as our site is best viewed on Chrome, maybe some of these torrents are best managed by Deluge or qBitTorrent? Just an idea...
Peace be with you...