Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Only the first file is downloaded when selecting multiple files from an RSS feed #343

Open
1 task done
d4k0 opened this issue Aug 5, 2024 · 6 comments
Open
1 task done

Only the first file is downloaded when selecting multiple files from an RSS feed #343

d4k0 opened this issue Aug 5, 2024 · 6 comments
Assignees
@dnzbk
Labels
bug Something isn't working

Comments

@d4k0
Copy link

d4k0 commented Aug 5, 2024

Is there already an issue for your problem?

  • I have checked older issues, open and closed

NZBGet Version

v24.2-stable

Platform

Windows

Environment

Device:
OS version: Windows 10 22H2 64-bit
CPU architecture: 64-bit
Includes libs or tool that apply: 7-Zip 24.07, UnRAR 7.01
Running in Docker: No

Current Behavior

I use an RSS feed from my indexer to download certain series, for example. If I now select several files for download in the RSS feed in NZBGet, only the top file is downloaded. The other files are marked as "DUPE" and moved to "History". I have to select them manually and select "Download Again" so that they are really downloaded.

The problem occurs with both NZBGet 24.1 and 24.2.

Expected Behavior

NZBGet should download all files right away and not mark all but one as "DUPE".

Steps To Reproduce

No response

Logs

No response

Extra information

  1. Select several files in the RSS feed for download:
    1_NZBGet_select_multiple_entries_from_RSS

  2. Only the first selected entry is downloaded:
    2_NZBGet_only_first_entry_is_downloaded

  3. The other entries are stored in the history tab as "DUPE":
    3_NZBGet_other_entries_are_marked_as_dupe

  4. The other entries must be selected manually in order to actually start them by clicking on "Download Again":
    4_NZBGet_other_entries_have_to_be_downloaded_again
@d4k0 d4k0 added the bug Something isn't working label Aug 5, 2024
@dnzbk dnzbk self-assigned this Aug 6, 2024
@dnzbk
Copy link
Collaborator

dnzbk commented Aug 6, 2024

I couldn't able to reproduce the bug, sorry.
What extensions are you using? Could they be the cause?
Could you send ([email protected]) the NZBs marked as "DUPE" for testing?
They can be found at ${MainDir}/nzb.
Thanks.

@d4k0
Copy link
Author

d4k0 commented Aug 6, 2024

I do not use any extensions:

NZBGet_extension_manager

I did a few more tests today. At first I thought that the indexer SceneNZBs.com or the the filter 1080p % 2160p for the RSS feed was the cause, as the problem only occurred there with the filter. But then I was able to reproduce the problem with NZBGeek without any filter:

NZBGet_tv_show_test_feed

I have created a GIF where you can see how I manually download several files via the RSS feed, some of which are immediately marked as duplicates:

NZBGet_duplicate_problem

I'm not sure, but apparently the first entries of an episode in the RSS feed are somehow "favored" (in this example, the 1080p version).

The following entries can then be found in the log:

Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E01.Are.You.Happy.in.Your.Life.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E01.Are.You.Happy.in.Your.Life.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E01.Are.You.Happy.in.Your.Life.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history
Tue Aug  6 21:41:38 2024	INFO	Queue Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024	INFO	Reordering files for Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024	INFO	Queue Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E02.Trip.of.a.Lifetime.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history
Tue Aug  6 21:41:38 2024	INFO	Queue Dark.Matter.2024.S01E03.The.Box.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024	INFO	Reordering files for Dark.Matter.2024.S01E03.The.Box.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024	INFO	Queue Dark.Matter.2024.S01E03.The.Box.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E03.The.Box.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E03.The.Box.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E03.The.Box.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history
Tue Aug  6 21:41:38 2024	INFO	Queue Dark.Matter.2024.S01E04.The.Corridor.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024	INFO	Reordering files for Dark.Matter.2024.S01E04.The.Corridor.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024	INFO	Queue Dark.Matter.2024.S01E04.The.Corridor.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested @ api.nzbgeek.info
Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E04.The.Corridor.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested is a duplicate to Dark.Matter.2024.S01E04.The.Corridor.1080p.ATVP.WEB-DL.DDP5.1.H.264-NTb-AsRequested
Tue Aug  6 21:41:38 2024	INFO	Collection Dark.Matter.2024.S01E04.The.Corridor.2160p.ATVP.WEB-DL.DDP5.1.H.265-NTb-AsRequested added to history

Here once again as screenshots:

NZBGet_download_tab

NZBGet_history_tab

The problem does not occur in SABnzbd.

Is this a bug or some function of NZBGet or the indexers?

@dnzbk
Copy link
Collaborator

dnzbk commented Aug 7, 2024

I need to investigate that more thoroughly.
Thank you for the report.

@dnzbk
Copy link
Collaborator

dnzbk commented Aug 14, 2024

I was able to reproduce such behavior, and I wouldn't say it's a bug. NZBget duplicate detection is quite advanced. In addition, rageid/season/episode it also compares imdb, tvdbid and tvmazeid fields to detect duplicates and in my cases this works well - NZBGet has successfully found the duplicated content and their Duplicate key were identical.
To find Duplicate key just click on NZB file name in HISTORY and click on Dupe-> button.

@d4k0
Copy link
Author

d4k0 commented Aug 17, 2024

Thanks for the research and the reference to the RSS documentation, from which I also learned some new things. I always thought that duplicate really meant the exact same download/file.

I have also tested it again over the last few days and gained a few new insights. Indeed, the duplicate detection seems to be the cause. If I deactivate the duplicate check under INCOMING NZBS --> DupeCheck, all selected files are always downloaded as expected.

NZBGet always seems to prefer the topmost selected entry in the RSS feed if it is the same episode. In the example from my last post, I downloaded a 1080p version and a 2160p version of the same episode, the duplicate key was then tvdbid=393159-S01-E01:

NZBGet_duplicate_key_second_example

Since the 1080p version is higher up in the RSS feed, it is “preferred” and the 2160p version is marked as a duplicate. So here it works as you described.

With the series from the first post, however, I don't understand why only the imdb value (imdb=28369634) is used as the duplicate key:

NZBGet_duplicate_key_first_example

If you download several (different) episodes, every episode except the top one selected in the RSS feed is then marked as a duplicate. I had a look at the RSS feed of the series, where the tvdbid and the respective season and episode would also be available:

<item>
	<title>Die.Landarztpraxis.S02E69.German.1080p.WEB.h264-WvF</title>
	<guid isPermaLink="true">https://scenenzbs.com/details/b1c0236dce8ed49c5a2f6b2c621aeb2b</guid>
	<link>https://scenenzbs.com/getnzb/b1c0236dce8ed49c5a2f6b2c621aeb2b.nzb&amp;i=5199&amp;r=***MyApiKey***</link>
	<comments>https://scenenzbs.com/details/b1c0236dce8ed49c5a2f6b2c621aeb2b#comments</comments> 	
	<pubDate>Sat, 10 Aug 2024 02:03:26 +0200</pubDate>
	<category>TV - DE-HD</category>
	<description><![CDATA[<div><ul><li>ID: b1c0236dce8ed49c5a2f6b2c621aeb2b</li><li>Name: <a href="https://scenenzbs.com/details/b1c0236dce8ed49c5a2f6b2c621aeb2b">Die.Landarztpraxis.S02E69.German.1080p.WEB.h264-WvF</a></li><li>Size: 1.26 GB </li><li>Attributes: Category - <a href="https://scenenzbs.com/browse?t=5140">TV - DE-HD</a></li><li>Groups: <a href="https://scenenzbs.com/browse?g="></a></li><li>Poster: Ds9QuuYtRewZJPiybFmHkPsuSRGW2fLj7i8tv3atWktN</li><li>PostDate: Sat, 10 Aug 2024 02:02:02 +0200</li><li>Password: Unknown</li></ul></div><div style="clear:both;">]]>
	</description>
	<enclosure url="https://scenenzbs.com/getnzb/b1c0236dce8ed49c5a2f6b2c621aeb2b.nzb&amp;i=5199&amp;r=***MyApiKey***" length="1357267947" type="application/x-nzb" />

	<newznab:attr name="category" value="5100" />
	<newznab:attr name="category" value="5140" />
	<newznab:attr name="size" value="1357267947" />
	<newznab:attr name="files" value="14" />
	<newznab:attr name="poster" value="Ds9QuuYtRewZJPiybFmHkPsuSRGW2fLj7i8tv3atWktN" />
	<newznab:attr name="season" value="S02" />
	<newznab:attr name="episode" value="E69" />
	<newznab:attr name="imdb" value="28369634" />
	<newznab:attr name="tvdbid" value="439270" />
	<newznab:attr name="grabs" value="29" />
	<newznab:attr name="comments" value="0" />
	<newznab:attr name="password" value="-1" />
	<newznab:attr name="usenetdate" value="Sat, 10 Aug 2024 02:02:02 +0200" />
	<newznab:attr name="group" value="" />
</item>
<item>
	<title>Die.Landarztpraxis.S02E71.German.1080p.WEB.h264-WvF</title>
	<guid isPermaLink="true">https://scenenzbs.com/details/b2d87184cb7b6618aea2835428115930</guid>
	<link>https://scenenzbs.com/getnzb/b2d87184cb7b6618aea2835428115930.nzb&amp;i=5199&amp;r=***MyApiKey***</link>
	<comments>https://scenenzbs.com/details/b2d87184cb7b6618aea2835428115930#comments</comments> 	
	<pubDate>Fri, 09 Aug 2024 23:56:36 +0200</pubDate>
	<category>TV - DE-HD</category>
	<description><![CDATA[<div><ul><li>ID: b2d87184cb7b6618aea2835428115930</li><li>Name: <a href="https://scenenzbs.com/details/b2d87184cb7b6618aea2835428115930">Die.Landarztpraxis.S02E71.German.1080p.WEB.h264-WvF</a></li><li>Size: 1.23 GB </li><li>Attributes: Category - <a href="https://scenenzbs.com/browse?t=5140">TV - DE-HD</a></li><li>Groups: <a href="https://scenenzbs.com/browse?g="></a></li><li>Poster: E75AjdxxQCiJG7gxcTLsx7jA6uLwdPzNcfQ7xoz1TAx1</li><li>PostDate: Fri, 09 Aug 2024 23:53:55 +0200</li><li>Password: Unknown</li></ul></div><div style="clear:both;">]]>
	</description>
	<enclosure url="https://scenenzbs.com/getnzb/b2d87184cb7b6618aea2835428115930.nzb&amp;i=5199&amp;r=***MyApiKey***" length="1317027165" type="application/x-nzb" />

	<newznab:attr name="category" value="5100" />
	<newznab:attr name="category" value="5140" />
	<newznab:attr name="size" value="1317027165" />
	<newznab:attr name="files" value="14" />
	<newznab:attr name="poster" value="E75AjdxxQCiJG7gxcTLsx7jA6uLwdPzNcfQ7xoz1TAx1" />
	<newznab:attr name="season" value="S02" />
	<newznab:attr name="episode" value="E71" />
	<newznab:attr name="imdb" value="28369634" />
	<newznab:attr name="tvdbid" value="439270" />
	<newznab:attr name="grabs" value="33" />
	<newznab:attr name="comments" value="0" />
	<newznab:attr name="password" value="-1" />
	<newznab:attr name="usenetdate" value="Fri, 09 Aug 2024 23:53:55 +0200" />
	<newznab:attr name="group" value="" />
</item>

I don't understand why only the imdb value is taken as the duplicate key here and why this is not supplemented by the season and episode to obtain a unique key. Does the imdb value have priority in NZBGet? Is this perhaps otherwise only used for movies, so that it is not a problem there? I also tried to add the parameter “extended=1” (the indexer uses Newznab), but nothing has changed in the RSS feed, the newznab:attr entries were already there before.

I also took a look at the RSS feed of other series, here usually only the tvdbid value is available (sometimes it is missing), I have not come across the imdb value in any other random samples. This is also the only series so far where I have encountered this problem.

@dnzbk
Copy link
Collaborator

dnzbk commented Aug 19, 2024 •
edited
Loading

Yes, imdbid is used mostly for movies, and if it exists, it has the highest priority.
From the doc:

- for movies: imdb=123456, where 123456 is imdb-id of the title;
- for series: rageid=12345-Sxx-Eyy, where 12345 is rageid of the series, xx is season number and yy is episode number. If rageid isn’t available, fields tvdbid and tvmazeid can be used instead.

But in your case, the series lists both imdb and tvdbid, which seems weird to me.
To be sure, you checked that the downloaded content really isn't the same, right?
If so, then I'll think about how to improve the Dupe detection algorithm.

extended=1 doesn't work for me either. Never used it before.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dnzbk dnzbk

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@d4k0 @dnzbk