pimhwang
/
ErsatzTV
mirror of https://github.com/ErsatzTV/ErsatzTV.git
1
0
Fork 0
Code Issues Projects Releases Wiki Activity
Browse Source

fix: local folder etags (#3004) 

* chore: update dependencies

* fix: local folder etags
pull/3005/head
Jason Dove 1 week ago committed by GitHub
parent
d5f113159f
commit
4a2aa1570c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
19 changed files with 83 additions and 38 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 2
      CHANGELOG.md
  2. 2
      ErsatzTV.Application/ErsatzTV.Application.csproj
  3. 6
      ErsatzTV.Core.Tests/ErsatzTV.Core.Tests.csproj
  4. 4
      ErsatzTV.Core/ErsatzTV.Core.csproj
  5. 4
      ErsatzTV.FFmpeg.Tests/ErsatzTV.FFmpeg.Tests.csproj
  6. 4
      ErsatzTV.FFmpeg/ErsatzTV.FFmpeg.csproj
  7. 4
      ErsatzTV.Infrastructure.Tests/ErsatzTV.Infrastructure.Tests.csproj
  8. 16
      ErsatzTV.Infrastructure/ErsatzTV.Infrastructure.csproj
  9. 4
      ErsatzTV.Scanner.Tests/ErsatzTV.Scanner.Tests.csproj
  10. 5
      ErsatzTV.Scanner/Core/Errors/ErrorProcessingFolder.cs
  11. 1
      ErsatzTV.Scanner/Core/Metadata/ImageFolderScanner.cs
  12. 12
      ErsatzTV.Scanner/Core/Metadata/MovieFolderScanner.cs
  13. 1
      ErsatzTV.Scanner/Core/Metadata/MusicVideoFolderScanner.cs
  14. 1
      ErsatzTV.Scanner/Core/Metadata/OtherVideoFolderScanner.cs
  15. 33
      ErsatzTV.Scanner/Core/Metadata/TelevisionFolderScanner.cs
  16. 2
      ErsatzTV.Scanner/ErsatzTV.Scanner.csproj
  17. 4
      ErsatzTV.Tests/ErsatzTV.Tests.csproj
  18. 6
      ErsatzTV.sln
  19. 10
      ErsatzTV/ErsatzTV.csproj

2
CHANGELOG.md
Unescape View File

@ -33,6 +33,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). @@ -33,6 +33,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- `<country>` from NFO metadata is now searchable with `country:` instead of `tag:`; deep scan a local library to convert existing items
- Fix multiple channel startup failure causes
- Fix seeking more than 24 hours into content
- Fix local movie and television folder scanner to only set etag when all files process successfully
- Previously, errors would be ignored so problematic files (e.g. malformed nfo files) would need to be touched for ETV to attempt reading them again
## [26.8.1] - 2026-08-29
### Security

2
ErsatzTV.Application/ErsatzTV.Application.csproj
Unescape View File

@ -11,7 +11,7 @@ @@ -11,7 +11,7 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliWrap" Version="3.10.4" />
<PackageReference Include="CliWrap" Version="3.10.5" />
<PackageReference Include="Humanizer.Core" Version="3.0.10" />
<PackageReference Include="MediatR" Version="[12.5.0]" />
<PackageReference Include="Microsoft.Extensions.Caching.Abstractions" Version="10.0.11" />

6
ErsatzTV.Core.Tests/ErsatzTV.Core.Tests.csproj
Unescape View File

@ -7,17 +7,17 @@ @@ -7,17 +7,17 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliWrap" Version="3.10.4" />
<PackageReference Include="CliWrap" Version="3.10.5" />
<PackageReference Include="LanguageExt.Core" Version="4.4.9" />
<PackageReference Include="Microsoft.Extensions.Caching.Memory" Version="10.0.11" />
<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="10.0.11" />
<PackageReference Include="Microsoft.Extensions.Logging" Version="10.0.11" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.11" />
<PackageReference Include="Microsoft.Extensions.Logging.Debug" Version="10.0.11" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.8.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="NSubstitute" Version="6.2.0" />
<PackageReference Include="NUnit" Version="4.6.1" />
<PackageReference Include="NUnit3TestAdapter" Version="6.2.0" />
<PackageReference Include="NUnit3TestAdapter" Version="6.3.0" />
<PackageReference Include="Serilog" Version="4.4.0" />
<PackageReference Include="Serilog.Extensions.Logging" Version="10.0.0" />
<PackageReference Include="Serilog.Sinks.Debug" Version="3.0.0" />

4
ErsatzTV.Core/ErsatzTV.Core.csproj
Unescape View File

@ -25,8 +25,8 @@ @@ -25,8 +25,8 @@
<PackageReference Include="Newtonsoft.Json" Version="13.0.4" />
<PackageReference Include="Serilog" Version="4.4.0" />
<PackageReference Include="Serilog.Sinks.Console" Version="6.1.1" />
<PackageReference Include="SkiaSharp" Version="4.151.1" />
<PackageReference Include="SkiaSharp.NativeAssets.Linux.NoDependencies" Version="4.151.1" />
<PackageReference Include="SkiaSharp" Version="4.151.2" />
<PackageReference Include="SkiaSharp.NativeAssets.Linux.NoDependencies" Version="4.151.2" />
<PackageReference Include="System.CommandLine" Version="2.0.11" />
<PackageReference Include="Testably.Abstractions" Version="10.3.0" />
<PackageReference Include="TimeSpanParserUtil" Version="1.2.0" />

4
ErsatzTV.FFmpeg.Tests/ErsatzTV.FFmpeg.Tests.csproj
Unescape View File

@ -10,10 +10,10 @@ @@ -10,10 +10,10 @@
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.DependencyModel" Version="10.0.11" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.11" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.8.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="NSubstitute" Version="6.2.0" />
<PackageReference Include="NUnit" Version="4.6.1" />
<PackageReference Include="NUnit3TestAdapter" Version="6.2.0" />
<PackageReference Include="NUnit3TestAdapter" Version="6.3.0" />
<PackageReference Include="Shouldly" Version="4.3.0" />
<PackageReference Include="coverlet.collector" Version="10.0.1">
<PrivateAssets>all</PrivateAssets>

4
ErsatzTV.FFmpeg/ErsatzTV.FFmpeg.csproj
Unescape View File

@ -11,10 +11,10 @@ @@ -11,10 +11,10 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="CliWrap" Version="3.10.4" />
<PackageReference Include="CliWrap" Version="3.10.5" />
<PackageReference Include="Hardware.Info" Version="110.0.0.1" />
<PackageReference Include="LanguageExt.Core" Version="4.4.9" />
<PackageReference Include="Lennox.NvEncSharp" Version="2.1.1" />
<PackageReference Include="Lennox.NvEncSharp" Version="2.2.0" />
<PackageReference Include="Microsoft.Extensions.Caching.Abstractions" Version="10.0.11" />
<PackageReference Include="Microsoft.Extensions.Logging.Abstractions" Version="10.0.11" />
</ItemGroup>

4
ErsatzTV.Infrastructure.Tests/ErsatzTV.Infrastructure.Tests.csproj
Unescape View File

@ -9,10 +9,10 @@ @@ -9,10 +9,10 @@
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.8.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="NSubstitute" Version="6.2.0" />
<PackageReference Include="NUnit" Version="4.6.1" />
<PackageReference Include="NUnit3TestAdapter" Version="6.2.0" />
<PackageReference Include="NUnit3TestAdapter" Version="6.3.0" />
<PackageReference Include="NUnit.Analyzers" Version="4.14.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>

16
ErsatzTV.Infrastructure/ErsatzTV.Infrastructure.csproj
Unescape View File

@ -13,12 +13,12 @@ @@ -13,12 +13,12 @@
<ItemGroup>
<PackageReference Include="Blurhash.SkiaSharp" Version="2.0.0" />
<PackageReference Include="CliWrap" Version="3.10.4" />
<PackageReference Include="CliWrap" Version="3.10.5" />
<PackageReference Include="Dapper" Version="2.1.79" />
<PackageReference Include="EFCore.BulkExtensions" Version="[9.0.2,10)" />
<PackageReference Include="Elastic.Clients.Elasticsearch" Version="9.5.0" />
<PackageReference Include="Elastic.Clients.Elasticsearch" Version="9.5.2" />
<PackageReference Include="Humanizer.Core" Version="3.0.10" />
<PackageReference Include="Jint" Version="4.15.3" />
<PackageReference Include="Jint" Version="4.16.1" />
<PackageReference Include="JsonSchema.Net" Version="9.4.0" />
<PackageReference Include="Lucene.Net" Version="4.8.0-beta00018" />
<PackageReference Include="Lucene.Net.Analysis.Common" Version="4.8.0-beta00018" />
@ -30,13 +30,13 @@ @@ -30,13 +30,13 @@
</PackageReference>
<PackageReference Include="Microsoft.EntityFrameworkCore.Relational" Version="9.0.15" />
<PackageReference Include="Newtonsoft.Json.Schema" Version="4.0.1" />
<PackageReference Include="Refit" Version="15.0.0" />
<PackageReference Include="Refit.Newtonsoft.Json" Version="15.0.0" />
<PackageReference Include="Refit.Xml" Version="15.0.0" />
<PackageReference Include="Refit" Version="15.2.0" />
<PackageReference Include="Refit.Newtonsoft.Json" Version="15.2.0" />
<PackageReference Include="Refit.Xml" Version="15.2.0" />
<PackageReference Include="RichTextKit.Stbear" Version="0.4.167.3" />
<PackageReference Include="Scriban.Signed" Version="7.2.6" />
<PackageReference Include="Scriban.Signed" Version="7.4.0" />
<PackageReference Include="SixLabors.ImageSharp" Version="[3.1.12,4)" />
<PackageReference Include="SkiaSharp" Version="4.151.1" />
<PackageReference Include="SkiaSharp" Version="4.151.2" />
<PackageReference Include="TagLibSharp" Version="2.3.0" />
<PackageReference Include="TimeZoneConverter" Version="7.2.0" />
</ItemGroup>

4
ErsatzTV.Scanner.Tests/ErsatzTV.Scanner.Tests.csproj
Unescape View File

@ -10,10 +10,10 @@ @@ -10,10 +10,10 @@
<ItemGroup>
<PackageReference Include="LanguageExt.Core" Version="4.4.9" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.8.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="NSubstitute" Version="6.2.0" />
<PackageReference Include="NUnit" Version="4.6.1" />
<PackageReference Include="NUnit3TestAdapter" Version="6.2.0" />
<PackageReference Include="NUnit3TestAdapter" Version="6.3.0" />
<PackageReference Include="NUnit.Analyzers" Version="4.14.0">
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>

5
ErsatzTV.Scanner/Core/Errors/ErrorProcessingFolder.cs
Unescape View File

@ -0,0 +1,5 @@ @@ -0,0 +1,5 @@
using ErsatzTV.Core;
namespace ErsatzTV.Scanner.Core.Errors;
public class ErrorProcessingFolder() : BaseError("Error processing folder");

1
ErsatzTV.Scanner/Core/Metadata/ImageFolderScanner.cs
Unescape View File

@ -211,6 +211,7 @@ public class ImageFolderScanner : LocalFolderScanner, IImageFolderScanner @@ -211,6 +211,7 @@ public class ImageFolderScanner : LocalFolderScanner, IImageFolderScanner
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
hasErrors = true;
}
}
}

12
ErsatzTV.Scanner/Core/Metadata/MovieFolderScanner.cs
Unescape View File

@ -174,6 +174,8 @@ public class MovieFolderScanner : LocalFolderScanner, IMovieFolderScanner @@ -174,6 +174,8 @@ public class MovieFolderScanner : LocalFolderScanner, IMovieFolderScanner
movieFolder);
}
var hasErrors = false;
foreach (string file in allFiles.OrderBy(identity))
{
// TODO: figure out how to rebuild playlists
@ -191,6 +193,7 @@ public class MovieFolderScanner : LocalFolderScanner, IMovieFolderScanner @@ -191,6 +193,7 @@ public class MovieFolderScanner : LocalFolderScanner, IMovieFolderScanner
foreach (BaseError error in maybeMovie.LeftToSeq())
{
_logger.LogWarning("Error processing movie at {Path}: {Error}", file, error.Value);
hasErrors = true;
}
foreach (MediaItemScanResult<Movie> result in maybeMovie.RightToSeq())
@ -200,12 +203,17 @@ public class MovieFolderScanner : LocalFolderScanner, IMovieFolderScanner @@ -200,12 +203,17 @@ public class MovieFolderScanner : LocalFolderScanner, IMovieFolderScanner
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
hasErrors = true;
}
}
await _libraryRepository.SetEtag(libraryPath, knownFolder, movieFolder, etag);
}
}
// only do this once per folder and only if all files processed successfully
if (!hasErrors)
{
await _libraryRepository.SetEtag(libraryPath, knownFolder, movieFolder, etag);
}
}
foreach (string path in await _movieRepository.FindMoviePaths(libraryPath))

1
ErsatzTV.Scanner/Core/Metadata/MusicVideoFolderScanner.cs
Unescape View File

@ -387,6 +387,7 @@ public class MusicVideoFolderScanner : LocalFolderScanner, IMusicVideoFolderScan @@ -387,6 +387,7 @@ public class MusicVideoFolderScanner : LocalFolderScanner, IMusicVideoFolderScan
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
hasErrors = true;
}
}
}

1
ErsatzTV.Scanner/Core/Metadata/OtherVideoFolderScanner.cs
Unescape View File

@ -207,6 +207,7 @@ public class OtherVideoFolderScanner : LocalFolderScanner, IOtherVideoFolderScan @@ -207,6 +207,7 @@ public class OtherVideoFolderScanner : LocalFolderScanner, IOtherVideoFolderScan
if (!await _scannerProxy.ReindexMediaItems([result.Item.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
hasErrors = true;
}
}
}

33
ErsatzTV.Scanner/Core/Metadata/TelevisionFolderScanner.cs
Unescape View File

@ -9,6 +9,7 @@ using ErsatzTV.Core.Interfaces.Images; @@ -9,6 +9,7 @@ using ErsatzTV.Core.Interfaces.Images;
using ErsatzTV.Core.Interfaces.Metadata;
using ErsatzTV.Core.Interfaces.Repositories;
using ErsatzTV.Core.Metadata;
using ErsatzTV.Scanner.Core.Errors;
using ErsatzTV.Scanner.Core.Interfaces;
using ErsatzTV.Scanner.Core.Interfaces.FFmpeg;
using ErsatzTV.Scanner.Core.Interfaces.Metadata;
@ -293,14 +294,20 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan @@ -293,14 +294,20 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan
return error;
}
await _libraryRepository.SetEtag(libraryPath, knownFolder, seasonFolder, etag);
season.Show = show;
if (!await _scannerProxy.ReindexMediaItems([season.Id], cancellationToken))
// only set etag if no errors were encountered with the folder, and if reindex succeeds
if (scanResult.IsRight)
{
_logger.LogWarning("Failed to reindex media items from scanner process");
if (await _scannerProxy.ReindexMediaItems([season.Id], cancellationToken))
{
await _libraryRepository.SetEtag(libraryPath, knownFolder, seasonFolder, etag);
}
else
{
_logger.LogWarning("Failed to reindex media items from scanner process");
}
}
season.Show = show;
}
}
}
@ -326,8 +333,15 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan @@ -326,8 +333,15 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan
.OrderBy(identity)
.ToList();
var hasErrors = false;
foreach (string file in allSeasonFiles)
{
if (cancellationToken.IsCancellationRequested)
{
return new ScanCanceled();
}
// TODO: figure out how to rebuild playlists
Either<BaseError, Episode> maybeEpisode = await _televisionRepository
.GetOrAddEpisode(season, libraryPath, seasonFolder, file, cancellationToken)
@ -344,6 +358,7 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan @@ -344,6 +358,7 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan
foreach (BaseError error in maybeEpisode.LeftToSeq())
{
_logger.LogWarning("Error processing episode at {Path}: {Error}", file, error.Value);
hasErrors = true;
}
foreach (Episode episode in maybeEpisode.RightToSeq())
@ -351,12 +366,18 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan @@ -351,12 +366,18 @@ public class TelevisionFolderScanner : LocalFolderScanner, ITelevisionFolderScan
if (!await _scannerProxy.ReindexMediaItems([episode.Id], cancellationToken))
{
_logger.LogWarning("Failed to reindex media items from scanner process");
hasErrors = true;
}
}
}
// TODO: remove missing episodes?
if (hasErrors)
{
return new ErrorProcessingFolder();
}
return Unit.Default;
}

2
ErsatzTV.Scanner/ErsatzTV.Scanner.csproj
Unescape View File

@ -21,7 +21,7 @@ @@ -21,7 +21,7 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="CliWrap" Version="3.10.4" />
<PackageReference Include="CliWrap" Version="3.10.5" />
<!-- <PackageReference Include="EntityFrameworkProfiler.Appender" Version="6.0.6049" />-->
<PackageReference Include="Humanizer.Core" Version="3.0.10" />
<PackageReference Include="LanguageExt.Core" Version="4.4.9" />

4
ErsatzTV.Tests/ErsatzTV.Tests.csproj
Unescape View File

@ -12,13 +12,13 @@ @@ -12,13 +12,13 @@
</ItemGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.8.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.9.0" />
<PackageReference Include="NUnit" Version="4.6.1" />
<PackageReference Include="NUnit.Analyzers" Version="4.14.0">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="NUnit3TestAdapter" Version="6.2.0" />
<PackageReference Include="NUnit3TestAdapter" Version="6.3.0" />
<PackageReference Include="Shouldly" Version="4.3.0" />
</ItemGroup>

6
ErsatzTV.sln
Unescape View File

@ -30,6 +30,12 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ErsatzTV.Core.Nullable", "E @@ -30,6 +30,12 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ErsatzTV.Core.Nullable", "E
EndProject
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "ErsatzTV.Tests", "ErsatzTV.Tests\ErsatzTV.Tests.csproj", "{56F56E76-CEF4-4639-B7BB-03FD201BB019}"
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Markdown", "Markdown", "{E4F90ABA-8D7B-4DD6-8D10-0DFC54F62611}"
ProjectSection(SolutionItems) = preProject
CHANGELOG.md = CHANGELOG.md
README.md = README.md
EndProjectSection
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|Any CPU = Debug|Any CPU

10
ErsatzTV/ErsatzTV.csproj
Unescape View File

@ -36,8 +36,8 @@ @@ -36,8 +36,8 @@
<PackageReference Include="Chronic.Core" Version="0.4.0" />
<PackageReference Include="FluentValidation" Version="12.1.1" />
<PackageReference Include="FluentValidation.AspNetCore" Version="11.3.1" />
<PackageReference Include="Heron.MudCalendar" Version="4.0.1" />
<PackageReference Include="HtmlSanitizer" Version="9.2.995" />
<PackageReference Include="Heron.MudCalendar" Version="4.1.0" />
<PackageReference Include="HtmlSanitizer" Version="9.2.1039" />
<PackageReference Include="LanguageExt.Core" Version="4.4.9" />
<PackageReference Include="Markdig" Version="1.3.2" />
<PackageReference Include="MediatR.Courier.DependencyInjection" Version="5.0.0" />
@ -54,10 +54,10 @@ @@ -54,10 +54,10 @@
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="MudBlazor" Version="9.8.0" />
<PackageReference Include="MudBlazor" Version="9.9.0" />
<PackageReference Include="NaturalSort.Extension" Version="4.4.1" />
<PackageReference Include="Refit.HttpClientFactory" Version="15.0.0" />
<PackageReference Include="Scalar.AspNetCore" Version="2.16.18" />
<PackageReference Include="Refit.HttpClientFactory" Version="15.2.0" />
<PackageReference Include="Scalar.AspNetCore" Version="2.17.2" />
<PackageReference Include="Serilog" Version="4.4.0" />
<PackageReference Include="Serilog.AspNetCore" Version="10.0.0" />
<PackageReference Include="Serilog.Settings.Configuration" Version="10.0.1" />

Write Preview
Loading…
Cancel
Save