Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement SharpBilinear scaler #102

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 64 additions & 5 deletions vsscale/scale.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,18 @@
from math import ceil, log2
from typing import Any, ClassVar, Literal

from stgpytools import inject_kwargs_params
from vsexprtools import complexpr_available, expr_func, norm_expr
from vskernels import Catrom, Hermite, LinearScaler, Mitchell, Scaler, ScalerT
from vskernels import (
Bilinear, BorderHandling, Catrom, ComplexScaler, Hermite, LinearScaler, Mitchell, Point, SampleGridModel, Scaler,
ScalerT
)
from vskernels.types import Center, LeftShift, Slope, TopShift
from vsrgtools import box_blur, gauss_blur
from vstools import (
DependencyNotFoundError, KwargsT, Matrix, MatrixT, PlanesT, ProcessVariableResClip, VSFunction, check_ref_clip,
check_variable, check_variable_format, clamp, core, depth, fallback, get_nvidia_version, get_prop, inject_self,
padder, vs
Dar, DependencyNotFoundError, KwargsT, Matrix, MatrixT, PlanesT, ProcessVariableResClip, Sar, VSFunction,
check_ref_clip, check_variable, check_variable_format, clamp, core, depth, fallback, get_nvidia_version, get_prop,
inject_self, padder, vs
)

from .helpers import GenericScaler
Expand All @@ -20,7 +25,8 @@
'DPID',
'SSIM',
'DLISR',
'Waifu2x'
'Waifu2x',
'SharpBilinear',
]


Expand Down Expand Up @@ -516,3 +522,56 @@ class SwinUnetPhotoV2(BaseWaifu2x):

class SwinUnetArtScan(BaseWaifu2x):
_model = 10


class SharpBilinear(ComplexScaler):
"""
Pre-supersample using Point, then scale to the target dimensions with Bilinear.

This is used for pixel art to preserve the "sharpness", while still allowing for
non-integer ratio scaling without messing up the intended look of the image.
"""

@inject_self.cached
@inject_kwargs_params
def scale( # type: ignore[override]
self, clip: vs.VideoNode, width: int | None = None, height: int | None = None,
shift: tuple[TopShift, LeftShift] = (0, 0),
*,
border_handling: BorderHandling = BorderHandling.MIRROR,
sample_grid_model: SampleGridModel = SampleGridModel.MATCH_EDGES,
sar: Sar | bool | float | None = None, dar: Dar | bool | float | None = None, keep_ar: bool | None = None,
linear: bool | None = None, sigmoid: bool | tuple[Slope, Center] = False,
**kwargs: Any
) -> vs.VideoNode:
target_width = width or clip.width
target_height = height or clip.height

# Ideally, we should always scale in linear light by default.
kernel_kwargs = dict(linear=fallback(linear, self.kwargs.get('linear', True)))

scale_kwargs = dict(
sar=sar, dar=dar, keep_ar=keep_ar,
sigmoid=sigmoid, border_handling=border_handling,
sample_grid_model=sample_grid_model
) | dict(linear=kernel_kwargs.get('linear', False)) | kwargs

if (clip.width, clip.height) == (target_width, target_height):
return clip

if target_width <= clip.width and target_height <= clip.height:
return Bilinear.scale(clip, target_width, target_height, shift, **scale_kwargs)

max_ratio = max(target_width / clip.width, target_height / clip.height)
int_ratio = 2 ** int(log2(max_ratio) + 0.5)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can I interest you in our newest invention, math.ceil

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am die goodbye

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it’s transpired that this was, of course, round, not math.ceil, oops

Copy link

@moi15moi moi15moi Oct 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, round isn't exactly the math round which can cause issue
Look here: https://realpython.com/python-rounding/#pythons-built-in-round-function

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, so it’s round-half-to-even? I didn’t know that. Thanks!

Anyway, @LightArrowsEXE, why is this forced to be a power of two? And do you actually want to overshoot the ratio and then downscale with bilinear, undershoot and then upscale with bilinear, or pick the closest integer in any direction?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, exactly, it is round-half-to-even.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@astiob I noticed while writing this that sometimes, it looks better when you downscale it vs. upscale it. I found that was the case if you had for example a factor of 2.5, upscaling looks better, but if you have for example a factor of 3.5, downscaling looks better.


ss_clip = Point(**kernel_kwargs).scale(
clip, clip.width * int_ratio, clip.height * int_ratio, shift, **scale_kwargs
)

if (ss_clip.width, ss_clip.height) == (target_width, target_height):
return clip

return Bilinear(**kernel_kwargs).scale(ss_clip, target_width, target_height, **scale_kwargs)

kernel_radius = 1
Loading