Check file size locally before upload

[weddleben]

What are your thoughts about adding a feature to the various SDKs that would check if the size of the file you want to upload exceeds the max file size set by the server (currently 100MB).

I just tried to upload a large file and was annoyed that it took forever to upload, only to get back an error. It would be much more convenient to know this up front.

One obvious problem: if in the future you decide to change the max file size, every SDK would need to be updated. A workaround for that could be to create an endpoint that returns the max file size, then use that to check your file. Maybe that's way too complicated.

This might be a dumb idea but I just ran into it so thought I'd mention it. If you don't like the idea, I would like to suggest including what the max file size is in message property of the 413 response.

Cheers

[VictoriqueMoe]

The file upload is handled directly by Multer. oddly enough that i thought then response is streamed and when the content-length is sent, the request would be rejected. but it seems that Multer does not stream when checking contenst size (the size of the content is supplied in Multer options) (see: https://github.com/waifuvault/WaifuVault/blob/main/src/Server.ts#L63).

I guess they decided not to read the content-length due to the fact that the fact that this header can not be trusted (that said. if you supply a URL, i do manually stream the content and read the content-length).

reflecting on it, the data may be streamed and be rejected the when stream hits 100mb. this means that uploading a 1GB file will only be rejected when the 100MB is uploaded (a sieze limit annoyingly enforced by cloudflare).

It is of my opinion that the sdk should not care about the size directly. as coupling JUST the size would seem rather odd. rather, i feel a restrictions endpoint to be put in place that will return a list of generic "rules" that an SDK must abide to.

of course in this case the only such rule would currently be illegal mime-types and the content size,

this will would be a hidden webservice not shown in swagger. but should be consumed by an SDK.

Any SDK that wishes to be offical should and must consume these rules.

does this solcve the issue?

In order to solve this a contributing.md file must be made with list of rules an SDK must follow. such as the content-lenth

RE: @nakedmcse

[weddleben]

that's what i meant! i love it! would love to hear what walker thinks

[VictoriqueMoe]

Too late. Already decided it's happening ;)

[nakedmcse]

The endpoint returning a set of restrictions as a json return value makes sense to me. Will the SDK actually check these restrictions, or just return them to the user to handle checking for themselves ?

[VictoriqueMoe]

It will check them and validate them. We will ensure this when accepting prs

[nakedmcse]

Cool sounds good to me.

[VictoriqueMoe]

@weddleben this feature has been implemented in the main application. I will propigate it to the sdk shortly

See https://waifuvault.moe/api-docs/ resource management for api

[weddleben]

that was fast! so currently checking mime type and content file size. wondering what else would benefit from client side checks?

[VictoriqueMoe]

I had to come up with an idea of what settings would be a restriction, as every restriction is a setting, but not every setting is a restriction.

Currently just those 2 are the only factors limiting an upload, but I guess you could also say the endpoint validation of the custom timeout format that is a regex value.