Support retrieving resources as flows #4500
Open
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Changes
fun getResourceAsFlow(String, Int): Flow<ByteArray>
, which retrieves the resource as a flow of byte array chunks.InputStream
.NSInputStream
.ReadableStreamBYOBReader
.ResourceIOException
, which wraps internal IO exceptions thrown while reading resources.Res
class containsgetAsFlow(String, Int)
, which invokesgetResourceAsFlow
.Res.getAsFlow
andkotlinx.io
in the files tab of the resource demo app.Segment.SIZE
ofkotlinx.io
orokio
.Motivation
I am making an app that uses deep learning. Although the actual app downloads the DL model from our server, we embed the DL model into our app as a resource for internal testing. I have been handling this by implementing resource reading with platform-specific APIs. I tried refactoring this part using the new resource API, but it didn't work because the app crashed with an OOM error when I used
Res.readBytes
.To prevent such an error, I need to read resources as chunks, so I need a proper IO API here. Since
kotlinx.io
already provides platform-agonistic IO, I tried to implement this withkotlinx.io
, butkotlinx.io
only provides synchronous IO and is incompatible with web targets, so I failed.So, I came up with the idea of using
Flow
for this, asFlow
is the only first-party Kotlin API to do something sequentially and asynchronously, as far as I know.Example