Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setBackend() required for ml5.neuralNetwork() bug #117

Open
shiffman opened this issue Mar 26, 2024 · 4 comments · May be fixed by #138
Open

setBackend() required for ml5.neuralNetwork() bug #117

shiffman opened this issue Mar 26, 2024 · 4 comments · May be fixed by #138

Comments

@shiffman
Copy link
Member

The ml5.neuralNetwork() examples require either:

ml5.setBackend("webgl");
ml5.setBackend("cpu");

Without one of the above, the examples break with an error related to webgpu, see #34. This came up again while reviewing #105. We'll leave it for now, but it would be great to remove the requirement from the examples and have them all work with webgl as default.

@lindapaiste
Copy link
Contributor

I think that all we need is an await tf.ready() statement in neuralNetwork init method. #105 (comment)

The ml5.setBackend function is an oddball because it initializes an asynchronous change but doesn't wait for it (tf.setBackend is async). So we need to wait for TF to be fully set up before using it.

@ziyuan-linn ziyuan-linn changed the title setBackground() required for ml5.neuralNetwork() bug setBackend() required for ml5.neuralNetwork() bug May 17, 2024
@ziyuan-linn
Copy link
Member

Adding await tf.ready() seems to work! However, a new error occurred now that we are using webgpu backend:

Error: WebGPU readSync is only available for CPU-resident tensors.
    at ms.readSync (pose-detection.esm.js:17:87509)
    at Engine.readSync (engine.js:943:1)
    at Tensor.dataSync (tensor.js:297:1)
    at Tensor.arraySync (tensor.js:228:29)
    at NeuralNetworkData.js:509:1
    at engine.js:328:1
    at Engine.scopedRun (engine.js:338:1)
    at Engine.tidy (engine.js:327:1)
    at Module.tidy (globals.js:175:18)
    at NeuralNetworkData.createOneHotEncodings (NeuralNetworkData.js:494:12)

I will try to investigate this further.

@lindapaiste
Copy link
Contributor

Over the long term I think that the ideal thing would be to avoid synchronous operations and use .array() instead of .arraySync(). However this function is called way deep in a chain so there's a lot of methods that would need to be converted to async for that to work.

It seems like the error has been fixed by TFJS, not sure what version it was released in. It might be that we need to bump our dependencies.
Fix PR: tensorflow/tfjs#7576
Issue: tensorflow/tfjs#5468

@ziyuan-linn
Copy link
Member

Thank you @lindapaiste! I will try to bump the dependencies and see what happens. I agree that switching to async would be ideal over the long term.

@ziyuan-linn ziyuan-linn linked a pull request Jun 15, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants