Garbage collector problem

[barisyild]

Garbage collector cannot delete textures from gpu memory when using this extension!
Is this a bug or is it normal?
If this is normal, can't the problem be fixed using WeakRef?

[greggman]

You arguably should not be counting on garbage collection for WebGL. As is says in the MDN article for WeakRef

---

Avoid where possible

Correct use of WeakRef takes careful thought, and it's best avoided if possible. It's also important to avoid relying on any specific behaviors not guaranteed by the specification. When, how, and whether garbage collection occurs is down to the implementation of any given JavaScript engine. Any behavior you observe in one engine may be different in another engine, in another version of the same engine, or even in a slightly different situation with the same version of the same engine. Garbage collection is a hard problem that JavaScript engine implementers are constantly refining and improving their solutions to.

Here are some specific points that the authors of the WeakRef proposal included in its explainer document:

Garbage collectors are complicated. If an application or library depends on GC cleaning up a WeakRef or calling a finalizer [cleanup callback] in a timely, predictable manner, it's likely to be disappointed: the cleanup may happen much later than expected, or not at all. Sources of variability include:

• One object might be garbage-collected much sooner than another object, even if they become unreachable at the same time, e.g., due to generational collection.
• Garbage collection work can be split up over time using incremental and concurrent techniques.
• Various runtime heuristics can be used to balance memory usage, responsiveness.
• The JavaScript engine may hold references to things which look like they are unreachable (e.g., in closures, or inline caches).
• Different JavaScript engines may do these things differently, or the same engine may change its algorithms across versions.
• Complex factors may lead to objects being held alive for unexpected amounts of time, such as use with certain APIs.

---

On top of all those reasons it's even worse with WebGL. AFAIK no JavaScript engine has any idea of the memory used by WebGL resources. In other words, Assume you have 150meg free of VRAM. If you allocate a 100meg texture, from JavaScript's POV all you've done is allocate a few bytes to track the the WebGL resource. The JavaScript engine has no idea of the 100meg if you've allocated in VRAM. If you then lose the reference and immediately allocate another 100meg texture the JavaScript engine has no idea it should stop the world and delete your unreferenced 100meg texture so that there's room to allocate a new one. Instead you'll likely just get an OUT_OF_MEMORY error from WebGL or you'll lose the WebGL context. Conversely if you gl.deleteTexture the texture it should work you'd have memory to allocate the new texture.

I really don't want to encourage bad practices via this library. This library requires you to delete your WebGL objects which is the right thing to do and it will report resource counts encouraging you to manually delete the objects.

[greggman]

Just as a test I wrote some code that allocated a 4Meg WebGL texture and watched for it to be garbage collected using WeakRef. On Safari it was collected 4 seconds later. On Firefox and Chrome it was never collected (well, I stopped waiting after 2 minutes). Note: I was not allocating more things, well, expect a new Promise and a new setTimeout event, both of which are allocations.

As another test I tried creating a Canvas and waiting to see when it was collected. On Safari it was collected in ~4 seconds. On Firefox ~9. On Chrome it was ~32 seconds. In this case, every second I created a new canvas and inserted in into the body of the document.

When I removed the creation of another canvas every second it was ~4 seconds on Safari, ~9 on Firefox, stop waiting after 8 minutes in Chrome

Anyway, this is just all evidence of the warnings above.

[greggman]

Also, just for fun, I tried implementing it. I couldn't get Chrome nor Firefox to garbage collect WebGL objects so there's no way to test 😅

But, also, unless there's some other way to implement it I don't know, I'm required to keep an array of all WeakRefs to every object and scan it (slow) because you don't get directly told when an object is collected, you can just check if it was. So, when someone calls ext.getMemoryInfo() then inside that

// pseudo code
for(const [ref, id] of arrayOfRefsToAllWebGLObjects) {
  if (!ref.deref()) {
    // This object was GCed. Remove its memory
    // Example: 1024x1024xRGBA texture is 4meg so lower the memory usage by 4meg
    // and remove it from the arrayOfRefsToAllWebGLObjects
  }
}

Maybe that's okay. It's just slow for a large project to go over that array. I guess if you're not calling ext.getMemoryInfo often it doesn't matter.

One more issue is that in order to work, webgl-memory would have to start tracking all attachments (see README on the fact that it doesn't). The reason is the browser can GC a WebGLBuffer, WebGLTexture, or WebGLRenderbuffer object but WebGL can still reference that object if it's attached to some other object. For example a buffer referenced in a vertex array or a texture referenced in a framebuffer

Here's an example (only works in Safari)
https://jsgist.org/?src=baa6464cd855205b42d7c24409f72475

The WebGLBuffer for the triangle is GCed but the triangle is still taking memory and still usable because it was attached to vertex array object. In order for WebGL-Memory to handle that it would require expanding it to track the state of all attachments and only record the freeing of the memory when the object is GCed by GL.

That's not to say it shouldn't be done. It's just more work than I'm willing to do at the moment.