For v3 of transformers.js, which image models does webgpu speed up?

[legut2]

Great work on all this! I thought I'd just quickly ask this here. I've been playing around with a demo of xenova/swin2SR-realworld-sr-x4-64-bsrgan-psnr and am wondering if I update it to use latest version of transformers.js will it see performance improvements from the webgpu support? I plan on publishing the demo as a write-up.

I'm curious about the image models that might see improvements from webgpu. Could you tell me which ones you've published that could see speed ups?

[xenova]

Hi there 👋 Vision models are perfect candidates for WebGPU speedups, like background removal, depth estimation, image classification, and many many more.

Let me know if you get it working!

[legut2]

Thank you for sharing! Currently I am using version 2 and will try out version 3 when I can. Previous version seems to take 25 seconds to upscale image on my macbook air from 2020. The project takes images and sends them as compressed images to then be upscaled on the web app side in order to save bandwidth. I'll let you know when I try v3 and how it compares when I get to it!

[legut2]

Unfortunately ran into an issue of upgrading to v3 and reported it at the end of this issue: huggingface/transformers.js#911

Mostly it seems to be an issue with using webpack with v3. Seems to work fine on v2.

[legut2]

Although I wasn't able to get it to run faster because of time constraints, I just published earlier today and got featured on the front page of a hardware site called hackster.io. 😁 I gave you and the contributors to transformers.js a credit at the end.