Ever wondered how those cool AI image tools work without uploading your photos? A team building ClearPix figured out how to run powerful AI models like LaMa and Real-ESRGAN directly in your web browser, overcoming huge technical challenges to offer instant, private image cleanup and upscaling.
Hey WondTech readers! Have you ever used an online tool to remove a watermark or upscale an image, and thought, 'How does it do that so fast without me uploading anything?' Well, a team behind a new suite of media cleanup tools called ClearPix has just shared some fascinating insights into making advanced AI models like LaMa Inpainting and Real-ESRGAN run *entirely* in your web browser. This means incredible privacy and speed for you, as your images never leave your device!
Building these tools, which let a 198MB LaMa model run inside your browser tab, wasn't easy. The biggest hurdle they faced was memory. Imagine your browser tab trying to hold a huge AI model, plus the image you're working on (a 4K image alone is about 33MB!), and all the temporary data. Chrome, for example, will just close your tab without warning if it runs out of memory. They also hit issues with graphics driver stability, where a WebGPU session might just freeze up mid-process.
Interestingly, sheer processing power (FLOPS) wasn't the main problem. Their profiling showed that browser-based AI is limited by how fast data can be moved around (memory-bandwidth-bound), not by raw calculation speed. This discovery changed how they chose their AI models. Instead of picking the «strongest» model, they went with options like Real-ESRGAN's compact `general-x4v3` variant. This model, though seemingly «weaker» on paper, performed an order of magnitude faster in the browser because its architecture was simply better suited for browser environments.
They used clever tricks to make this possible. For instance, they purposefully pinned their main inpainting model to WebAssembly (WASM) and used «tiled inference» to break down large images into smaller pieces, processing them without overwhelming your browser's memory. This prevents your tab from crashing even when handling big pictures. Their most important rule? «Profile before you touch anything» – meaning they always test thoroughly to see what’s actually slowing things down before trying to optimize. It's a huge step forward for web-based AI, giving you powerful tools right at your fingertips without the need for uploads or special hardware.
Building these tools, which let a 198MB LaMa model run inside your browser tab, wasn't easy. The biggest hurdle they faced was memory. Imagine your browser tab trying to hold a huge AI model, plus the image you're working on (a 4K image alone is about 33MB!), and all the temporary data. Chrome, for example, will just close your tab without warning if it runs out of memory. They also hit issues with graphics driver stability, where a WebGPU session might just freeze up mid-process.
Interestingly, sheer processing power (FLOPS) wasn't the main problem. Their profiling showed that browser-based AI is limited by how fast data can be moved around (memory-bandwidth-bound), not by raw calculation speed. This discovery changed how they chose their AI models. Instead of picking the «strongest» model, they went with options like Real-ESRGAN's compact `general-x4v3` variant. This model, though seemingly «weaker» on paper, performed an order of magnitude faster in the browser because its architecture was simply better suited for browser environments.
They used clever tricks to make this possible. For instance, they purposefully pinned their main inpainting model to WebAssembly (WASM) and used «tiled inference» to break down large images into smaller pieces, processing them without overwhelming your browser's memory. This prevents your tab from crashing even when handling big pictures. Their most important rule? «Profile before you touch anything» – meaning they always test thoroughly to see what’s actually slowing things down before trying to optimize. It's a huge step forward for web-based AI, giving you powerful tools right at your fingertips without the need for uploads or special hardware.
