Generating image placeholders using geometric primitives

Published by almstdigital 28 May 2019

Generating image placeholders using geometric primitives

When working on Decentium I couldn't resist scratching an itch I've had for a long time now: building a placeholder generation and loading system for images.

For those of you not into web development placeholder images are used in conjunction with lazy loading to greatly speed up load times of image-heavy websites. Lazy loading means that you don't start loading an image until the user wants to look at it.

Lazy loading alone works great to improve load times but it creates a jarring experience for the user since the images will pop into existence as they are reading making the content jump. This is where the placeholder comes in. A simple approach is to use a grey box that has the same size as the image, that prevents the content movement but it is still pretty distracting.

A more advanced approach is to downsample the image and upsample it again client side when displayed as a placeholder. This combined with heavy JPEG compression and a gaussian blur filter generates a pretty good looking result while keeping the placeholder size small enough that they can be embedded.

Initially I wanted to use this approach but instead of scaling down the images use a neural network trained on the ImageNet db with a loss function that compares the original to an upscaled and blurred version of the output layer which would be something like a 20x20 grid of colours.

I still think this would be a interesting (and fun to implement) approach that could add some personality to the placeholder images. Working on Decentium has some unique constraints however – all code has to run either client side in the users browser or in the distributed network where you pay for CPU-time by the nanosecond – even simple image processing tasks would be very costly and I don't think users would want to pay for generating images that are barley seen just to scratch my itch, no matter how much personality they have.

That leaves doing it client-side, I briefly looked into using tensorflow.js but that quickly felt like a rabbit hole so deep I would never get back out.

Using geometric primitives as placeholders

Doing some searching on the subject I came across this great article about placeholders by José M. Pérez that introduced me to Michael Fogleman's Primitive - a piece of software that given an image will give you an approximation of it using a bunch of different coloured and sized geometric primitives.

Self-portrait using 100 circles generated with Primitive

This looked very promising, and fortunately for me Sam Twidale had already written a clone of Primitive called Geometrize that had a haxe library that I could transpile to TypeScript and cannibalise.

I added a component that extracts all images from the article as the last step of publishing, this component draws each image to a HTML canvas and reads the pixel data so that it can be fed into the Geometrize library. For each image it generates 200 triangles that are packed using nine 8-bit integers (x and y coordinates for each corner and the colour) and finally stores them on the blockchain along the article as metadata.

Each placeholder weighs in at 1.8kb which is pretty reasonable. About the same size you'd get with the downsampled JPEG method. I could probably get away with 100 triangles or less if I added some blur but I like the polygon look.

When articles are rendered the data is unpacked and the triangles are converted into SVG polygons that are inlined in the post body and watched with the Intersection Observer API to swap them out for the original when the user scrolls them into view.

This works incredibly well, on a decent connection your brain barley notices that the images where swapped out. I had to resist the urge to add a fade-in animation to show off the placeholders...

Here are some large images that should take long enough to load for the effect to be noticeable:

Side by side comparisons if you missed it:

As far as I have been able to find this is the first automated use of "geometric" SVG placeholders and I'm certain it's the first use where they are generated client-side and included in a blockchain.

If you're interested in Decentium but don't to get an EOS account or just want to test the placeholder generation feel free to use the TestNet version - username: test password: hejsan if asked - sign in with private key: 5JgSkwCUKrDvy5F4w6mPT5jVZhaQmqoz9FzgKFtRfCkc3JL9gKN

Thanks for reading!