Decouple data with how it's shown in the DOM

Introduction #

A recent chat with a mobile dev about how android views work got me thinking how in-efficient most DOM rendering is. More often than not if a thousand items exist in an array - a thousand items will be rendered in the DOM. There’s a one-to-one relationship between the array and the DOM. Every item in the array has a corresponding DOM element. Which is crazy when you think about it.

We have techniques like infinite scroll and pagination to limit how many things are rendered - yet these don't fulfil all use cases, and have some caveats. For example scroll bars in infinite scroll change size and position whenever new elements are appended. Scroll bars are highly underrated. They manage a user's expectation of how much scrolling needs to be done!

What we need to do is decouple the data we want to show with the way it's shown. A perfect example is an input with a dropdown list.

Usually these are achieved by giving the list a height and setting its overflow to auto. This allows the list to scroll when there are more things to show than space to show them. But why render things the user can't see in the first place - i.e the items above and below the fold.

Theory #

The key point of this post is to decouple data with how it's shown. Instead of thinking we are scrolling up or down a list of DOM nodes stacked on top of one another - we should treat the scroll as an interaction that merely controls what's in view. In essence this is how List Views work in Android - and I wanted to see if the same principles could be used on the web.

Let's continue using the dropdown list as an example, and set some guidelines. The data we want to show is an array containing 1000 items. The list needs to show 1000 items but with fewer DOM elements. And the scrollbar needs to behave just like it would if 1000 elements were rendered at once.

In order to do this the number of DOM elements in the list needs to be capped. Let's say no more than 10 elements can be rendered. So we have 1000 items to show with only 10 elements to show them in. In other words the list is a subset - showing only a portion of the data in the array.

Implementation #

The initial render of the subset is pretty straight forward - splice the first 10 items in the array and show them in the list. Give the list a pseudo height (1); which is the height it would be if all items in the array were rendered (this makes the scrollbar accurate). Then when the user scrolls up/down, or presses up/down on keyboard change the data in the subset (2) + (3) and update the DOM.

1. Height of a single DOM item * array length
2. Scroll/press up - reduce the start and end index of the subset by 1
3. Scroll/press down - increase the start and end index of the subset by 1

The above handles the mechanics of keeping the subset (data) and list (DOM) in sync - however it’s missing something. If we left it at that the result would look like this (I'm scrolling slowly so it's easier to see):

As you can see scrolling down swaps out the items correctly, but the new ones below can’t be seen. This is because of the pseudo height applied to the list. What needs to happen is when the list scrolls up/down it also has to move up/down to stay in view. The list can be moved using CSS transform translate. And the distance to move is the start index of the subset multiplied by the height of a single DOM element in the list. This distance is also deducted from the lists height - and voila:

And lastly #

Couple of things to point out. The pseudo height of the list and it’s top translation need to be recalculated and applied whenever the data changes (i.e when it’s filtered). Also the example above assumed the subset contained 10 items - which won't always be the case. What actually happens is the number of DOM elements in the list mirrors the length of the subset. So when the subset contains 5 items only 5 elements are rendered in the DOM.

Final thing is keyboard shortcuts. Pressing up or down navigates through the items in the list. When the shortcut goes to an item out of bounds (before first index and after last index of the subset) the scroll position of the list is set using scrollTop. This in turn calls the methods already in place to handle scrolling - which updates the subset and moves the list up/down.

And that's that. I’m happy to announce the implementation can be found on github :) and demos can be found here. Hope its all made sense and I'd love to get your feedback!