live programming
Here we deal with things that are particular to willshake development. We’ll add some features that are useful while you’re making changes to the program. This document should not be included in production.
Development is a “cross-cutting concern” for our purposes. Elsewhere, we are sensitive to dependencies—not here. In other words, this document may assume the presence of any other feature at any time. So this document is said to “build on” the system, but everything is fair game.
This should not be done at all in production. As it is, it just exits when on live.
The first thing we’ll do is inject a “development” script into the document:
<require module="development" />
Using zz because scripts should go at the end; this ensures that the append occurs last.
Now we’ll write that script, using a placeholder that we can reference in this document. When this starts getting unwieldy, we’ll use the old “folder of scripts” approach instead:
define([], () => {
if (/willshake\.net/.test(window.location.hostname))
return;
console.log("development mode");
<<development script>>
}());
The hostname
check is of course a cheap way to prevent this from happening on
live. Better would be to exclude this document completely for production, so
that the script is not shipped, not to mention downloaded.
1 “live reload” changes in the browser
As part of our ongoing effort to poorly approximate the power of live
programming, we use a technique that has been unabashedly called “live
reloading.” That is: you use a browser to look at your web site; you change the
program; and the browser automatically updates what you see. In some
cases—such as style and image updates—the change can be applied in-place;
for everything else, the whole page is requested, rendered, compressed,
transmitted, decompressed, parsed, interpreted, laid out, composited, and
displayed again. The beauty of “live” reloading is that you don’t have to press
Ctrl+R
to make this happen; you can just sit and watch.
To serve us in this noble cause we enlist the livereloadx
package1, which you can install like so:
npm install -g livereloadx
This package is based on the old LiveReload system by Andrey
Tarantsov.2 It uses WebSockets to communicate between a special
script and a special server so that it can reload assets (or the entire page)
when the site’s disk files change. By default, it only tracks common file types
(html, css, etc). Since we use a lot of xsl
, we have to add that explicitly.
cd site; livereloadx --include '*.{xsl,svg}'
Once the server is running, the LiveReload client script is available for
download. Generally, it will run on the same machine that is serving the site,
but on a different port number (so it doesn’t conflict with the actual site).
So, if we want to remain agnostic of the hostname, we have to load the script
dynamically. The hostname will often be localhost
, but the beauty of this
method is that it also works when accessing the site from a remote client.
Thus, we just use whatever hostname was used to view the site.
var livereload_script = document.createElement('script');
livereload_script.src = "//" + window.location.hostname + ":35729/livereload.js?snipver=2";
document.body.appendChild(livereload_script);
That’s it. Whenever a file in the site’s directory changes, any browser that has it open will hear about it and act accordingly.
Special exceptions to the security policy are needed for this to work. Right
now we don’t bother to do this either additively or conditionally. See content
security policy. Further to that, note that although we are “protocol agnostic”
above (by using //
), those security exceptions are in fact explicit as to
protocol.
1.1 preserve scroll position when reloading page
The whole point of live reloading is to see changes applied in-place. But as
we’ve said, this often means reloading the whole page. One defect of the
livereload.js
script is that it doesn’t retain the scroll position when
reloading. Instead, a page that is partly scrolled at the time of a change will
jump back to the top. This defeats the purpose of live reloading, since action
is needed to return to the place of interest (which is presumably where you were
looking). The browser itself will often restore your scroll position when you
return to a page, but not, as a rule, when you reload it.3
Whether this is ideal for ordinary usage is not our concern; for development
reloads, it’s clearly needed.4
Fortunately, the LiveReload
object can be patched at runtime to effect this
change.
First, we need a way to pass information (in this case, the scroll position)
from one incarnation of the page to the next. Variables won’t work, since the
javascript execution environment is thrown away upon each load. Fortunately,
HTML5 introduces a couple of persistence mechanisms: localStorage
and
sessionStorage
. The only difference between them is that localStorage
will
survive closing and reopening the browser, so for this purpose, sessionStorage
is adequate. These storage API’s are designed to share the same permissions as
cookies for a given domain. Trying to access them on a domain where cookies are
not allowed will throw a SecurityException
. So for this technique to work,
cookies must be enabled, and we need to catch the exception so that the script
will not always crash when they aren’t. Note that it’s not quite enough to
simply reference the storage object; you actually have to use it to confirm that
it’s working.
function get_session_storage() {
const KEY = 'test_value', VALUE = 1;
var storage;
try {
(storage = window.sessionStorage).setItem(KEY, VALUE);
if (storage.getItem(KEY) == VALUE) {
storage.removeItem(KEY);
return storage;
}
} catch (exception) {
console.log("Session storage is not available:", exception);
}
}
If we should ever want to use persistent storage for user-facing features, this method can be moved into production.
Remember, we use layers. Only one layer is scrollable at a time, but it changes from place to place. (See “the-layers”).
function get_scroll_layer() {
return document.querySelector('.active.scroll-layer') || document.documentElement;
}
Now, we replace LiveReload’s reloadPage
function with our own version, which
first saves the current scroll position.
livereload_script.addEventListener('load', () => {
const reloader = LiveReload.reloader;
const __base_method = reloader.reloadPage;
reloader.reloadPage = function() {
const storage = get_session_storage();
if (storage)
storage.setItem('last_scroll_y', get_scroll_layer().scrollTop);
__base_method.apply(reloader, [].slice.call(arguments));
};
});
Finally, when the page loads, we’ll check for a saved scroll position and then try to restore it.
(function() {
const storage = get_session_storage();
if (storage) {
const y = storage.getItem('last_scroll_y');
if (y) {
console.log("restoring persisted scroll position", y);
get_scroll_layer().scrollTop = y;
}
storage.removeItem('last_scroll_y');
}
}());
1.2 a program change should only cause a reload when you’re looking at it
We integrate the program itself with the web site. It’s neat. But this causes
a problem for live reloading. Suppose that I change a style rule in one of the
documents. One of the nice features of livereload
is that it will update
stylesheets in-place. But the program in which I changed the style rule will
also be updated, and since this is not a CSS change, it will trigger a full
reload. We don’t want that.
But what’s the alternative? We could tell livereloadx
to ignore the directory
where programs are published. But we don’t want that either, because it’s
useful to see the published program while you’re working on it—it’s a kind of
dogfooding for the whole concept. So what do we do?
We’ll try something similar to the approach used for scrolling. We’ll hack
LiveReload
at runtime.
livereload_script.addEventListener('load', () => {
const reloader = LiveReload.reloader;
const __base_method = reloader.reload;
reloader.reload = function(path, options) {
var about_file = path.match(/^static\/doc\/about\/([^\/]+)\.xml$/);
if (about_file
&& window.location.pathname != '/about/' + about_file[1]) {
console.log('Skipping reload of document', about_file);
return;
}
__base_method.apply(reloader, [].slice.call(arguments));
};
});
1.3 reload object
elements
LiveReload does not support reloading of object
elements, which can be used to
“embed” external entities into the page. If nothing else, this is useful for
“interactive” SVG graphics (that is, SVG graphics with functioning links and
scriptable elements). In fact, the current version of LiveReload doesn’t
recognize SVG as a type of image to reload at all. So we may eventually need to
deal with both cases here. For the moment, this just reloads objects.
livereload_script.addEventListener('load', function() {
const reloader = LiveReload.reloader;
const __base_method = reloader.reload;
reloader.reload = function(path, options) {
if (path.match(/\.svg$/)) {
// Note that the path sent to this method is not rooted. Also, we
// use "starts with" instead of an exact match because we'll be
// adding a query to the end.
const objs = document.querySelectorAll('object[data^="/' + path + '"]');
[].slice.call(objs).forEach(obj => {
// Yeah, reloader has a generateCacheBustUrl method, but it
// complains about the `this' reference no matter how I invoke
// it.
//
// Also, yes, the object has a `data` property that you can set
// directly. But this takes the liberty of setting the `data`
// *attribute* to an absolute path, making it harder to find
// this element again next time it changes (since we're using
// attribute selectors).
obj.setAttribute(
'data',
obj.getAttribute('data')
.replace(/(\?.*)?$/, '?t=' + new Date().getTime()))
});
return;
}
__base_method.apply(reloader, [].slice.call(arguments));
};
});
Footnotes:
NPM, “livereloadx: An implementation of the LiveReload 2 server in Node.js” https://www.npmjs.com/package/livereloadx
LiveReload: “The Web Developer Wonderland (a happy land where browsers don’t need a Refresh button)" http://livereload.com/
Bugzilla Bug 838523: “When reloading a page, Firefox doesn’t retain the scroll position” https://bugzilla.mozilla.org/show_bug.cgi?id=838523
See, e.g., Bugzilla Bug 979989: “Maintain scroll position after page reload” https://bugzilla.mozilla.org/show_bug.cgi?id=979989