layers

Merge this back into hyperspace.

The problem of cyberspace, as things stand, is that we don’t live in a flat world, yet we’re constantly dealing with flat screens.

One approach to this problem is to ignore it. You do this by creating a virtual world that mimics three-dimensional space, and use your flat screen as a “window” into that world. That window would show a flat projection of the virtual space, except that it’s a window you can never open to go through it. That’s obviously a bad solution, and I won’t say any more about it.

Another approach to this problem is to use layers.

Everybody knows what layers are—right? Whenever you have a stack of flat things, those things are called “layers.”

So the idea of “layers” is an intuitive way to describe a physical arrangement.

It’s also a good compromise between unbounded space and a flat screen.

The individual layers may be flat, but taken together, they have depth, meaning that people can still use their sense of space to understand the domain. Instead of laying everything out in a plane, you can stack things.

If you happen to be interested in documents, it may not be such a bad compromise. It allows you to organize lots of information in a tractable way—literally. Layers let you get “traction” on an entire swath of content at one time.

1 layers in HTML/CSS/JavaScript

For better or worse, layers are not a first-class concept in HTML. As far as a web page is concerned, the thing we’re calling a “layer” is just another element.

Fortunately, layers are a first-class concept in compositing engines, also known as “compositors.” Compositors are graphics-processing programs that “compose” the images that you see on the screen, usually from a stack of layers. The layers are often partly transparent, which is why they need to be composited in the first place.

So for our purposes, a “layer” is like a transparency—potentially very large—with some images on it.

It is helpful to think of a layer as a set of flat things suspended in a transparent material. Granted, this is not something you’re likely to encounter in the world, but it is easy enough to imagine.

It’s helpful because this is basically how “layers” work in CSS. Transparent areas are solid—you can’t “reach through” a layer to something behind it. Well, you can, by using pointer-events, but this introduces other problems. And for a layer that scrolls, even pointer-events won’t help you, since it will make the layer basically unusable. This, combined with the fact that pointer-events is not perfectly supported everywhere, and we are compelled to start by considering only such layers, that is, ones which are essentially mounted on glass.

2 proposal

What’s the point of layers?

How do you communicate the existence and relationship of layers?

The relationship between two layers is always that one is in front of the other—otherwise, they are effectively not layered.

So how do you know that you’re looking at two different layers?

  • you can see the bottom layer behind the top one
    • because it’s offset
    • because it’s translucent
    • because parts of it are transparent
  • because they move independently

2.1 characteristics of layers

New layers will always be introduced in front of existing ones—"closer” to you. This is somewhat like a “stack,” in that you have to pop things off of the stack to get back down to where you were.

New layers will generally (if not always) represent more specific contexts.

example_layers.svg
Figure 1: Example layers

Applying this “top-down” thinking to the question of layers, we proceed through the following considerations:

  1. free space
  2. layers
  3. the kind of layers that we can actually implement

Free space is a non-starter, as we’ve already said. We’d like to keep these considerations as much as possible in the “ideal” space, where we’ve no concern for implementation. By arguing in this “top down” way, we can create designs that are driven by their inherit merits, which will not change quickly, rather than being dictated by real world constraints, which will change more quickly. But ultimately, implementation problems are going to bubble up and inform (i.e. restrict) the models that we can consider.

2.2 definitions and requirements (redux)

Ideally, layers would be intuitive. Anything that looks and feels like a layer, should be a layer. It seems odd that we should even have to say that—of course a layer is a layer. Yet in practice, as long as we’re targeting the web, it’s not going to be that simple.

Intuitively, a layer is a set of things that move together.

But not quite… what about the “about” documents. As things stand, those are in a “layer” (scrolling container), but they don’t scroll together. There’s no way that you could look at the document and its map and not think of them as separate layers. Yet we only have one scroll parent, and hence only one “layer” (by the DOM-based definition).

This is what we want. It’s influenced by the fact that we identify documents as our main form of content.

  • must be able to have long-form content that is wheel- (and gesture-) scrollable from anywhere on the page…
  • even when there is fixed content above, below, or to the left of the
  • and that fixed content must itself be affordable
  • would like if we can also mix two scrolling areas, with one extending out from under the other on the left side

So: when there are two scrolling areas (which we don’t currently have an instance of), then dragging (or wheeling in) those areas will cause scrolling of of that area—as you would expect.

When there is only one scrolling area, then—unless we override this with script—dragging or wheeling anywhere on the page will affect that scrolling area, even if you’re dragging or wheeling over non-scrolling content.

3 properties

By default, user agents make html the scrolling container, and certain special behaviors accrue to this assumption, particularly as relate to mouse and keyboard input. When we break such defaults, we use this condition.

html
	scroll-behavior smooth
	overflow-x hidden
	&.not-scroll-layer
		overflow hidden

Normally, I’d use overflow-y scroll for a layer that supports scrolling, to ensure that the scrollbar is always visible, even when it’s not needed. Without an explicit setting, the default is overflow auto. The only benefit to using auto is that you don’t get a scrollbar when it’s not needed. But layers that are designed to support scrolling pretty much always need it, and using auto has unwanted side-effects as content is added and removed. First, the scrollbar “flickers” when it’s removed and restored, which is ugly and distracting. Second, it causes the recalculation of viewport-based units (chiefly, vw). Reassigning these units will cause unwanted layouts and paints. It turns out that the spec addresses this specifically.

This is also the reason for using 100vw instead of 100% in full-screen, which is in turn used by scroll-layer(): it prevents elements from changing size just because a scrollbar is added or removed on the root (html) element. Layers should always have the full width of the viewport, and most layers have their own persistent scrollbar, meaning that they will match the width of other layers regardless of which layer is currently scrollable. Putting layers in fixed position also ensures that they get tucked “under” any other scrolling layer, rather than being “parented” by another scrolling container (in which case you’d see two scrollbars—a problem strenuously avoided here).

Note also that using -webkit-overflow-scrolling touch will cause problems, to wit, that the page becomes suddenly blank, among others.

http://patrickmuff.ch/blog/2014/10/01/how-we-fixed-the-webkit-overflow-scrolling-touch-bug-on-ios/

That said, there still major problems with scrolling on iOS.

Layers occupy the entire viewport, always. (At least, as things stand.)

This is kind of like a docking rule. But it’s only useful for layers because only layers can co-opt the entire viewport.

full-screen(left = 0)
	position fixed
	top 0
	left left
	width 100vw
	height 100vh

scroll-layer(compositor = true, smooth = true, left = 0)
	full-screen(left: left)
	overflow-y scroll                  // vs auto to prevent scrollbar flicker
	scroll-behavior smooth if smooth

	// Without this, Chrome is likely to repaint on every scroll.
	if compositor
		transform translateZ(0)

Move the scroll-layer() properties to .scroll-layer itself and just apply the class where needed. This is a transitional approach because the scroll-layer class is still abused in places. .scroll-layer should not be a transient class. Once a scroll layer, always a scroll layer.

Yeah, well, it ain’t that way now.

4 scrolling

scroll (n.)
c. 1400, “roll of parchment or paper,” altered (by association with rolle “roll”) from scrowe (c. 1200), from Anglo-French escrowe, Old French escroe “scrap, roll of parchment,” from Frankish *skroda “shred” or a similar Germanic source, from Proto-Germanic *skrauth- (cognates: Old English screada “piece cut off, cutting, scrap;” see shred (n.)).1

Just about all of the problems presented by layers are related to scrolling.

Dealing with scrolling will take some scripting. Let’s make a module.

<require module="space" />

Layers and scrolling scripts

// There are only a couple of jquery references, and I'll probably eliminate
// them.
define(['jquery'], $ => {
	<<closest tag>>
	<<scrolling>>

	// No exports
	return {};
});

4.1 the active scrolling layer

“Active” is a bad word for this because it doesn’t imply that there can be only one at a time. “Foremost” would be a better candidate, or maybe even “default.”

Also, the “activeness” of a scroll layer (see below) should not have anything to do with the layer’s own properties (i.e., its overflow setting). The active setting is really just part of the keyboard hack noted below, although at the moment it’s also used to control properties of descendants of the layer.

Exactly one scrolling layer is active at all times.

If one is not specifically marked (with the active class), then it’s assumed to be the document itself.

function active_scroll_layer() {
	return document.querySelector('.active.scroll-layer') ||
		document.documentElement;
}

4.1.1 TODO keyboard support

This is what you might call a “corner case.” But it’s one I care about.

If you visit a page where the active scroll layer is not the body, you won’t be able to scroll it with the keyboard right away.

You may not know it, but you can navigate the web with your keyboard. You can scroll with the ↑ up and down ↓ arrows, the Page Up and Page Down keys (on some keyboards), and even the spacebar! Special-purpose browsers (presumably) accept voice commands.

But in order for that to work, the browser has to know what thing to scroll. When a page loads, the thing that receives scrolling commands will always be the the focused element, which in turn will always be the document itself. (As far as I know, anyway.2)

This isn’t noticeable if you’re using a pointing device or touch screen. But if you try to use the keyboard, nothing will happen. In that case, you can work around this yourself by focussing the layer some other way, such as clicking the screen or (usually) pressing Tab.

But that’s annoying. So when script is available, a script is used to focus the element explicitly.

Note that focusing an arbitrary element (i.e. non-input) requires its tabindex to be set.3

on_load_complete(() => {
	const active_layer = active_scroll_layer();
	if (active_layer)
		// TODO: This works in Chrome and IE, but not Firefox.
		active_layer.focus();
});

This needs to be done on new locations, not just the initial load.

Move this to a module.

// This also matches empty... was for script element but how does it apply
// to document readyState?
var LOAD_COMPLETE = /loaded|complete|/;
function on_load_complete(action) {
	if (LOAD_COMPLETE.test(document.readyState))
		action();
	else
		window.addEventListener('load', action);
}

4.2 reset scrolling state

Scrollable layers are kind of like sliding doors. They’re not “spring-loaded"—they stay where you left them.

This is only true if you’re returning to a layer that’s been hidden in the interim—usually a deeper layer than the one you’re returning to. For layers that remain visible while new layers are added and then removed, this is definitely not wanted. But how could you tell the difference? Right now you can use a dummy hash to bypass this, but that’s a kludge.

The main layers of willshake are re-used by different content as you move around to different places. If you leave a layer partly scrolled, it will still be partly scrolled when you go back to it, even if you’re in a “different” place now. Whereas, you’d expect to be at the top of the page.

Dealing with this is pretty simple. Whenever you go somewhere, reset the scroll position of the active layer—unless there was a “hash.” Hashes (a.k.a. “anchors”) indicate a specific location within the document and are dealt with elsewhere.

navigating.on(change => {
	if (!change.new_place.hash)
		window.requestAnimationFrame(() => {
			active_scroll_layer().scrollTop = 0;
		});
});

4.3 pointing stuff

function doc(name, event) {
	let args = [name];

	if (event.changedTouches && event.changedTouches.length) {
		const changed1 = event.changedTouches[0];
		args = args.concat(['changed'],
						   Math.round(changed1.clientX),
						   Math.round(changed1.clientY),
						   changed1.force);
	}

	if (event.touches && event.touches.length) {
		const touch1 = event.touches[0];
		args = args.concat(['touch'],
						   Math.round(touch1.clientX),
						   Math.round(touch1.clientY),
						   touch1.force);
	}

	console.log.apply(console, args);
}
function started_pointing(event) {
	var link = closest_tag(event.target, 'A');
	if (link) {
		doc('start', event);
		link.classList.add('pointing');
	}
}

function stopped_pointing(event) {
	var link = closest_tag(event.target, 'A');
	if (link) {
		doc('stop', event);
		link.classList.remove('pointing');
	}
}

document.body.addEventListener('touchstart', started_pointing);
document.body.addEventListener('touchend', stopped_pointing);
document.body.addEventListener('touchmove', stopped_pointing);

4.4 BUG tapping links doesn’t work consistently

I’m filing this here because I suspect that it’s related to the above script. Tapping seems to work if you don’t move the pointer at all. Every time I stop trying to reproduce this, I start getting it again.

In fact, I strongly suspect that the above technique doesn’t do any good at all. Yet, I haven’t positively confirmed that it causes any problem not already present in most mobile browsers (viz, the problem it’s trying to help solve).

4.5 BUG links stay “pointed at” after activated

4.6 BUG firefox hash anchor thing

This is old code that’s been disabled for probably some 15 Firefox versions. I’d have to repro this before restoring it. If I can’t, I’ll get rid of it.

if (window.location.hash && window.addEventListener)
	window.addEventListener(
		// Yes this is a “wierd assignment,” but using .reload() here causes
		// infinite loop.
		'load', () => (window.location.href = window.location.href));

Firefox does not wait until the page is fully loaded before scrolling to the hash link4, meaning that for elements which haven’t been positioned yet, anchor links won’t work. Although this appears to affect firefox only, we’re not sniffing for Gecko until this proves problematic for others.

In practice, this is only for initial page loads. After that, we have script to intercept links and scroll anchors into position.

4.7 TODO restore scroll position on popstate

The browser doesn’t do this for push/pop state changes, but we could. I’d prefer it like 100% of the time.

I’d say this belonged to getflow, but getflow doesn’t know what the real scroll layer is. This is somewhat related to reset scrolling state.

5 well-known layers

In the future, new features could add layers. The objective here has been to deal with layers in a general way, without knowing exactly what layers may exist.

5.1 the document layers

Documents are kind of a special case, though. They serve as a well-known place where document content should go.

In the initial state (when you enter the site), there is no document open. These layers are here as placeholders when a document is opened.

body <append>
	<div id="document-background-layer">
		<div id="document-background" class="background" />
	</div>
	<section id="document-layer" />
</append>

The document background is a separate partition between the exhibit and the document.

Note that #document-background-layer has separate rules in play-background.styl. We want to use one source of truth, while still keeping layer-specific business here.

The document has two layers, one for its background.

@import docking
@import layers
@import order-of-main-layers
#document-background-layer
	scroll-layer()
	z-index $documentBackgroundLayer

#document-layer
	position relative                  // enable z-index
	z-index $documentLayer

The document layers are supposed to be lexically foremost—that is, they should come before other layers in the actual source. This is based on the assumption that other layers will have mainly navigational content. When a document is open, it is more important than the navigation which led to it.

When viewing the site without stylesheets, this has the natural result that the expected content is the first thing you see.

However, it has the opposite effect in space. Elements that are lexically later are, by default, in front of ones that come earlier. So an explicit layering scheme is necessary to reverse that.

5.1.1 the document layers are not affordable

As an EXCEPTION to below (huh?), the document background does accept pointer events when active, only to block click-through to below layers.

The document layers sit in front of other layers even when they have no content. This prevents us from having to “show” the layers when a document is opened, which would require a reflow and could disrupt transitions. This rule prevents them from being detectable in any way when no document is open.

But note that even when a document is open, these layers still do not take user input. This enables other affordable layers (in particular, the beacon) to sit between the document and its background.

#document-background-layer
	pointer-events none

As a result, anything in the document layer that needs to receive user input must declare as much explicitly. Naturally, this will apply to the doucment itself.

.document
	pointer-events auto

But this is something of a stopgap. It’s likely that we’d rather apply the pointer-events on a container of the document, since the .document itself doesn’t always cover the page, meaning that “wheel” scrolling against the background will scroll the background instead of the document. This may or may not be expected, I’m not sure yet.

5.2 stacking order

Again, this is just about the stacking order of the known layers. Other layers could be added in the future, without impacting this.

the-main-layers.svg
Figure 2: The fundamental layers

These elements are appended and prepended to the body in various places, so for better or worse their physical order (also known as document order) is somewhat arbitrary. As long as the elements are statically-positioned, they will follow one after the other without overlap. As soon as we make them positioned elements, then the stacking order becomes important.

By default, the physical order of sibling elements will determine their stacking order, where top-to-bottom means back-to-front. Remember that we want to maintain the physical order that makes the most sense when there are no stylesheets. So if the physical order doesn’t happen to yield the stacking order that we want (and it doesn’t), then we resort to the z-index property. We define the layer indexes as variables so that we can see them all in one place.

$firmamentLayer = 0
$exhibitLayer = 1
$documentBackgroundLayer = 2
$beaconLayer = 3
$documentLayer = 4

6 parallax

When you’re moving through space, nearby things pass quickly, and distant things move slowly. This is called the parallax effect.

This is an effect of the things’ physical arrangement in space. But the effect is so familiar that you can simulate a spatial arrangement by showing things (like layers) moving at different speeds.

The parallax effect is not limited to scrolling. It can apply to animations or generally any scripted sequence. But scrolling is a special case because the web has something like native support for parallax scrolling, via the transform-style: preserve-3d property.

This addition to the platform was presumably motivated by the fact that people were using javascript to achieve the effect, with uneven results. Native support for three-dimensional effects would make them smoother and thus more effective at creating the illusion. And indeed, parallax scrolling, such as it is used in games, can be extremely effective at creating a sense of space in two-dimensional projections. But I’d be the first to admit that, on the web in practice, it’s a gimmick with no semantic value. I would be the first, that is, if Dede M. Frederick of Purdue University hadn’t beat me to it. He studied the effects of parallax scrolling on “user experience” on the web and concluded that

although parallax scrolling enhanced certain aspects of the user experience, it did not necessarily improve the overall user experience.5

The paper spends a great deal of its attention on the “pleasure” associated with the parallax effect. But much of that pleasure is attributed to its “novelty.” Personally, I find parallax effects pleasing, even when they are pointless. And when it comes to the subject of this domain—Shakespeare—pleasure is the ticket; no other purpose is needed. But this is the architectural level, and architecture has no place for anything pointless. If willshake will use parallax, it will have meaning.

6.1 parallax versus momentum scroll on iOS

There’s a hitch on iOS:

-webkit-overflow-scrolling:touch; breaks css 3d perspective6

What is -webkit-overflow-scrolling-touch? Well, some Webkit-based browsers (made by Apple) won’t apply “momentum” scrolling to elements other than the body unless they see this property.7 At the moment, I can’t imagine when I wouldn’t want scrolling elements to behave consistently with the body.

But this bug means that you can have parallax scrolling, or you can have momentum scrolling.

Keith Clark, the guy who made this technique well-known, deals with it by simply turning off the parallax trick on browsers that support -webkit-overflow-scrolling-touch, considering the latter more important.8

I think that’s a good idea, so let’s do it. If you want to use parallax, just put it inside of a parallax() block, and this iOS detection will be applied.

parallax()
	@supports ((perspective: 1px) and (not (-webkit-overflow-scrolling: touch)))
		{block}

Finally, I don’t think this is something you can overcome with javascript because iOS (as I understand it) throttles scroll events when momentum scroll is enabled, meaning that a script-based parallax would be janky at best. I haven’t tried it, though. I may need to, though, because have I mentioned that this is all screwed up in Firefox as well? Different story.

7 handle SVG links like regular (getflow) links

I’m disposed to moving this into getflow itself. When would you not want this?

Just like HTML, SVG has a “link” element. Naturally, it’s far more verbose, but it’s a link.

<svg height="50" width="250"
		xmlns="http://www.w3.org/2000/svg"
		xmlns:xlink="http://www.w3.org/1999/xlink">
	<a xlink:href="/plays/Ham"
		 xlink:title="Hamlet">
		<rect x="0" y="0" width="100%" height="100%" fill="#ACE" />
		<text x="0" y="50%">The Tragedie of Hamlet</text>
	</a>
</svg>

When that SVG image is used inside of an HTML document, you’d expect those links to work just like reguar links.

There are several ways to use SVG inside of HTML. It can be inline, meaning that the actual text of the SVG is part of the HTML markup. That comes out like this:

The Tragedie of Hamlet

You can also reference the SVG file. One way to do that is with the old img tag.

<img src="/static/images/sample_svg_with_link.svg" />

which comes out looking like this.

You can also “embed” the SVG. That looks like this. sample_svg_with_link.svg

Embedding the image creates a new “browsing context.” That’s the way that most SVG images are used in willshake, because that’s the only way to intercept their links.

It shouldn’t make any difference whether the svg.

Links inside of inline SVG are handled “naturally” by getflow. If we want to treat links inside of external SVG the same way (and of course we do), these extra measures are needed, since “interactive” SVG requires it to be embedded as an object (or embed), and embedded objects will not bubble click events up to the main document (where getflow would handle them).

To accomplish this, we have to check the page for such objects after every state transition (location change), since the change may have introduced new content. Some extra care is taken here to ensure that the requisite handlers wil persist in the face of such changes as well as updates to the objects themselves.

<require module="svg_links" />

This is ref-ed in two places.

function closest_tag(element, tag) {
	while (element) {
		if (element.nodeName == tag)
			return element;
		element = element.parentNode;
	}
	return null;
}
require(['getflow'], getflow => {
	<<closest tag>>

	// This is similar to getflow's document click handler.
	function svg_clicked(event) {
		const link = closest_tag(event.target, 'a');

		if (link && link.href && link.href.baseVal) {
			// This is the getflow from the parent window
			getflow.go(link.href.baseVal);
			event.preventDefault();         
		}
	}

	function ensure_svg_click_handler() {
		const svg = this.getSVGDocument();
		if (svg)
			svg.addEventListener('click', svg_clicked);
	}

	function setup_svg_links() {
		[].slice.call(
			document.querySelectorAll('object[type="image/svg+xml"]'))
			.forEach(obj => {
				ensure_svg_click_handler.apply(obj);

				// If the SVG hasn't loaded yet, try again when it has.  Add
				// this even if it's already loaded, because it might reload
				// later (at least during development).
				obj.addEventListener('load', ensure_svg_click_handler);
			});
	}

	window.navigating.on(setup_svg_links);
	setup_svg_links();
});

Note that duplicate listeners will not be registered even if the above happens multiple times.

If multiple identical EventListeners are registered on the same EventTarget with the same parameters, the duplicate instances are discarded. They do not cause the EventListener to be called twice, and they do not need to be removed manually with the removeEventListener method.9

7.1 BUG svg links still don’t work if you go back to the page they’re on

8 some images

Layers:

  • File:%D0%A1%D1%81-license-layers.png
  • File:InternetProtocolStack.png
  • File:2012-06-Stasimuseum_Berlin_Tagungsraum_02_anagoria.JPG
  • File:Xubuntu710_06_Xfce_Windows_parameters_transparency.png
  • File:USGS_The_National_Map.jpg

Scrolls/scrolling:

  • File:Panorama_of_Persian_by_Piasetsky,_Hermitage,_fot_I_Nowicka.JPG
  • File:Zauberrolle_1.jpg
  • File:National_Treasure_of_South_Korea_196_(Avatamsaka_sutra_in_ink_on_white_paper).jpg
  • File:NewShuteHousePlan.jpg
  • File:Blue_Sliding_Door_(Closeup).jpg (a pretty picture, though doesn’t illustrate much, I guess)

Footnotes:

1

scroll”, Online Etymology Dictionary

3

HTML5 § 7.4 “Focus”, October 2014

4

See, among many other discussions, https://bugzilla.mozilla.org/show_bug.cgi?id=277969#c1

5

Dede M. Frederick (18 April 2013). “The Effects Of Parallax Scrolling On User Experience And Preference In Web DesignPurdue University. Purdue University. Retrieved 17 April 2014. (PDF).

8

Keith Clark, “Practical CSS Parallax,” October 2015 http://keithclark.co.uk/articles/practical-css-parallax/

9

§ “Multiple identical event listeners” in “EventTarget.addEventListener()" MDN https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener#Multiple_identical_event_listeners

about willshake

Project “willshake” is an ongoing effort to bring the beauty and pleasure of Shakespeare to new media.

Please report problems on the issue tracker. For anything else, public@gavinpc.com

Willshake is an experiment in literate programming—not because it’s about literature, but because the program is written for a human audience.

Following is a visualization of the system. Each circle represents a document that is responsible for some part of the system. You can open the documents by touching the circles.

Starting with the project philosophy as a foundation, the layers are built up (or down, as it were): the programming system, the platform, the framework, the features, and so on. Everything that you see in the site is put there by these documents—even this message.

Again, this is an experiment. The documents contain a lot of “thinking out loud” and a lot of old thinking. The goal is not to make it perfect, but to maintain a reflective process that supports its own evolution.

graph of the program

about

Shakespeare

An edition of the plays and poems of Shakespeare.

the works