change rules

What are change rules? I’ll admit right away, it’s something we made up. It’s not part of “the web platform” as people know it.

Change rules are little statements saying what lives in each part of the web site. They’re fully functional (no mutation, no side-effects) and fully declarative. If you can bear working with them (and I daresay most people couldn’t), they make a web site “easier to reason about.” And these days, people love talking about reasoning about web sites and all other manner of programs. What makes all this possible is a little framework called getflow.

1 okay, so how do we actually get html?

how-do-we-get-html.svg
Figure 1: How do we get HTML

2 get getflow

So let’s start by getting getflow to the web site. On the client side, getflow is distributed as a single script. We just copy it to the web site.

: $(ROOT)/getflow/<client_min> \
|> ^ ship getflow client^ cp %<client_min> %o \
|> $(SITE_SCRIPT)/getflow.js

That’s it. Note that the web server also uses getflow, but this is all that’s needed for client operation (i.e. a browser).

Get the unminified version, too, for development. On the wierd name, see compressing and shipping.

: $(ROOT)/getflow/<client> \
|> ^ ship getflow client (full source) ^ cp %<client> %o \
|> $(SITE_SCRIPT)/getflow.js.src.js

3 make getflow “blueprints”

All of the routes and change rules get bundled together into one file, which getflow calls the “blueprints” (or “master plans”). Why? Why not? It’s convenient to have everything in one place. This way, with just two downloads (the getflow client, and this bundle), the web site can start responding right away to any link that is clicked. Yes, additional downloads may be necessary for most locations, but some action can be started immediately, which everyone agrees is healthy for today’s users’ sense of well-being.

: $(ROOT)/change_rules/bundled/bundle.xsl \
  $(ROOT)/program/routes.xml \
  | $(ROOT)/change_rules/<bundled> \
|> ^ bundle blueprints^ xsltproc %f > %o \
|> $(SITE)/getflow.xml $(SITE)/<getflow.xml>

This takes as input the routes definitions. Then it bundles the change rules into the route elements.

What is bundle.xsl? It’s a transform that will “inflate” the routes with the relevant change rules.

We’ll start with an identity template, which copies everything from the input as-is. This is a standard XSL thing, so let’s write it to its own transform.

<xsl:template match="node()|@*">
	<xsl:copy>
		<xsl:apply-templates select="node()|@*" />
	</xsl:copy>
</xsl:template>

And use it in the bundle transform.

And… elsewhere.

Now, when we encounter a route element, we insert the contents of the change rules.

<xsl:template match="route">
  <xsl:copy>
		<!-- Keep all attributes except "to", since we're replacing it with the
		     contets.  In practice, we don't use attributes on route, anyway. -->
    <xsl:apply-templates select="@*[not(name()='to')]" />
    <xsl:apply-templates select="node()" />
    <xsl:apply-templates select="document(concat(@to, '.xml'))/r/node()" />
  </xsl:copy>
</xsl:template>

3.1 self-closing tags

Making web pages has always been a very forgiving art form. Traditionally, browsers have quietly overlooked even the most egregiously unintelligible markup, putting something on the screen at all costs. This is a humane practice, which partly accounts for the success of the web. Yes, “computer people” think it’s perfectly reasonable for a program to barf out an Error message if it encounters a single misplaced character. But HTML is supposed to used by normal people.1

And yet, for some reason, browsers are very fussy about self-closing tags. Web pages can get completely screwed up if certain tags use this shorthand. This is a classic case of HTML ain’t XML. In XML, an empty element written with a single, self-closing tag

is absolutely equivalent to an element written with a separate closing tag:

<p></p>

If you’re just using the document represented by these tags, they are indistinguishable. It’s like the difference between print and cursive, if you’re just hearing someone read text aloud. The “difference” between them exists only in the text, and it is thrown away when they are read into memory (i.e. when an XML reader deserializes them). There is nowhere in the Document Object Model (DOM) that you could ask which way it was written. Period.

But of course that’s not the end of the story, because HTML ain’t XML. HTML is a vocabulary, and different terms get different treatment. Some elements are allowed to close themselves. But if a browser sees a paragraph written as <p/>, woe be to the elements that follow it. So we use a special rule to to prevent most elements from closing themselves when they are empty—regardless of how they were originally written. Of course, we can always write things the “right way.” But this rule is still necessary because we will sometimes use XML processors to generate HTML, and they won’t always be respectful (or aware) of the distinction.

<xsl:template priority="0"
		match="*
	   [not(node())]
	   [not(self::route or self::br or self::link or self::meta or self::input or self::img)]">
  <xsl:copy>
    <xsl:apply-templates select="@*" />
    <xsl:comment>t</xsl:comment>
  </xsl:copy>
</xsl:template>

This has to be done for getflow verbs as well (class, xslt), otherwise the client will parse the structure incorrectly. But the priority is zero because it should not preëmpt anything for which we have a specific definition.

This is not needed on the server, and I’d rather not have to do it at all. It may be possible to avoid it by having the browser parse getflow.xml as text/html, but my gut says that’s certain to cause other (worse) problems.

3.2 no top-level comments in templates

Currently, comments are not supported as top-level content of insert rules (viz, before, after, prepend, and append). Their behavior not just undefined—I think they will crash the getflow server. So I really shouldn’t use them at all. But during development I still find that I “comment things out,” and I don’t want it to break everything.

<xsl:template match="before/comment()" />
<xsl:template match="after/comment()" />
<xsl:template match="prepend/comment()" />
<xsl:template match="append/comment()" />

Otherwise, comments are supported.

3.3 add change rule import feature

As long as we’re changing the meaning of the change rules, let’s add an import feature, where we want to re-use the same change rule.

<xsl:template match="import" priority="1">
  <xsl:apply-templates select="document(@href)/r/node()" />
</xsl:template>

Note that the changes are expanded before transmission, so use carefully.

Is the priority really necessary there? Is this feature needed at all, now that we have macros?

3.4 add shorthand for transform names

All of the transforms are .xsl files located under /static/transforms. So instead of always saying the full path and extension, we can just state the name:

<xsl:template match="xslt/@name">
  <xsl:attribute name="transform">
    <xsl:value-of select="concat('/static/transforms/', ., '.xsl')"/>
  </xsl:attribute>
</xsl:template>

4 support additive change rules

Each location in the site has a route and a set of change rules. When you add a new location, you create a file with the initial change rules for that location, and you add a route which points to that file. For example, at the /plays path, as it is initially defined, we mark the plays region as active.

But suppose that a feature defined in the future wanted to extend this behavior. In our additive building model, we don’t want to change the existing definition of the rules for /plays—we just want to add to them. Here, we allow any document to add new rules to any set:

# Need a separate copy of this here because it's referenced relatively... ugh.
: $(PROGRAM)/transforms/identity.xsl \
|> !link_from |> $(ROOT)/change_rules/identity $(ROOT)/change_rules/<identity>

service make_change_rules \
: foreach $(ROOT)/change_rules/*.xml \
  | $(ROOT)/change_rules/more/<all> \
    $(ROOT)/change_rules/bundled/<bundler> \
    $(ROOT)/change_rules/bundled/<transform> \
|> ^o group change rules %B^ \
   echo "%<all>" | %<bundler> %<transform> %f > %o \
|> $(ROOT)/change_rules/bundled/%b \
   $(ROOT)/change_rules/<bundled>

So for example, if you want to extend a set of change rules called plays.xml, you can add a file called more/plays--new-feature.xml, and this will be included in the set.

The bundling is done with a little script, which also adds opening and closing tags to make the result usable as an XML document:

transform="$1"
main_file="$2"                                  # project-relative path
file="${main_file##*/}"                 # strip path
name="${file%.*}"                               # strip extension

# Reads the file list from the input to this command.  Sorting is important
# because sometimes filenames are used to control the order, and the list as
# provided by Tup is not sorted.
for more in $(tr ' ' '\n' | sort); do
	more_file="${more##*/}"          # strip path
	more_name="${more_file%--*}" # strip suffix
	if [ $name = $more_name ]; then
		more_for_this="$more_for_this $more"
	fi
done

make_bundle() {
	echo '<r>'
	cat "$main_file" $more_for_this
	echo '</r>'
}

make_bundle | xsltproc "$transform" -

In the process, we also run a transform on those rules. The next section covers the purpose of this.

5 support the addition of shorthand

As we saw earlier, we can actually change the change rules that we’ve written, just by adding templates to that pre-process. This could be useful. Suppose that there were some bit of markup that we wanted to use in a number of places, even with variations. We could add a rule to recognize a shorthand for it, and expand it.

But that shouldn’t stop us from creating our own features! Besides, how could we expect getflow to know what we wanted? In fact, we don’t expect ourselves to know what we might want in the future, when it comes to writing change rules. So here we’re going to try to do something about that.

(Yes, if we were using a Lisp, then we wouldn’t have this problem at all. But we aren’t using a Lisp.)

Shorthand will sometimes be convenient. Let’s make it possible to define some shorthand. Then will define some shorthands. And we’ll make sure that new features can create new shorthands.

: $(ROOT)/change_rules/make_transformer \
  | $(ROOT)/change_rules/transforms/<all> \
|> %f %<all> > %o \
|> $(ROOT)/change_rules/bundled/transform.xsl \
   $(ROOT)/change_rules/bundled/<transform>

Assemble all of the defined macros into one transform, and we’ll throw in some string functions.

echo "<?xml version='1.0' encoding='utf-8'?>"
echo "<xsl:transform version='1.0'"
echo " xmlns:xsl='http://www.w3.org/1999/XSL/Transform'"
echo " xmlns:strings='http://exslt.org/strings'"
echo " extension-element-prefixes='strings'>"
echo "<xsl:output omit-xml-declaration='yes' />"
list="$(echo $* | tr ' ' '\n' | sort)"
cat $list
echo "</xsl:transform>"

I know. That’s XML for ya.

6 ship transforms

Although getflow has “templates” for most rules, the templates have no processing power of their own (other than to interpolate the result of XPath expressions). Instead, it lets you call an XSL template. Here we copy the transforms to the web site in the expected location.

service ship_transforms : foreach $(TRANSFORMS)/* \
|> !copy_to |> $(SITE_TRANSFORMS)/%b  $(SITE_TRANSFORMS)/<all>

At present, this “clearing house” directory is used only for transforms that come from tangled code blocks. We could instead tangle directly to the site. However, doing so would cause a reload whenever a document containing a transform was tangled, whether or not it had changed. The tangle rule uses Tup’s “output flag” for preventing subsequent build rules from executing, so this rule will not execute unless the file has actually changed.

On the other hand, I don’t want changing one transform to cause all of them to be recopied. In other words, I want each tangled transform to map to a rule; so this is done as a service (not just by reading the group).

7 update :target styles on state change

I’m not sure the best place to do this.

“Anchors,” also known as “hashes” and “fragment identifiers,” are a useful (if not absolutely essential) part of URI’s. They let you point to a specific place within a document (provided that the author has given it a unique ID).

Likewise, :target selectors are very useful for highlighting a specific point in the current page.

But when you use pushState to change the page address, :target selectors don’t update to reflect the new target. This has been the case for a long time, and apparently that’s not going to change.2

I agree with Paul Irish that this is how it should work, though.

My first thought is, what if you just set location.hash to itself after a page transition?

Yes, that works, but it jumps to the anchor now. Whereas I have special handling to scroll to an anchor… so you’d have to do it after that, if the scrolling takes place, and immediately, otherwise.

8 issues

8.1 TODO add a visual cue of loading

Like, make the beacon glow, or something. Most transitions should take long most of the time, but inevitably, some will.

Footnotes:

1

For a summary of the situation as it was in 2008 (much of which has not changed), see “Is writing self closing tags for elements not traditionally empty bad practice?” StackOverflow (2008) http://stackoverflow.com/a/348818

See also “recognizing a new extra input” tup-users group (2015). https://groups.google.com/d/msg/tup-users/F0s62gAkF9A/d0rIj42PAwAJ

2

Bug 83490 - history pushState doesn’t affect :target selector. In an amazing coincidence, this four-year-old bug was RESOLVED INVALID just minutes ago, as I write this on January 25, 2016.

about willshake

Project “willshake” is an ongoing effort to bring the beauty and pleasure of Shakespeare to new media.

Please report problems on the issue tracker. For anything else, public@gavinpc.com

Willshake is an experiment in literate programming—not because it’s about literature, but because the program is written for a human audience.

Following is a visualization of the system. Each circle represents a document that is responsible for some part of the system. You can open the documents by touching the circles.

Starting with the project philosophy as a foundation, the layers are built up (or down, as it were): the programming system, the platform, the framework, the features, and so on. Everything that you see in the site is put there by these documents—even this message.

Again, this is an experiment. The documents contain a lot of “thinking out loud” and a lot of old thinking. The goal is not to make it perfect, but to maintain a reflective process that supports its own evolution.

graph of the program

about

Shakespeare

An edition of the plays and poems of Shakespeare.

the works