self documenting
Programs are made of bits. The manner in which we present these bits greatly affects our sense of how they are put together.
Brian Foote and Joseph Yoder, “Big Ball of Mud”
Most of willshake’s master files are either documents or data. This document is about publishing the documents, where “publishing” means getting them from the form in which they are written into the form best suited for reading.
1 motivation
The whole point of literate programming is that you create something readable for humans (not just programmers). But it’s not just about being readable. Where will humans get the thing to read in the first place? If the answer is “by cloning the repository and building the documents,” you’ve defeated the purpose, because non-programmers will never do that. So it seems to me that literate programming also requires you to publish the documents in a way that is just as easy to get to as the product itself.
And what better place to do that than the web site itself? It’s already designed for the viewing of documents.
What I write as the Literate Programming source text is not the human-readable output (articles on Literate Programming tend to show only this beautiful output) but the rather messy source to the weaving, markup and all.1
In this section, I’ll create this “beautiful output.” I don’t currently publish the “messy source,” but I do see it as analogous to the relationship between modern editions of Shakespeare (the “beautiful output”) and the source editions (the Folio, also beautiful but perhaps messy to modern eyes).
2 logical location
It’s always been useful to have a way to reference a specific part of a specific book. Yes, you have page number, but that changes with each new edition. You’d need a scheme for referencing the content, not the arbitrary properties of the printing. The Bible has had this feature for a long time, but most other books have not followed suit. Since the Internet and the web, our lack of such a system has become even more noticeable.
Meanwhile, in programming, it’s also useful to have an exact way to reference any part of the program. It’s useful for all the same reasons as it is for other books, but programs can take formal referencing schemes even further. If one part of a program can locate and use other parts of a program as data, then it can extend the program through transformations.
The documents we’re talking about here are not related to Shakespeare per se; they are about willshake. As such, they all belong under the “about” section.
<route path="/about/{document}" to="about_document" />
3 viewing a document
We want to embed the document itself into the site. That’s what the document layer is for.
This was also removing scroll-layer
from #exhibit
, but that’s not right, right?
#about-layer <class remove="active" />
html <class remove="not-scroll-layer" />
#document-layer <prepend>
<article id="top-of-paper"
class="about document" data-program="{document}">
<xslt name="identity"
input="/static/doc/about/{document}.xml" />
</article>
</prepend>
<!-- The document file name is usable as a title. -->
<title><eval>translate(document, '_', ' ')</eval></title>
The document has already been pre-processed to death, so it’s ready to drop in as-is.
4 presentation
So what does a document look like? This section covers that, in two parts.
- Document “hosting,” i.e. the integration of documents into space.
- Document content, i.e. formatting of the documents themselves.
4.1 integration
How does a document fit into the space?
This is what we’re going for.
The left side is reserved. Presumably—everything else being equal—whatever was there before will still be there. So being able to see it will amount to “preserving context,” which is useful for staying oriented.
The right margin is reserved for sidenotes, or blocks (such as figures and code) that may benefit from a greater width than the main text column.
We could also do edge-to-edge blocks, which would temporarily cover the context on the left. It might be worth doing in some cases, but for now I’m just trying to get these right.
The document view is centered around a column of text, which should be a comfortable reading width, though not quite as wide as a “normal” book.
@require typesetting
@require centering
$flowRems = $readingRems * 2/3
$flowMargin = 2em
centered-block(width: $flowRems)
This establishes what 100% means within the document.
padding $flowMargin
box-sizing border-box
position relative // support layering
So with that you can define a “wide” element as occupying half of the viewport’s width, plus and additional half of the document flow’s width.
.wide
margin-left -2em // match document padding
margin-right calc(50% - 50vw)
Special case for the source container.
.org-src-container
width calc(50vw + 50%)
box-sizing border-box
// From Tufte
$documentColor = #FFFFF8 // off white towards yellow
The document is a material thing.
@require document_colors
background $documentColor
box-shadow 0 0 .5em
The shadow extends out on all sides, so that the document appears to float above any material below it, regardless of what edge you happen to be looking at.
Now, as the viewport gets smaller, the margins on either side of the main flow will get smaller. When the viewport is narrower than the preferred width, then the flow itself will shrink, and there will be no margins at all.
// Create compositing layer in Chrome
transform translateZ(0)
margin-bottom 2em
But if we’re going “out of the box,” why not go all the way?
.full
margin 1em calc(50% - 50vw)
With that, you can do the “edge-to-edge” imagery that’s so hot now, without giving up the context.
But for that to look good, it’ll take a little cleaning up.
@require fonts
@require colors
@require document-map-colors
.wide
.full
box-sizing border-box
box-shadow 0 0 1em #111
background $documentMapDarkColor
background blend($resourcesColor, $documentMapDarkColor, .25)
border-radius 1em
overflow hidden
background-clip border-box
img
box-shadow 0 0 1em #111
p:first-child
margin-top 0
// Probably a caption
p:last-child:not(:first-child)
body-font()
color #FFE
@require responsive
+narrow-screen()
padding 1em
4.1.1 BUG “wide” and “full” classes not working on figure
You get the border radius and box shadow, but not the larger size.
4.2 advertisement
DISABLED. I’d rather link to the PDF.
As another pointer to the presence of more material, let’s add to the about region message:
#mission-statement <append>
<span>You can read our <a href="/about/the_project#top-of-paper">philosophy</a>.</span>
</append>
4.3 signs for the documents
This is disabled right now… it was a plain-text list of the documents. The SVG diagram is more useful generally, but this could have a place, too—for example as a fallback for non-stylesheet browsers (better, in this case, even where SVG is supported by such browsers, since the links will function).
Also note that if you do restore this, you’ll need to ship documents.xml
.
#about <append>
<section id="the-program">
<h2>the program</h2>
<p>willshake.net is a literate program</p>
<xslt name="program-links" input="/static/doc/documents.xml" />
</section>
</append>
That assumes the document index has been copied to the web site.
: $(PUBLISHED)/<documents.xml> \
|> ^o ship document index ^ \
cp %<documents.xml> %o \
|> $(SITE_DOCS)/documents.xml
Now for that transform:
<xsl:template match="/">
<ul><xsl:apply-templates /></ul>
</xsl:template>
<!-- Only match documents with titles. This skips "included" files. -->
<xsl:template match="document[title]">
<li>
<a href="/about/{@key}">
<xsl:value-of select="title" />
<xsl:if test="description">
<xsl:text> </xsl:text>
<span class="description">
<xsl:value-of select="description" />
</span>
</xsl:if>
</a>
</li>
</xsl:template>
5 hypertext integration
I know, “integration” sounds scary. This is about making the documents work, particularly with respect to links, in the context of the web site where it will live, which has a somewhat different logical structure than the physical documents.
5.1 adjust link addresses
Part of the value of literate programming is in cross-referencing. As long as
we can address locations in the document, we can do so by using links. During
HTML export, Org will render inter-document links with the assumption that each
document is exported to an equivalently-named .html
file. We are instead
placing them under /about
, with no extension.
<!-- Links to the about documents themselves -->
<xsl:template
mode="org-copy"
match="a/@href[
'.html' = substring(., string-length(.) - 4, 5)
and not(starts-with(., 'http'))]">
<!-- The link may include a subsystem path, which is ignored for this
purpose. -->
<xsl:variable name="with-path" select="substring-before(., '.html')" />
<xsl:variable name="file">
<xsl:for-each select="strings:tokenize($with-path, '/')">
<xsl:if test="position() = last()">
<xsl:value-of select="." />
</xsl:if>
</xsl:for-each>
</xsl:variable>
<!-- Support included documents -->
<xsl:variable name="name"
select="substring-before(concat($file, '__'), '__')" />
<xsl:attribute name="href">
<xsl:value-of select="concat('/about/', $name)"/>
</xsl:attribute>
<<validate document reference>>
</xsl:template>
We just assume that any link ending with .html
and not starting with is a link
to a document—as long as it doesn’t also start with http
, which would indicate
that it’s a link to an external site.
Regarding the double-underscore business. Willshake’s literate document system
supports the use of Org’s INCLUDE feature. For export purposes, included
documents are part of the containing document, and have no independent
existence. But some links will point to included documents. Rather than making
the links “incorrect” (by pointing instead to the containing document), I rely
on the naming convention that included documents use, which is
{parent}__{child}
, where parent
is the name of the containing document.
5.2 rebase site references
Anything in the org document that points to the site
folder can be retargeted
trivially to its child directory:
<xsl:template mode="org-copy" match="img/@src[starts-with(., '../site/')]">
<xsl:param name="attribute" select="'src'" />
<xsl:attribute name="{$attribute}">
<xsl:value-of select="substring-after(., '../site')"/>
</xsl:attribute>
</xsl:template>
This will generally be used for images.
5.3 support development domains
The willshake web site lives at https://willshake.net, of course. But when
working on willshake, you’ll usually run a local copy of the site, at an address
like localhost:8000
. But what if a link uses an absolute path to the real
domain, like this:
https://willshake.net/plays/AYL/4.1#This_looks
That will take you to a different site, which is not what you want. So we change it to a relative path here,
/plays/AYL/4.1#This_looks
This way, the link will be treated as internal.
You might ask, why not just use a relative path in the first place? Because we also publish the documents to non-web-based formats, such as PDF, where URI’s need to be absolute. This transformation is only for the web.
But if this is only needed for development, why do it in production, too? Because it yields shorter paths where the full address is unnecessary.
<!-- Rebase link paths. -->
<xsl:template mode="org-copy" match="a/@href[starts-with(., 'https://willshake.net/')]">
<xsl:attribute name="href">
<xsl:value-of select="substring-after(., 'https://willshake.net')"/>
</xsl:attribute>
</xsl:template>
6 document features
This section is mostly about the formatting of the actual document contents. It should be relatively agnostic of the container.
Everything in the document should be hyphenated.
@require fonts
book-font()
hyphens auto
6.1 headings
h1, h2
font-weight normal
Sections headings are links to themselves.
<xsl:template mode="org-copy" match="h2|h3|h4|h5|h6">
<xsl:variable name="id" select="@id" />
<xsl:copy>
<xsl:apply-templates mode="org-copy" select="@*" />
<a href="#{$id}">
<xsl:apply-templates mode="org-copy" select="node()" />
</a>
</xsl:copy>
</xsl:template>
6.2 images
I think “a thousand words” gets a bad rap in that old saying about pictures. I am happy to have both, and I’m not alone.
Number one, it has to be easy to use images from the document, where “use” means that they show up on the site.
And as long as we’re using Org documents, it’s convenient if the image can also be seen while editing. The simplest way to achieve that is to use image references that are relative to the document. But of course that breaks the first priority, since those references will be no good when the document is in another context such as the web site. By rebasing certain paths, we have it both ways.
6.2.1 marginal images
Thanks to the collection, it’s easy to get and use images around here, and there’s no reason that it should be limited to Shakespeare-related content.
Further to the notion of “a picture and a thousand words,” it would be nice if we could set pictures in the margin—the “side” area that will be reserved for such things (see reading a document). It would be hard to overstate how useful this technique can be, given the right images and enough labor.2
We kind of have that with our “TBD” thing already. Here is the medium-sized image projection example.
Then, we’ll need a way to put things into the margin. You can do that with an
#+ATTR_HTML: :class marginal
before an image link. Org will export this as a
figure with the marginal
class.
.marginal
position absolute
left 100%
width 100%
max-width s('calc(50vw - %s)' % ($flowRems / 2))
margin-left ($flowMargin * 3/4)
6.3 links
Give some indication that links are links, which we don’t by default. This should only apply within narrative text.
p, ul, ol, blockquote
a
text-decoration underline
Yeah, this doesn’t play well with descenders, and so on.3
Of course, I’ve generally avoided using links in the text directly, anyway, favoring links in footnotes, and not hyperlinked text at that (favoring plain URL’s, since these can still be used from printed PDF’s). It’s a good time to revisit that. You can extract the URL during PDF export, without polluting the web documents.
This is kind of a temporary measure. Some code blocks (which don’t wrap) are very long and this bubbles up to affect the computed size of the whole layer.
pre
overflow-x auto
This is more or less for the same reason as above, but it’s less of a hack.
img, object
max-width 100%
Figure images in documents should be centered horizontally.
figure
img, object
display block
margin 0 auto
// why this?
&:not(.marginal)
img, object
border-radius .25em
The link is rendered as plain HTML of course.
6.4 quotes
Block quotes come from other documents—or at least other sources. Here, we make them look like a separate material from the document in which they are quoted.
The risk is that these sheets won’t look like part of the document “flow.” Unlike asides, block quotes are not flow-interrupting; they are supposed to be read in sequence. That’s a downside to this visual cue, which should keep us from using too dramatic of an effect.
blockquote
line-height 1.25
box-shadow 0 0 .5em #111 // make a sheet
// see source block
background tint($documentColor, 50%)
padding .5em 2em
box-sizing content-box
border-radius 1em
border-bottom-right-radius 0
We do occasionally quote some verse around here. Org actually has a special construct for it, which differs from a regular blockquote in that line breaks are preserved as written.
.verse
line-height 1.25
margin-left 4em
6.4.1 TODO formally indicate source
HTML doesn’t require you to provide a citation for a blockquote, and Org doesn’t even have a structure for this.
But I want to adopt a convention that will let me formally indicate the source of quotes. It may be that a footnote at the end of the quote would be the best way to do it.
6.5 lists
Do we actually use dl
’s anywhere?
dt
font-style italic
6.6 special constructs
As I look over the blocks I’ve demarked, several categories arise:
- tbd
- Notes (essentially to myself) about unsatisfactory things that are actionable—or should be soon.
- apologias
- Explanations about (usually) unsatisfactory things that have no immediate or even forseeable remedy.
The tbd’s can always be marginal, because the expectation is that they’ll be removed soon! So you don’t have to worry about how they negotiate a place with other content.
There is another way to look at all of the remaining notes, including the apologias, which is orthogonal to that grouping, and that is whether they must be in the flow or whether they can be marginal.
Another consideration is how long the note is. Longer passages in the margin just don’t work well. It’s not just the fact that absolutely positioned boxes can end up overlapping one another (which you could avoid by judicious use); the length is an indication that it’s probably not really an aside.
It’s possible that with a better layout strategy, there could be a way to
preserve the aside-nature of the blocks that are currently longish aside
’s, but
as things stand, they clearly go back into the document flow.
6.6.1 asides
See note in publish the documents. The class version (.aside
) is needed in
addition to the tag version (aside
) because we’re not really exporting html5
elements.
An aside is a remark that is not addressed to everyone. Hamlet’s first line is an aside. Of course, we’re talking here about asides in documents, but the idea is similar: that asides are not for everyone. The HTML5 specification says
The
aside
element represents a section of a page that consists of content that is tangentially related to the content around theaside
element, and which could be considered separate from that content. Such sections are often represented as sidebars in printed typography.4
Bring together the aside-related code, and generally organize all this by construct, not by language.
So, if we use an aside
(which we go out of our way to do), we are signaling that
it may be taken out of the document flow. Still, it is part of the document,
and moreover—unlike block quotes—its original source is the document. So it
really means something different than a block quote, and should not use the same
visual cue.
6.6.2 tbd
@require pointing
@require document-map-colors // see note in map section
.aside
aside
padding-left 1em
padding-right @padding-left
border-left 4px solid $documentMapColor
&.tbd
position absolute
left calc(100% - 1em)
//margin-left 1em
width calc(50vw - 50% - 1em)
line-height 1.2
box-shadow 0 0 2px
background $documentColor // since we can't inherit
z-index 1 // get in front of positioned things in main flow
// In case they do overlap
+user_pointing_at()
z-index 2
6.6.3 notes
Very often in the course of these discussions, I find myself having to express a rather numbing amount of detail. In many cases, a passage cries out as being especially contingent on ephemera, if not trivia. Although I’m calling these sections “notes” right now, my general sense is that they are mostly just demarkers of incidental complexity—/apologias/ for the way things are.
They may be seen as removable in some way. And yet they aren’t quite unimportant enough to be footnotes.
So what? How should they look?
They aren’t “affordable” in that you can’t do anything with them, so they shouldn’t have any signifiers that would suggest that they did.
But really, why couldn’t these be “side notes”? They could be side notes without being “asides.” I think this would be a distinction within the existing notes. Some of them are just too long to be put in the margin, whatever their semantics. Also, those longer-form notes could conceivably have marginal contents of their own (if, for example, we put footnotes in the margin, which I’m planning to).
Most ways of accentuating these sections call more attention to them, which is kind of the opposite of what I want.
They’re not “asides,” since they’re part of the narrative flow, albeit “parenthetically.” So they’re shown as if the whole block were in parentheses.
@require document-map-colors // see note in map section
.note
padding 0 1em
text-align justify
font-size 90%
line-height 1.2
// "Parentheses"
border-left .25em solid $documentMapColor
border-right @border-left
border-radius .75em / 50%
6.7 dialogs
—Hey, what’s better than a monologue?
—A dialog?
—What’s better than a dialog?
—Shakespeare?
—Okay, well this ain’t Shakespeare, but it’s a nice break from the narrative, don’t you think?
—If you say so.
Just for fun, let’s use dialogs. It’s kind of a zen teacher/student thing. For
the structure itself, no extra code is needed. Just by using a DIALOG
block,
#+BEGIN_DIALOG
---Who's there?
---Nay, answer me. Stand and unfold yourself.
---Long live the king.
---Barnardo?
---He.
#+END_DIALOG
Org export will create a div
with class dialog
. Everything inside of it will be
formatted as elsewhere.
To create the back-and-forth effect, the paragraphs are set alternately to the left and right, much like a typical text messaging app.
speaker(set, side)
p:nth-child({set})
float side
text-align side
border-bottom-{side}-radius 0
Otherwise, they are the same: little speech bubbles.
@require document-map-colors
.dialog
> p
margin .25em 0
padding .5em 1em
line-height 1.25
border-radius 1em
clear both
max-width 66%
background lighten($documentMapColor, 80%)
// Assume first character is a dash, and hide it
text-indent -1em
&:first-letter
color transparent
&:not(.flipped)
speaker(odd, left)
speaker(even, right)
&.flipped
speaker(even, left)
speaker(odd, right)
// The old "clearfix" trick
&:after
content ''
display block
clear both
As of now, the bubbles just have a light gray background. Use any stronger color, and you have to reverse the text, which I think is too “dramatic” for this. So I don’t even use two different shades for the speakers, because there isn’t much of a range that is workable here. The same goes for using a background color on the whole section, and making the speech bubbles white, as we do in the plays. It just creates a break in the document flow that is too abrupt.
Because of the max-width
, these can sometimes wrap in an ugly
way.5
Also, the :first-letter
selector isn’t working in Firefox. I’ve seen this
happen before when the character is a non-ASCII character. Not sure what to do
about it.
—But wait, what if you wanted to start on the other side?
—Then you mark the dialog as “flipped”.
—How do you do that?
—With #+ATTR_HTML: :class flipped
.
—Nice.
For other formats (such as PDF), the contents will be completely unaffected.
6.8 figures
Figures are first-class in HTML (as of HTML5). Mozilla Developer Network, my unofficial reference of choice, describes it this way:
The HTML
<figure>
element represents self-contained content, frequently with a caption (<figcaption>
), and is typically referenced as a single unit. While it is related to the main flow, its position is independent of the main flow. Usually this is an image, an illustration, a diagram, a code snippet, or a schema that is referenced in the main text, but that can be moved to another page or to an appendix without affecting the main flow.6
At the moment, all images in these documents are automatically considered figures. It’s an open question as to whether I should be able to distinguish between these cases. As the official specification notes,
When a
figure
is referred to from the main content of the document by identifying it by its caption (e.g. by figure number), it enables such content to be easily moved away from that primary content, e.g. to the side of the page, to dedicated pages, or to an appendix, without affecting the flow of the document.If a
figure
element is referenced by its relative position, e.g. “in the photograph above” or “as the next figure shows”, then moving the figure would disrupt the page’s meaning. Authors are encouraged to consider using labels to refer to figures, rather than using such relative references, so that the page can easily be restyled without affecting the page’s meaning.
Although I agree with the general point, I have not reviewed whether I have legitamite cases of the latter kind.
figure
margin 0 // beat figure default
// already set elsewhere, right? to auto, which is effectively the same
//img
// margin 0
figcaption
margin .5em 1em
line-height 1.2
font-size smaller
text-align center
.figure-number
display block
font-style italic
Marginal figures are floating off of the document sheet, so they need to be their own material.
figure.marginal
border-radius .25em
background lighten($documentMapColor, 66%)
box-shadow -.25em .25em .5em #111
6.8.1 Org and HTML5 figures and captions
This can be filed under “compensating for Org export,” a category which at some point will start to rival the complexity of implementing my own export. To be fair, Org may do a better job of this if “fancy” elements are enabled, but they’re not. See HTML publication options.
Figures are currently rendered by Org export as a div.figure
with the contents
wrapped inside of a p
, followed by another p
for the caption. So a bit of
schlepping is needed to translate that into HTML5.
<xsl:template mode="org-copy" match="*[@class='figure']">
<figure>
<xsl:apply-templates mode="org-copy" select="node()|@*" />
</figure>
</xsl:template>
<xsl:template mode="org-copy" match="*[@class='figure']/p[2]">
<figcaption>
<xsl:apply-templates mode="org-copy" select="node()|@*" />
</figcaption>
</xsl:template>
The image is also needlessly wrapped inside of a paragraph element.
<xsl:template mode="org-copy" match="*[@class='figure']/p[img]">
<xsl:apply-templates mode="org-copy" select="node()|@*" />
</xsl:template>
Also, when I use HTML_ATTR: :class
before an image, Org applies the class to the
img
, not the container. This bubbles the class from the img
to the containing
figure
.
<xsl:template mode="org-copy" match="@class[.='figure']">
<xsl:copy-of select="..//img/@class" />
</xsl:template>
<xsl:template mode="org-copy" match="*[@class='figure']//img/@class" />
6.9 section status
I am using keywords for bugs and proposals and such, but I don’t think this is working now. Not because of this, but because the keywords I use aren’t set at export time.
Mostly for internal use, but no harm in publishing.
@require colors
keyword($color)
color $color
text-shadow 1px 1px 0 #555
&:before
content ''
position absolute
right 100%
margin-right 1em
width 1em
height 100%
background $color
box-shadow 1px 1px 1px #555
$badColor = desaturate(red, 50%)
.tbd
@extend .document .aside
border-left-color $badColor
.outline-2
.outline-3
.outline-4
position relative // support section status markers
#table-of-contents li
.todo:before
display none
.todo
font-family sans-serif;
.STUB
keyword($badColor)
.DRAFT
keyword(desaturate(yellow, 50%))
.PUBLIC
keyword(desaturate(green, 50%))
6.10 IDEA epigrams
I use epigrams a lot, and it seems that they could do with some special formatting. At the least, they are usually right-aligned.
Footnotes:
David M. MacMillan and Rollande Krandall, “‘Minimalist’ Literate Programming in graphein”
See, for example, the document version of Maciej Cegłowski’s fantastic presentation “Web Design: The First 100 Years” (2014). http://idlewords.com/talks/web_design_first_100_years.htm
Dave Liepmann & Edward Tufte, “Tufte CSS.” https://edwardtufte.github.io/tufte-css/
HTML5 § 4.3.5 “The aside
element”. W3C Recommendation
28 October 2014. http://www.w3.org/TR/html5/sections.html#the-aside-element
This problem and its lack of a solution are addressed by the question, “max-width adjusts to fit text?” http://stackoverflow.com/q/13672760
“<figure>”, Mozilla Developer Network. See also the
somewhat more turgid official specification of § “The figure
element”.