Elsewhere, we talk about the naming of resources in the collection. We assign to each image a unique name, called its key (or sometimes identifier).

Why? Because we’ll need to reference the image, of course. While most of the images come from external systems that themselves have unique identifiers for the images, these vary wildly in their format and level of detail.

We observe the following guidelines when assigning an image key. It’s not an exact science.

The main images in our “catalog” are those related to Shakespeare’s plays. The general form for these is {source}-{target},

Where {source} is, in order of preference (as applicable):

1. Artist last name (use spelling in English Wikipedia address)
2. Actor last name
3. Production company short name

And so on. Also,

• use both artist last names when there is, e.g.,
  • {engraver}-{painter}
  • {painter}-{actor}
  • but not for a photographer and actor, then just use the actor

and {target} is, likewise,

• {play}, if the image is to the play generally
• {play}-poster, for posters
• {play}-{section}, where a particular section is depicted
• {role}, where a particular role is depicted in isolation (that is, where you can’t tell which scene it is)
• {role}-{role}, where two roles are likewise depicted
• {play}-{role}, for the obscurer case (e.g. the “Juliet” in MM) where above would be ambiguous
• {target}-{version} where version specifies medium or otherwise disambiguates from a similar entry

Note that some keys begin with digits. This means that you can’t use them at the beginning of a DOM identifier without causing problems.

Image records contain some standard and some special metadata.

Here, for example, is catalog-Abbey-Ham-3.2.jpg.xml:

<by>
	<artist person="Edwin_Austin_Abbey" />
</by>
<of play="Ham" section="3.2" anchor="Do_you_think_I" />
<from commons-file="The_Play_Scene_in_Hamlet.jpg" />
<note>
	interesting but lower-res version at The_Play_Szene_in_“Hamlet”.jpg
</note>

The whole purpose of the image records is to store information about the image. But since we use a file-based build system, there are certain things that we want to know about the image a priori—without even opening the record. We encode those bits in the filename, like so:

{source type}-{name}.{format}.xml

So, an image record like

catalog-Abbey-Ham-3.2.jpg.xml

would break down as shown in this figure.

We will match on these patterns to route the files through the build process. At each point we make sure that the files have an extension representing their actual content.

Local images are the easiest to deal with. Not only do we already have the file handy, but we’re also going to make the assumption that they’re all going to be used as-is (i.e. without projections).

: foreach $(ROOT)/assets/images/* \
|> !link_from \
|> $(SITE)/static/images/%b

For “catalog” type media, we don’t store the download location directly. Instead, we store the name that uniquely identifies the image’s record in the remote catalog system. That system will have some kind of API that allows us to look up their metadata for that image, which, among other things, will include its ultimate download location.

So we start by getting the other system’s metadata for each catalog image.

: foreach $(IMAGE_RECORDS)/catalog-*.xml \
  | $(PROGRAM)/get-resource \
    $(PROGRAM)/images/* \
|> ^o get image metadata %B^ \
   $(PROGRAM)/images/get_image_metadata \
   $(PROGRAM)/get-resource \
   $(PROGRAM)/images/commons-query-imageinfo "%f" "%o" "%g" \
|> $(IMAGE_METADATA)/%g.xml

This build rule breaks down the image filename structure that we described earlier. First, it only deals with catalog- images. The image’s name and format are captured by the globbing star (*), which is made available in the %g flag. The %g that we pass to the command is actually not used by the system, but supports aliasing in the internet cache; see get-resource.

tup error: %e is only valid with a foreach rule for files that have extensions.
 -- Path: './database/images/catalog-1629-Puck.jpg.xml'
tup error: Unable to parse :-rule from run script: ': foreach $(IMAGE_RECORDS)/catalog-*.*   '

Getting the metadata means

1. figuring out the exact API call to make (that is, what URL)
2. calling it and storing the result

Step 2 will be handled by get-resource, so that we can take advantage of its caching. This is especially important for remote systems that may limit the number of calls you can make (which Wikimedia Commons does).

import sys
from io import BytesIO
from subprocess import call
from lxml import etree
from urllib import parse

(__, get_resource, query_file,
 image_record, out_file, name) = sys.argv

API_URL = "http://commons.wikimedia.org/w/api.php"

# Would be less roundabout if only you could parse a fragment!
with open(image_record, 'r', encoding='utf-8') as f:
    record = ('<r>' + f.read() + '</r>')
    context = etree.iterparse(BytesIO(record.encode('utf-8')), tag='from', events=['start'])
    __, element = next(context)
    commons_file = element.get('commons-file')

# We store the query in a one-parameter-per-line format.  The values may need
# escaping, but we need to leave the equal signs alone.
with open(query_file, 'r') as f:
    query = '&'.join(parse.quote(line.strip(), safe='=') for line in f)

url = API_URL + '?' + query + parse.quote(commons_file)

call([get_resource, url.encode('utf-8'), out_file, name + '__metadata'])

Right now, the only remote catalog system we deal with is Wikimedia Commons. The build rule passes the following file, containing the API parameters for a basic metadata request:

action=query
prop=imageinfo
format=xml
iiprop=comment|parsedcomment|url|size|dimensions|mime|thumbmime|mediatype|metadata|archivename|bitdepth
iilimit=10
titles=File:

It expects the filename you’re inquiring about to be appended at the end. (It’s also formatted for readability; see the script.)

If all goes well—like, the file actually exists, for example—the query returns an XML document containing certain information about the image. Specifically, it will include the fields we requested with the iiprop parameter. This is kind of “meta-metadata” in the sense that it’s mostly about the remote system’s record of the image (not so much about the image itself). (You can see an example here.) The only thing relevant to our present purpose is url—the URL of the actual image file— which can be easily extracted with a simple XPath expression.

<xsl:output method="text" />
<xsl:template match="/">
	<xsl:value-of select="(//ii)[1]/@url"/>
</xsl:template>

So we can download now, right? Sure. But first we write the download location to its own file.

: foreach $(IMAGE_METADATA)/*.xml \
  | $(PROGRAM)/images/* \
|> ^o get catalog image url %B^ \
   xsltproc $(PROGRAM)/images/extract_commons_image_url.xsl "%f" > "%o" \
|> $(IMAGE_LOCATION)/%B.txt

This apparently overzealous step is very effective in preventing needless calls to get-resource, since downstream build actions will not be triggered unless the download location actually changes, regardless of what refactoring we may do upstream.

Now we know where the images live. Let’s go get them one by one.

: foreach $(IMAGE_LOCATION)/* \
  | $(PROGRAM)/get-resource \
|> ^o get image resource %B ^ \
   $(PROGRAM)/get-resource "`cat '%f'`" "%o" "%B" \
|> $(IMAGES)/%B

This is where we get the things people usually think of as “metadata"—the title, artist, year, medium, and so on. It’s currently here for reference and not formally used at this time. The metadata that we have in our image records was entered manually from the web site.

For example, suppose you wanted to use an image on the web site. Sure, you can set the image’s display size using CSS, but the original image could be several megabytes. It’s much more efficient to pre-scale images. A projection that limits images to 700 pixels on either side can be written in a few lines.

in="$1"
out="$2"
convert "$in" -resize '700>' -format jpg "$out"

So touching the file database/image-projections/Sargent-Terry-Lady_Macbeth@margin.jpg.txt will create a scaled-down version of that image.

Second, you have to reference it somewhere. The first step will cause the image to be distributed with willshake, but it still won’t appear anywhere unless you point to it!

Like any digital media, images have to be encoded in some format.¹

For our purposes, we consider all of the various image formats to be interchangeable.

As noted above, we indicate the source format of an image in the filename of its record, and we preserve this throughout the process of retrieving it. But once we have the image, we reserve the right to convert it to any other format. In practice, we never use images directly in their original format, but convert them to JPEG. We use JPEG everywhere, both for the convenience of uniformity and for the efficiency of superior compression.

With all this in mind, we set out to define the “programmatic grid” whereby such detailing can be effected.

The “Howard” (or “Spirit”) images, i.e. the 500 or so plates from The Spirit of the Plays of Shakespeare by Frank Howard, which cover the whole canon, deserve special consideration. The full plates are a perfect size for a landscape-oriented “handheld” display (of the smallest size we consider), but they become illegible at portrait size due to their nature as line drawings (since the pixel size becomes smaller than many of the lines, although note that I haven’t tried this on a high-resolution device).

Without further ado, let’s make some details.

We’re going to define a usage type called detail. Each image will get to decide certain things about how it gets detailed, but the output size will always be the same.

This will work by writing the image’s specific preferences to a file like

$(IMAGE_USAGE)/detail/{image}

For the moment, let’s just scale them down without reading the details of the details.

: foreach $(IMAGE_USAGE)/detail/* \
  | $(IMAGES)/* \
|> convert $(IMAGES)/%b.* \
	   -resize '512' \
	   -format jpg %o \
|> $(SITE_IMAGES)/detail/%b.jpg

This program will crop an image given a rectangle that is defined by reference to the original image’s size. In other words, the inputs are ratios, not pixels.

import sys
from subprocess import call, check_output, CalledProcessError

if len(sys.argv) < 7:
    sys.exit('usage: in-image out-image dx dy dw dh');

in_image = sys.argv[1]
out_image = sys.argv[2]
dx, dy, dw, dh = map(float, sys.argv[3:7])

try:
    size = check_output(
	['convert', in_image,
	 '-format', "%w %h", 'info:']
    ).decode('utf-8').strip()

except CalledProcessError as e:
    sys.exit("error: couldn't get image dimensions: "
	     + str(e.output))

width, height = map(int, size.split(' '))

px, py = width * dx, height * dy
pw, ph = width * dw, height * dh

# print("width = {}".format(width))
# print("height = {}".format(height))
# print("px = {}".format(px))
# print("py = {}".format(py))
# print("pw = {}".format(pw))
# print("ph = {}".format(ph))

call(['convert', in_image,
      '-crop', '{}x{}+{}+{}'.format(pw, ph, px, py),
      '+repage', out_image])

# That is equivalent to the less-explicit
# 
# call(['convert',
#       '{}[{}x{}+{}+{}]'.format(in_image, pw, ph, px, py),
#       out_image])

This is a general cropping program that doesn’t use the bizzare specification that I define above.