photo (3)

John Hurst

Version 1.11.4


Related Programs

camera3 - camera module with common functions for other photo operations;
download3 - collect image files from camera; filePics - (this program has been decommissioned);
photo3 - process images in directory to build web pages (this document); tardis3 - change time and date stamps on processed photos.
EXIF format - details of the EXIF file formats.
EXIF Tips - Tips & tricks to batch edit EXIF metadata of photos.

Table of Contents

1 The Main Program
1.1 Setup
1.2 Define miscellaneous subroutines
1.3 Collect the Command Line Options
1.4 Perform the Top Level Visit
1.5 write list file
1.6 Plan for new data structure
2 The visit routine
2.1 initialize variables for visit routine
2.2 extract date field as default for ordering
2.3 sort fnames into directories and images
2.4 open index.xml and process directory information
2.5 get ordering information from album XML file
2.6 write tree element to index.xml
2.7 Visit All Directories
2.8 scan all images and process them
2.9 wind up index file
3 Descriptions
3.1 read descriptions file and create dictionary
3.2 wind up descriptions file
4 the addToIndex routine definition
5 the addToList routine definition
6 the makeImage routine
7 the makeSubImages routine
7.1 New SubImage generalized size definitions
8 the retrieve support routines definition
9 The getAlbumParameters routine
10 The index.xml file format
11 Makefile
12 Appendices
12.1 Chunk Indices
12.2 Macro Indices
12.3 Indentifier Indices
12.4 Document History

1. The Main Program is a Python program that organizes digital photographic images into a set of browsable web pages. The images are assumed to be organized into directories. (A separate program,, performs this function while downloading images from the cameras.) These directories form the basis for browsing amongst the generated web pages.

Within each directory, an index.xml file is generated. This XML file is renderable into HTML by a separate XSL program, album.xsl, also available separately. The format and structure of these directories is user-controlled, and photos can be arranged by subject, date, location, etc., etc.. Within a directory, photos are arranged by the order specified in an associated descriptions file, containing the image name and an image description. The initial sequence of image names is generated automatically for any image found in the directory, but not present in the descriptions file. The descriptions file is created if one is not found. See the section The index.xml file format for details.

Within each directory, an album.xml file describes the content of the directory, and is used to generate the associated index.xml file. The album.xml file contains the path name of the directory relative to the root directory of all photos managed by the system, along with the directory description and name, and a path to a thumbnail image that will represent the directory. This thumbnail image can be one of the photos contained within the directory, or elsewhere, and thus can be stylised to suit the directory contents.

"" 1.1 =
<interpreter definition 1.2> <banner 1.3> <basic usage information 1.4> <todos 1.5> <imports 1.6> <initialization 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1> <define miscellaneous subroutines 1.14,1.15,1.20,2.13,9.1> <the addToList routine definition 5.1> <the makeImage routine definition 6.1> <the makeSubImages routine definition 7.2> <the addToIndex routine definition 4.1> <the visit routine definition 2.1> <the retrieve routine definition 8.2> <collect the command line options 1.16> <perform the top level visit 1.17> <write list file 1.21> **** Chunk omitted!

1.1 Setup

<interpreter definition 1.2> = #! /home/ajh/binln/python3
Chunk referenced in 1.1
<banner 1.3> =
######################################################################## # # # p h o t o . p y # # # ########################################################################
Chunk referenced in 1.1
<basic usage information 1.4> =
# A program to build photo album web pages. # # John Hurst # version <current version 12.1>, <current date 12.2> # # This program recursively scans a set of directories (given as # command line parameters) for images that comprise a photo album to # be rendered into a set of web pages. A file 'index.xml' is created # in each (sub)directory, containing a list of images and # subdirectories. This file can be rendered by an XML stylesheet # 'album.xsl' into HTML pages.
Chunk referenced in 1.1
<todos 1.5> =
# TODO # 20060306 flag descriptions that do not have full details (and auto # copy this to a 'NOT COMPLETE' line in higher level # album.xml data). # # 20190228 handling of thumbnail photo is bad - first photo keeps getting # re-instated as thumbnail, rather than that specified in album.xml # # 20200304 when album is more recent than index, and photo is run # over the directory, the camera model info is removed, and # you have to recreate it with a -f
Chunk referenced in 1.1
<imports 1.6> =
import cgi from EXIF3 import ImageMetaData import PIL #from PIL import Image #from PIL.ExifTags import TAGS import datetime import getopt import hashlib import os import os.path import re import stat import subprocess import sys import time import xml.dom from xml.dom import Node from xml.dom.minidom import parse from xml.parsers.expat import ExpatError
Chunk referenced in 1.1

<initialization 1.7> =
indexdict={} debug=0
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

Initialize the debug flag. This is currently set manually, but eventually we expect to make it a command line option.

<initialization 1.8> =
lists = {}
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

Initialize the variable 'lists'. This is a python directory, where the keys are (os) directories paths rooted at the album subdirectory (the cli parameters), and the entries are lists of image file names within that directory. This is used to build a list of images within each (os) directory traversed. The list/directory is initially empty.

<initialization 1.9> =
descriptions={} sounds={} protected={}
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

Initialize the variable 'descriptions'. This is a python directory, where the keys are image file names, and the entries are descriptions/captions for the image concerned. It is built from the eponymously named file 'descriptions' in the directory currently being scanned.

<initialization 1.10> =
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

Define the XSL script used to render all XML files built by this program.

<initialization 1.11> =
ignorePat=re.compile("(.*(_\d+x\d+)\..*$)|(.xvpics)") medmatch=re.compile(".*_640x480.JPG") bigmatch=re.compile(".*_1600x1200.JPG") thumbmatch=re.compile(".*_128x128.JPG") identifyPat = re.compile(".* (\d+)x(\d+)( |\+).*") treepat=re.compile('.*/(20[/\d]+)')
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

<initialization 1.12> =
maxmissing = 0 maxdir = ''
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

maxmissing, maxdir are variables that track the directory with the maximum number of missing descriptions. This is purely a convenient device to point the user to where most documenting effort is required!

<initialization 1.13> =
(sysname,nodename,release,version,machine)=os.uname() if sysname=='Linux': EXIV2='/usr/bin/exiv2' IDENTIFY='/usr/bin/identify' CONVERT='/usr/bin/convert' elif sysname=='Darwin': EXIV2='/usr/local/bin/exiv2' IDENTIFY='/usr/local/bin/identify' CONVERT='/usr/local/bin/convert' else: print("You have not yet programmed this module for use on {} systems".format(sysname))
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

1.2 Define miscellaneous subroutines

<define miscellaneous subroutines 1.14> =
jpgPat=re.compile('(.*)\.(j|J)(p|P)(g|G)$') def trimJPG(name): res=jpgPat.match(name) if res: return name
Chunk referenced in 1.1
Chunk defined in 1.14,1.15,1.20,2.13,9.1
<define miscellaneous subroutines 1.15> =
def attrString(el): if el.hasAttributes(): str='' nl=el.attributes if debug: print("looking at attributes of length %d" % (nl.length)) for i in range(nl.length): attr=nl.item(i) str+=' %s="%s"' % (,attr.value) return str else: return '' def flatString(elem): if elem.nodeType==Node.TEXT_NODE: return elem.nodeValue elif elem.nodeType==Node.ELEMENT_NODE: attrs=attrString(elem) str="" for el in elem.childNodes: str+=str+flatString(el) return "<%s%s>%s</%s>" % (elem.tagName,attrs,str,elem.tagName) else: return "[unknown nodeType]" def flatStringNodes(nodelist): str='' for node in nodelist: str+=flatString(node) return str #def get_exif(fn): # ret = {} # i = # info = i._getexif() # if info: # for tag, value in info.items(): # decoded = TAGS.get(tag, tag) # ret[decoded] = value # return ret
Chunk referenced in 1.1
Chunk defined in 1.14,1.15,1.20,2.13,9.1

1.3 Collect the Command Line Options

There are four command line options:

turn on debugging
Force the generate of the image XML files. This is useful if other relevant components have changed, such as the XSL files, which are not known about here.
Do not generate the large subimages.
Recursively traverse any subdirectories for further images.
Generate only the thumbnail subimages.
<collect the command line options 1.16> =
try: (opts,args) = getopt.getopt(sys.argv[1:],'dflrs:tv') except getopt.GetoptError as err: print("Don't know that option: {}".format(err)) opts=[]; args=[] forceXmls=recurse=thumbsOnly=False; large=True for (option,value) in opts: if option == '-d': debug=True elif option == '-f': forceXmls=True elif option == '-r': recurse=True elif option == '-l': large=False elif option == '-t': thumbsOnly=True elif option == '-v': print("<current version 12.1>") sys.exit(0)
Chunk referenced in 1.1

1.4 Perform the Top Level Visit

<perform the top level visit 1.17> =
visitdirs = args # if there are no directories to visit, include at least current directory if len(visitdirs)==0: visitdirs = ['.'] for dir in visitdirs: checkdir = os.path.abspath(dir) (h,t) = os.path.split(checkdir) (titl,nph,nalb,thm,descr,nmiss,order)=visit(0,checkdir,'','',lists) atTop=False while not atTop: reldir=os.path.relpath(checkdir) msg = "Directory %s has %d images " % (reldir,nph) msg += "and %d missing descriptions" % (nmiss) print(msg) <explore next higher level to update counts 1.18>
Chunk referenced in 1.1

We visit every directory given as arguments to the program invocation. If no arguments are given, then visit the current directory.

All the real work of visiting a directory and computing the index XML files is done by the visit procedure, which returns a tuple of potentially changed values that are of relevance to higher level directory index XML files. These are, respectively: the title of the visited directory; the number of photos or images contained in the directory and its sub-directories; the number of albums ditto; the thumbnail to characterize the directory; the description of the directory; and the number of missing descriptions in the directory and its sub-directories.

These values are then propogated to higher level photo albums (directories) via the chunk <explore next higher level to update counts >, which also recomputes updated values of the tuple described above.

<explore next higher level to update counts 1.18> =
try: higherdir=checkdir+'/..' highername=higherdir+'/index.xml' higherindex=open(highername) higherdom=parse(higherindex) higherindex.close() if debug: print("parsed %s" % (highername)) direlements=higherdom.getElementsByTagName('directory') countalbums=countphotos=countmissing=0 maxmissing=0;maxdir="" <scan all directory elements, updating counts 1.19> (titl,nph,nalb,thm,descr,nmiss)=\ ('',countphotos,countalbums,'','',countmissing) try: summary=higherdom.getElementsByTagName('summary')[0] summary.setAttribute('photos',"%d" % (nph)) summary.setAttribute('albums',"%d" % (nalb)) summary.setAttribute('missing',"%d" % (nmiss)) summary.setAttribute('maxmissing',"%d" % (maxmissing)) summary.setAttribute('maxdir',maxdir) except: pass # no summary to worry about newxml=higherdom.toxml() if debug: print(newxml) higherindex=open(highername,'w') higherindex.write(newxml) higherindex.close() checkdir=os.path.abspath(higherdir) (h,t) = os.path.split(checkdir) except IOError: #print("No higher level index.xml file") atTop=True except ExpatError: print("XML error in higher level index.xml file") atTop=True
Chunk referenced in 1.17

The next higher directory is identified and the index.xml file parsed. This is used to identify all the directory elements, which are scanned to update the key parameters. The title, thumbnail and descriptions are reset, since these will not change in the higher level summaries.

The summary element is then recomputed, and the index.xml file rewritten.

Note that this rewriting may not be necessary if none of the key variables have changed. This is not currently tested.

<scan all directory elements, updating counts 1.19> =
for direl in direlements: if direl.getAttribute('name')==t: if debug: print("found directory %s!" % (t)) if titl: direl.setAttribute('title',titl) direl.setAttribute('albums',"%d" % nalb) direl.setAttribute('images',"%d" % nph) if thm: direl.setAttribute('thumb',t+'/'+thm) if descr: direl.setAttribute('description',descr) if nmiss==0: if direl.hasAttribute('missing'): direl.removeAttribute('missing') else: direl.setAttribute('missing',"%d" % nmiss) countalbums+= collectAttribute(direl,'albums') countalbums+= 1 # include one album for this directory countphotos+= collectAttribute(direl,'images') miss=collectAttribute(direl,'missing') countmissing+=miss if miss>maxmissing: maxmissing=miss maxdir=direl.getAttribute('name')
Chunk referenced in 1.18
<define miscellaneous subroutines 1.20> =
def collectAttribute(d,a): if d.hasAttribute(a): v=d.getAttribute(a) try: return(int(v)) except: print("Cannot convert attribute '%s' with value '%s'" % (a,v)) return(-1) else: return 0
Chunk referenced in 1.1
Chunk defined in 1.14,1.15,1.20,2.13,9.1

1.5 write list file

<write list file 1.21> =
keys=lists.keys() for k in keys: try: listn=k+"/list" listf=open(listn,"w") except: print("Cannot open %s" % (listn)) continue for a in lists[k]: listf.write(a+"\n") listf.close()
Chunk referenced in 1.1

Open the file list in each of the directories visited, and write a list of images found in that directory to the file. This may not be entirely accurate, and needs to be confirmed that it does work correctly. It has not been used recently, and may be superflous to needs. It has now been commented out (v1.2.3).

1.6 Plan for new data structure

In order to develop this program further, it is suggested that a full data structure for each image and album be developed. This data structure would be populated in some way, and would then provide the data for writing the index.xml file. How is it populated?

  1. from the current index.xml file
  2. from the album.xml file
  3. from the image file
  4. from the descriptions file
  5. from the command line (?)

There would be a list of such data components for each directory visited (hence a local variable to routine visit). This list could be sorted on one or more of its attributes to provide a range of views.

Components of the new data structure:

name the name of the image or album
title a title or caption for the image/album
date/time the date and time of the image
threads (new theme) a list of thread names
shutter/aperture shutter speed and aperture opening
film speed
flash used

2. The visit routine

<the visit routine definition 2.1> =
def blankTemplate(title,descr,date): print ("blankTemplate(title={},descr={},date={})".format(title,descr,date)) return '''<!DOCTYPE album SYSTEM "/home/ajh/lib/dtd/album.dtd"> <album> <title> {} </title> <thumbnail></thumbnail> <shortdesc> {} </shortdesc> <longdesc> Box </longdesc> <date> {} </date> <order></order> </album> '''.format(title,descr,date) def visit(level,dirname, prev, next, lists): global descriptions, sounds, protected, indexdict <initialize variables for visit routine 2.2> <extract date field as default for ordering 2.3> <read descriptions file and create dictionary 3.2> fnames=os.listdir(".") fnames.sort() dirs=[]; <sort fnames into directories and images 2.4> <get information from album and index files 2.5,2.7> #print("1-thumbnail={}".format(thumb)) basename=os.path.basename(dirname.rstrip('/')) indent=' '*level print('%sScanning directory "%s" ... (%s)' % (indent,basename,title)) <write tree element to index.xml 2.12> {Note 2.1.1} #print("2-thumbnail={}".format(thumb)) if not thumb: if len(images)>0: thumb=images[0] print("No thumbnail in directory {}".format(dirname)) {Note 2.1.2} if thumb[-4:len(thumb)].lower()=='.jpg': thumb=thumb[0:-4] indexout+=' <thumbnail>%s</thumbnail>\n' % (thumb) #print("3-thumbnail={}".format(thumb)) <visit all directories 2.14> #sys.stdout.write("%d" % (len(images))) <scan all images and process them 2.18> #print("Order value is {}".format(indexorder)) if not indexorder: print(("Order value ({}) undefined in {}".format(indexorder,dirname))) indexout+=' <summary photos="%d" albums="%d" missing="%d" \ maxmissing="%d" maxdir="%s" minmissing="%d" \ mindir="%s" order="%s"/>\n' % \ (nphotos,nalbums,missingdescrs,maxmissing,maxdir,\ minmissing,mindir,indexorder) <wind up index file 2.19> <wind up descriptions file 3.3> os.chdir(saved) return (title,nphotos,nalbums,thumb,description,missingdescrs,indexorder)
Chunk referenced in 1.1
{Note 2.1.1}
if there is no thumbnail image defined in the album.xml file, use the first image in the directory as the default
{Note 2.1.2}
remove any trailing image type extension from thumbnail name

The visit routine is called to visit and process images in the given directory dirname. It returns a tuple of parameters describing the images within the directory.

2.1 initialize variables for visit routine

<initialize variables for visit routine 2.2> =
nphotos=nalbums=missingdescrs=gotdescrfile=0 listmissingimages=[] albumdom=None; title=""; thumb=description="" maxmissing=0; maxdir=""; minmissing=999999; mindir="" saved = os.getcwd() indexorder='' # make this local to whole routine indexout='' # make this local to whole routine indexin='' # make this local to whole routine try: os.chdir(dirname) except OSError: print("*** Cannot open that directory - do you have the correct path?") sys.exit(1) mdescr=-1 images=[]
Chunk referenced in 2.1

A list of the local variables (in alphabetic order).

The XML-parsed album file.
The description of this album.
Set if the descriptions file was successfully read.
A string value used to defined the lexical ordering of directories withing the index.xml file.
A string variable containing the previous contents of the index.xml file.
A string variable containing the partially assembled contents of the index.xml file.
A list of those images for which a description was found, but no image file.
The number of images for which no description was found in the descriptions file.
The number of subalbums (subdirectories containing images) found in this subdirectory.
The number of image files found in this subdirectory (album).
The title of this album (subdirectory).
The thumbnail image used to represent this album.

2.2 extract date field as default for ordering

<extract date field as default for ordering 2.3> =
#print(dirname) testdir=dirname # little kludge to allow for dateline crossing (east->west) if testdir[-1] in ['a','b']: testdir=testdir[0:-1] res=True mtchstr='' while res: res=re.match('.*/(\d+)$',testdir) if res: testdir=re.sub('/''$','',testdir) #print(mtchstr) if mtchstr: indexorder=mtchstr #print(("got indexorder= {} for {}".format(indexorder,dirname))) else: print("date extraction failed on directory {}".format(testdir))
Chunk referenced in 2.1

Initialize the indexorder to a default value. A moot point is whether this needs to be the full date (as here), or just the number of this directory, since the latter should be sufficient to define order withing the page.

2.3 sort fnames into directories and images

<sort fnames into directories and images 2.4> =
dirIgnores=re.compile("(movies)|(sounds)") if debug: print(images) for f in fnames: res=ignorePat.match(f) if res: if debug: print("ignoring %s" % (f)) continue if os.path.isdir(f): if not dirIgnores.match(f): dirs.append(f) else: print(' '*level,"skipping subdirectory %s" % (f)) continue if os.path.isfile(f): (r,e)=os.path.splitext(f) if e.lower() in ['.jpg','.avi']: if r not in images: images.append(f) listmissingimages.append(f) continue
Chunk referenced in 2.1

Sort the list of all files in this directory into directories and images. We detect the first from an os.path.isdir call, and the second by checking its extension, which must be either .JPG or .jpg (or capitalisations in between).

2.4 open index.xml and process directory information

Every directory in the photo album should contain two auxiliary files: album.xml and index.xml. The first is created by the user and contains meta level information about the directory, such as title and description of the contents. The second is generated by this program, and contains both information derived from album.xml, the image files in this directory, and information from any nested sub-directories.

If no index.xml file exists, one is created from the album.xml file. Once the index file is created, subsequent scans of this directory need only access this index file to collect information about the directory and its subdirectories, unless the album.xml has been modified in the meantime, in which case it must be rescanned and the index file rebuilt.

<get information from album and index files 2.5> =
# first check modification times of both album and index files indexpath=os.path.abspath("index.xml") albumpath=os.path.abspath("album.xml") if os.path.exists(indexpath): istats=os.stat(indexpath) imod=istats.st_mtime imoddatetime=datetime.datetime.fromtimestamp(imod) #print("index.xml modified at {}".format(imoddatetime.strftime("%Y%m%d:%H%M%S"))) else: # no index file, any album file (if it exists) is older imoddatetime=datetime.datetime(datetime.MINYEAR,1,1) if os.path.exists(albumpath): astats=os.stat(albumpath) amod=astats.st_mtime amoddatetime=datetime.datetime.fromtimestamp(amod) #print("album.xml modified at {}".format(amoddatetime.strftime("%Y%m%d:%H%M%S"))) else: # no album file, any index file (if it exists) is younger if not os.path.isfile(albumpath): # here is where we should create a brand new album, populated by # things we currently know. # first, define parameters (currently blank) descr="" monthname=["Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec"] # old version of pattern, ? #res=re.match(".*/(\d{4})/((\d{2})/(\d{2})?)?\/album.xml$",albumpath) res=re.match(".*/(\d{4})/((\d{2})(/(\d{2}))?)?\/album.xml$",albumpath) if res: if month: monthtext=monthname[int(month)-1] if day: date="{}{}{}".format(year,month,day) title="{} {} {}".format(day,monthtext,year) elif month: date="{}{}".format(year,month) title="{} {}".format(monthtext,year) else: date="{}".format(year) title="{}".format(year) else: print("Could not extract date from {}".format(albumpath)) date="" title="" # second, a blank template with parameters: template=blankTemplate(title,descr,date) print("Created new album with Date={},title={}".format(date,title)) albumfile=open(albumpath,'w') albumfile.write(template) albumfile.close()
Chunk referenced in 2.1
Chunk defined in 2.5,2.7

We need to know if the album.xml file has been more recently modified compared to the index.xml file, or indeed, if either or both exist. The file stats are retrieved in order to make this comparision. In the case that the album.xml file does not exist, a template is created for the user to edit, and it is populated with such information as can be automatically determined. In this version, the title and date are extracted from the current path name, where directories are named by the date hierachy: year, month, day. Other conventions are possible, and are open to the user to edit as appropriate. Currently the description is left blank. (A suggested possibility is to extract a description from a higher level directory.)

<album template 2.6> =
<?xml version="1.0" ?> <!DOCTYPE album SYSTEM "/home/ajh/lib/dtd/album.dtd"> <album> <title> {} </title> <thumbnail></thumbnail> <shortdesc> {} </shortdesc> <date> {} </date> <order></order> </album>

(Note that the closing 'd' tag must immediately follow the closing CDATA tag.) This code fragment defines the string used as a template to create a new album when required. It is pre-populated with the title, short description, and date.

<get information from album and index files 2.7> =
# first check if album defines a thumbnail try: albumdom=parse("album.xml") except: print("Cannot parse album.xml in directory %s" % dirname) # something went wrong with the album.xml syntax sys.exit(1) # check the thumbnail thumbs=albumdom.getElementsByTagName('thumbnail') fc=thumbs[0].firstChild if fc: thumb=fc.nodeValue else: print(' '*level,"There is no thumbnail defined in %s/album.xml" % (dirname)) # extract information from current index file, # and populate the indexdict dictionary try: indexf=open("index.xml","r") indexf.close() #print("read index.xml") indexdom=parse("index.xml") #print("parsed index.xml") except: indexin=''; indexdom=None # an index file exists, so read its data if indexdom: #print(indexdom.toprettyxml()) indexdict={} indexarr=indexdom.getElementsByTagName('image') for image in indexarr: #print(image.toprettyxml()) name=image.getAttribute('name') gps=camera='' if image.hasAttribute('gps'): gps='yesIthink' if image.hasAttribute('model'): camera=image.getAttribute('model') indexdict[name]=(gps,camera) if debug: print(indexdict) if amoddatetime>=imoddatetime: <album more recent than index 2.8> else: <index more recent than album 2.9>
Chunk referenced in 2.1
Chunk defined in 2.5,2.7

Continuing the code to extract information from the index and album files, We first chack for a thumbnail, and print a warning message if none exists. Then open the index file (assuming one exists), and extract from it relevant current information, such as GPS and camera information. This avoids having to otherwise scan all image files to extract this information. The chunk ends with the key decision as to which of the two files, album and index, is more recent. In the first case, we must regenerate the index file; in the second case, all that is needed is to rescan the index file.

<album more recent than index 2.8> =
# new album file, process it to get indexfile print("album more recent than index in directory {}".format(dirname)) if albumdom: indexout='' indexout+='<?xml version="1.0" ?>\n' indexout+='<?xml-stylesheet type="text/xsl" ' indexout+='href="%s"?>\n' % ALBUMXSL indexout+='<album>\n' indexout+=' <title>' try: titles=albumdom.getElementsByTagName('title') title=titles[0].firstChild.nodeValue except: title="" indexout+=title indexout+='</title>\n' indexout+=' <prev>%s</prev>\n' % (prev) indexout+=' <next>%s</next>\n' % (next) albumdescr=albumdom.getElementsByTagName('description') if not albumdescr: try: albumdescr=albumdom.getElementsByTagName('shortdesc') except: albumdescr="" try: description=albumdescr[0].firstChild.nodeValue description=description.strip() except: description="" indexout+=' <description>' indexout+=description indexout+='</description>\n' blogdescr=albumdom.getElementsByTagName('blog') #print("Blog Descriptor {}".format(blogdescr)) if blogdescr: blog=blogdescr[0].firstChild.nodeValue blog=blog.strip() print("Found a blog link {}".format(blog)) else: blog="" indexout+=' <blog>{}</blog>\n'.format(blog) commentary=albumdom.getElementsByTagName('commentary') if commentary: commentary=commentary[0] str=commentary.toprettyxml() else: commentary=albumdom.getElementsByTagName('longdesc') str=""; l=commentary.length for i in range(l): n = commentary.item(i) children = n.childNodes for c in children: if debug: print("Looking at child node %s" % c) if c.nodeValue: str+=c.nodeValue if c.nodeName=='nl': # this is a kludge to capture desired nl str+='<nl/>' indexout+=' <commentary>' indexout+=str indexout+='</commentary>\n' #print("4-thumbnail={}".format(thumb)) <extract ordering information 2.11>
Chunk referenced in 2.7

If an album.xml file exists and is more recent than the index file (including the possibility that it has just been created by the previous chunk), it is scanned and a new contents image (indexout) for the index.xml file built. This string is completed and written out later (see <wind up index file 2.19>).

<index more recent than album 2.9> =
#print("index is more recent") # index is definitive, get its information and make new index.xml indexout='' indexout+='<?xml version="1.0" ?>\n' indexout+='<?xml-stylesheet type="text/xsl" ' indexout+='href="%s"?>\n' % ALBUMXSL indexout+='<album>\n' try: indexf=open("index.xml","r") indexf.close() #print("read index.xml") indexdom=parse("index.xml") #print("parsed index.xml") except: print("Could not parse index file in directory {}".format(dirname)) indexin=''; indexdom=None <extract info from current index 2.10>
Chunk referenced in 2.7

Here the index file is more recent, and so the album file is ignored. The index.xml file is read, and a new contents string is started. This string is completed and written out later (see <wind up index file 2.19>).

Note the possibility that no index file exists, or it cannot be read. In this case,

<extract info from current index 2.10> =
# stuff here to extract values: title,prev,next,description,commentary title=prev=next=description=commentary=blog='' titlelist=indexdom.getElementsByTagName('title') try: title=titlelist[0].firstChild.nodeValue except: title='' prevlist=indexdom.getElementsByTagName('prev') #print(prevlist) if prevlist and prevlist[0].firstChild: prev=prevlist[0].firstChild.nodeValue nextlist=indexdom.getElementsByTagName('next') if nextlist and nextlist[0].firstChild: next=nextlist[0].firstChild.nodeValue descriptionlist=indexdom.getElementsByTagName('description') if descriptionlist and descriptionlist[0].firstChild: description=descriptionlist[0].firstChild.nodeValue bloglist=indexdom.getElementsByTagName('blog') if bloglist and bloglist[0].firstChild: blog=bloglist[0].firstChild.nodeValue commentarylist=indexdom.getElementsByTagName('commentary') if commentarylist and commentarylist[0].firstChild: commentary=commentarylist[0].firstChild.nodeValue thumblist=indexdom.getElementsByTagName('thumbnail') if thumblist and thumblist[0].firstChild: thumb=thumblist[0].firstChild.nodeValue summarynode=indexdom.getElementsByTagName('summary') orderval=summarynode[0].getAttribute('order') if orderval: try: indexorder=int(orderval) except: indexorder=0 # stuff here to save values indexout+=" <title>{}</title>\n".format(title) indexout+=" <prev>{}</prev>\n".format(prev) indexout+=" <next>{}</next>\n".format(next) indexout+=" <description>{}</description>\n".format(description) indexout+=" <blog>{}</blog>\n".format(blog) indexout+=" <commentary>{}</commentary>\n".format(commentary) indexout+=" <thumbnail>{}</thumbnail>\n".format(thumb) #print("5-thumbnail={}".format(thumb))
Chunk referenced in 2.9

Now we (hopefully) have an index file, read it and extract the previous information into a new index file template, ready for output along with any further information added subsequently.

2.5 get ordering information from album XML file

<extract ordering information 2.11> =
try: orderNode=albumdom.getElementsByTagName('order') indexorder=orderNode[0].firstChild.nodeValue except: # order field is empty, try date dateNode=albumdom.getElementsByTagName('date') if dateNode and dateNode[0].firstChild: indexorder=dateNode[0].firstChild.nodeValue.strip() else: indexorder='' try: indexorder=int(indexorder) except: indexorder=0
Chunk referenced in 2.8

The ordering of albums within a page is determined by the order element within the album.xml file. This is an alphameric value which is converted to an integer if possible. Hence the sort order is determined by the lexical ordering of strings if both are strings, and numerical ordering if both are integers. (The sort order is undefined if a mixture of alphameric and integer values are used.)

2.6 write tree element to index.xml

<write tree element to index.xml 2.12> =
treepath=os.getcwd() (prev,next)=getPrevNext(treepath) if debug: print("treepath={}, prev={}, next={}".format(treepath,prev,next)) indexout+=' <tree prev="{}" next="{}">'.format(prev,next) indexout+=treepath indexout+='</tree>\n'
Chunk referenced in 2.1

The tree element carries information about the previous and next subdirectories. These are found by looking at the current directory path

<define miscellaneous subroutines 2.13> =
def getPrevNext(treePath): (super,dir)=os.path.split(treePath) dirlist=os.listdir(super) #print(dirlist) if 'Pictures' in dirlist: return ('','') d2=[] for d in dirlist: fulld=super+'/'+d #print(fulld,os.path.isdir(fulld)) if os.path.isdir(fulld): d2.append(d) dirlist=d2 dirlist.sort() i=dirlist.index(dir) maxi=len(dirlist)-1 (higherp,highern)=getPrevNext(super) #print("dir={},i={},maxi={},higherp={},highern={}".format(dir,i,maxi,higherp,highern)) if maxi==0: prev='../'+higherp next='../'+highern elif i==0: prev='../'+higherp next=dirlist[i+1] elif i==maxi: next='../'+highern prev=dirlist[i-1] else: prev=dirlist[i-1] next=dirlist[i+1] return(prev,next)
Chunk referenced in 1.1
Chunk defined in 1.14,1.15,1.20,2.13,9.1

2.7 Visit All Directories

<visit all directories 2.14> =
dircollection=[] for d in dirs: missd=0 thisdir=dirs.index(d) prev=thisdir-1 if prev < 0: prev=0 prev=dirs[prev] next=thisdir+1 if next >= len(dirs): next=len(dirs)-1 next=dirs[next] if recurse: if debug: print("in dir %s, about to visit %s ..." % (dirname,d)) ad=os.path.abspath(d) (dtitle,nimages,ndirs,thumbn,ddescr,missd,order)=visit(level+1,ad,prev,next,lists) else: res=retrieve(d) if res: (dtitle,nimages,ndirs,thumbn,ddescr,missd,order)=res else: continue try: order=int(order) except: pass <update directory parameters 2.15> {Note 2.14.1} if not thumb: thumb=d+"/"+thumbn # must make sure that thumbn is not empty if not thumbn: thumbn='' if thumbn and thumbn[-4:len(thumbn)].lower()=='.jpg': thumbn=thumbn[0:-4] <update index.xml information 2.16> <sort directories into required order 2.17>
Chunk referenced in 2.1
{Note 2.14.1}
if there is no thumbnail name, that implies the relevant index.xml file is not available. In the past, we skipped this entry, but we do need to make some placeholder to show that this directory does exist.

For each subdirectory in the currently visited directory, we have two choices: either to recurse into that subdirectory and gather (possibly new) information about what it contains; or to simply retrieve previously stored information from its index.xml file. In the latter case, any new information will be skipped. This is clearly a faster option.

The subdirectory information is then used to update current directory information, and to update the synopsis index.xml file. If a recursive visit is performed, the subdirectory's album.xml file (if it exists) is used to create a new index.xml file.

<update directory parameters 2.15> =
nphotos+=nimages; nalbums+=ndirs+1 missingdescrs+=missd if missd>maxmissing: maxmissing = missd maxdir = d if missd>0 and missd<minmissing: minmissing = missd mindir = d
Chunk referenced in 2.14

Upon return from visiting a lower level directory, we have a collection of parameters that identify details of that lower level directory: its title, number of images, number of subdirectories, the name of its thumbnail, a description of its contents, the number of missing image descriptions, and the order it should be placed in the listing of subdirectories. These are used to update in turn the corresponding values for the currently visited directory, which is performed in this chunk.

<update index.xml information 2.16> =
thisdirentry='' thisdirentry+=' <directory title="%s" name="%s"\n' % (dtitle,d) thisdirentry+=' albums="%d"\n' % (ndirs) thisdirentry+=' prev="%s"\n' % (prev) thisdirentry+=' next="%s"\n' % (next) thisdirentry+=' images="%d"\n' % (nimages) if missd: thisdirentry+=' missing="%d"\n' % (missd) thisdirentry+=' thumb="%s"\n' % (d+"/"+thumbn) thisdirentry+=' order="%s"\n' % (order) thisdirentry+=' description="%s"/>\n' % (ddescr) dircollection.append((order,thisdirentry))
Chunk referenced in 2.14
<sort directories into required order 2.17> =
# got all directories, flush them to index file in specified order #srtfn=lambda o,d:o dircollection.sort()#key=srtfn) for (o,i) in dircollection: indexout+=i #print("order {}, value {}".format(o,i))
Chunk referenced in 2.14

We use the order element defined in the album.xml document (and transcribed into the corresponding index.xml document) to sort the directories. Since the list of directories is just a set of filenames, we create a tuple containing the order value and the associated index.xml element, aka directory, and sort that with an appropriate comparison function

2.8 scan all images and process them

<scan all images and process them 2.18> =
inum=0 lastimage=len(images)-1 for i in images: iprev=images[0]; inext=images[lastimage] if inum>0: iprev=images[inum-1] if inum<lastimage: inext=images[inum+1] (descr,hasGPS,camera)=makeSubImages(level,lists,i,iprev,inext,mdescr) if descr==-1:{Note 2.18.1} continue if not descr: missingdescrs+=1 addToList(lists,i) indexout+=addToIndex(lists,i,descr,hasGPS,camera) inum+=1 nphotos+=len(images)
Chunk referenced in 2.1
{Note 2.18.1}
dud descriptions file entry, ignore it

2.9 wind up index file

<wind up index file 2.19> =
indexout+='</album>\n' #indexof = open("index-in.xml","w") #indexof.write(indexin) #indexof.close() if indexout.strip() != indexin.strip(): indexof = open("index.xml","w") indexof.write(indexout) indexof.close() print(' '*level,"... index.xml")
Chunk referenced in 2.1

3. Descriptions

This is a revision of the literate structure to bring all descriptions related material together.

<initialization 3.1> =
descrpat = re.compile("([^ *]+?)(\.JPG)?( |\*)(.*?)( # (.*))?$")
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

descrpat is a pattern to match image names. Note that the first pattern must be non-greedy, or the '.JPG' gets swallowed, and the fourth pattern likewise, or the ' # ' gets swallowed.

The third pattern has been added (v1.3.0) to flag protected images that must not be shown publically.

3.1 read descriptions file and create dictionary

<read descriptions file and create dictionary 3.2> =
try: descrf=open("descriptions","r") mdescr=os.stat("descriptions")[stat.ST_MTIME] gotdescrfile=1 for l in descrf.readlines(): res=descrpat.match(l) if res: if not ext: ext=".JPG" protect=('*') if protect: protected[filen]=True if debug: print(filen,ext,comment,soundname) descriptions[filen]=imgtitle if comment: sounds[filen]=soundname print(' '*level,"Got a sound file:%s" % (soundname)) images.append(filen) else: images.append(l.rstrip()) except IOError: print("cannot open descriptions file in directory %s" % dirname) descriptions={} sounds={} protected={}
Chunk referenced in 2.1

Build the python dictionary of descriptions. We attempt to open a file in the current directory called descriptions, and use that to initialize the dictionary. The file has the format of one entry per line for each photo, with the name of the photo at the start of the line (with or without the .JPG extension), followed by at least one blank, followed by the descritive text, terminated by the end of line.

3.2 wind up descriptions file

<wind up descriptions file 3.3> =
if not gotdescrfile: if os.path.exists('descriptions'): print("OOPS!! attempt to overwrite existing descriptions file") else: descr=open('descriptions','w') for i in images: descr.write(trimJPG(i)+' \n') descr.close() elif listmissingimages: print("There are images not mentioned in the descriptions file:") print("appending them to the descriptions file.") descr=open('descriptions','a') for i in listmissingimages: descr.write(trimJPG(i)+' \n') descr.close()
Chunk referenced in 2.1

We finalise the descriptions file. Two scenarios are relevant: either a) the descriptions file does not exist, in which case we build a new one, with just the image names of those images found in the directory scan, or b) the descriptions file exists, but there are images found not mentioned in it. In this latter case, we append image names to the file. No attempt is made to insert them in their 'correct' position.

If there are no changes required, the descriptions file is untouched. The existence of the descriptions file is determined by whether the flag gotdescrfile is set.

4. the addToIndex routine definition

<the addToIndex routine definition 4.1> =
def addToIndex(lists,imagename,descr,hasGPS,model): (r,e)=os.path.splitext(imagename) # this was inserted to try an control the length of captions. It # is dangerous, however, as it can munge the well-formedness of # any contained XML. #if len(descr)>200: # descr=descr[0:200] + ' ...' gps='' #if hasGPS: gps=' gps="yes"' mod='' if model: model=re.sub('Canon ','',model) model=re.sub('PowerShot ','',model) mod=' model="%s"' % (model) return ' <image name="%s"%s%s>%s</image>\n' % (r,mod,gps,descr) # was cgi.escape(descr)
Chunk referenced in 1.1

I've changed my mind a couple of times about escaping the description string. Currently the string is not escaped, meaning that any special XML characters (ampersand, less than, etc.) must be entered in escaped form. But this model does allow URLs and the like to appear in the description captions.

5. the addToList routine definition

<the addToList routine definition 5.1> =
# 'addToList' is called to add an image name to the list of images. # See above for the format of the data structure 'list'. It is # assumed that the image name is a full path from the album root ((one # of) the CLI parameter(s)). The directory part is removed and used # to match against the keys of 'lists'. A new entry is created if # this path has not previously been seen. The image name is then # added to this list, if it is not already there. def addToList(lists,imagefile): (base,tail)=os.path.split(os.path.abspath(imagefile)) (name,ext)=os.path.splitext(tail) if base in lists: a = lists[base] else: a = [] lists[base] = a if not name in a: a.append(name)
Chunk referenced in 1.1

6. the makeImage routine

<the makeImage routine definition 6.1> =
def makeImage(level,orgwd,orght,newwd,newht,newname,orgname,orgmod): # orgwd: original image width # orght: original image height # newwd: new image width # newht: new image height # newname: new image file name # orgname: original image file name # orgmod: original image modification date/time # if debug: print("makeImage(%d,%d,%d,%d,%s,%s,%s)" % \ (orgwd,orght,newwd,newht,newname,orgname,orgmod)) if orght>orgwd: # portrait mode, swap height and width t=newht newht=newwd newwd=t if orght>newht: # check that resolution is large enough newmod=0 if os.path.isfile(newname): newmod=os.stat(newname)[stat.ST_MTIME] if orgmod>newmod: print(' '*level,"Writing %s at %dx%d ..." % (newname,newwd,newht)) size=" -size %dx%d " % (orgwd,orght) resize=" -resize %dx%d " % (newwd,newht) if debug: print(size+orgname+resize+newname) os.system(CONVERT+size+orgname+resize+newname) pass return
Chunk referenced in 1.1

7. the makeSubImages routine

This routine is responsible for building all the smaller images from the master image.

7.1 New SubImage generalized size definitions

We define here data structures to handle generalized sub-image size definitions. Not yet active.

<initialization 7.1> =
imageSizeDefs=[(128,128,"thumb"),(384,288,"small"),\ (640,480,"medium"),(1200,900,"large"),(0,0,"original")] imageSizes=[]; derivedImageMatch=[] for (wd,ht,sizeName) in imageSizeDefs: imageSizes.append("%dx%d" % (wd,ht)) derivedImageMatch.append(re.compile(".*_%dx%d.JPG" % (wd,ht)))
Chunk referenced in 1.1
Chunk defined in 1.7,1.8,1.9,1.10,1.11,1.12,1.13,3.1,7.1

Initialize the image size parameters. imageSizeDefs is normative, and defines all required image sizes. The first entry is deemed to be the thumbnail size, and an entry of (0,0) refers to the original size. imageSizes is an equivalent list of strings in the form %dx%d, and derivedImageMatch is an equivalent list of patterns to match image names of those sizes. (Except the last, which fix!)

<the makeSubImages routine definition 7.2> =
def makeSubImages(level,lists,imagename,iprev,inext,mdescr): global descriptions,forceXmls,debug def getRational(exifField,rational=False): if exifField in exif_data: datum=exif_data[exifField] if type(datum) is PIL.TiffImagePlugin.IFDRational: n=datum.numerator d=datum.denominator else: (n,d)=datum if rational: if d==1: val=n else: val='{}/{}'.format(n,d) else: val=n/d #print("{} is {}".format(exifField,val)) return val else: return 0.0 def getRat(tuple): (n,d)=tuple r=n/d return r (r,e)=os.path.splitext(imagename) if not e: e='.JPG' imagename="%s%s" % (r,e) if not os.path.isfile(imagename): e='.jpg' imagename="%s%s" % (r,e) if debug: print("making sub images for %s, e=%s, forceXmls=%s" % (imagename,e,forceXmls)) thumbn=r+"_128x128.JPG" smalln=r+"_384x288.JPG" medn=r+"_640x480.JPG" bign=r+"_1200x900.JPG" viewn=r+".xml" <return on missing image file 7.3> <get all modification times 7.4> <makeSubImages: make any sub images required 7.5> # check if there is a new description olddescr=descr="" try: oldxmldom=parse(viewn) olddescr=oldxmldom.getElementsByTagName('description') olddescr=flatStringNodes(olddescr[0].childNodes) olddescr=olddescr.strip() except: print("Could not get old description for %s" % viewn) pass if r in descriptions: descr=descriptions[r].strip() # create the new IMAGE.xml if the descriptions file exists # (mdescr>0) and is more recent than IMAGE.xml, or the IMAGE.JPG # is more recent than IMAGE.xml, and there is a new description if debug: print("mdescr=%d, mview=%d, mfile=%d" % (mdescr,mview,mfile)) if r in indexdict: (hasGPS,camera)=indexdict[r] else: hasGPS=camera='' #print(indexdict) #print("checking %s, it has %s gps" % (r,hasGPS)) if forceXmls or \ (mview==0) or \ ( (olddescr!=descr) and \ ( (mdescr>0 and mdescr>mview) or \ (mfile>mview)\ )\ ): <makeSubImages: write image xml file 7.6> return (descr,hasGPS,camera)
Chunk referenced in 1.1
<return on missing image file 7.3> =
if not os.path.isfile(imagename): if r in descriptions: descr=descriptions[r].strip() return (descr,False,None) else: print("descriptions entry '%s' : image not found" % (imagename)) return (None,False,None)
Chunk referenced in 7.2

Check that the image file exists. If it doesn't, then there are two possible reasons. Firstly, the image file has been removed for space reasons, in which case there will be an entry in the descriptions database, and we return that without attempting to make any sub images.

Alternatively, no such descriptions entry means the image is genuinely missing, and therefore we should print a warning message, and return a missing image flag.

<get all modification times 7.4> =
mfile=os.stat(imagename)[stat.ST_MTIME] mthumb=msmall=mmed=mbig=mfile-1 mview=0 if os.path.isfile(thumbn): mthumb=os.stat(thumbn)[stat.ST_MTIME] if os.path.isfile(smalln): msmall=os.stat(smalln)[stat.ST_MTIME] if os.path.isfile(medn): mmed=os.stat(medn)[stat.ST_MTIME] if os.path.isfile(bign): mbig=os.stat(bign)[stat.ST_MTIME] if os.path.isfile(viewn): mview=os.stat(viewn)[stat.ST_MTIME] if debug: print("%d > (%d,%d,%d)" % (mfile,mthumb,mmed,mbig))
Chunk referenced in 7.2

Get all the modification times. The default modification times are that mview is the oldest, mfile is the youngest, and all the others are in between.

<makeSubImages: make any sub images required 7.5> =
if mfile>min(mthumb,msmall,mmed,mbig): p=subprocess.Popen([IDENTIFY,imagename],stdout=subprocess.PIPE,stderr=subprocess.PIPE) (cmd_out,cmd_stderr)=p.communicate(None) if cmd_stderr and "Invalid SOS parameters" not in str(cmd_stderr): print("error in identify for %s ... (%s)" % (imagename,cmd_stderr)) #return (None,False) names=[thumbn,medn,bign,imagename] (b,t)=os.path.split(imagename) if debug: print("base=%s, tail=%s" % (b,t)) if debug: print("Making images for %s" % (imagename)) cmd_out=cmd_out.decode(encoding='UTF-8') res=identifyPat.match(cmd_out) if res: wd=int( ht=int( smallwd=384.0; medwd=640.0; largewd=1200.0 smallht=288.0; medht=480.0; largeht=900.0 if wd>ht: smallwd=int(smallwd*(float(wd)/float(ht))) medwd=int(medwd*(float(wd)/float(ht))) largewd=int(largewd*(float(wd)/float(ht))) else: smallht=int(smallwd*(float(ht)/float(wd))) medht=int(medwd*(float(ht)/float(wd))) largeht=int(largewd*(float(ht)/float(wd))) makeImage(level,wd,ht,128,128,thumbn,imagename,mfile) if False: # [was] not thumbsOnly: makeImage(level,wd,ht,smallwd,smallht,smalln,imagename,mfile) makeImage(level,wd,ht,medwd,medht,medn,imagename,mfile) if large: makeImage(level,wd,ht,largewd,largeht,bign,imagename,mfile) count=1
Chunk referenced in 7.2

(20110717:172812) The variables smallwd, medwd, largeht are introduced to ensure that the resulting images all have a consistent height, if not width. This was introduced because panoramas came out with the same width as convention 4x3s, making the height impossible small.

<makeSubImages: write image xml file 7.6> =
print(' '*level,"Writing %s ..." % (viewn),end='') imagef=open(imagename,'rb') exiftags={} try: meta_data=ImageMetaData(imagename) #latlng =meta_data.get_lat_lng() exif_data = meta_data.get_exif_data() except: print("Some error in extracting image metadata for {}".\ format(imagename)) exif_data=None #try: # exiftags=EXIF.process_file(imagef) #except (ValueError,TypeError): # print("cannot extract exif data from %s" % imagename) if exif_data: for tag in exif_data.keys(): if debug: print("%s = %s" % (tag,exif_data[tag])) imagexmlf = open(viewn,"w") imagexmlf.write('<?xml version="1.0" ?>\n') imagexmlf.write('<?xml-stylesheet type="text/xsl" ') imagexmlf.write('href="%s"?>\n' % ALBUMXSL) imagexmlf.write('<album>\n') (rp,ep)=os.path.splitext(iprev) (rn,en)=os.path.splitext(inext) treepath=os.getcwd()+'/' res=treepat.match(treepath) if res: imagexmlf.write(' <tree>%s</tree>\n' % ( imagexmlf.write(' <view name="%s" \n' % (r)) imagexmlf.write(' prev="%s" next="%s"\n' % (rp,rn)) if r in protected: print(" protecting image {}".format(r)) imagexmlf.write(' access="localhost"\n') if r in sounds: snd=sounds[r] print(" Adding sound: %s" % (snd)) imagexmlf.write(' audio="sounds/SND_%s.WAV"\n' % (snd)) imagexmlf.write(' >\n') imagexmlf.write(' <description>\n') imagexmlf.write(' %s\n' % descr) # was cgi.escape(descr) imagexmlf.write(' </description>\n') if 'Model' in exif_data: camera=exif_data['Model'].__str__().strip() imagexmlf.write(' <camera>%s</camera>\n' % camera) hasGPS=False if 'DateTimeOriginal' in exif_data: datetime=exif_data['DateTimeOriginal'] imagexmlf.write(' <datetime>%s</datetime>\n' % datetime) #print(exif_data) shutter=getRational('ExposureTime',rational=True) imagexmlf.write(' <shutter>%s</shutter>\n' % shutter) aperture=getRational('FNumber') imagexmlf.write(' <aperture>%s</aperture>\n' % aperture) focal=getRational('FocalLength') imagexmlf.write(' <focallength>%s</focallength>\n' % focal) if 'ISOSpeedRatings' in exif_data: speed=exif_data['ISOSpeedRatings'] imagexmlf.write(' <ISOspeed>%s</ISOspeed>\n' % speed) if 'GPSInfo' in exif_data: gpsdata=exif_data['GPSInfo'] if 'GPSLatitude' not in gpsdata: hasGPS=False else: hasGPS=True #print(gpsdata) lat=gpsdata['GPSLatitude'] lng=gpsdata['GPSLongitude'] if lat: (latdegrees,latminutes,latseconds)=lat latdegrees=getRat(latdegrees) latminutes=getRat(latminutes) latseconds=getRat(latseconds) latRef=gpsdata['GPSLatitudeRef'] imagexmlf.write(' <latitude hemi="%s" degrees="%d" minutes="%d" seconds="%f"/>\n' % \ (latRef,latdegrees,latminutes,latseconds)) print('Lat:%d %d %7.4f %s ' % (latdegrees,latminutes,latseconds,latRef),end='') if lng: (longdegrees,longminutes,longseconds)=lng longdegrees=getRat(longdegrees) longminutes=getRat(longminutes) longseconds=getRat(longseconds) longRef=gpsdata['GPSLongitudeRef'] imagexmlf.write(' <longitude hemi="%s" degrees="%d" minutes="%d" seconds="%f"/>\n' % \ (longRef,longdegrees,longminutes,longseconds)) print('Long:%d %d %7.4f %s' % (longdegrees,longminutes,longseconds,longRef),end='') imagexmlf.write(' </view>\n') imagexmlf.write('</album>\n') imagexmlf.close() print('')
Chunk referenced in 7.2

See comment under <the addToIndex routine definition 4.1> regarding escaping the description string.

8. the retrieve support routines definition

<the retrieve support routines definition 8.1> =
def getNodeValue(dom,field,missing): try: val = dom.getElementsByTagName(field).item(0).firstChild.nodeValue except: val=missing return val def retrieveField(field): try: val = index.getElementsByTagName(field).item(0).firstChild.nodeValue except: val="[could not retrieve %s]" % (field) return val
Chunk referenced in 8.2

These two support routines for retrieve encapsulate data extraction from the DOM model. They perform the same basic operation, differeing only in the default parameters required.

<the retrieve routine definition 8.2> =
def retrieve(d): <the retrieve support routines definition 8.1> missing=0 try: index = parse(d+'/index.xml') except: (etype,eval,etrace)=sys.exc_info() print("Did not find index.xml in directory %s" % (d)) #print(" (error type %s)" % (etype)) return ('',0,0,0,'',0,0) if debug: print("Successfully parsed index.xml in directory %s" % (d)) try: album = parse(d+'/album.xml') except: (etype,eval,etrace)=sys.exc_info() print("Cannot open album.xml in directory %s because %s" % (d,etype)) album=None title = retrieveField('title') description = retrieveField('description') imageElems=index.getElementsByTagName('image') summary=index.getElementsByTagName('summary').item(0) name=summary.getAttribute('name') photos=int(summary.getAttribute('photos')) albums=int(summary.getAttribute('albums')) missing=int(summary.getAttribute('missing')) order=summary.getAttribute('order') if not order: order=name #print("Order for directory {} is >{}<".format(d,order,name)) # sort out the vexed problem of the thumbnail if album: albumThumbs=album.getElementsByTagName('thumbnail') else: albumThumbs='' thumbDefault=getNodeValue(index,'thumbnail','') thumb=thumbDefault if not thumbDefault: thumbDefault=getNodeValue(album,'thumbnail','') if debug: print(" albumThumb: %s" % (thumbDefault)) if not thumbDefault: thumb=thumbDefault miss=index.getElementsByTagName('missingdescriptions') if miss: missing=int(miss.item(0).firstChild.nodeValue) if 0: print("retrieved:") print(" title: %s" % (title)) print(" photos: %d" % (photos)) print(" albums: %d" % (albums)) print(" thumb: %s" % (thumb)) print(" description: %s" % (description)) return (title,photos,albums,thumb,description,missing,order)
Chunk referenced in 1.1

retrieve is called when we do not wish to recursively visit a subdirectory. Instead, relevant details from the subdirectory are extracted ('retrieved') from an index.xml file, which has been constructed when the relevant subdirectory was visited.

There is a slight problem with identifying a thumbnail for this album, since the definite source is that in the album.xml file. However, this may be blank, in which has we need to promote the first thumbnail within the directory (or subdirectories where the directory has no images of its own).

9. The getAlbumParameters routine

<define miscellaneous subroutines 9.1> =
def attrStr(dom,tag): tags=dom.getElementsByTagName(tag) if not tags: return None firsttag=tags[0] try: str=firsttag.firstChild.nodeValue return str except: return None def getAlbumParameters(fname): dom=parse(fname) if dom: # title description commentary tree directory* thumbnail image* summary title=attrStr(dom,'title') description=attrStr(dom,'description') commentary=attrStr(dom,'commentary') tree=attrStr(dom,'tree') directory=attrStr(dom,'directory') thumbnail=attrStr(dom,'thumbnail') image=attrStr(dom,'image') summary=attrStr(dom,'summary') #albums=attrStr(dom,'albums') #images=attrStr(dom,'images') #missing=attrStr(dom,'missing') #maxmissing=attrStr(dom,'maxmissing') #photos=attrStr(dom,'photos') return (title,description,commentary,tree,directory,thumbnail,image,summary)
Chunk referenced in 1.1
Chunk defined in 1.14,1.15,1.20,2.13,9.1

10. The index.xml file format

The following description defines the collection of XML elements used to build an index.xml file.

Note that lowercase names are the names of the element tags; capitalized names are element content; names starting with @ are attributes (with string values: attributes that must be digit strings are so identified).

indexfile   = title description commentary tree 
              directory* thumbnail image*
title       = TextNode .
description = TextNode .
commentary  = TextNode .
tree        = TextNode .
directory   = @title @name @albums @images @missing @thumb @description Empty .
thumbnail   = TextNode .
image       = @name @model TextNode .
summary     = @albums @maxdir @maxmissing @missing @photos .
@albums     = Digits .
@images     = Digits .
@missing    = Digits .
@maxmissing = Digits .
@photos     = Digits .

Explanation of attributes:

The title of the directory or album (for the human reader). This attribute is derived from the title element of the specified (sub-)directory
The name of the directory or image (the Unix name). In the case of an image, only the basename is given.
digit string defining the number of sub-directories
digit string defining the number of photos/images in this directory and any sub-directories (albums and sub-albums)
digit string defining the number of missing descriptions for all enclosed images
for an image, the camera model on which the image was taken

11. Makefile

This Makefile integrates with the other photo tool Makefiles

"MakefilePhoto3" 11.1 =
install-photo3: chmod 755 cp -p $(HOME)/bin/ photo3.tangle

12. Appendices

12.1 Chunk Indices

File Name Defined in
MakefilePhoto3 11.1 1.1

12.2 Macro Indices

Chunk Name Defined in Used in
album more recent than index 2.8 2.7
album template 2.6
banner 1.3 1.1
basic usage information 1.4 1.1
collect the command line options 1.16 1.1
current date 12.2 1.4
current version 12.1 1.4, 1.16
define miscellaneous subroutines 1.14, 1.15, 1.20, 2.13, 9.1 1.1
define miscellaneous subroutines 1.14, 1.15, 1.20, 2.13, 9.1 1.1
define miscellaneous subroutines 1.14, 1.15, 1.20, 2.13, 9.1 1.1
define miscellaneous subroutines 1.14, 1.15, 1.20, 2.13, 9.1 1.1
explore next higher level to update counts 1.18 1.17
extract date field as default for ordering 2.3 2.1
extract info from current index 2.10 2.9
extract ordering information 2.11 2.8
get all modification times 7.4 7.2
get information from album and index files 2.5, 2.7 2.1
get information from album and index files 2.5, 2.7 2.1
imports 1.6 1.1
index more recent than album 2.9 2.7
initialization 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 1.13, 3.1, 7.1 1.1
initialization 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 1.13, 3.1, 7.1 1.1
initialization 1.7, 1.8, 1.9, 1.10, 1.11, 1.12, 1.13, 3.1, 7.1 1.1
initialize variables for visit routine 2.2 2.1
interpreter definition 1.2 1.1
makeSubImages: make any sub images required 7.5 7.2
makeSubImages: write image xml file 7.6 7.2
perform the top level visit 1.17 1.1
read descriptions file and create dictionary 3.2 2.1
return on missing image file 7.3 7.2
scan all directory elements, updating counts 1.19 1.18
scan all images and process them 2.18 2.1
sort directories into required order 2.17 2.14
sort fnames into directories and images 2.4 2.1
the addToIndex routine definition 4.1 1.1
the addToList routine definition 5.1 1.1
the makeImage routine definition 6.1 1.1
the makeSubImages routine definition 7.2 1.1
the retrieve routine definition 8.2 1.1
the retrieve support routines definition 8.1 8.2
the visit routine definition 2.1 1.1
todos 1.5 1.1
update directory parameters 2.15 2.14
update index.xml information 2.16 2.14
visit all directories 2.14 2.1
wind up descriptions file 3.3 2.1
wind up index file 2.19 2.1
write list file 1.21 1.1
write tree element to index.xml 2.12 2.1

12.3 Indentifier Indices

Identifier Defined in Used in
addToIndex 4.1 2.18
addToList 5.1
debug 1.7
descriptions 1.9 2.1, 3.2, 3.2, 7.2, 7.2, 7.2
descrpat 3.1 3.2
dirname 2.1
getNodeValue 8.1
ignorePat 1.11 2.4
indexin 2.2
indexorder 2.2
indexout 2.2
listmissingimages 2.2 2.4, 3.3, 3.3
lists 1.8
makeImage 6.1 7.5, 7.5, 7.5, 7.5
makeSubImages 7.2 2.18
retrieve 8.2 2.14
retrieveField 8.1
visit 2.1 1.17, 2.14

12.4 Document History

20060314:183617 ajh 1.0.0 add single directory pass as default, -r to recursively scan subdirectories (major revision)
20060418:170216 ajh 1.0.1 add exception handling to a number of XML accesses
20060502:161718 ajh 1.1.0 add various options to omit processing all sizes
20060512:174933 ajh 1.1.1 add image names to descriptions file if not already there
20060515:064337 ajh 1.1.2 fixed bug with entry in decriptions not having an image file
20060601:165937 ajh 1.1.3 add tree field, and -f option to force generation of .xml files
20060603:231034 ajh 1.2.0 remove double previous/next, since now done dynamically in XSLT sheet.
20060608:175540 ajh 1.2.1 fix bug in descriptions with http escape chars
20060611:184930 ajh 1.2.2 fix bug when missing images
20060616:160512 ajh 1.2.2 clean up some description handling
20060628:114452 ajh 1.2.3 start non-destructive update of index files
20060630:065926 ajh 1.2.4 fix bug in scoping of retrieve support routines
20061022:102528 ajh 1.2.5 return non-escaping of description strings
20070419:153749 ajh 1.2.6 ignore movies in (recursive) scanning for directories
20070812:092952 ajh 1.2.7 missing files are not omitted from album if a description exists
20080210:224120 ajh 1.2.8 minor bug in generating sound links: restructured descriptions section
20090914:164528 ajh 1.3.0 add protected flag to descriptions file, generates access attribute in xml file
20090923:152614 ajh 1.4.0 traverse all directories back to root, filling in updated values
20090924:134427 ajh 1.4.1 fixed bug in new code
20110602:180115 ajh 1.4.2 upgrade popen to use subprocess module
20110714:094436 ajh 1.4.3 Improve commentary
20110714:191909 ajh 1.5.0 add GPS data
20110716:192730 ajh 1.5.1 fix whole numbered seconds for latitude
20110717:163005 ajh 1.5.2 fix whole numbered seconds for longitude
20110721:174705 ajh 1.5.3 add camera model
20110806:163853 ajh 1.5.4 tidy GPS reporting
20110819:115130 ajh 1.5.5 add handling of iPhone GPS coords
20180320:142934 ajh 1.6.0 revision of apparently (?) erroneous handling of index.xml
20180322:140330 ajh 1.6.1 reinstate ordering of albums
20180518:105356 ajh 1.6.2 add EOS REBEL T3i to models
20180903:150122 ajh 1.7.0 (undocumented changes)
20190228:131738 ajh 1.7.1 Clean up processing of album and index files, and restructure literate program.
20190302:171252 ajh 1.8.0 Some rather drastic restructuring of program logic in a somewhat unsuccessful attempt to further refine the literate program. It is working, but only just.
20200325:172816 ajh 1.8.1 some bug fixes
20200402:132355 ajh 1.8.2 fig bug with index more recent than album
20200512:101820 ajh 1.8.3 Renamed to photo2 as the final python2.7 version. Subsequent versions are modified to python3.6ff.
20200512:102547 ajh 1.9.0 Converted to run under python3
20200702:113036 ajh 1.9.1 fixed bug where new directories were being ignored because of lack of thumbnails
20200704:114858 ajh 1.9.2 fixed bug whereby prev and next not calculated correctly if directory is a single sub-directory of next higher directory.
20201114:143238 ajh 1.10.0 Fixed EXIF data
20201114:183703 ajh 1.10.1 fixed bug whereby redoing photo changes what should be an unchanged index.xml
20210216:174628 ajh 1.11.0 add blog element to album processing
20210907:153035 ajh 1.11.1 Fixed extraction of rational exif data, and solved indenting album template
20210909:164858 ajh 1.11.2 More rational data fixes, caused by change in the EXIF3 protocols
20220515:101533 ajh 1.11.3 ditto, GPS data
20220923:171754 ajh 1.11.4 allow for double dates when crossing dateline
<current version 12.1> = 1.11.4
Chunk referenced in 1.4 1.16
<current date 12.2> = 20220923:171754
Chunk referenced in 1.4