XLP Literate Program for all cron jobs
A.J.Hurst
Version 0.6.6
Table of Contents
1. Introduction
This program generates, installs and initiates all cron jobs
associated with Hurst computers.
A number of systems have been decommissioned. Legacy software
relating to those decommissioned systems has been moved to the
subdirectory
$HOME/Computers.technical/cron/decommissioned.
<cron header 1.1> =##
## **********************************************************
## * DO NOT EDIT THIS FILE! *
## * Use $HOME/Computers/technical/cron.xlp instead *
## **********************************************************
##
<cron title 1.2> =##################################
## CRON FILE for machine ##
##################################
<cron trailer 1.3> =##
## The End
##
2. The Newport Cron
The newport cron is used to ensure that my current workspace is
backed up each night.
"cron.newport" 2.1 =
<cron title 1.2>(machine='NEWPORT ')
<cron header 1.1>
HOME=/home/ajh
LOGS=/home/ajh/logs/newport
DATE=`/home/ajh/bin/date`
INCRLOG=/home/ajh/logs/newport/backup/$DATE
PYTHONLIB=/home/ajh/lib/python
PYTHONPATH=/home/ajh/lib/python:/home/ajh/Computers/python:/usr/local/lib/python
PYTHON=/home/ajh/Computers/python
SHELL=/bin/bash
PATH=.:/home/ajh/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
XML_CATALOG_FILES=/home/ajh/etc/xml/catalog
#Mins Hour Day Month Weekday
# backup to backup disk
30 0 * * * $HOME/bin/backup >>$HOME/logs/backup.log 2>&1
<cron trailer 1.3>
3. The Burnley Cron
The burnley cron is used to ensure that my current workspace is
backed up to spencer each night.
"cron.burnley" 3.1 =
<cron title 1.2>(machine='BURNLEY ')
<cron header 1.1>
HOME=/home/ajh
LOGS=/home/ajh/logs/burnley
DATE=`/home/ajh/bin/date`
INCRLOG=/home/ajh/logs/burnley/backup/$DATE
PYTHONLIB=/home/ajh/lib/python
PYTHONPATH=/home/ajh/lib/python:/home/ajh/Computers/python:/usr/local/lib/python
PYTHON=/home/ajh/Computers/python
SHELL=/bin/bash
PATH=.:/home/ajh/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
XML_CATALOG_FILES=/home/ajh/etc/xml/catalog
#Mins Hour Day Month Weekday
# backup to spencer
30 0 * * * $HOME/bin/sync to spencer >>$HOME/logs/sync.log
<cron trailer 1.3>
4. The Spencer Cron
spencer is the disk farm, and these few cron jobs are related to
performing backups.
"cron.spencer" 4.1 =
<cron title 1.2>(machine='SPENCER ')
<cron header 1.1>
HOME=/home/ajh
LOGS=$HOME/logs
DATE=`$HOME/bin/date`
INCRLOG=$HOME/logs/spencer/backup/$DATE
PYTHONPATH=./$HOME/lib/python:/home/ajh/Computers/python:/usr/local/lib/python
SHELL=/bin/bash
PATH=.:/sw/sbin:/sw/bin:$HOME/bin/macosx:$HOME/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin
XML_CATALOG_FILES=$HOME/etc/xml/catalog
MAILTO=ajh@ajhurst.org
#Mins Hour Day Month Weekday
20 1 * * * rsync -auv $HOME/* /lizard/ajh/
0 0 * * * $HOME/bin/backup >>$HOME/logs/backup.log
0 * * * * sleep 17 ; wget -O - http://freedns.afraid.org/dynamic/update.php?ZkhMd3JlaWZhS2FCSldOVmxReXQ6MTcyMzkwNzI= >>/tmp/freedns_@_ajh_id_au.log 2>&1 &
# wiki backup: start this a couple of minutes after the cleanWiki script should have run
31 3 * * * /usr/bin/rsync -auv --exclude=surgeprotect --exclude=__session__ gwuc-wiki.gwuc.org.au:/var/wiki/share/moin/server/data/* /lizard/backup/wiki/
0 7 * 1 * mv /lizard/backup/wiki /lizard/backup/wiki-`/bin/date "+%Y%m%d"` ; mkdir /lizard/backup/wiki
#0 1 * * * /usr/bin/rsync -auv root@gorg.njhurst.com:/lizard/njh/* /lizard/njh/ >>$LOGS/njhXfer.log
#0 1 * * * /usr/bin/rsync -auv gorg.njhurst.com:/lizard/bunny/* /lizard/bunny/ >>$LOGS/bunnyXfer.log
#0 2 * * * /usr/bin/rsync -auv /home/ajh/Pictures/Albums/* gorg.njhurst.com:/lizard/ajh/Pictures/Albums/ >>$LOGS/picsXfer.log
<cron trailer 1.3>
The backup cron job is temporarily commented out as it
appears to be failing regularly. The reasons for this are not
clear, and need investigating.
20191125:142117 Re-inserted: the backup script has been revised
and tested, so we shall see how well it works now.
5. The Albens Cron
Under development
"cron.albens" 5.1 =
<cron title 1.2>(machine='ALBENS')
<cron header 1.1>
HOME=/home/ajh
LOCAL=/home/ajh/local/albens
DATE=`/home/ajh/bin/date`
SHELL=/bin/bash
PATH=.:/sw/sbin:/sw/bin:/home/ajh/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin
#Mins Hour Day Month Weekday
0 1 * * * find /home/ajh/local/albens/htmls -mtime +14 -type f -exec rm {} \;
0 2 * * * find /home/ajh/local/albens/counters -mtime +14 -name personal-albums-\* -exec rm {} \;
<cron trailer 1.3>
The two cron entries do clean-up on the web page logging. The
first is to remove archived html files that have been cached
from the corresponding xml file rendering, and are older than 14
days.
The second entry removes counters from the photo albums. These
tend to be generated by web crawlers, and are of no intrinsic
interest. As they tend to clog up the counter directory, they
are removed, also after 14 days.
6. The Dimboola Cron
Not currently in use.
"cron.dimboola" 6.1 =
<cron title 1.2>(machine='DIMBOOLA')
<cron header 1.1>
HOME=/Users/ajh
LOGS=/Users/ajh/logs/dimboola
DATE=`/Users/ajh/bin/date`
INCRLOG=/Users/ajh/logs/dimboola/backup/$DATE
PYTHONLIB=/Users/ajh/lib/python:/Users/ajh/Church/Computers/rosters
PYTHONPATH=/Users/ajh/lib/python:/home/ajh/Computers/python:/usr/local/lib/python
PYTHON=/Users/ajh/Computers/python
ROSTER=/Users/ajh/Church/Computers/rosters
HOUSE=/Users/ajh/Computers/House
SHELL=/bin/bash
PATH=.:/sw/sbin:/sw/bin:/Users/ajh/bin/macosx:/Users/ajh/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/bin
XML_CATALOG_FILES=/Users/ajh/etc/xml/catalog
PYTHONPATH=/home/ajh/lib/python:/usr/local/lib/python
#Mins Hour Day Month Weekday
# Pictures/albums/2016/07 synching
#0 3 * * 1 rsync -auv albens:/home/ajh/Pictures/Albums/2016/07/* $HOME/Pictures/Albums/2016/07/
# backup to spencer
30 0 * * * $HOME/bin/sync to spencer
<cron trailer 1.3>
7. The Makefile
"Makefile" 7.1 =SOURCEDIR=$(HOME)/Computers/technical/cron
default=program
GenFiles = .program install-program
include ${HOME}/etc/MakeXLP
install: install-echuca install-spencer install-wolseley
start: start-echuca start-spencer start-wolseley
all: install start
clean: litclean
-rm $(GenFiles)
start-%: install-%
if [ "$(HOST)" = "$*" ] ; then \
crontab $(SOURCEDIR)/cron.$* ;\
else \
ssh $* crontab $(SOURCEDIR)/cron.$* ;\
fi
touch start-$*
install-%: make-%
if [ "$(HOST)" != "$*" ] ; then \
rsync -auv cron.$* $*:$(SOURCEDIR)/ ;\
fi
touch install-$*
make-%: cron.tangle
touch make-$*
#cron.tangle: cron.xlp # from MakeXLP
8. Migration
A lot of the cron jobs defined above use legacy or out-of-date
software. In order to manage the migration to new systems, the
following dependencies are identified (by cron reference):
| Cron |
Main |
Location |
Dependencies |
| Albens |
allRosterMail |
/home/ajh/Church/Computers/rosters |
copy, getopt, sys, subprocess (all standard libraries) |
| ERegnans |
genRoster |
/home/ajh/Church/Computers/rosters |
calendarNames (lib/python), SaltDuties (.), rosterAccess3 (.),
datetime, getopt, math, re, sys (all standard libraries) |
| ERegnans |
rosterAccess3 |
/home/ajh/Church/Computers/rosters |
calendarNames (lib/python), wikiAccess (.), editmoin (lib/python),
datetime, math, os, os.path, re, string, sys, urllib2 (all standard libraries) |
9. Document History
| 20130820:111537 |
ajh |
0.0 |
first draft, built from individual cron files |
| 20131125:222602 |
ajh |
0.1 |
omit central crons |
| 20131227:165448 |
ajh |
0.2 |
added garedelyon cron table, and migration section |
| 20140127:141504 |
ajh |
0.3 |
replace central with lilydale |
| 20140425:114511 |
ajh |
0.4 |
replace lilydale with ringwood |
| 20150420:174011 |
ajh |
0.5 |
add orsay, start to remove obsolete code (ringwood) |
| 20150606:120513 |
ajh |
0.5.1 |
add bastille, take over from orsay |
| 20150717:094715 |
ajh |
0.5.2 |
add lilydale, take over from bastille |
| 20170223:153456 |
ajh |
0.6.0 |
General clean, remove a lot of obsolete entries
|
| 20170510:120728 |
ajh |
0.6.1 |
further tidy ups (remove Lilydale), and revisions of content
|
| 20170906:131409 |
ajh |
0.6.2 |
revised spencer cron, and fix HOME address for spencer
|
| 20171022:124649 |
ajh |
0.6.3 |
change spencer backup time |
| 20180706:182105 |
ajh |
0.6.4 |
add burnley, remove dimboola |
| 20181114:171620 |
ajh |
0.6.5 |
add newport |
| 20181115:104546 |
ajh |
0.6.6 |
remove echuca and wolseley crons |
10. Indices
10.1 Files
| File Name |
Defined in |
| Makefile |
7.1 |
| cron.albens |
5.1 |
| cron.burnley |
3.1 |
| cron.dimboola |
6.1 |
| cron.newport |
2.1 |
| cron.spencer |
4.1 |
10.2 Chunks
10.3 Identifiers
| Identifier |
Defined in |
Used in |