HouseMade - The Hurst HouseHold Heater Helpmate


Version 4.3.2


Table of Contents

1 Introduction
1.1 Overview
1.2 TODOs
1.3 History
1.4 Progress since 2015
1.5 Philosophies
2 Key Data Structures
2.1 Edit Warning
2.2 House Definitions Module
2.2.1 HouseDefinitions: server connections and interfaces
2.2.2 HouseDefinitions: general routines
2.2.3 the isDay function definition
3 The Hardware System
3.1 I/O Allocation for the Raspberry Pi
3.2 Inverse GPIO Pin Allocation
3.3 The Device Tree
3.4 Restarting the Hardware
3.4.1 Set the Pin Definitions
3.4.2 Restart the Hardware
3.5 the program
3.6 The program
3.7 The program
4 The Relay Driver
4.1 RelayDriver: init method
4.2 RelayDriver: write method
4.3 RelayDriver: read method
4.4 Relay Pinouts
5 The Relay Server
5.1 Relay Server Code
5.1.1 RelayServer: connect to the HardwareServer
5.1.2 Define the convert State to String function
5.1.3 RelayServer: define the RPC-Server interface
5.1.4 relayserver: getState
5.1.5 relayserver: quiescent
5.1.6 relayserver: readDoor
5.1.7 relayserver: setState
5.1.8 relayserver: setBit
5.1.9 relayserver: setBitOn
5.1.10 relayserver: setBitOff
5.1.11 HouseData define getTank
5.1.12 relayserver: getTimer
5.1.13 relayserver: resetTimer
5.1.14 relayserver: getSolar
5.1.15 relayserver: start
5.1.16 relayserver: countDown
5.2 Starting the Relay Server
5.3 The Relay Controller Code
6 The Events Module
6.1 The Event Class
6.2 The EventList class
6.3 Events: main routine for testing code
7 The Event Server
7.1 Event Server calling points
7.2 Event Server: serverprocess routine
7.3 Event Server: main routine
8 The Event Scheduler
8.1 EventScheduler: collect ChookDoor times
8.2 EventScheduler: collect GardenSteps times
8.3 EventScheduler: main loop
9 The Event Editor
9.1 The EventEditor Instructions
9.2 EventEditor: get current events routine
9.3 EventEditor: Make Home Page
9.4 EventEditor: Make Edit Page
10 The Chook Door
10.1 ChookDoor: misc routines
10.2 ChookDoor: class ChookDoor
10.2.1 class ChookDoor: init
10.2.2 class ChookDoor: load
10.2.3 class ChookDoor: compute
10.2.4 class ChookDoor: save
10.2.5 class ChookDoor: Open Chook Door
10.2.6 class ChookDoor: Close Chook Door
10.2.7 class ChookDoor: chookDoor
10.2.8 class ChookDoor: doorState
10.2.9 class ChookDoor: handleEvent
10.2.10 class ChookDoor: run
10.2.11 class ChookDoor: stop
10.3 ChookDoor: main
11 The Garden Steps Lighting
12 The Garden Watering System
12.1 Garden Run
13 The Ring Main Relay Handler
14 The Spare Relay Handler
15 The Web Interface
15.1 The cgi application
15.2 The HouseMade module
15.2.1 The house interface
15.2.2 Get Local Information
15.2.3 Get Relay Information
15.2.4 Get Events Information
15.2.5 Legacy Code
15.2.6 The Relay Information section
15.2.7 define the Generate Weather Data routine
15.2.8 define the Generate Solar Data routine
15.2.9 define the Generate Tank Data routine
15.2.10 Collect Date and Time Data
15.2.11 Check the Client Connection
15.2.12 Generate the Web Page Content
15.2.13 Make the Temperature Panel
15.3 The HeatingModule module
15.3.1 Define the heatingData Class
15.3.2 Collect Parameters and Update
15.3.3 Web: heating: handle each argument
15.3.4 Build Widths for Web Page Table
15.3.5 Build the web page
16 External Hardware
16.1 The ChookDoor Controller
16.2 The Proving Circuitry Monitoring System
17 Test Programs
17.1 Check RPC Operation
18 The Log Files
19 Installing and Starting the HouseMade Software
19.1 Introduction
19.2 Details
19.2.1 Start the Hardware Server
19.2.2 Relay Server
20 The Cron Jobs
21 Validation and Maintenance Scripts
22 Makefile
23 Document History
24 Indices
24.1 Files
24.2 Chunks
24.3 Identifiers

1. Introduction

These programs have been distilled from Nathan Hurst's original Nautilus Shell suite of programs. The main differences are a) the change of name, b) the documentation, and c) consistency in logging structures (where possible). It has also undergone several reconstructions, most recently (as described by this document) to support a single Raspberry Pi based system (as opposed to a distributed system using a BeagleBone for the relay driver and a Raspberry Pi for all the higher level software).

1.1 Overview

There are a number of programs in this suite, and they are grouped into the following categories:

The Hardware Subsystem
The hardware interface to the relay drivers.
The Relay Control System
Uses a Raspberry Pi to provide the underlying hardware interface to the 16 relays used to switch the various house circuits.
The Relay Server
Provides an RPC interface to controlling the house relays.
The Event Server
Provides an RPC interface to controlling the various house events.
The Web Interface
Provides an easy to use interface to the program suite, using two key programs: house and eventEditor.
The Chook Door System
Controls the opening and shutting of the Chook House Door.
The Garden Steps System
Controls the switching of the garden steps lights.
The Garden Watering System
Controls the watering of various garden irrigation outlets.

1.2 TODOs

1.3 History

The original House Computer was set up on an Intel 386 box, named redfern, in keeping with my philosophy of naming all my computers after railway junctions. But redfern did not have enough expansion capability, and used the rapidly dating EISA bus architecture, so it was not long before it was replaced with a larger chassis using a 486 and PCI bus. This machine was named central, and was the mainstay of the house computer system for many years.

The earliest evidence for the operation of these two systems is a file dated 23 May 1999, which I believe was written for the central system. Whatever, by mid 2001 the central system was certainly running, which makes me think that redfern dates are from early 1999, while central was probably commissioned in late 1999 to early 2000. Central was a single system, and ran a suite of programs known as Nautilus (aka "Water Shell", originally because it was a "shell" controlling just the garden watering operations), serving both the web page control systems (through Apache and locally written cgi scripts), as well as the various logging subsystems (temperature, humidity, solar panel insolation, and rainwater tank levels.

Central's disk system was the weakest link in the system, since it died in June 2012, some 12 years after commissioning. This is regarded as a fairly robust operation, and set the benchmark for future systems.

Before it died, however, work had been underway to replace the logging operations through a low-power mini-controller 486 based system, known as garedelyon, with the original intention to move all functionality to garedelyon. However, garedelyon's operating system was not up to running a full web server, and so it became just the data logging component of the system. Garedelyon had several RS232 and USB ports, and took over the responsibility for performing all the logging operations. A remote RPC mechanism allowed the web server system to communicate data between the two systems. Once central had died, the web serving functions were moved to various other machines, ending up on an old PowerBook Apple laptop, known as ringwood.

In January 2015, while we we away overseas, ringwood's battery died, taking the system, including the mini-controller garedelyon and weather station, with it. While I had a spare mini-controller, it was not worth replacing the laptop, and it was decided to move the entire system back to a single controller, based upon the new Beaglebone Black that I had acquired while overseas. This new machine was known as orsay.

In the meantime, while the hardware for orsay was being developed, my Acer laptop known as lilydale was pressed into service, this time running a limited number of functions. Part of this limitation was due to there no longer being any way to communicate with RS232 ports, as used by the weather/tank/solar logging systems on garedelyon. A major part of the hardware redesign occasioned by the switch to using a Beaglebone was the need to run a USB Hub, and to connect all the RS232 ports via RS232/USB dongles.

The re-design of the system was sufficiently complete by mid April 2015 to bring it on-line. Major changes include moving all functionality to the one system (thus reverting to a similar framework to the original Central/Nautilus system), and a shift to using Flask as the main web interface. This major rewrite was renamed HouseMade because of its much wider functionality, and controlled all of the house heating, the watering system, logging and display of house data (solar, water tank), the web interfaces, and more recently, the chook house door.

Unfortunately orsay was a little unreliable, crashing more frequently than it should, and eventually dying altogether in mid Jun 2015 (exact date not recorded). The replacement machine (known as bastille) suffered similar problems, and failed on 10 Jul 2015. The reasons for these failures are not clear, but are thought to be related to power supply instabilities.

Rather than risk another $100 piece of hardware, I reverted (ostensibly temporarily) to using my Acer laptop, running Ubuntu 14.04, "Trusty Tahr". This did require a few days work to get it going properly, but there were a number of significant improvements as a consequence:

  1. The tank logging was migrated to a Python program, thus creating the potential for some more smarts in that subsection.
  2. The USB issues previously identified have been resolved. See section USB Resolution. These issues were all a consequence of the need to eliminate all RS232 code, and switch to a USB only system. The Beaglebone Black was known to have a significant bug in its USB subsystem, and this may have contributed to the unreliable behaviour reported above.
  3. The chook door mechanism was implemented, and was operational.
  4. The hostname had been hardcoded into the code, since I did not expect that it would be changing so rapidly. It has now been factored out, and replaced by the abstract title of central, a tribute to the original system which lasted for some 12 years.

1.4 Progress since 2015

In Oct 2016 we moved house, since we had embarked upon a complete renovation of the existing house, and we needed to vacate the premises. All the house automation system had to be dismantled, since every room in the house was undergoing some sort of renovation, and the cellar was no exception. For the remainder of 2016, all of 2017, and the first few months of 2018, the HouseMade system was out of action.

Progress was slow in reinstating the system, partly because of the disruption to the cellar, and partly because most of the components were in storage. The cellar had been dug out to expand the space available, and most of our spare time in 2019 was spent in making it more habitable. A new floor went in, a baseboard for the new model railway constructed (see History), drawers and shelves installed and a new (secondhand) rack to house the house server and central replacement brought in. It made a big difference to the space! This was largely complete by mid 2020, by which time the chook door opening mechanism had been reinstated, and recabled.

A complete redesign of the system was undertaken, reusing some components from the previous design, but now based upon an event-driven server-based model. This is described below in the relevant sections. The new hardware for this version of the system was a BeagleBone Black (kerang), which ran the relay drivers, and was the only processor that talks directly to hardware. kerang provided a primitive RPC interface to turn relays on and off, and in turn was driven by a decentralized Relay Server controlling the relay driver, and a decentralized Event Server, providing generalized event services to web pages and the like. These servers have been designed and tested on my desktop, but can and have been migrated to separate systems, with the stable version being known as terang, which provided http services throughout the household network to program and control the various relay devices and the systems that they control. terang was a Raspberry Pi Model 4 running NOOBS, a Unix-like system.

This system was operational by Dec 2020, driving the chook door, garden lights (a new addition to the controlled enviroment), and the garden irrigation system. Further development plans are now at the stage of active work, and this version of the documentation describes these improvements.

As of Mar 2021, the house computer system now runs on a new Raspberry Pi Model 4B, known as reuilly, occasioned by the need to expand the number of relays that it drives (now 16).

1.5 Philosophies

The house computer complex is just that - complex. Some design principles are in order. One of the difficulties of design has been the need to maintain several different systems, for different reasons. The main systems are (in abstract terms): the data logger, the house controller, and the house web server (Web Page Management Software), and more lately, the solar system (Solar Panel and Battery Analysis system).

The first principle is then all data logging to be handled by the data logging system (as far as possible). Where this is not possible, the system responsible for collecting the data should transfer it to the logging system as soon as possible, and the logging system should be regarded as the definitive source for any data requests. While this principle was originally framed to permit it to be handled by a separate machine, it can reside anywhere on the house network.

The second principle is that any request to change the state of the house must be passed through the house controller, even if it is not the final direct control mechanism. The state of the house is defined by the state of the (currently) 12 relays controlled by the controller system. Again, there is no requirement that this functionality be co-located with the others.

The third principle is that all web traffic is handled by the web system. A key factor in deciding how this functionality is handled is whether the web server is secure enough (can external users change the heating or open the chook door, for example), and secondly whether it could handle the additional traffic imposed by an externally available web server.

The fourth principle is that, as far as possible, each system component should be independent of the others, exchanging information through well-defined protocols, and using RPC mechanisms to all components to be distributed as required. For example, although both the house and solar systems have significant web interfaces, they send information to the web server as pre-structured HTML pages, and stand-alone cgi interfaces for information flow in the opposite direction.

The third principle (web traffic) has been softened in the sense that there can be multiple web servers, but only one is seen as definitive. For example, the house controller is the definitive server for all house and solar enquiries, but other web servers can run their own house or solar platforms, as long as it is clear which is the definitive system. For example, the externally facing server (, running on spencer) can offer house or solar enquiries, but it derives its data from reuilly, and is also limited in what changes it can make through the house computer (to avoid nefarious non-authorized changes).

2. Key Data Structures

2.1 Edit Warning

Provide a simple inclusion to flag that all derived files are extracted from this file, rather than stand-alone.

<edit warning 2.1> =
## ********************************************************** ## * do NOT EDIT THIS FILE! * ## * Use $HOME/Computers/House/HouseMade.xlp instead * ## **********************************************************
Chunk referenced in 17.1
<LOGDIR 2.2> = /home/ajh/logs
Chunk referenced in 2.3

2.2 House Definitions Module

The house definitions module,, gathers together in one place those constants that are common to all systems. Note that no shared variables may be handled by this module, since it is not shared across the various systems as a single instance. (All declared "variables" are actually constants, but python does not allow explicit constant declarations.)

Note that the name of house computer (which in version 4 runs all of the house software) is defined globally as CENTRAL (in honour of the original machine), but is currently located on physical machine reuilly.

"" 2.3 =
# this is the HouseDefinitions module import xmlrpc.client import datetime import os import re import sys <server definitions 2.4> <RelayNameTable 2.5> latitude = -37.8731753 # for 5 Fran Ct, Glen Waverley longitude = 145.1643351 NEFname="/home/ajh/Computers/House/events.txt" LOGDIR="<LOGDIR 2.2>/housemade" # base temperature setting if not explicitly changed ThermostatSetting=14 # although where this comes from in the new regime is not clear! # heating system desired temperature - set in cgi-bin/ aimtemp=10 # until further notice colours=['#00f', # 10 '#04f','#08f','#0cf','#0ff', # 11-14 '#0fc','#0f8','#0f4','#0f0', # 15-18 '#4f0','#8f0','#cf0','#ff0', # 19-22 '#fc0','#f80','#f40','#f00'] # 23-26 <HouseDefinitions: server connections and interfaces 2.7> <HouseDefinitions: general routines 2.8>
<server definitions 2.4> =
CENTRAL="reuilly.local" CENTRALIP="" # other servers NEWPORT="newport.local" ; NEWPORTIP="" REUILLY="reuilly.local" ; REUILLYIP="" JEPARIT="jeparit.local" ; JEPARITIP="" SPENCER="spencer.local" ; SPENCERIP="" #specific task servers HARDWARE=REUILLY ; HARDWAREIP=REUILLYIP EVENT =REUILLY ; EVENTIP =REUILLYIP RELAY =REUILLY ; RELAYIP =REUILLYIP MServer=f'http://{NEWPORT}/~ajh/cgi-bin/' # Main (web) server SServer='http://%s:5000/solar' % (CENTRAL) # SolarServer TServer='http://%s:5000/tank' % (CENTRAL) # TankServer # not currently implemented CServer=f'http://{NEWPORT}/weather' # ClimateServer # prototype system only WServer=f'http://{NEWPORT}/~ajh/cgi-bin/' # HeatingServer # prototype system only
Chunk referenced in 2.3

The server definitions are gathered in one place here so there changes may be made expeditiously, without having to track down all instances of specific servers.

<RelayNameTable 2.5> =
# system-wide definition of the house-controlling relay complement RelayNames=[ 'ChookUp', # 0 - the order of these is important 'ChookDown', # 1 'SouthVegBed', # 2 'NorthVegBed', # 3 'GardenSteps', # 4 'RingMain', # 5 'Spare6', # 6 'Spare7', # 7 'Spare8', # 8 'Spare9', # 9 'Spare10', # 10 # 'Spare11', # 11 # 'Spare12', # 12 # 'Spare13', # 13 # 'Spare14', # 14 # 'Spare15', # 15 ] NumberOfRelays = len(RelayNames) # changed in v3.1.1 RelayTable={} for i in range(NumberOfRelays): RelayTable[RelayNames[i]]=i
Chunk referenced in 2.3 2.6

There are in fact 16 relays in the system. Relay names 11 to 15 have been commented out simply to preserve space and keep the interface as simple as possible. They can be easily reinstated as necessary.

"" 2.6 =
<RelayNameTable 2.5>

The relays are given user-friendly names so that they can be referred to easily.

2.2.1 HouseDefinitions: server connections and interfaces

<HouseDefinitions: server connections and interfaces 2.7> =
# server running this script try: ThisServer=os.environ['SERVER_NAME'] except: ThisServer=NEWPORT print(f"Cannot find SERVER_NAME, using {ThisServer} instead") # Hardware Server HardwareServerIP=HARDWAREIP ; HardwareServerPort=9999 HardwareServerAdr=(HardwareServerIP,HardwareServerPort) HServer='http://%s:%s' % (HardwareServerIP,HardwareServerPort) # EventServer EventServerIP=EVENTIP ; EventServerPort=8002 EventServerAdr=(EventServerIP,EventServerPort) EServer='http://%s:%s' % (EventServerIP,EventServerPort) # RelayServer RelayServerIP=RELAYIP ; RelayServerPort=8001 RelayServerAdr=(RelayServerIP,RelayServerPort) RServer='http://%s:%s' % (RelayServerIP,RelayServerPort) NTempBlocks=6 # max number of distinct temperature blocks allowed RelayServerGood=True RelayServer=xmlrpc.client.ServerProxy(RServer) # check that the server is running by testing one of its interfaces try: RelayServer.getState() except: # bad response, let users know RelayServerGood=False EventServerGood=True EventServer=xmlrpc.client.ServerProxy(EServer) # check that the server is running by testing one of its interfaces try: dummy=EventServer.moreEvents() except: # bad response, let users know EventServerGood=False
Chunk referenced in 2.3

These are all the definitions required to talk to the various pieces of code around the place. Several of them are not yet implemented.

2.2.2 HouseDefinitions: general routines

<HouseDefinitions: general routines 2.8> =
logging=True def logMsg(msg,NewLine=False,logfile='house.log'): #if NewLine: msg+='\n' if logging: logfile=open(LOGDIR+'/'+logfile,'a') if NewLine: logfile.write("\n") logfile.write("{}: {}\n".format(now.strftime("%H:%M:%S"),msg)) logfile.close() else: print(msg, end=' ') <HouseMade: isDay definition 2.9> def setColourOld(temp): # return colours[temp-10] if temp>=HouseDefinitions.ThermostatSetting: return 'red' else: return 'blue' def setTemperatureOld(arg): t=int(arg) if t>HouseDefinitions.ThermostatSetting: t=HouseDefinitions.ThermostatSetting if t<HouseDefinitions.ThermostatSetting: t=10 return t def setColour(temp): return colours[temp-10] def setTemperature(arg): global aimtemp aimtemp=int(arg) return aimtemp def getTemperature(): global aimtemp return aimtemp
Chunk referenced in 2.3

The routines setColour and setTemperature are defined to localize these two calculations for the house and timer modules. They will be revised once the temperature adjustment system is rebuilt to its full potential.

2.2.3 the isDay function definition

<HouseMade: isDay definition 2.9> =
def isDay(d,spec): # return True if d in day list spec if d=='*': return True if spec=='*': spec='0-6' sp=spec.split(',') dlist=[] for s in sp: res=re.match('(\d)-(\d)',s) if res: a=int( b=int( dlist.extend(range(a,b+1)) else: dlist.append(int(s)) d=int(d) #print (f"isDay checks for {d} in {dlist} ({d in dlist})") return d in dlist
Chunk referenced in 2.8

The function isDay is related to the ability to schedule events differently according to the day of the week. It returns True if the nominated day of the week (d) is included in the list of days spec. spec can be a single day, a comma separated list of days, or a range of days. The BNF is

      spec = day | day ',' spec | group | group ',' spec .
      group = day '-' day .
      day = ['0' | '1' | '2' | '3' | '4' | '5' | '6' ] .
'0' represents Sunday (the first day of the week, John 20:1), through to '6' representing Saturday.

3. The Hardware System

There are three software components to this system:

The bottom layer software to hardware interface.
An interface to the outside world, providing primitive calls to control the attached relays. This sets up a server on port HouseDefinitions.HardwareServerPort
A simple program to test the server interface.

3.1 I/O Allocation for the Raspberry Pi

The relay control operations are being migrated to a Raspberry Pi (Model 4 B). One major change in this process is to add an additional 8-channel relay board, now allowing for up to 16 switchable connexions. To do this, utilize the following pinouts:

Module A Module B
Relay No GPIO Pins Relay No GPIO Pins

There is also a need for inputs to the system. As yet, the exact nature of these inputs is to be determined, but there are at least 2: Chook Door proof closed, and Chook Door proof open. Currently the gpio pin allocations are:
Input Number GPIO Pin usage
1 17 Chook Door proof open
2 27 Chook Door proof closed
3 4 request Ring Main (auto off)
4 21 request garden steps (putative, photo detector on gate, auto off)
TBA 14 DS18B20 thermometer probe
0 (EEPROM, avoid)
1 (EEPROM, clock)
2 (serial, prefer avoid)
3 (serial, prefer avoid)
GPIO 22, 18 and 15 are fried

3.2 Inverse GPIO Pin Allocation

GPIO00 EEPROM GPIO01 EEPROM GPIO02 Serial Clock GPIO03 Serial Clock GPIO04 Input 3 GPIO05 Relay 4 GPIO06 Relay 5 GPIO07 Relay 13
GPIO08 Relay 12 GPIO09 Relay 2 GPIO10 Relay 1 GPIO11 Relay 3 GPIO12 Relay 14 GPIO13 Relay 6 GPIO14 DS18B20 thermometer GPIO15 (fried)
GPIO16 Relay 15 GPIO17 Input 1 GPIO18 (fried) GPIO19 Relay 7 GPIO20 Relay 16 GPIO21 Input 4 GPIO22 (fried) GPIO23 Relay 9
GPIO24 Relay 10 GPIO25 Relay 11 GPIO26 Relay 8 GPIO27 Input 2 GPIO28 Input 3 GPIO29 Relay 4 GPIO30 Relay 5 GPIO31 Relay 13

3.3 The Device Tree

Like many such devices, the Raspberry Pi has software configurable hardware, configured by the /boot/config.txt file. We set here the hardware pullup/down resistor configuration.

For documentation on how to write configuration file entries, see the Device Tree Configuration file.

Note that the permissions on /boot/config.txt do not allow direct updating of the file. It must be edited by root. The changes that need to be made are identified at the end of the following file.

Note also that the module w1-gpio, together with w1-therm, must be added to the /etc/modules file, so that they are loaded at boot time. Do this with the following code:

        sh -c 'grep "w1_therm" /etc/modules || echo "w1_therm" >> /etc/modules'
        sh -c 'grep "w1_gpio" /etc/modules || echo "w1_gpio" >> /etc/modules'

"config.txt" 3.1 =
# For more options and information see # # Some settings may impact device functionality. See link above for details # uncomment if you get no picture on HDMI for a default "safe" mode #hdmi_safe=1 # uncomment this if your display has a black border of unused pixels visible # and your display can output without overscan #disable_overscan=1 # uncomment the following to adjust overscan. Use positive numbers if console # goes off screen, and negative if there is too much border #overscan_left=16 #overscan_right=16 #overscan_top=16 #overscan_bottom=16 # uncomment to force a console size. By default it will be display's size minus # overscan. #framebuffer_width=1280 #framebuffer_height=720 # uncomment if hdmi display is not detected and composite is being output #hdmi_force_hotplug=1 # uncomment to force a specific HDMI mode (this will force VGA) #hdmi_group=1 #hdmi_mode=1 # uncomment to force a HDMI mode rather than DVI. This can make audio work in # DMT (computer monitor) modes #hdmi_drive=2 # uncomment to increase signal to HDMI, if you have interference, blanking, or # no display #config_hdmi_boost=4 #ajh mod 20210503 config_hdmi_boost=4 # uncomment for composite PAL #sdtv_mode=2 #uncomment to overclock the arm. 700 MHz is the default. #arm_freq=800 # Uncomment some or all of these to enable the optional hardware interfaces #dtparam=i2c_arm=on #dtparam=i2s=on #dtparam=spi=on # Uncomment this to enable infrared communication. #dtoverlay=gpio-ir,gpio_pin=17 #dtoverlay=gpio-ir-tx,gpio_pin=18 # Additional overlays and parameters are documented /boot/overlays/README # Enable audio (loads snd_bcm2835) dtparam=audio=on [pi4] # Enable DRM VC4 V3D driver on top of the dispmanx display stack dtoverlay=vc4-fkms-v3d max_framebuffers=2 [all] #dtoverlay=vc4-fkms-v3d # ajh 20230111:111736 HouseMade mods to change input pulldowns to pullups gpio=4,17,18,21,27=pu gpio=4,17,18,21,27=ip # ajh 20230111:113242 HouseMade mods to use gpio14 as DS18B20 thermometer input dtoverlay=w1-gpio,gpiopin=14

3.4 Restarting the Hardware

The Raspberry Pi allows software reconfiguration of its I/O pins. These must be set before any software that controls the pins can run. As the settings require superuser privilege, this reconfiguration process must be run as root.

Note also that device settings in the hardware are lost on a reboot. Accordingly, the I/O pin settings must be redeclared on reboot. To restart the system from scratch (i.e., after a reboot), run this script:

The script also changes some of the pin setting permissions so that users may also interact with the hardware.

3.4.1 Set the Pin Definitions

Note within the pin setting script, the gpio14 pin is reserved for a DS18B20 thermometer probe, and is not available as a general purpose pin. Its setting is handled by an entry in the /boot/config.txt file. See Robert Elder's page on interfacing the DS18B20 to a Raspberry Pi.

<gpioPinsModA 3.2> = '10 9 11 5 6 13 19 26'
Chunk referenced in 3.6 3.7
<gpioPinsModB 3.3> = '23 24 25 8 7 12 16 20'
Chunk referenced in 3.6 3.7
<gpioInputPins 3.4> = '17 27 4 21'
Chunk referenced in 3.6 3.7
<gpioSparePins 3.5> = '0 1 2 3'

These definitions collect the Raspberry Pi's GPIO pin definitions in one place. There are two 8-channel relay modules, known as MODULEA and MODULEB, for a total of 16 independently switchable relays.

"" 3.6 =
#!/bin/bash GPIO=/sys/class/gpio MODULEA=<gpioPinsModA 3.2> MODULEB=<gpioPinsModB 3.3> INPUTS=<gpioInputPins 3.4> # output pins for i in $MODULEA $MODULEB ; do echo $i >$GPIO/export done # input pins for i in $INPUTS ; do echo $i >$GPIO/export done # make the outputs for i in $MODULEA $MODULEB ; do echo "out" >$GPIO/gpio$i/direction echo 1 >$GPIO/gpio$i/value done # make the inputs for i in $INPUTS ; do echo "in" >$GPIO/gpio$i/direction done # make user accessible chmod g+w /sys chmod g+w /sys/class chmod g+w /sys/class/gpio chgrp adm /sys chgrp adm /sys/class chgrp adm /sys/class/gpio # repeat for loop, to allow for race conditions in setting permissions for i in $MODULEA $MODULEB $INPUTS ; do chgrp adm $GPIO/gpio$i chgrp adm $GPIO/gpio$i/value done # repeat for loop, to allow for race conditions in setting permissions for i in $MODULEA $MODULEB $INPUTS ; do chmod g+w $GPIO/gpio$i chmod g+w $GPIO/gpio$i/value done

The key thing to note is the list of pin numbers MODULEA and MODULEB, which define output pins. These numbers are used to set gpio definitions, and then to set directions (outputs) and initialize these outputs to 1 (which is the "off" state for the driven relays).

Input pins need no initialization.

Finally, we set various permissions to make sure that users (and not just root) can use these definitions.

3.4.2 Restart the Hardware

Once the pin definitions are restored, the Hardware and associated servers can be restarted. This is done with a simple

            $ ~/Computers/House/ &
Remember to activate the server!
            $ ~/Computers/House/ virile
See below for the code for the HardwareServer.

(20230702:162737) A new script has been added to simplify restarting the HouseMade. This checks each piece of software in order to see if it is running, and if not, to (re)start it. It is important that each software process is stable before starting the next one.

3.5 the program

"" 3.7 =
ModuleApins=<gpioPinsModA 3.2> ModuleBpins=<gpioPinsModB 3.3> InputGpioPins=<gpioInputPins 3.4> ma=ModuleApins.split(' ') mb=ModuleBpins.split(' ') ip=InputGpioPins.split(' ') ma=list(map(int,ma)) mb=list(map(int,mb)) InputPins=list(map(int,ip)) OutputPins=ma.copy() OutputPins.extend(mb) ExtPins=[17,27,4,21] class relayDriver(): def __init__(self,virile): noPins=len(OutputPins) niPins=len(InputPins) self.noPins=noPins self.niPins=niPins self.valueOFiles=[] f=[None for i in range(noPins)] for i in range(noPins): gpio="/sys/class/gpio/gpio{}".format(OutputPins[i]) f[i]=open("{}/value".format(gpio),'w') self.valueOFiles.append(f[i]) self.ovalues=[0 for i in range(noPins)] self.reads=[0 for i in range(noPins)] # do we really need this? self.IFileNames=[] for i in range(niPins): gpio="/sys/class/gpio/gpio{}/value".format(InputPins[i]) self.IFileNames.append(gpio) self.ivalues=[0 for i in range(niPins)] self.virile=virile def switch(self,relay,value): if relay >= self.noPins: return if value: v=0 else: v=1 if self.virile: self.valueOFiles[relay].write(str(v)) self.valueOFiles[relay].flush() self.ovalues[relay]=(1-v) # complement, since 1 represents relay off def settings(self): return "".join(list(map(str,self.ovalues))) def test(self,In): # return value of external input number In if In <= self.niPins: # map external input to gpio pin no gpiopin=ExtPins[In-1] # get position in read list i=InputPins.index(gpiopin) f=open(self.IFileNames[i],'r') self.ivalues[i] f.close() return self.ivalues[i] else: return None def inputs(self): # return res a string of 1s and 0s for each active input res='' for i in range(self.niPins): self.ivalues[i]=self.test(i+1) res=res+"{}".format(self.ivalues[i]) return res def makeVirile(self): self.virile=True def makeSterile(self): self.virile=False def __str__(self): str="" for i in range(self.noPins): if self.ovalues[i]: str+="1" else: str+="0" return str

This code provides a class that drives the relays directly through the GPIO pins of the HardwareBone. The init method of the class creates a file for each GPIO pin in use. This file is available for writing or readiOutputPinsng, as appropriate. For relays, by writing the appropriate value, 1 for on or 0 for off, to the file turns the nominated relay attached to the corresponding GPIO pin (labelled gpiopin number) on or off.

The second method, switch, turns a single bit/relay on or off. The input request is a pair of digits, the first of which defines the relay number to be used (0-origin), and the second digit of which defines the on (1) or off (0) desired state of this relay. Note that the file has to be flushed after writing, for the new data value to be transferred immediately. We preserve the new state in the class entity values.

The third and fourth methods access the Raspberry Pi inputs. test access the given input and returns a 1 or 0, as apprpriate. read accesses all inputs and returns a composite string with one character (0 or 1) for each input.

The fifth and sixth methods control the activity of the driver. It can be switched into an ineffective mode, called sterile, by invoking the method makeSterile, and conversely, made active by invoking the method makeVirile. The flag entity virile controls this activity, and simply enables/disables the file writes in the switch method.

The __str__ method renders the current saved state into a character string, which is used to return the new state to the calling environment. An 'o' in this string indicates that the corresponding relay is enabled, and inactive relays are shown as '.'. (An equivalent method for inputs is the read method. Perhaps this should be integrated into the __str__ method?)

3.6 The program

"" 3.8 =
#!/home/ajh/binln/python3 import datetime import re import sys import HardwareDriver import socketserver HardwareServerAdr=('',9999) driver=HardwareDriver.relayDriver(True) # initially working laststate='open' def putDoorState(val): global laststate laststate=val return def getDoorState(): global laststate return laststate class MyRelayServer(socketserver.BaseRequestHandler): ''' The request handler class for our server. It is instantiated once per connection to the server, and must override the handle() method to implement communication to the client. ''' def handle(self): # self.request is the TCP socket connected to the client line = self.request.recv(1024).strip() line = str(line,'utf-8') #print("{} wrote:".format(self.client_address[0])) print("current state: {}, request: {}".format(driver,line)) if line: res=re.match('(\d+) +(\d)',line) if res: # switch a relay # before relay is switched, check chookdoor state state=driver.inputs() if state=='10': putDoorState('closed') if state=='01': putDoorState('open') relay=int( value=int( driver.switch(relay,value) else: if line=='virile': driver.makeVirile() print("driver is now active!") elif line=='sterile': driver.makeSterile() print("driver disabled") elif line=="reset": print(driver) for i in range(driver.noPins): driver.switch(i,False) elif line=='settings': rtn=bytes(driver.settings(),'utf-8') print("returned: {}".format(rtn)) self.request.sendall(rtn) return elif line=='inputs': rtn=driver.inputs() print("returned: {}".format(rtn)) rtn=bytes(rtn,'utf-8') self.request.sendall(rtn) return elif line=='readDoor': res=driver.inputs() doorstate=getDoorState() print("reading door, inputs={}, door={}".format(res[0:2],doorstate)) if res[0:2]=='11': if doorstate=='open': rtn='closing' elif doorstate=='closed': rtn='opening' elif res[0:2]=='10': putDoorState('closed') rtn='closed' elif res[0:2]=='01': putDoorState('open') rtn='open' # don't return normal state print("returned: {}".format(rtn)) rtn=bytes(rtn,'utf-8') self.request.sendall(rtn) return elif line=='quit': self.finish() self.close() sys.exit(0) else: print("did not recognize request:{}.".format(line)) # just print and send back the new driver state rtn=bytes(driver.__str__(),'utf-8') self.request.sendall(rtn) if __name__ == "__main__": # Create the server, binding to localhost on port 9999 server = socketserver.TCPServer(HardwareServerAdr, MyRelayServer) # Activate the server; this will keep running until you # interrupt the program with Ctrl-C print("{} HardwareServer starts".format(now)) server.serve_forever()

This code runs a server to interface with the relay driver code ( It imports the driver class, and creates a TCP socket server that reads and passes a relay state string directly through to the driver. This is done for reasons of simplicity and reliability. Calls on the server handler handle take one of the following parameters:

set the server to make active changes according to the requests of subsequent calls
disable any subsequent calls from activating changes
turn all relays off
return the current state of the relays as a string of 1s and 0s
return the current state of the inputs as a string of 1s and 0s
return the current state of the ChookDoor as one of the set ['open','closing','closed','opening']

If the call on the server is empty, then return the current state as a string representation of 1s and 0s (equivalent to 'settings').

The readDoor request returns the state of the Chook Door, as determined by the previous and current states. To do this, it maintains a global variable DoorState, which can have one of given four states, depending on the following conditions:

the chook door proof input is low
the chook door proof input is low
both proof circuits are high, and the chook DoorState was previously closed.
both proof circuits are high, and the chook DoorState was previously open.
Note that it is impossible for both proof circuits to be simultaneously low.

3.7 The program

"" 3.9 =
#!/home/ajh/binln/python3 import socket import sys HardwareServerAdr=('',9999) data = " ".join(sys.argv[1:]) # Create a socket (SOCK_STREAM means a TCP socket) sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) try: # Connect to server and send data sock.connect(HardwareServerAdr) req=bytes(data + "\n",'utf-8') sock.sendall(req) # Receive data from the server and shut down received = sock.recv(1024) finally: sock.close() print("Sent: {}".format(data)) print("Received: {}".format(received))

A simple little program to test the operation of the Hardware relay software. It reads a pair of digits from the invoking CLI line, and passes them through to the relay driver via the server.

4. The Relay Driver

This is a new development (as of 20220320:153553), which is to replace both the HardwareBone (kerang) and Relay Server/Event Scheduler/Event Manager (terang) system with a single Raspberry Pi 4 Model B system running all of those system components. The main reason for this is the rationalization of the hardware interface (previously done by kerang) into a replacement terang system, now know as reuilly. terang will be kept as a backup system, as it is identical to the new reuilly, but will be kept running until the new system is fully implemented and debugged.

But first, the new relay driver subsystem.

"" 4.1 =
import RPi.GPIO as GPIO RelayToGPIO={'R01':'GPIO02', 'R02':'GPIO03', 'R03':'GPIO04', 'R04':'GPIO17',\ 'R05':'GPIO27', 'R06':'GPIO22', 'R07':'GPIO10', 'R08':'GPIO09' } def pinno(relayname): gpio=RelayToGPIO[relayname] num=int(gpio[4:]) return num class driver(): <RelayDriver: init method 4.2> <RelayDriver: write method 4.3> <RelayDriver: read method 4.4>

There are three methods in the relay driver class: init, write, and read: init sets up the Raspberry Pi to perform the various I/O operations; write sets a given output pin to high or low; and read returns the value on an input pin.

4.1 RelayDriver: init method

<RelayDriver: init method 4.2> =
def __init__(self,outputs,inputs): self.outputs=outputs GPIO.setmode(GPIO.BCM) GPIO.setwarnings(False) for o in outputs: GPIO.setup(o,GPIO.OUT) for i in inputs: GPIO.setup(o,GPIO.IN) pass
Chunk referenced in 4.1

4.2 RelayDriver: write method

<RelayDriver: write method 4.3> =
def write(self,gpio,value): if gpio not in outputs: return None possiblevalues=[GPIO.LOW,GPIO.HIGH] GPIO.output(gpio,possiblevalues[value])
Chunk referenced in 4.1

4.3 RelayDriver: read method

<RelayDriver: read method 4.4> =
def read(self,gpio): if gpio not in inputs: return None val=GPIO.input(gpio) # possible translation of input value here return val
Chunk referenced in 4.1

4.4 Relay Pinouts

The matching of GPIO pins (General Purpose I/O pins) to actual pin numbers on the Raspberry Pi is not intuitively obvious, so here is the mapping to be used:

3v3 1
GPIO02 3 R01
GPIO03 5 R02
GPIO04 7 R03
GPIO17 11 R04
GPIO27 13 R05
GPIO22 15 R06
3V3 17
GPIO10 19 R07
GPIO09 21 R08

And now inverted:

R01 3 GPIO02
R02 5 GPIO03
R03 7 GPIO04
R04 11 GPIO17
R05 13 GPIO27
R06 15 GPIO22
17 3V3
R07 19 GPIO10
R08 21 GPIO09

5. The Relay Server

Currently a Hardware Bone Black (see previous section) is used to interface to the house relays, and as general data logger and server for the various house functions.

In this table, the relays are numbered left to right on the computer house panel. Bit 0 is the most significant (or leftmost) bit.

Relay Name Function Wire Colour
0 ChookUp open the Chook Door blue
1 ChookDown close the Chook Door brown
2 SouthVegBed Garden Bed next to Lounge window blue
3 GardenSteps Garden Path lighting black
4 NorthVegBed Garden Bed opposite Lounge window green
5 Spare5
6 Spare6
7 Spare7

There are currently 8 relays wired for operation, but only 5 in use. Possibilities for the new relays include:

The Hardware Bone also uses a number of inputs. There are 6 inputs currently wired, but only 2 are in use.

Input Name Function Wire Colour
1 Spare1
2 Spare2
3 Spare3
4 Spare4
5 Proof Open n.o., close when chook door is open blue
6 Proof Closed n.o., close when chook door is closed brown

(n.o. = normally open)

5.1 Relay Server Code

The relay server runs all the time, offering a RPC interface to the relay driver. The relay state is represented as a n-element list, where each element represents the relay state as an integer 1 (relay on) or 0 (relay off). It can be controlled either by passing a full state list, or by turning individual bits on and off. The latter is to be preferred, to avoid parallel interaction conflicts.

"" 5.1 =
#!/home/ajh/binln/python3 import datetime import os import re import socket import sys import subprocess import threading import time import ChookDoor from xmlrpc.server import SimpleXMLRPCServer from xmlrpc.server import SimpleXMLRPCRequestHandler from HouseDefinitions import * # Restrict to a particular path. class RequestHandler(SimpleXMLRPCRequestHandler): rpc_paths = ('/RPC2',) # Create Relay Server listening port server = SimpleXMLRPCServer(('', RelayServerPort), requestHandler=RequestHandler, logRequests=False) server.register_introspection_functions() print("RelayServer registers RPC") # open the logfile logname=LOGDIR+"/RelayServer.log" logs=open(logname,'a') <RelayServer: connect to the HardwareServer 5.2> # define the relay state try: state=serverSend('settings') except: print("Cannot talk to the Hardware Server - have you started it?") print('Error Message was "{}"'.format(sys.exc_info()[1])) sys.exit(1) currentState=[0 for i in range(NumberOfRelays)] for i in range(NumberOfRelays): if state[i]=='o': currentState[i]=1 currentTime =[0 for i in range(NumberOfRelays)] # time on in seconds nonZeroTimes=[] # those relays on for some time redundantChanges=[0 for i in range(NumberOfRelays)] # count idempotent ops <relayserver: strState 5.3> <relayserver: define the RPC-Server interface 5.4> # Define and Register the quiescent function <relayserver: quiescent 5.6> # Define and Register the readDoor function <relayserver: readDoor 5.7> # Define and Register the getState function <relayserver: getState 5.5> # Define and Register the setState function <relayserver: setState 5.8> # Define and Register the setBit function <relayserver: setBit 5.9> # Define and Register the setBitOn function <relayserver: setBitOn 5.10> # Define and Register the setBitOff function <relayserver: setBitOff 5.11> # Define and Register the getTank function <relayserver: define getTank 5.12> # Define and Register the getTimer function <relayserver: getTimer 5.13> # Define and Register the start function <relayserver: resetTimer 5.14> # Define and Register the resetTimer function <relayserver: start 5.16> # define the count down timers process <relayserver: countDown 5.17> # Define and Register the getSolar function <relayserver: getSolar 5.15> # Run the server's main loop"%Y%m%d:%H%M%S") logMsg("RelayServer starts",NewLine=True) # counters run in a separate thread. counters=countDown() logMsg("starting countDown thread") counters.start() print("RelayServer starts serving") server.serve_forever() counters.join() logs.close()

5.1.1 RelayServer: connect to the HardwareServer

<RelayServer: connect to the HardwareServer 5.2> =
def serverSend(data): # Create a socket (SOCK_STREAM means a TCP socket) sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) try: # Connect to server and send data sock.connect(HardwareServerAdr) sock.sendall("{}\n".format(data).encode()) # Receive data from the server and shut down received = sock.recv(1024).decode() finally: sock.close() return received
Chunk referenced in 5.1

This interface to the Hardware Server is now deprecated - use the relayChannel call instead <relayserver: define the RPC-Server interface 5.4>.

5.1.2 Define the convert State to String function

<relayserver: strState 5.3> =
# define the convert state to string function def strState(state): str='' for i in range(NumberOfRelays): if currentState[i]==0: str += '0' else: str += '1' return str
Chunk referenced in 5.1

This short routine converts the internal representation of the current state of the relays into a string form suitable for printing.

5.1.3 RelayServer: define the RPC-Server interface

<relayserver: define the RPC-Server interface 5.4> =
# define the relay control server interface def relayChannel(data): nowTime=now.strftime("%Y%m%d:%H%M") logMsg("relayChannel sends {}".format(data)) try: sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # Connect to server and send data sock.connect(HardwareServerAdr) sock.sendall(bytes(data,'utf-8')) # Receive data from the server and shut down received = str(sock.recv(1024),'utf-8') finally: sock.close() logMsg("relayChannel returns {}".format(received)) return received
Chunk referenced in 5.1

Note that this interface is the preferred form of communication with the HardwareServer, and replaces the serverSend function, now deprecated.

This chunk defines how the Relay Server (an RPC interface server) talks to the low level Hardware Server (a raw HTTP server). The Relay Server provides high level operations, that allow individual relays to be turned on and off, while the low-level server has an interface that is more primitive.

This allows other programs to talk more directly to the low-level relay interface, without needing the complexity of the RPC interfaces.

These RelayServer RPC interfaces are:

digit+ (0|1)
Set the relay identified by the digit string to the state indicated by a 0 or 1.
Reads the state of the chicken door.
Returns an array of bits (0,1) indicating the current state of the relays, as defined by the low-level interface (and thus allowing for asynchronous interactions with the relays).
Resets the relays according to the bits in the array newState, where a '0' indicates the relay is (now) to be turned off, and a '1' indicates the relay is (now) to be turned on. (deprecated)
Set the relay identified by relay (numbered 0 and up) to the new state newState.
Set the relay identified by relay (numbered 0 and up) to 'On'.
Set the relay identified by relay (numbered 0 and up) to 'Off'.

5.1.4 relayserver: getState

<relayserver: getState 5.5> =
def getState(): state=serverSend('settings') for i in range(NumberOfRelays): if state[i]=='1': currentState[i]=1 return currentState server.register_function(getState, 'getState')
Chunk referenced in 5.1

The getState function returns the current state of all relays as a word of zeroes and ones, counting relay 0 as the MSB and showing a 1 where the relay is currently on.

5.1.5 relayserver: quiescent

<relayserver: quiescent 5.6> =
def quiescent(): return not nonZeroTimes server.register_function(quiescent, 'quiescent')
Chunk referenced in 5.1

quiescent returns True if there is currently no timers running.

5.1.6 relayserver: readDoor

<relayserver: readDoor 5.7> =
def readDoor(): state=serverSend('readDoor') return state server.register_function(readDoor, 'readDoor')
Chunk referenced in 5.1

readDoor returns the state of the ChookDoor, as a string drawn from the set ['open', 'closing', 'closed', 'opening']

5.1.7 relayserver: setState

<relayserver: setState 5.8> =
def setState(newState): logMsg("DEPRECATED: setState({})".format(newState)) nrels=len(newState) s='' for i in range(nrels): if currentState[i]!=newState[i]: relay=i value=newState[i] s="{} {}".format(relay,value) logMsg("relayChannel called with {} for relay {}".format(s,i)) relayChannel(s) currentState[i]=newState[i] logMsg("setState({})=>{}".format(newState,currentState)) return (currentState,"OK") server.register_function(setState, 'setState')
Chunk referenced in 5.1

The setState function is deprecated, as it now requires too many redundant calls to the Hardware Server to change bits that are not changes. It is preferred that the setBit, and its two subordinates, setBitOn and setBitOff are used instead.

5.1.8 relayserver: setBit

<relayserver: setBit 5.9> =
def setBit(bitNo,value):"%Y%m%d:%H%M%S") currentState=getState() if bitNo>=NumberOfRelays: errmsg="{} bad bit number {} in call to setBit".format(now,bitNo) logMsg(errmsg) return (currentState, errmsg) % (currentState, errmsg) data="{} {}".format(bitNo,value) relayChannel(data) currentState[bitNo]=value return (currentState,"OK") server.register_function(setBit, 'setBit')
Chunk referenced in 5.1

This new routine is intended to coalesce the operations of setBitOn and setBitOff, by passing in an extra parameter newValue.

setBit sets the relay control word to its current state, and with bit number bitNo set to newValue. This is done simply to keep the currentState variable up-to-date: the actual change is made by a call to the HardwareServer to set relay n to 0 or 1.

5.1.9 relayserver: setBitOn

<relayserver: setBitOn 5.10> =
def setBitOn(bitNo): try: return setBit(bitNo,1) except: msg="setBitOn server function failed for bit number {}".format(bitNo) print(msg) logs.write(msg+'\n') logs.flush() server.register_function(setBitOn, 'setBitOn')
Chunk referenced in 5.1

setBitOn sets the relay specified by bit number bitNo to 1 (on).

5.1.10 relayserver: setBitOff

<relayserver: setBitOff 5.11> =
def setBitOff(bitNo): try: return setBit(bitNo,0) except: msg="setBitOff server function failed for bit number {}".format(bitNo) print(msg) logs.write(msg+'\n') logs.flush() server.register_function(setBitOff, 'setBitOff')
Chunk referenced in 5.1

setBitOff sets the relay specified by bit number bitNo to 0 (off).

5.1.11 HouseData define getTank

Here we make a stab at applying a temperature compensation to the water level. It assumes that the compensation is linear in both temperature and level, a somewhat bold assumption.

<relayserver: define getTank 5.12> =
# define the get water level function def getTank(): # dynamically import tank, so that we get latest data settings import tank"%Y%m%d:%H%M%S") statefile=LOGDIR+'/tankState' p=open(statefile) l=p.readline() p.close() res=re.match('^(\d\d\d\d\d\d\d\d:\d\d\d\d\d\d) +(\d+) ',l) if res: level=int( else: level=-1 # now temperature compensate the level, and calibrate litres=tank.convert(level) #logs.write("%s getTank()=>%s (from level=%s)\n" % (now,litres,level)) #logs.flush(); os.fsync(logs.fileno()) return litres server.register_function(getTank, 'getTank')
Chunk referenced in 5.1

5.1.12 relayserver: getTimer

<relayserver: getTimer 5.13> =
def getTimer(bitNo): remTime=currentTime[bitNo] return remTime server.register_function(getTimer, 'getTimer')
Chunk referenced in 5.1

5.1.13 relayserver: resetTimer

<relayserver: resetTimer 5.14> =
def resetTimer(bitNo): currentTime[bitNo]=0 setBitOff(bitNo) if bitNo in nonZeroTimes: nonZeroTimes.remove(bitNo) return 0 server.register_function(resetTimer, 'resetTimer')
Chunk referenced in 5.1

5.1.14 relayserver: getSolar

<relayserver: getSolar 5.15> =
def getSolar(regNo):"%Y%m%d:%H%M%S") reg=solar.float_register(regNo) #logs.write("%s: getSolar(%d)=>%4.1f\n" % (now,regNo,reg)) #logs.flush(); os.fsync(logs.fileno()) return reg server.register_function(getSolar, 'getSolar')
Chunk referenced in 5.1

5.1.15 relayserver: start

<relayserver: start 5.16> =
def start(bitNo,timeon):"%Y%m%d:%H%M%S") currentState=getState() logMsg("RelayServer.start bit {} timer for {} seconds".format(bitNo,timeon)) if bitNo>=NumberOfRelays: errmsg="%s bad bit number %d in call to start" logMsg(errmsg % (now,bitNo)) return (currentState, errmsg) % (now,bitNo) setBitOn(bitNo) s=strState(currentState) logMsg("startTimer(%d,%4.1f), newstate=%s (%s)" % (bitNo,timeon,s,RelayNames[bitNo])) # design decision: timeon is relative, not absolute currentTime[bitNo]+=timeon if bitNo not in nonZeroTimes: nonZeroTimes.append(bitNo) # turning the bit off is taken care of by the countDown process return (currentState,"OK") server.register_function(start, 'start')
Chunk referenced in 5.1

5.1.16 relayserver: countDown

<relayserver: countDown 5.17> =
class countDown(threading.Thread): def __init__(self): threading.Thread.__init__(self) def run(self): while True: if nonZeroTimes: currentState=getState() for bitNo in nonZeroTimes: currentTime[bitNo]-=1 val=currentTime[bitNo] #logMsg("countDown timer decrements bit {} to {}".format(bitNo,val)) if currentTime[bitNo]==0: # turn this bit off and log the fact setBitOff(bitNo) logMsg("stopTimer(%d), (%s)" % (bitNo,RelayNames[bitNo])) # remove from nonZeroTimes nonZeroTimes.remove(bitNo) time.sleep(1) # sleep until next cycle logMsg("SHOULD NOT HAPPEN: countDown timer loop exits")
Chunk referenced in 5.1

The purpose of auxilary process countDown is to maintain the count of how long each relay is to be held closed, if it was started with a start call. Note that it is possible to also set relays on and off without a time (using the setBitOn and setBitOff RPC calls), in which case the time remaining is shown as 0 (zero). Such calls will not be automatically turned off, since they do not set the list of nonZeroTimes.

Once a second, the process awakes, and decrements any non-zero counts. These non-zero counts are indicated in the list nonZeroTimes as a partial optimization to avoid testing all relay counts, and to avoid ambiguity about which relays are independently turned on.

5.2 Starting the Relay Server

This short script encapsulates all that is necessary to (re)start the relay server. It records the process ID in the file relayProcess so that when it is restarted, any previous invocation is removed properly.

It is invoked by the make script as make start-relay.

"" 5.18 =
LOGDIR='<u name="LOGDIR"/>' HOUSE='/home/ajh/Computers/House' BINLN=${HOME}/binln if [ -f ${LOGDIR}/relayProcess ] ; then for p in `cat ${LOGDIR}/relayProcess` ; do kill -9 $p done fi rm ${LOGDIR}/relayProcess # use logging version for now ${BINLN}/python3 ${HOUSE}/ >${LOGDIR}/RelayServer.log 2>&1 & ps aux | grep "" | grep -v grep | awk '{print $2}' >${LOGDIR}/relayProcess

5.3 The Relay Controller Code

This is a simple standalone program used by cron jobs to turn on relays at various times (mainly watering) for fixed periods of time. It calls the RelayServer via RPC calls to actually drive the relays, and really serves only as a CLI parameter handler. The relay name, and the time it is to turn on are supplied by two CLI parameters. If other than two parameters (besides the program name) are supplied, a default is used.

This program is used only for testing the Relay Server, and is not used by the HouseMade system per se.

"" 5.19 =
#!/home/ajh/binln/python3 import time import os import sys import xmlrpc.client from HouseDefinitions import * def main(device,timeRunning): # get relay bit for device try: bitNo=RelayTable[device] except KeyError: print("bad relay function key %s" % (device)) sys.exit(1) # start the relay for timeRunning seconds (state,ok)=RelayServer.start(bitNo,timeRunning) print("relays set to %s" % (state)) pass if __name__ == '__main__': if len(sys.argv)==3: # get relay name device=sys.argv[1] # get relay time timer=int(sys.argv[2]) else: print("insuffcient parameters! Need relay name and timer") sys.exit(1) main(device,timer)

6. The Events Module

This module contains definitions of the Event and EventList classes, suitable for importing into other modules. At the moment, there is no standalone (main) function.

"" 6.1 =
<Event class: definition 6.2> <EventList class: definition 6.4> <Events: main routine for testing code 6.11>

6.1 The Event Class

A significant shift in the design (from HouseMade v2.2.0) is the introduction of the Event class, to encapsulate data pertaining to an event. An event can now occur at any time in the future, and is day of the week aware, as well as time of the year. This has implications for seasonal variations, such as sunrise/sunset times, and for summer/winter changes.

From version 3.0, events are also defined by a dictionary model, using the class dictionary as a vehicle for swapping between the two representation. This is because when events are being passed to and from the EventServer, the rpc mechanism requires a dictionary representation of events.

<Event class: definition 6.2> =
from HouseDefinitions import isDay,logMsg class Event(): def __init__(self,dictn=None,time=None,weekday=None,\ device=None,operation=None): self.time=time if weekday: self.weekday=weekday else: self.weekday='*' self.device=device self.operation=operation if dictn: for k in dictn.keys(): setattr(self,k,dictn[k]) pass def dictRep(self,ev): for key in ev.keys(): setattr(self,key,ev[key]) return def __str__(self): rtn='{' rtn+='weekday={}, '.format(self.weekday) rtn+='time={}, '.format(self.time) rtn+='device={}, '.format(self.device) rtn+='operation={}'.format(self.operation) rtn+='}' return rtn <Event class: compare two events 6.3>
Chunk referenced in 6.1

The Event class provides data that defines an event in the system. There are a number of attributes, and an event can be accessed either as an Event object, through its entities, or as a dictionary, accessed through its keys. The method dictRep allows conversion from the entity-based representation to a dictionary, and the dictn optional parameter allows creation of a new event from the dictionary representation.

The attributes are defined as follows:

A value in the range 0-6, where 0 represents Sunday, and 6 represents Saturday. This is for events that only occur on specific days of the week. A value of None (or '*') indicates that the event occurs on every day of the week. (currently only '*' implemented)
A value in the form HH:MM, representing time in a 24 hour format, and defining the time of day at which an event occurs.
The name of the device for which the event occurs. This is a text field, and currently has the values ChookDoor, NorthVegBed, SouthVegBed, and GardenLights.
The parameter for the event. This is also a text field, and has the format

operation=text [',' text]*
text=<any string of characters except ','>

text is defined by the handler routine, but is normally one of the values on, off, up, down, or a string of digits, representing an integer value.

Note that an event can be created with zero or more initial values for these attributes through the use of optional parameters. The optional parameter dictn can be used with a dictionary of values (with keys as for the fields defied above) to initialize the event. These values override any defined with the other values, since the dictionary is evaluated after the static representations are initialized.

<Event class: compare two events 6.3> =
def compare(self,a): if a.time > self.time: return -1 elif a.time < self.time: return 1 else: return 0 def __gt__(self,other): return def __lt__(self,other): return def __eq__(self,other): return self.time==other.time
Chunk referenced in 6.2

Compare this event against another. For comparison purposes, only the times are relevant. If this event occurs first, return -1. If it occurs at the same time as event a, return 0. Otherwise, return 1.

6.2 The EventList class

<EventList class: definition 6.4> =
EVENTFILE="/home/ajh/etc/events.txt" import re class EventList(): def __init__(self): self.list=[] def __str__(self): rtn='' for e in self.list: rtn+="{}\n".format(e) return rtn <EventList class: add event 6.5> <EventList class: delete event 6.6> <EventList class: nextEvent 6.8> <EventList class: sort events 6.7> <EventList class: load events 6.9> <EventList class: save events 6.10>
Chunk referenced in 6.1

The EventList maintains a list of events, sorted chronologically. Past events are possible, and are ignored for scheduling purposes.

<EventList class: add event 6.5> =
def add(self,e,dupl=False,matchattrs=None): # e is an Event # dupl is boolean, True=>duplicates allowed # matchattrs is a list of attributes # a duplicate is defined as two events that have the same value # for each of the attributes matchattrs #print("EventList: add event({},{},{}) called".format(e,dupl,matchattrs)) if not self.list: self.list=[e]; return if matchattrs=='all': matchattrs=['weekday','time','device','operation'] if not matchattrs: matchattrs=['device','operation'] i=-1 for l in self.list: i+=1 #print ("check event {}, type is {}, dupl={}".format(l,type(l),dupl,matchattrs)) same=True # until proven false for attr in matchattrs: if getattr(l,attr)!=getattr(e,attr): same=False #print("distinction found on attr {}".format(attr)) break if same: #print("EventList: add event finds duplicate={} at position {}".format(e,i)) pass if same and not dupl: #print("EventList: add event {} not added, duplicates disallowed".format(e)) return elif not same: #print("EventList: add event {} unique, adding it".format(e)) pass if < 0: i=self.list.index(l) self.list.insert(i,e) #print("EventList: add event {} added".format(e)) return i # collect fall-thru - add this event at end self.list.append(e) #print("EventList: add event {} added".format(e))
Chunk referenced in 6.4

When adding an event, it is placed into the list according to its time attribute, so that the preceding event has a time value that is the same (see <Event class: compare two events >, and the following event has a time that is the same or later.

The dupl parameter, when True, allows duplicate events to be added. Normally, duplicate events cannot be added when this parameter is False. A duplicate is defined as matching on a set of given attributes, given as a list of attributes. If this parameter is an empty list or None, then the match is defined only on device and operation, NOT time.

The add interface returns the index of the new event in the event list if successful, None otherwise.

<EventList class: delete event 6.6> =
def delete(self,a): # a is an Event self.list.remove(a) return
Chunk referenced in 6.4

Remove a given event from the list.

<EventList class: sort events 6.7> =
def sort(self): old=self.list self.list=[] for e in old: self.add(e,dupl=True) return
Chunk referenced in 6.4

(Re)sort the event list. This is done with an insertion sort, to ensure that the list is recreated in the same order as when events are added one by one.

<EventList class: nextEvent 6.8> =
def nextEvent(self,now,today='*'): # now now includes day of week # return index of first event with time =/> now # event.weekday must match today if it is not '*' print(f"nextEvent called with now={now}, today={today}") #print(self.list) i=0 for e in self.list: print("nextEvent looks at event {}, {}".format(i,e)) if isDay(today,e.weekday): print(f'{now},{e.time},{now<=e.time}') if now<=e.time: print("now ({}) is less/equal to e.time ({})".format(now,e.time)) print("nextEvent returns event {}".format(i)) return i i+=1 return None
Chunk referenced in 6.4

Return the first event in the list with time equal or later to the given time stamp now. This has been complicated by the fact that in version 4.1, events can now be specified by day of the week. If the day of the week parameter is '*', then all days of the week are involved. For any other value drawn from the set ['0','1','2','3','4','5','6'], then only that day of the week is to be scheduled (0=Sunday, 6=Saturday).

Return None if there are no such events. (This is a change in the definition - previously the last event would be returned in this case, which violates the postcondition.)

<EventList class: load events 6.9> =
def load(self,loadfile=EVENTFILE): f=open(loadfile,'r') for l in f.readlines(): res=re.match('{weekday=(.*), time=(.*), device=(.*), operation=(.*)}',l) if res: # new in version 4.1.0: take note of day field if d=='None': d='*' if d: d=d.split(',') d.sort() for dy in d: ev=Event(weekday=dy,time=t,device=dv,operation=op) #print("adding sub-day event: {}".format(ev)) self.add(ev,matchattrs='all') else: ev=Event(weekday=d,time=t,device=dv,operation=op) self.add(ev) else: print("Cannot parse {}".format(l)) f.close() #print("loaded events: {}".format(self.list)) return
Chunk referenced in 6.4

Load events from a file. Existing events are retained, and new events are inserted to preserve the chronological ordering.

<EventList class: save events 6.10> =
def save(self,savefile=EVENTFILE): f=open(savefile,'w') content=self.__str__() f.write(content) f.close() return
Chunk referenced in 6.4

Save all current events to a file, which may be later reloaded with load. Be aware that on reloading, any existing events with the same parameters will become duplicate events.

6.3 Events: main routine for testing code

<Events: main routine for testing code 6.11> =
def main(): def checkNext(day,time): print("\n\ncheckNext({},{})".format(day,time)) next=el.nextEvent(time,today=day) print("found match at {}".format(next)) el=EventList() # make an empty EventList el.load('/home/ajh/etc/events.txt') # and preload it e=Event() # create an empty Event e.time='17:30' # set some of its attributes e.weekday='*' # set everyday e.device='Spare6' e.operation='on' el.add(e) # add an element print('check loaded eventlist:\n{}'.format(el)) checkNext('*','13:40') checkNext('*','19:20') checkNext('6','07:20') checkNext('0','07:20') checkNext('*','07:20') checkNext('2','14:20') if __name__=='__main__': main()
Chunk referenced in 6.1

A new feature - each module in this suite is to have a main routine that provides testing facilities for the defined features of the module. This main routine creates Events and EventLists, populates them, and runs a test of each method in the classes.

7. The Event Server

The EventServer is a development of the EventManager program (versions 3.0 and previous) as a means of providing a generic event service. A key difference is that the EventServer provides a remote procedure call interface to the key data structures identifying the events that are managed by the HouseMade system. These events can be accessed and managed through identified RPC interfaces, thus preserved the integrity of the data.

"" 7.1 =
#!/home/ajh/binln/python3 import datetime import sys import time from HouseDefinitions import * from Events import Event,EventList class ShutDown(Exception): pass from multiprocessing import Process from xmlrpc.server import SimpleXMLRPCServer from xmlrpc.server import SimpleXMLRPCRequestHandler # data structures el=EventList() # Restrict to a particular path. class RequestHandler(SimpleXMLRPCRequestHandler): rpc_paths = ('/RPC2',) # Create Event Server listening port port=('',EventServerPort) server = SimpleXMLRPCServer(port, requestHandler=RequestHandler, allow_none=True) server.register_introspection_functions() #print("EventServer registers RPC") # identify the next event. None means not yet identified. # Caveat! nextEventPointer can take the value 0, meaning the first # event. Be careful to distinguish 0 from None, they both test False! nextEventPointer=None lastEventPointer=-1 <Event Server: calling points 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16> <Event Server: serverprocess routine 7.17> <Event Server: main routine 7.18> if __name__ == '__main__': main()

7.1 Event Server calling points

<Event Server: calling points 7.2> =
def add(evd,dupl,handle=None): global nextEventPointer # evd comes in as a dictionary, must convert it ... ev=Event() # first make empty event for key in evd.keys(): setattr(ev,key,evd[key]) print("EventServer: adding event {}".format(ev)) # note that add always adds in order newindex=el.add(ev,dupl=dupl) # must make sure that nextEventPointer is (potentially) updated curtime=now.strftime("%H:%M") curday=(now.isoweekday()+7) % 7 if nextEventPointer: oldnext=nextEventPointer else: oldnext=-1 nextEventPointer=el.nextEvent(curtime,today=curday) logMsg(f"EventServer adjusts nextEvent from {oldnext} to {nextEventPointer}") logMsg("server adds event {}".format(ev)) return newindex server.register_function(add, 'add')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16

The add event interface is complicated by the fact that complex data structures (such as an event) cannot be passed directly over the RPC interface, but must be converted first to a dictionary representation. Hence the copying of attributes/dictionary entries as the first step. Once the event has been reconstructed, it is added to the event list through the eventlist call el.add. dupl is a boolean flag that identifies (if True) that the new event may be duplicated if it matches another event in the event list, otherwise it will be ignored. The index into the list of the newly added entry is returned.

<Event Server: calling points 7.3> =
def remove(evn): if type(evn) is int: # delete event number evn if evn<len(el.list): del el.list[evn] logMsg("server removes event number {}".format(evn)) return elif type(evn) is Event: # find event that matches evn in device and operation pass # needs work! server.register_function(remove, 'remove')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16

The remove interface removes event number evn from the event list. Although the code suggests that the event to be renoved can be specified verbatim, this has not yet been proved to be necessary. It may well be removed itself at some stage.

<Event Server: calling points 7.4> =
def setNext(curtime,curday): '''Use the current day and time to identify the next event to be scheduled, and then set nextEventPointer to point to this event. nextEventPointer is advanced by getNext.''' global nextEventPointer if not curtime: curtime=now.strftime("%H:%M") nextEventPointer=el.nextEvent(curtime,today=curday) server.register_function(setNext, 'setNext')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16

In this model, the setNext interface proved to be somewhat difficult to implement. This version defers most of the hard work to the event list class, and all that happens here is that we make sure that the calling parameters are set correctly. The curday parameter is a case in point: it can either be '*', meaning any and all days participate in the operation, or it may be a specific day of the week (see code fragment <EventList class: nextEvent 6.8>), when only those events scheduled for that day are considered.

<Event Server: calling points 7.5> =
def advanceNext(): '''move to the next event in the list''' global nextEventPointer,lastEventPointer if nextEventPointer is None: nextEventPointer=0 elif nextEventPointer < len(el.list): lastEventPointer=nextEventPointer nextEventPointer+=1 return server.register_function(advanceNext, 'advanceNext')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16

Note that advanceNext takes no notice of the day of the week (unlike setNext), but simply steps through the complete list event by event. It is up to the calling environment to decide if the next event is what is wanted.

<Event Server: calling points 7.6> =
def showNext(): '''return the next event in the list. The nextEventPointer is not advanced.''' global nextEventPointer,lastEventPointer #logMsg("EventServer.showNext() called") if nextEventPointer is None: return None if nextEventPointer >= len(el.list): return None ev=el.list[nextEventPointer] if nextEventPointer!=lastEventPointer: logMsg("EventServer.showNext: next event {} is {}".format(nextEventPointer,ev)) lastEventPointer=nextEventPointer return ev server.register_function(showNext, 'showNext')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.7> =
def moreEvents(): global nextEventPointer #logMsg("EventServer.moreEvents() called") if nextEventPointer is not None and nextEventPointer < len(el.list): return True else: return False server.register_function(moreEvents, 'moreEvents')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.8> =
def printEvents(): print("There are {} events:".format(len(el.list))) for ev in el.list: print(ev) return server.register_function(printEvents, 'printEvents')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.9> =
def listEvents(): logMsg(f"EventServer.listEvents() called") retstr=f"There are {len(el.list)} events:\n" for ev in el.list: retstr+='{}\n'.format(ev) return retstr server.register_function(listEvents, 'listEvents')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.10> =
def getEvent(i): #logMsg(f"EventServer.getEvent({i}) called") if i>=len(el.list): return None else: return el.list[i] server.register_function(getEvent, 'getEvent')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.11> =
def matchEvents(ev): # return a list of event numbers matching ev in device and operation # NOTE ev is a dictionary representation of an event logMsg("EventServer.matchEvents() called") l=[]; i=-1 try: for e in el.list: i+=1 if e.device==ev['device'] and e.operation==ev['operation']: l.append(i) except: fails='matchEvents fails with error {}'.format(sys.exc_info()) logMsg(fails) print(fails) return l server.register_function(matchEvents, 'matchEvents')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.12> =
def matchEvent(ev): # return the index of the first event matching ev in device and operation # NOTE ev is a dictionary representation of an event logMsg("EventServer.matchEvent() called") i=-1 try: for e in el.list: i+=1 if e.device==ev['device'] and e.operation==ev['operation']: return i except: fails='matchEvent fails with error {}'.format(sys.exc_info()) logMsg(fails) print(fails) return None server.register_function(matchEvent, 'matchEvent')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.13> =
def sortEvents(): el.sort() server.register_function(sortEvents, 'sortEvents')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.14> =
def loadEvents(): el.load() server.register_function(loadEvents, 'loadEvents')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.15> =
def saveEvents(): server.register_function(saveEvents, 'saveEvents')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16
<Event Server: calling points 7.16> =
def registerCallback(device,routine): dispatcher[device]=routine server.register_function(registerCallback, 'registerCallback')
Chunk referenced in 7.1
Chunk defined in 7.2,7.3,7.4,7.5,7.6,7.7,7.8,7.9,7.10,7.11,7.12,7.13,7.14,7.15,7.16

These are the entry points for calls on the EventServer. The logMsg are commented out until I can work out how to output messages from a server subprocess (it involves using a Queue or a Pipe, see The Python3 Library).

7.2 Event Server: serverprocess routine

<Event Server: serverprocess routine 7.17> =
def serverprocess(logMsg): try: server.serve_forever() except KeyboardInterrupt: pass
Chunk referenced in 7.1

The serverprocess routine is the heart of the event server. Its role is to package up the separate process of running the server, while allowing the event manager to manage its events. Not sure about the KeyboardInterrupt, though.

7.3 Event Server: main routine

<Event Server: main routine 7.18> =
def main(testing=True,forReal=True): logMsg("EventServer starts serving on port {}".format(port),NewLine=True) serverp = Process(target=serverprocess, args=(logMsg,)) serverp.start() try: serverp.join() except KeyboardInterrupt: logMsg("Keyboard Interrupt received! ") nowstr=now.strftime("%Y%m%d:%H%M%S") logMsg("EventServer stops serving (main)") logs.close()
Chunk referenced in 7.1

8. The Event Scheduler

This program takes the place of the old EventManager, and uses the EventServer to maintain the collection of events to be scheduled. Its basic algorithm is much the same: look at the next event from the current time, wait until its time to be scheduled has arrived, and then schedule it.

Events may be added to the event list between now and the scheduling time. Providing that the scheduled time is in the future, the es.showNext call (which is dynamically computed) should collect it. If the added event time is in the past, then it is ignored. If the added event's time is now, then it is indeterminate (a race condition exists) as to whether the event is scheduled, and no guarantees are made.

"" 8.1 =
#!/home/ajh/binln/python3 import datetime from Events import Event,EventList from HouseDefinitions import * import re import sys import time import xmlrpc.client import GardenSteps import GardenWater import ChookDoor import RingMain import Spares nowTime=now.strftime("%H:%M") nowdate=now.strftime("%Y%m%d:%H%M") nowDay=(now.isoweekday()+7) % 7 # log the start of operations msg="EventScheduler starts at {}, ".format(nowdate) msg+="using EventServer {}".format(EServer) logMsg(msg,NewLine=True) # connect to the server es=xmlrpc.client.ServerProxy(EServer,allow_none=True) # dispatcher is a dictionary of call-back routines indexed by device, and # called when scheduled with the given parameter for the event. dispatcher={} gs=GardenSteps.GardenSteps() dispatcher['GardenSteps']=gs.handleEvent gw=GardenWater.GardenWater() dispatcher['SouthVegBed']=gw.handleEvent dispatcher['NorthVegBed']=gw.handleEvent cd=ChookDoor.ChookDoor() dispatcher['ChookDoor']=cd.handleEvent rm=RingMain.RingMain() dispatcher['RingMain']=rm.handleEvent rl=Spares.SpareRelay() for spNo in range(6,16): dispatcher['Spare{}'.format(spNo)]=rl.handleEvent # now load the events for the day logMsg("SETUP TODAY SPECIFIC EVENTS") #logMsg("Current events are:") #logMsg(es.listEvents()) logMsg("load yesterday's events") es.loadEvents() #logMsg("New current events are:") #logMsg(es.listEvents()) <EventScheduler: collect ChookDoor times 8.2> <EventScheduler: collect GardenSteps times 8.3> <EventScheduler: main loop 8.4> logMsg("EventScheduler runs out of events, terminating\n") es.saveEvents()

Each of the devices controlled by the Scheduler have a dispatcher entry point which is called when an operation on that device is scheduled. These are known as the handleEvent methods attached to each device. It has the responsibility for determining exactly what the 'operation' scheduled by the event translates to, whether it be 'off' or 'on', 'up' or 'down', etc..

Most of this is straightforward, except for those events that vary with the seasons. Both the ChookDoor and the GardenSteps are affected by the hours of daylight, and hence the timing of the events need to be computed by the appropriate module. This is performed by a call to 'collect (device) times'.

8.1 EventScheduler: collect ChookDoor times

<EventScheduler: collect ChookDoor times 8.2> =
# here is where we need to update chookdoor opening and closing times openev=Event(device='ChookDoor',operation='up') # get matching event cdup=es.matchEvent(openev) logMsg("ChookDoor up matching event is {}".format(cdup)) if cdup>=0: es.remove(cdup) logMsg("ChookDoor up matching event {} removed".format(cdup)) # now insert a new replacement openev newev=Event(time=cd.dooropen,device='ChookDoor',operation='up') es.add(newev.__dict__,True) logMsg("ChookDoor up new event added") #logMsg(es.listEvents()) # now make the closedoor event closeev=Event(device='ChookDoor',operation='down') # get matching event cddown=es.matchEvent(closeev) logMsg("ChookDoor down matching event is {}".format(cddown)) if cddown: es.remove(cddown) logMsg("ChookDoor down matching event {} removed".format(cddown)) # now insert a new replacement closeev newev=Event(time=cd.doorshut,device='ChookDoor',operation='down') es.add(newev.__dict__,True) logMsg("ChookDoor down new event added") #logMsg(es.listEvents())
Chunk referenced in 8.1

The EventScheduler has to collect from the ChookDoor object the current opening and closing times, which are computed at ChookDoor initialization. We check the current event list to see what door times are recorded, updating them if they exist, and creating new ones if they do not.

8.2 EventScheduler: collect GardenSteps times

<EventScheduler: collect GardenSteps times 8.3> =
# here is where we need to update gardensteps opening and closing times # first make the new on event onev=Event(device='GardenSteps',operation='on') # get matching event gson=es.matchEvent(onev) if gson: es.remove(gson) # now insert a new replacement gson event newev=Event(time=gs.onTime,device='GardenSteps',operation='on') es.add(newev.__dict__,True) # gsoff time is overridden only if self.Auto is True if gs.Auto: closeev=Event(device='GardenSteps',operation='off') # get matching event gsoff=es.matchEvent(closeev) if gsoff: es.remove(gsoff) # now insert a new replacement closeev newev=Event(time=gs.offTime,device='GardenSteps',operation='off') es.add(newev.__dict__,True)
Chunk referenced in 8.1

The EventScheduler has to collect from the Gardensteps object the current opening and closing times, which are computed at GardenSteps initialization. We check the current event list to see what times are recorded, updating them if they exist, and creating new ones if they do not.

8.3 EventScheduler: main loop

<EventScheduler: main loop 8.4> =
print("EventScheduler starts main loop") logMsg("\nEventScheduler starts main loop") es.setNext(nowTime,nowDay) try: while es.moreEvents(): ev=es.showNext() etime=ev['time'] eday=ev['weekday'] while nowTime!=etime: ev=es.showNext() # check that event list has not been updated etime=ev['time'] logMsg("clock ticks forward from {} to {}".format(nowTime,etime)) nowTime=now.strftime("%H:%M") secs2zero=60-now.second time.sleep(secs2zero) nowTime=now.strftime("%H:%M") logMsg("next event is {}".format(ev)) if isDay(nowDay,eday): print("schedule event {} at time {} on day {}".format(ev,etime,eday)) logMsg("EventScheduler dispatches event {}".format(ev)) dev=ev['device'] ; op=ev['operation'] if dev in dispatcher: dispatcher[dev](dev + ' ' + op) logMsg("Event {}({}) dispatched".format(dev,op)) else: logMsg("No handler for event {}".format(ev)) else: print("EVENT NOT SCHEDULED: {}".format(ev)) es.advanceNext() ev=es.showNext() print() except KeyboardInterrupt: logMsg("EventScheduler terminated by KeyboardInterrupt") es.saveEvents() sys.exit(0)
Chunk referenced in 8.1

The main event scheduler loop scans over the event list in chronological order, calling the event server as events become due.

If two or more events are scheduled for the same time, we must be careful that both do get scheduled. Hence the outer loop has responsibility for advancing over the list of events, and only in the case that the next event is not scheduled for the current time (while nowTime!=etime) do we allow the clock to be advanced.

9. The Event Editor

This is a new component to the HouseMade system. It provides a web interface to the EventServer, allowing the user to view and edit the current list of events.

"" 9.1 =
#!/home/ajh/binln/python3 import sys sys.path.append('/home/ajh/Computers/House') import cgi import datetime from HouseDefinitions import * import os,sys from Events import Event,EventList import xmlrpc.client nowstr=now.strftime("%Y%m%d:%H%M") import cgitb cgitb.enable() logMsg("eventEditor started") es=xmlrpc.client.ServerProxy(EServer,allow_none=True) print("Content-type: text/html\n\n") # check if EventServer is running if not EventServerGood: print('EventServer not running - aborting') logMsg("eventEditor aborted - EventServer not running") sys.exit(1) form=cgi.FieldStorage() #print(form) print('<h1 style="text-align:center; background-color:lightgreen">HouseMade Events Editor</h1\n') <EventEditor: print instructions 9.4> <EventEditor: define get current events routine 9.5> <EventEditor: define make home page routine 9.6> <EventEditor: define make edit page routine 9.7>
Chunk defined in 9.1,9.2,9.3

Define global stuff, then the various routines to perform the EventEditor operations.

"" 9.2 =
try: el=getAllEvents() except ConnectionRefusedError: print('<p style="background-color:red; font-style:italic;color:white">',end='') print('Connection to EventServer refused - is it running?</p>') sys.exit(0) #print(el) if 'action' in form: request=form['action'].value else: request='home' entry=None if 'entry' in form: entry=int(form['entry'].value) weekday=time=device=operation='' if 'weekday' in form: day=form['weekday'].value if 'time' in form: time=form['time'].value if not ':' in time: time="{:s}:{:s}".format(time[0:2],time[2:4]) if 'device' in form: device=form['device'].value if 'operation' in form: operation=form['operation'].value #print("<p>Request is for {}, entry={}</p>".format(request,entry))
Chunk defined in 9.1,9.2,9.3

Now collect the events from the server, and the edit parameters from the URL form.

"" 9.3 =
if request=='home': page=makeHomePage() elif request=='add': # Need to know where to add this entry (given by value of entry) # added after entry number 'entry' with time set to that of entry number entry # adding to empty list is a special case if len(el.list)==0: # make first entry event a=Event() a.time='00:00' else: # get event at this entry number e=es.getEvent(entry) a=Event(dictn=e) logMsg("eventEditor: Adding event {}".format(a)) newindex=es.add(a,True) el=getAllEvents() #print(el.list) page=makeHomePage(new=newindex) elif request=='edit': page=makeEditPage(entry,weekday=weekday,time=time,device=device,operation=operation) elif request=='delete': print("<p>deleting entry number {}</p>".format(entry)) es.remove(entry) page=makeHomePage() else: page="<p>Action <i>{}</i> not implemented</p>".format(request) print("{}".format(page))
Chunk defined in 9.1,9.2,9.3

Examine the request, and perform the appropriate task. add adds a new event to the table, using the current event as the default value. edit updates any edited information, and delete deletes the given element.

9.1 The EventEditor Instructions

<EventEditor: print instructions 9.4> =
print(''' <p> Events are shown, one per line in the table. '*' for <b>Month</b> and <b>Day</b> indicate every month/day is involved. Months are not implemented, but different days of the week can be shown as lists of numbers drawn from 0-6, where 0 represents Sunday (the first day of the week), and 6 represents Saturday (the last day of the week). So 0,6 (or 6,0) represents the weekend. </p> <p> <b>Times</b> are to be entered in 24 hour format, either as 'HH:MM' or as 'HHMM' (but are stored and displayed in the first format). The <b>Device</b> will be a pull-down from a menu, but is currently implemented only as a text field. Similarly for the <b>Operation</b> field. </p> <p> The <b>Change</b> column buttons save/enter any changed data on the given line. <b>Add</b> adds a new event, initialized to the data on the given line. <b>Delete</b> will delete the event. Note that if you need to add an event before the first entry, simply click on <b>Add</b> for the first entry, and edit the time field to be sooner than the given time. </p> <p> The <b>NOW</b> line indicates the current time in the sequence of events, and the next line will be the next event to be scheduled. </p> ''')
Chunk referenced in 9.1

9.2 EventEditor: get current events routine

<EventEditor: define get current events routine 9.5> =
# get all the current events def getAllEvents(): i=0 es.sortEvents() x=es.getEvent(0) el=EventList() while x: #print(x) ev=Event(dictn=x) el.add(ev,dupl=True) i+=1 x=es.getEvent(i) #el.sort() #print('<p>',el) return el
Chunk referenced in 9.1

9.3 EventEditor: Make Home Page

<EventEditor: define make home page routine 9.6> =
def makeHomePage(new=None): es.sortEvents() el=getAllEvents() numEvents=len(el.list) nowTime=now.strftime("%H:%M") #print("<p>at start of makeHomePage, el={}".format(el)) page='<p>Current events are:</p>\n' page+=' <table style="margin-left:40pt;" border="4pt solid green">\n' page+=' <tr><th>WeekDay (Sun=0)</th><th>Time</th><th>Device</th><th>Operation</th>' addstr='Add<br/>After' if numEvents==0: addstr='<form action="http://{}/~ajh/cgi-bin/" method="post">'.format(CENTRAL) addstr+='<button type="submit" value="0">Add</button>' addstr+='</form>' page+=' <th>Save<br/>Changes</th><th>{}</th><th>Delete this entry</th>'.format(addstr) page+=' </tr>\n' i=0 ; doneNow=False for ev in el.list: col='' if new and new==i: col=' bgcolor="green"' line=' <tr{}>\n'.format(col) if ev.time and not doneNow and nowTime < ev.time: line+='<td colspan="2" align="center" bgcolor="pink">NOW</td>' line+='<td>{}</td>'.format(nowTime) line+='<td colspan="5" align="center" bgcolor="pink">NOW</td></tr><tr>\n' doneNow=True month=ev.month if not month: month='*' line+=' <form action="http://{}/~ajh/cgi-bin/{}" method="post">\n'.format(CENTRAL,i) day=ev.weekday if not day: day='*' line+=' <td align="center"><input type="text" name="day" value="{}"></input></td>\n'.format(day) time=ev.time line+=' <td halign="center"><input align="center" type="text" name="time" value="{}"></input></td>\n'.format(time) device=ev.device line+=' <td align="center"><input type="text" name="device" value="{}"></input></td>\n'.format(device) operation=ev.operation line+=' <td align="center">' line+=' <input type="text" name="operation" value="{}"></input>'.format(operation) line+=' </td>\n' # Enter column line+=' <td align="center">\n' line+=' <button type="submit" value="{}">Enter</button></td>\n'.format(i) line+=' </td>\n' line+=' </form>\n' # Add column line+=' <form action="http://{}/~ajh/cgi-bin/{}" method="post">\n'.format(CENTRAL,i) line+=' <td align="center" valign="middle">\n' line+=' <button type="submit" value="{}">Add</button>\n'.format(i) line+=' </td>\n' line+=' </form>\n' # Delete column line+=' <form action="http://{}/~ajh/cgi-bin/{}" method="post">\n'.format(CENTRAL,i) line+=' <td align="center" style="valign:bottom">\n' line+=' <button type="submit" value="{}">Delete</button>\n'.format(i) line+=' </td>\n' line+=' </form>\n' line+=' </tr>\n' page+="{}".format(line) i+=1 if not doneNow: page+='<td colspan="8" align="center" bgcolor="pink">NOW</td></tr><tr>\n' page+=' </table>\n' page+='</p>\n' page+='<p><a href="http://{}/~ajh/cgi-bin/">(Return to HouseMade)</a></p>\n'.format(CENTRAL) return page
Chunk referenced in 9.1

9.4 EventEditor: Make Edit Page

<EventEditor: define make edit page routine 9.7> =
def makeEditPage(entry,time=None,weekday=None,device=None,operation=None): el=getAllEvents() if entry>= len(el.list): ev=Event() else: ev=el.list[entry] es.remove(entry) ev.weekday=weekday ev.time=time ev.device=device ev.operation=operation es.add(ev,True) logMsg("eventEditor returns normally") return makeHomePage()
Chunk referenced in 9.1

10. The Chook Door

This section is being rebuilt, and obsoletes the old section of the same name. The philosophy used is quite different, and is based on an event handling model. This module is the first to be rebuilt according to this model.

This code has been migrated from HouseMade and slightly modified to now work with the EventServer model. The original has been changed to ChookDoor to avoid confusion. Currently (version 1.0.4) the interface does not handle sunrise and sunset times.

The chook house (described separately in the Chickens page) has a door that is automatically controlled, and opens and shuts in accordance with sunrise and sunset times throughout the year.

The EventScheduler module has the responsibility for driving the chook door events. Each module requiring action at a particular time needs to register with the event scheduler with a request to be alerted when a particular time is reached. This may be a time of day, or every minute, or every hour, or even a day of the week.

This ChookDoormodule just acts as a passive interface, providing the necessary code to actually perform the chook door operation. The are two base level operations: openDoor and closeDoor, with obvious meanings. They communicate by means of a socket connection on port 9999 to the relay server, accessed through the generic interface serverSend.

A third operation chookDoor(p) uses a parameter p to indicate whether an open or close operation is required. Either of 'open'/1 can be used to invoke the opening, or 'close'/'shut'/0 to close the door.

A fourth operation, doorState, can be used to interrogate the current door status.

The routine handleEvent, is the generic interface to the EventScheduler, and bundles all the driver routines.

"" 10.1 =
import socket import sys import time #import currentState import datetime #import eventManager import getopt import re import sys from HouseDefinitions import * from suntime import Sun, SunTimeException class BadChook(BaseException): pass compute=0 version='1.0.0' latitude = -37.8732083333333 # for Glen Waverley longitude = 145.164671666667 chookFileName='/home/ajh/Computers/House/suntimes.txt' timezone=datetime.timezone(datetime.timedelta(hours=10)) <ChookDoor: misc routines 10.2> <ChookDoor: class ChookDoor 10.3> <ChookDoor: main 10.15> if __name__ == '__main__': (vals,path)=getopt.getopt(sys.argv[1:],'cVn=', ['compute','now=','version']) for (opt,val) in vals: if opt=='-c' or opt=='--compute': compute=1 if opt=='-n' or opt=='--now': now=datetime.datetime.strptime(val,"%Y%m%d:%H%M") if opt=='-V' or opt=='--version': print(version) sys.exit(0) main()

10.1 ChookDoor: misc routines

<ChookDoor: misc routines 10.2> =
def parse(pat,line): res=re.match(pat,line) if res: return else: return ''
Chunk referenced in 10.1

10.2 ChookDoor: class ChookDoor

<ChookDoor: class ChookDoor 10.3> =
class ChookDoor(): <class ChookDoor: init 10.4> <class ChookDoor: load 10.5> <class ChookDoor: compute 10.6> <class ChookDoor: save 10.7> <class ChookDoor: openDoor 10.8> <class ChookDoor: closeDoor 10.9> <class ChookDoor: chookDoor 10.10> <class ChookDoor: doorState 10.11> <class ChookDoor: handleEvent 10.12> <class ChookDoor: run 10.13> <class ChookDoor: stop 10.14>
Chunk referenced in 10.1

10.2.1 class ChookDoor: init

<class ChookDoor: init 10.4> =
def __init__(self): self.debug=False self.lastDoorState='unknown' self.opendelay=120 self.shutdelay=20 self.lastrun='' self.current='open' self.compute() # update the door times
Chunk referenced in 10.3

10.2.2 class ChookDoor: load

<class ChookDoor: load 10.5> =
def load(self): opendelay=0; shutdelay=0 try: suntimefile=open(chookFileName,'r') innow=suntimefile.readline() self.lastrun=parse('now += (.*)$',innow) inopdel=suntimefile.readline() opendelay=parse('opendelay += (.*)$',inopdel) inshdel=suntimefile.readline() shutdelay=parse('shutdelay += (.*)$',inshdel) inrise=suntimefile.readline() self.sunrise=parse('sunrise += (.*)$',inrise) inset=suntimefile.readline() self.sunset=parse('sunset += (.*)$',inset) inopen=suntimefile.readline() self.dooropen=parse('dooropen += (.*)$',inopen) inshut=suntimefile.readline() self.doorshut=parse('doorshut += (.*)$',inshut) incurrent=suntimefile.readline() self.current=parse('door is +(.*)$',incurrent) suntimefile.close() except IOError: pass pass
Chunk referenced in 10.3

10.2.3 class ChookDoor: compute

<class ChookDoor: compute 10.6> =
def compute(self): yesterday=today-datetime.timedelta(days=1) # NEW sunrise/set calculation sun = Sun(latitude, longitude) # Get today's sunrise and sunset in UTC # have to actually compute yesterday's sunrise, as suntime gets the date wrong! sunrise = sun.get_local_sunrise_time(yesterday) sunset = sun.get_local_sunset_time(today) dooropen=sunrise+datetime.timedelta(0,0,0,0,int(self.opendelay)) doorshut=sunset+datetime.timedelta(0,0,0,0,int(self.shutdelay)) self.dooropen=dooropen.strftime("%H:%M") self.doorshut=doorshut.strftime("%H:%M") self.sunrise=sunrise.strftime("%H:%M") self.sunset=sunset.strftime("%H:%M") if self.whichsrday='today' else: self.whichsrday='tomorrow' if self.whichssday='today' else: self.whichssday='tomorrow'
Chunk referenced in 10.3

10.2.4 class ChookDoor: save

<class ChookDoor: save 10.7> =
def save(self): suntimefile=open(chookFileName,'w') suntimefile.write("now = %s\n" % ("%Y%m%d:%H%M"))) # now suntimefile.write("opendelay = %s\n" % (self.opendelay)) # opendelay suntimefile.write("shutdelay = %s\n" % (self.shutdelay)) # shutdelay suntimefile.write("sunrise = %s\n" % (self.sunrise)) # sunrise suntimefile.write("sunset = %s\n" % (self.sunset)) # sunset suntimefile.write("dooropen = %s\n" % (self.dooropen)) # dooropen suntimefile.write("doorshut = %s\n" % (self.doorshut)) # doorshut suntimefile.write("door is %s\n" % (self.current)) # current state suntimefile.close() pass
Chunk referenced in 10.3

10.2.5 class ChookDoor: Open Chook Door

<class ChookDoor: openDoor 10.8> =
def openDoor(self): RelayServer.setBitOn(RelayTable['ChookUp']) time.sleep(2) RelayServer.setBitOff(RelayTable['ChookUp']) self.lastDoorState='close' logMsg("ChookDoor has been opened")
Chunk referenced in 10.3

10.2.6 class ChookDoor: Close Chook Door

<class ChookDoor: closeDoor 10.9> =
def closeDoor(self): RelayServer.setBitOn(RelayTable['ChookDown']) time.sleep(2) RelayServer.setBitOff(RelayTable['ChookDown']) self.lastDoorState='open' logMsg("ChookDoor has been closed")
Chunk referenced in 10.3

10.2.7 class ChookDoor: chookDoor

<class ChookDoor: chookDoor 10.10> =
def chookDoor(self,p): if str(p) in ['open','up','1']: self.openDoor() elif str(p) in ['close','shut','down','0']: self.closeDoor() else: raise(BadChook)
Chunk referenced in 10.3

10.2.8 class ChookDoor: doorState

<class ChookDoor: doorState 10.11> =
def doorState(self): r=RelayServer.readDoor() return r
Chunk referenced in 10.3

10.2.9 class ChookDoor: handleEvent

<class ChookDoor: handleEvent 10.12> =
def handleEvent(self,parms): ps=parms.split(' ') device=ps[0] onoff=ps[1] try: self.chookDoor(onoff) except: raise(BadChook)
Chunk referenced in 10.3

10.2.10 class ChookDoor: run

<class ChookDoor: run 10.13> =
def run(self,em,debug): self.debug=debug self.load() self.compute() # strip open and close times to hours:minutes res=re.match('(\d{2}):(\d{2})',self.dooropen)':' res=re.match('(\d{2}):(\d{2})',self.doorshut)':' openev=('*',op,'chookdoor','open',self.handleEvent) em.registerEvent(openev,self.handleEvent) shutev=('*',sh,'chookdoor','close',self.handleEvent) em.registerEvent(shutev,self.handleEvent)
Chunk referenced in 10.3

The run routine serves only to create the two main chook door events, opening and closing, registered with the event manager em passed as a parameter. It is intended to be called from the event manager at the start of each day.

10.2.11 class ChookDoor: stop

<class ChookDoor: stop 10.14> =
def stop(self): # just print a message for now logMsg("ChookDoor handler now terminating") print("ChookDoor handler now terminating")
Chunk referenced in 10.3

The stop method is called when all events for the day have been completed. This gives a chance for any event driven module to clean up and save any relevant information before the EventManager terminates and no control thread exists any more.

10.3 ChookDoor: main

<ChookDoor: main 10.15> =
def main(): print("Running ChookDoor.main") chooks=ChookDoor() chooks.load() chooks.compute() print("chooks lastrun = %s" % (chooks.lastrun)) print("now = %s" % (now.strftime("%Y%m%d:%H%M"))) print("opendelay = %s" % (chooks.opendelay)) print("shutdelay = %s" % (chooks.shutdelay)) print("today sunrise = %s" % (chooks.sunrise)) print("today sunset = %s" % (chooks.sunset)) print("dooropen = %s" % (chooks.dooropen)) print("doorshut = %s" % (chooks.doorshut)) print("chook door is %s" % (chooks.current))
Chunk referenced in 10.1

11. The Garden Steps Lighting

This module follows the same model as the (revised) ChookDoor module, namely that it defines GardenSteps.handleEvent method that interfaces to the EventScheduler.

"" 11.1 =
#!/home/ajh/binln/python3 from HouseDefinitions import * from suntime import Sun, SunTimeException import datetime class GardenSteps(): def __init__(self): self.debug=0 self.ondelay=5 # delay from sunset in minutes self.offdelay=300 # 5 hours of Garden Steps lights, if no manual entry self.compute() def compute(self): timezone=datetime.timezone(datetime.timedelta(hours=10)) yesterday=today-datetime.timedelta(days=1) # NEW sunrise/set calculation sun = Sun(latitude, longitude) # GardenSteps does not need sunrise time sunset = sun.get_local_sunset_time(today) ontm=sunset+datetime.timedelta(0,0,0,0,int(self.ondelay)) offtm=sunset+datetime.timedelta(0,0,0,0,int(self.offdelay)) # compute desired on and off times self.onTime=ontm.strftime("%H:%M") self.offTime=offtm.strftime("%H:%M") # user specified self.Auto=False def switchOn(self,timer=0): if not self.debug: logMsg("Garden Steps lights switched on") if timer>0: RelayServer.setBitOn(RelayTable['GardenSteps'],timer) else: RelayServer.setBitOn(RelayTable['GardenSteps']) else: print("(debug) Garden Steps lights are switched on") return def switchOff(self): if not self.debug: logMsg("Garden Steps lights switched off") RelayServer.setBitOff(RelayTable['GardenSteps']) else: print("(debug) Garden Steps lights are switched off") return def switch(self,onoff): if onoff: self.switchOn() else: self.switchOff() return def handleEvent(self,parms): ps=parms.split(' ') device=ps[0] onoff=ps[1] if onoff in ['1','on']: self.switchOn() elif onoff in ['0','off']: self.switchOff() def run(self,em,debug): self.debug=debug # register these events onEv=('*',self.onTime,'gardensteps','on',None) em.registerEvent(onEv,self.handleEvent) offEv=('*',self.offTime,'gardensteps','off',None) em.registerEvent(offEv,self.handleEvent) # save the on and off times for the web page stepsTimesf=open('/home/ajh/Computers/House/stepsTimes.txt','w') stepsTimesf.write("{} (On Time)\n".format(self.onTime)) stepsTimesf.write("{} (Off Time)\n".format(self.offTime)) stepsTimesf.close() return def stop(step): logMsg("GardenSteps handler now terminating") # not much required as of yet return

12. The Garden Watering System

This module follows the same model as the (revised) ChookDoor module, namely that it defines GardenWater.handleEvent method that interfaces to the EventScheduler.

A major difference with the GardenSteps module, however, is that there is more than one sprinkler. Hence the sprinkler involved has to be passed as a parameter to the event handling routine, and the on/off routines. There are two sprinklers: the SouthVegBed and the NorthVegBed. The dispatcher in the EventScheduler defines a handler for each distinct water section, even though in each case the handler is the same (GardenWater.handleEvent). The distinction between the different watering sections is through the device name, passed as a parameter through the handleEvent methos.

"" 12.1 =
#!/home/ajh/binln/python3 from HouseDefinitions import * from suntime import Sun, SunTimeException import datetime import re class GardenWater(): def __init__(self): self.debug=0 self.onTime='09:00' self.offTime='09:15' def switchOn(self,sprinkler): if not self.debug: logMsg("Garden Water Sprinkler {} turned on".format(sprinkler)) RelayServer.setBitOn(RelayTable[sprinkler]) else: print("(debug) Garden Water Sprinkler {} turned on".format(sprinkler)) return def switchOff(self,sprinkler): if not self.debug: logMsg("Garden Water Sprinkler {} turned off".format(sprinkler)) RelayServer.setBitOff(RelayTable[sprinkler]) else: print("(debug) Garden Water Sprinkler {} turned off".format(sprinkler)) return def switch(self,onoff,sprinkler): if onoff: self.switchOn(sprinkler) else: self.switchOff(sprinkler) return def handleEvent(self,parms): logMsg("Garden Watering parameters are {}".format(parms)) ps=parms.split(' ') device=ps[0] onoff=ps[1] if onoff in ['1','on']: self.switchOn(device) elif onoff in ['0','off']: self.switchOff(device) <GardenRun 12.2> def stop(step): logMsg("GardenWater handler now terminating") # not much required as of yet return

12.1 Garden Run

<GardenRun 12.2> =
def run(self,em,debug): self.debug=debug # register these events onEv=('*',self.onTime,'gardenwater','SouthVegBed on',None) em.registerEvent(onEv,self.handleEvent) offEv=('*',self.offTime,'gardenwater','SouthVegBed off',None) em.registerEvent(offEv,self.handleEvent) # register these events onEv=('*',self.onTime,'gardenwater','NorthVegBed on',None) em.registerEvent(onEv,self.handleEvent) offEv=('*',self.offTime,'gardenwater','NorthVegBed off',None) em.registerEvent(offEv,self.handleEvent) # save the on and off times for the web page waterTimesf=open('/home/ajh/Computers/House/waterTimes.txt','w') waterTimesf.write("{} (On Time for {})\n".format(self.onTime,'SouthVegBed')) waterTimesf.write("{} (Off Time for {})\n".format(self.offTime,'SouthVegBed')) waterTimesf.write("{} (On Time for {})\n".format(self.onTime,'NorthVegBed')) waterTimesf.write("{} (Off Time for {})\n".format(self.offTime,'NorthVegBed')) waterTimesf.close() return
Chunk referenced in 12.1

The run routine registers the various event handlers for the Garden Watering System. At the moment, some default entries for on and off events are registered, although it is not clear that these are necessary for the proper functioning of the module, since the registerEvent routine should be called by the event registering process at the startup of the EventScheduler.

At the moment, it is a case of "if it is working, don't change it".

13. The Ring Main Relay Handler

This module follows the same model as the GardenSteps method that interfaces to the EventScheduler. It is provided to interface the ring main to the automatic system.

"" 13.1 =
#!/home/ajh/binln/python3 from HouseDefinitions import * import datetime class RingMain(): def __init__(self): self.debug=0 self.ondelay=0 self.offdelay=0 def switchOn(self): if not self.debug: logMsg("Ring Main relay switched on at {}".format(now)) RelayServer.setBitOn(RelayTable['RingMain']) else: print("(debug) Ring Main relay is switched on") return def switchOff(self): if not self.debug: logMsg("Ring Main relay switched off at {}".format(now)) RelayServer.setBitOff(RelayTable['RingMain']) else: print("(debug) RingMain relay is switched off") return def switch(self,onoff): if onoff: self.switchOn() else: self.switchOff() return def handleEvent(self,parms): logMsg("RingMain parameters are '{}'".format(parms)) ps=parms.split(' ') device=ps[0] onoff=ps[1] if onoff in ['1','on']: self.switchOn() elif onoff in ['0','off']: self.switchOff() else: logMsg("bad parameter {} to Ring Main handler".format(parms)) def run(self,em,debug): self.debug=debug # no events to register return def stop(step): logMsg("Ring Main handler now terminating") # not much required as of yet return

See section Event Scheduler for details of how this module is used.

14. The Spare Relay Handler

This module follows the same model as the GardenSteps method that interfaces to the EventScheduler. It is provided so that all the spare relays identified may be tested.

"" 14.1 =
#!/home/ajh/binln/python3 from HouseDefinitions import * import datetime class SpareRelay(): def __init__(self): self.debug=0 self.ondelay=0 self.offdelay=0 def switchOn(self,spare): if spare not in RelayTable: print("no such spare relay {}".format(spare)) return if not self.debug: print("Spare relay {} switched on at {}".format(spare,now)) RelayServer.setBitOn(RelayTable[spare]) else: print("(debug) Spare relay {} is switched on".format(spare)) return def switchOff(self,spare): if spare not in RelayTable: print("no such spare relay {}".format(spare)) return if not self.debug: print("Spare relay {} switched off at {}".format(spare,now)) RelayServer.setBitOff(RelayTable[spare]) else: print("(debug) Spare relay {} is switched off".format(spare)) return def switch(self,onoff,spare): if spare not in RelayTable: print("no such spare relay {}".format(spare)) return if onoff: self.switchOn(spare) else: self.switchOff(spare) return def handleEvent(self,parms): ps=parms.split(' ') spare=ps[0] onoff=ps[1] if ',' in ops: ops2=ops.split(',') spare+=ops2[0] onoff=ops2[1] else: print("bad parameters in {}".format(ops)) print("ops={},spare={},onoff={}".format(ops,spare,onoff)) if spare in RelayTable: if onoff in ['1','on']: self.switchOn(spare) elif onoff in ['0','off']: self.switchOff(spare) else: print("no such spare relay {}".format(spare)) def run(self,em,spare,debug): if spare not in RelayTable: print("no such spare relay {}".format(spare)) return self.debug=debug # register these events onEv=('*',self.onTime,'Spare','on',None) em.registerEvent(onEv,self.handleEvent) offEv=('*',self.offTime,'Spare','off',None) em.registerEvent(offEv,self.handleEvent) return def stop(step): print("Spare relay handler now terminating") # not much required as of yet return

15. The Web Interface

The web interface is a cgi application running on the house computer reuilly, and providing a conventional web page via a port 80 call (the http port number). This web page provides mechanisms to control the house relays manually, and to schedule events that control them automatically.

To preserve security of the system, and prevent unauthorized access, this web server will only operate house functions if it is invoked from a machine on the 10.0.0 network (the private house network). In the longer term, username/password authority may be added.

15.1 The cgi application

"" 15.1 =
#!/home/ajh/binln/python3 import sys sys.path.append('/home/ajh/Computers/House') import cgi import datetime import HouseMade import os import re import urllib.request # sanitize copies the requested text, but replaces all live references # to form requests with an impotent domain name def sanitize(url): h=urllib.request.urlopen(url) print("Content-type: text/html\n\n")'utf-8') lines=lines.split('\n') for l in lines: l=re.sub('reuilly','gotohell',l.strip()) print(l) sys.exit(0) server=os.environ['SERVER_NAME'] form=cgi.FieldStorage() # here check if special treatment is required if server=='': if 'pw' in form: password=form['pw'].value if password!='Jemima2014': sanitize("http://reuilly/~ajh/cgi-bin/") elif 'li' in form: # special case to turn garden lights on for 5 mins steps.switchOn(timer=300) else: sanitize("http://reuilly/~ajh/cgi-bin/") # end of special case for server '' nowstr=now.strftime("%Y%m%d:%H%M") import cgitb cgitb.enable() print("Content-type: text/html\n\n") remadr=os.environ['REMOTE_ADDR'] #print("%s@%s: house arguments=%s" % (server,remadr,form)) #print(os.environ),server,form) print(page)

This cgi script simply collects a few parameters, and then passes control to the house interface in the HouseMade module. The latter module has the responsibility of generating the web page, which is returned as a string to be printed (rendered) by this cgi script.

15.2 The HouseMade module

"" 15.2 =
#!/usr/bin/python ## H o u s e M a d e . p y ## ## ********************************************************** ## * do NOT EDIT THIS FILE! * ## * Use $HOME/Computers/House/HouseMade.xlp instead * ## ********************************************************** # this is where to find any programs invoked in this module import sys sys.path.append('/home/ajh/Computers/House/') sys.path.append('/home/ajh/Computers/Sources/Solar/') import ChookDoor import datetime import EnergyChart import GardenSteps from HeatingModule import * import cgi,math,string import os import re import subprocess import time import HouseDefinitions from HouseDefinitions import \ CENTRAL,EventServer,isDay,logMsg,MServer,NumberOfRelays,\ RelayNames,RelayServer,RelayTable,ThermostatSetting,ThisServer, \ setTemperature,getTemperature import urllib.request <HouseMade: define the Generate Solar Data routine 15.10> <HouseMade: define the house interface 15.3> if __name__=='__main__': house() ## ## The End ##

This module defines the key elements for the HouseMade system. They are the various subsections that concern the data generating functions: solar, weather, tank, and heating (although some of these are currently decommissioned). Each of these is accessed through a specific routine, called by the last component, the routine, which is the web interface called as a cgi script.

15.2.1 The house interface

<HouseMade: define the house interface 15.3> =
def house(remadr,server,args): import os DEBUG=False limited=False # set True when only limited operations allowed # sanitize copies the requested text, but replaces all live references # to form requests with an impotent domain name def sanitize(url): h=urllib.request.urlopen(url) #print("Content-type: text/html\n\n")'utf-8') lines=lines.split('\n') for l in lines: l=re.sub('reuilly','gotohell',l.strip()) print(l) sys.exit(0) # these statements provide timing information for the various # phases of the web page generation timeZero=time.clock_gettime(time.CLOCK_MONOTONIC) lasttime=timeZero def timing(loc): nonlocal lasttime tnow=time.clock_gettime(time.CLOCK_MONOTONIC) totalt=tnow-timeZero incrt=tnow-lasttime lasttime=tnow #sys.stderr.write(f'{loc:20} {totalt:10} ({incrt})\n') timing('start') if args.keys(): logMsg("house call, parms: remadr={}, server={}, args={}".format(remadr,server,args)) <HouseMade: collect date and time data 15.12> <HouseMade: check client connection 15.13> timing('local info') ################################################## LOCAL INFORMATION ######### # Now get and display some local information. <HouseMade: get local information 15.4> # localinfo is a string containing local information timing('relay info') ################################################## RELAY INFORMATION ######### # Now get and display the relay state information. # determine what relays are currently switched on <HouseMade: get relay information 15.5> # relayStateStr is a string containing the relay state information timing('events info') ################################################## EVENTS INFORMATION ######## # Get the list of events from the Event Server and display them <HouseMade: get events information 15.6> # eventsInfo is a string containing the events information ################################################## OTHER INFORMATION ######### # # From here on is fairly irrelevant at the moment, and is here only # as legacy code. It will be tidied up in due course. <HouseMade: legacy code for 15.7> # Currently no useful information returned # reload the page every second if a timer is active, otherwise only every minute if active: redirect="1;URL='%s'" % (MServer) else: redirect="60" timing('house page') {Note 15.3.1} serverlink=MServer thisserver=ThisServer # same problem other='newport' if server in ['newport','newport.local','localhost']: other='reuilly.local' elif server in ['reuilly','reuilly.local']: other='newport.local' housepage=<HouseMade: generate the web page content 15.14> timing('end') return housepage
Chunk referenced in 15.2
{Note 15.3.1}
these idempotent assignments are because MServer/ThisServer are not local variables, and we have to get their content into local variables for use in the next 'u' link!

The house routine has the responsibility of generating the HouseMade web page. There are currently four key sections: local, relay, events, and other. (The latter covers legacy software that may at some stage be re-instated.) Each of these is handled by a code section that returns a variable populated with web-page ready XML code. These are then assembled by the final assignment to the variable housepage, which is returned as the value of the procedure, passed back to the module, and from there accessed by the actual cgi script (q.v.).

15.2.2 Get Local Information

<HouseMade: get local information 15.4> =
# Firstly, chook relevant times import ChookDoor chooks=ChookDoor.ChookDoor() chooks.compute() #chooks.load() localrise=chooks.sunrise localset=chooks.sunset gateopentime=chooks.dooropen gateshuttime=chooks.doorshut steps=GardenSteps.GardenSteps() stepsOn=steps.onTime #stepsOff=steps.offTime # now user specified dayOpen=chooks.whichsrday.capitalize() dayShut=chooks.whichssday import GardenWater water=GardenWater.GardenWater() waterOn=water.onTime waterOff=water.offTime localinfo=''
Chunk referenced in 15.3

15.2.3 Get Relay Information

<HouseMade: get relay information 15.5> =
RelayState=HouseDefinitions.RelayServer.getState() # here process any switching requests # respond to any argument requests - can only do if RPC server present timing('relay state') active=False if args and HouseDefinitions.RelayServerGood: currState=HouseDefinitions.RelayServer.getState() for relay in HouseDefinitions.RelayNames: if relay in args: active=True bitNo=HouseDefinitions.RelayTable[relay] newState=args[relay].value if newState in ['off','on']: doWhat={'off':HouseDefinitions.RelayServer.setBitOff,'on':HouseDefinitions.RelayServer.setBitOn} change=doWhat[newState](bitNo) logMsg("change bit %d(%s) to %s" % (bitNo,relay,newState)) if newState=='off': # reset the timer as well HouseDefinitions.RelayServer.resetTimer(bitNo) else: # start timer with time == newstate timerCount=int(newState) HouseDefinitions.RelayServer.start(bitNo,timerCount) logMsg("timer started for bit %d(%s) for %d" % (bitNo,relay,timerCount)) timing(f'loop, relay no={relay}') elif limited: # limited operations currently only turning lights on print("<H2>Lights On!</H2>\n") HouseDefinitions.RelayServer.start(RelayTable['GardenSteps'],600) # and that is all that can be done sys.exit(0) # just to confirm any changes RelayState=HouseDefinitions.RelayServer.getState() #print(RelayState);time.sleep(5) chookdoorlabel=chooks.doorState() # just to confirm any changes RelayState=HouseDefinitions.RelayServer.getState() currentcircuits=[] for i in range(NumberOfRelays): if RelayState[i]: currentcircuits.append(RelayNames[i]) if len(currentcircuits) > 1: currentcircuits = "the " + ', '.join(currentcircuits[:-1]) + " and " + currentcircuits[-1] elif len(currentcircuits) > 0: currentcircuits = "the " + currentcircuits[0] else: currentcircuits = "no" relayStateStr=''' <p> Currently %(currentcircuits)s circuits are on. </p> ''' % vars()
Chunk referenced in 15.3

The relay information is currently provider by the server running on kerang, which at the moment only relates to the chook door opening and closing. This will be made more generic in the near future.

15.2.4 Get Events Information

<HouseMade: get events information 15.6> =
eventsInfo='' i=0; evlist=[] while True: ev=EventServer.getEvent(i) if ev: evlist.append(ev);i+=1 else: break if i>0: nowTime=now.strftime("%H:%M") day=now.isoweekday() % 7 # Sunday is first day (0) of week name=['Sun','Mon','Tue','Wed','Thu','Fri','Sat'][day] eventsInfo+=f'<h3>Scheduled events for today (day {day}, {name})</h3>\n' eventsInfo+='<table>\n' nextHead=f'</table><h3>{nowTime} Now</h3><table>' noPrevs=True for e in evlist: eday=e['weekday'] if isDay(day,eday): if e['time']>nowTime: if noPrevs: eventsInfo+='<p>None</p>\n' noPrevs=False eventsInfo+=nextHead nextHead='' else: noPrevs=False eventsInfo+='<tr>\n' eventsInfo+='<td width="100px">{}</td>'.format(e['weekday']) eventsInfo+='<td width="100px">{}</td>'.format(e['time']) eventsInfo+='<td width="200px">{}</td>'.format(e['device']) eventsInfo+='<td width="100px">{}</td>'.format(e['operation']) eventsInfo+='</tr>\n' eventsInfo+='<table>\n' else: eventsInfo += '<p>No events found</p>\n' # end of generating event list eventsInfo += '<p>(<a href="http://reuilly.local/~ajh/cgi-bin/">' eventsInfo+='Edit Events</a>)' eventsInfo+=' (<a href="http://reuilly.local/~ajh/cgi-bin/">' eventsInfo+='Save Events</a>)</p>\n'
Chunk referenced in 15.3

15.2.5 Legacy Code

<HouseMade: legacy code for 15.7> =
#(aimtemp,onoff) = (ThermostatSetting,'off') #garedelyon.getHeating() #res=0 #onoffcolor='blue' # legacy code - will be reinstated some day #if onoff=='on': onoffcolor='red' import os,string timing('temperature') ################### # make the adjust temperature button panel <house make temperature panel 15.15> timing('relay control') ################### Relay Control <HouseMade: Relay Control 15.8> # the following are null, as the respective sections are commented out ################### Heating Section heatingsection=adjustPanel ################### Water Storage tanksection='' # tank([]) ################### Solar Power timing('solar start') solarsection=solar(now) timing('solar end') ################### Climate weathersection='' # weather([]) ###################
Chunk referenced in 15.3

15.2.6 The Relay Information section

<HouseMade: Relay Control 15.8> =
RelayServerGood=True if RelayServerGood: row="<tr><th>Name</th><th>bit No</th><th>On/Off</th>" row+="<th>Timer</th><th colspan='8'>Run For</th></tr>\n" ################### Chook Door chookmode='' ; chookdooractive=False if chookdoorlabel == 'closed': chookdoorcolour="lime" chookdooractive=False elif chookdoorlabel == 'open': chookdoorcolour="orangered" chookdooractive=False elif chookdoorlabel == 'closing': chookdoorcolour="greenyellow" chookmode='style="fade"' chookdooractive=True elif chookdoorlabel == 'opening': chookdoorcolour="pink" chookmode='style="fade"' chookdooractive=True else: chookdoorcolour="grey" row += '<tr bgcolor="%s">\n' % (chookdoorcolour) row += ' <td>ChookDoor</td>' row += '<td colspan="11" align="center" %s>%s</td>\n' % (chookmode,chookdoorlabel) row += '</tr>\n' timing('relay table') for key in sorted(RelayTable, key=RelayTable.get): bitNo=RelayTable[key] thisBit=RelayState[bitNo] thisState=['off','on'][thisBit] newState=['on','off'][thisBit] thisColour=['lightblue','red'][thisBit] row+=' <form action="http://{}/~ajh/cgi-bin/" method="post">\n'.format(CENTRAL) row += ' <tr bgcolor="%s">\n' % (thisColour) row += ' <td>%s</td>\n' % (key) row += ' <td bgcolor="%s">%s</td>\n' % (thisColour,bitNo) # bit number row += ' <td bgcolor="%s">\n' % (thisColour) row += ' <input type="hidden" name="state" value="{}"/>\n'.format(newState) row += ' <input type="hidden" name="relay" value="{}"/>\n'.format(bitNo) row += ' <button name="%s" value="%s" type="submit">\n' % (key,newState) row += ' %s\n </td>\n' % (thisState) timeLeft=RelayServer.getTimer(bitNo) if timeLeft>0: active=True row += ' <td bgcolor="%s" width="50px">%d</td>\n' % (thisColour,timeLeft) for t in [30,60,120,300,600,1200,1800,3600]: if t<60: if key[0:5]=='Chook': t=45 buttontxt="%d secs" % (t) elif t>=3600: buttontxt="%d hour" % (t/3600) else: buttontxt="%d min" % (t/60) t1=t row += ' <td><button name="%s" value="%d" type="submit">%s</td>\n' % (key,t1,buttontxt) row += " </tr>\n" row += " </form>\n" timing('end relay table') ######## end of building RelayTable form entries relaycontrol=""" <table border="1"> %(row)s </table> """ % vars() else: relaycontrol='<p>No relay information available</p>'
Chunk referenced in 15.7

15.2.7 define the Generate Weather Data routine

<HouseMade: define the Generate Weather Data routine 15.9> =
def weather(args): WServer='http://%s:5000/weather' % (CENTRAL) MServer='http://%s/~ajh/cgi-bin/' % (CENTRAL) MAXMINFILE=LOGDIR+'/maxmintemps.log' curtemp = w.inside.temp curhumid = w.inside.humidity outtemp = w.outside.temp outhumid = w.outside.humidity yesterday=starttime-datetime.timedelta(days=1) yesterday=yesterday.strftime("%Y%m%d") today=starttime.strftime("%Y%m%d") # check maximum and minimum maxminpat='(\d\d\d\d\d\d\d\d)' # date only maxminpat+=' +([0-9.]+)' # maximum temp maxminpat+=' +([0-9:]+)' # maximum temp time maxminpat+=' +([0-9.]+)' # minimum temp maxminpat+=' +([0-9:]+)' # minimum temp time maxminpat=re.compile(maxminpat) f=open(MAXMINFILE,'r') maxmintable={} for l in f.readlines(): #print("read maxmin line of %s" % (l)) res=maxminpat.match(l) if res: max=float( min=float( maxmintable[d]=(max,maxat,min,minat) else: print("cannot parse %s" % (l)) f.close() if maxmintable.has_key(yesterday): (yestermax,yestermaxat,yestermin,yesterminat)=maxmintable[yesterday] else: print("<P>Min/Max temperatures not available for yesterday</P>\n") (yestermax,yestermaxat,yestermin,yesterminat)=(0.0, '00:00', 0.0, '00:00') if maxmintable.has_key(today): (max,maxat,min,minat)=maxmintable[today] else: print("<P>Min/Max temperatures not available for today</P>\n") (max,maxat,min,minat)=(0.0, '00:00', 0.0, '00:00') # get desired temperature house=currentState.HouseState() house.load() #aimtemp=house.get('thermostat') onoffcolor="black" WServer='http://%s:5000/weather' % (CENTRAL) # WeatherServer weathersection=""" <h2><a href="%(WServer)s">Weather</a></h2> <image src="personal/tempplot.png"/> <form method="POST" action=""> <p> Outside it is %(outtemp).1fC and %(outhumid)d%% humid. It is currently %(curtemp).1fC and %(curhumid)d%% humid inside. <span style="color:%(onoffcolor)s">The heating is aiming for <input type="text" size="6" name="temp" value="%(aimtemp).1f"></input> C.</span> </p> </form> <p> Temperature Extremes on the outside: <table align="center" width="80%%"> <tr><th>Yesterday</th><th>Today</th></tr> <tr> <td align="center">maximum=%(yestermax)s at %(yestermaxat)s</td> <td align="center">maximum=%(max)s at %(maxat)s</td> </tr> <tr> <td align="center">minimum=%(yestermin)s at %(yesterminat)s</td> <td align="center">minimum=%(min)s at %(minat)s</td> </tr> </table> </p> <p><a href="MServer">Back to House</a></p> """ % vars() return weathersection

This routine collects up the climate/temperature/heating data and builds a web page to display it. The web page is returned as a string, allowing it to be directly called by the Flask module.

The maxminTemp() call will return an empty dictionary if it cannot find the maximum and minimum temperatures, so we need key ckecks to avoid run time errors.

15.2.8 define the Generate Solar Data routine

<HouseMade: define the Generate Solar Data routine 15.10> =
def solar(today): tim="{:4.2f}".format(today.hour+today.minute/60.0) hostname=os.environ['SERVER_NAME'] fname=today.strftime("%Y%m%d") path='/home/ajh/public_html/images/solar' imgFile=f'{path}/{fname}.jpg' #if hostname[0:7]!='reuilly': # imgFile=f'/lizard/ajh/{path}/{fname}.jpg' rtn='' try: sts=os.stat(imgFile) except FileNotFoundError: rtn=f"hostname={hostname},imgFile={imgFile}\n" rtn+=f'<h3>Current Solar Energy Data NOT FOUND</h3>\n' return rtn imglink=f'{today:%Y%m%d}.jpg' rtn+=f'<img src="{imglink}" width="100%"/>' mtime=time.localtime(sts.st_mtime) mktime=f"{mtime.tm_hour}:{mtime.tm_min:02}" datafile=open(f'/home/ajh/logs/maxsol/{fname}.log','r') rtn+=f'<h3>{mktime}: Current Solar Energy Data; \n' pat='(\-?[0-9.]+)' # 10 lots of optionally signed float numbers, comma separated datapat=9*f"{pat},"+pat res=re.match(datapat,dataline) if res: (maxb,tgen,tload,tgrid,bp,bc,p,f,l,minb)=res.groups() bc=float(bc) maxb=float(maxb); p=float(p); f=float(f); l=float(l) tgrid=float(tgrid); tgen=float(tgen); tload=float(tload) rtn+=f' battery: {bp}%@{bc:.3}kW, solar: {p:.3} kW, ' rtn+=f'grid: {-f:.3} kW, load: {l:.3} kW</h3>\n' rtn+=f'To Date: total grid consumption: {tgrid:.3} kWh, ' rtn+=f'total solar generation: {tgen:.3} kWh, ' rtn+=f'total household load: {tload:.3} kWh, ' rtn+=f'maximum battery charge = {maxb:.3}% ' rtn+=f'(minimum={minb:.4}%)\n' else: rtn+='Could not extract data from datafile\n' return rtn
Chunk referenced in 15.2

Collect the current day's solar power information for display and generate the related text.

the date (format YYYYMMDD)
the time (format HHMMSS)
battery percentage charge
battery charge (watts, +ve charging, -ve discharging)
Solar Edge panel generation (logged separately from Fronius)
Fronius panel generation (logged separately from Solar Edge)
total panel generation (sum of Sp + Fp)
feed in to grid (+ve export, -ve import)
Solar Edge load (as logged by Solar Edge software)
Fronius load (as logged by Fronius software)
total load (sum of Sl + Fl)
power balance (p-bc+f-l) - this should be zero
the minimum battery charge

15.2.9 define the Generate Tank Data routine

<HouseMade: define the Generate Tank Data section 15.11> =
def tank(args): TServer='http://%s:5000/tank' % (CENTRAL) MServer='http://%s:5000/house' % (CENTRAL) # get tank data tankvolume=0.0 # RelayServer.getTank() tanktemp=volts=0.0 tankfull=4500 tankpercent = (tankvolume/float(tankfull))*100 tanksection=""" <h2><a href="%(TServer)s">Tank Storage</a></h2> <image src="personal/tankplot.png"/> <p> The rain water tank is currently at %(tankvolume).1fl(/%(tankfull)dl) = %(tankpercent).1f%%. Check the 7 day graph: </p> <image src="personal/tankplot7.png"/> <p><a href="%(MServer)s">Back to House</a></p> """ % vars() return tanksection

This section now installed on lilydale.

15.2.10 Collect Date and Time Data

<HouseMade: collect date and time data 15.12> =
# collect date and time data (year, month, day, hour, minute, second, weekday, yday, DST) = time.localtime(time.time()) #tm = time.asctime(time.localtime(time.time())) + \ # ["", "(Daylight savings)"][DST] tm=tm.strftime("%a, %d %b %Y, %H:%M:%S") # compute fraction of a day now dayfrac=(60.0*now.hour+now.minute)/1440.0 #print(dayfrac) jobtime=str(now-starttime) # isoweek is Mon-Sun, 1-7, but want 0-origin starting with Sun # Sun Mon Tue Wed Thu Fri Sat # iso: 7 1 2 3 4 5 6 # 0-org 0 1 2 3 4 5 6 weekday=now.isoweekday() % 7
Chunk referenced in 15.3 15.16

The date and time at which this program is run is useful for logging, so collect it at the start of operations. The variable jobtime is intended to check how intensive use of this code may become.

15.2.11 Check the Client Connection

<HouseMade: check client connection 15.13> =
if 'SSH_CONNECTION' in os.environ: clientIP=os.environ['SSH_CONNECTION'] res=re.match('^(\d+\.\d+\.\d+\.\d+).*$',clientIP) if res: else: clientIP='' if DEBUG: print(os.environ) print(clientIP) clientIP='' res=re.match('10\.0',clientIP) # here check if special treatment is required if server=='': limited=True if 'pw' in args: sys.stderr.write("called with pw\n") password=args['pw'].value if password!='Jemima2014': sanitize("http://reuilly/~ajh/cgi-bin/") else: sys.stderr.write("password accepted\n") elif 'li' in args: sys.stderr.write("called with li\n") # special case to turn garden lights on for 10 mins HouseDefinitions.RelayServer.start(RelayTable['GardenSteps'],600) sys.stderr.write("Garden Steps turned on for 10 mins\n") else: sys.stderr.write("called with no parms\n") sanitize("http://reuilly/~ajh/cgi-bin/") # end of special case for server '' if not HouseDefinitions.RelayServerGood: print("<p>Cannot talk to the RelayServer - have you started it?</p>") sys.exit(1) # handle any special parameters if 'saveEvents' in args: EventServer.saveEvents() # just fall through to continue with HouseMade
Chunk referenced in 15.3

This page is intended to be world-wide-web accessible, and hence we must establish the credentials of the client. If it is on the local network, no problem, but external users must authenticate (username/passwd) before being allowed to alter any parameters. (Currently not implemented.)

The IP address is used in the first instance, as given by the environment variable 'SSH_CONNECTION'. If we can extract the 4-block IP address, well and good, otherwise make it the local mask. IP addresses on the local network (10.0.0.*) are allowed. All others pay money, and their names are taken.

15.2.12 Generate the Web Page Content

<HouseMade: generate the web page content 15.14> =
F""" <HTML> <HEAD> <LINK REL="SHORTCUT ICON" HREF="favicon.ico"> <meta http-equiv="Refresh" content="%(redirect)s"> <meta http-equiv="Pragma" content="no-cache"> <TITLE>HouseMade</TITLE> </HEAD> <BODY> <h1> <a href="%(serverlink)s"> HouseMade: the Hurst House Heater Helpmate on Reuilly </a> </h1> <p> HouseMade is running on %(thisserver)s (<a href="http://{other}/~ajh/cgi-bin/">Switch</a>), and thinks it is currently %(tm)s. You might want to see what rain is <a href=""> happening in melbourne</a>, or the <a href=""> local forecast</a> and <a href=""> temperatures</a>. </p> %(localinfo)s <table width="100%%"> <tr> <td> %(relaycontrol)s %(relayStateStr)s </td> <td> %(eventsInfo)s </td> </tr> <tr> %(heatingsection)s %(weathersection)s %(tanksection)s </tr> </table> </td> </tr> </table> %(solarsection)s </BODY> </HTML> """ % vars()
Chunk referenced in 15.3

This is where the framework of the web page is generated. Most of the content is generated elsewhere as strings to be inserted into this template, hence the global dictionary call on vars at the end, with variable content being added via %s formatting imperatives.

15.2.13 Make the Temperature Panel

<house make temperature panel 15.15> =
# make the adjust temperature button panel # this is dynamically constructed to show the current aiming temperature. buttonColours=['blue','#10e','#20d','#30c','#40b','#50a','#609','#708', '#807','#906','#a05','#b04','#c03','#d02','#e01','red'] tempdata=heatingData() # from HeatingModule tempdata.load() aimtemp=tempdata.getcurrtemp() aimIndex=math.trunc(aimtemp+0.5)-12 if aimtemp<=12.5: aimIndex=0 elif aimtemp>=22.5: aimIndex=11 buttonColours[aimIndex]='yellow' adjustPanel=f''' <p>aiming temperature is {aimtemp}.</p> <form action="{WServer}" method="post"> <td><button name="button" value="" type="submit">Submit</button></td> <td bgcolor="%s"> <button name="button" value="cooler" type="submit">COOLER</button> </td> <td bgcolor="%s"><button name="button" value="13" type="submit">13C</button></td> <td bgcolor="%s"><button name="button" value="14" type="submit">14C</button></td> <td bgcolor="%s"><button name="button" value="15" type="submit">15C</button></td> <td bgcolor="%s"><button name="button" value="16" type="submit">16C</button></td> <td bgcolor="%s"><button name="button" value="17" type="submit">17C</button></td> <td bgcolor="%s"><button name="button" value="18" type="submit">18C</button></td> <td bgcolor="%s"><button name="button" value="19" type="submit">19C</button></td> <td bgcolor="%s"><button name="button" value="20" type="submit">20C</button></td> <td bgcolor="%s"><button name="button" value="21" type="submit">21C</button></td> <td bgcolor="%s"><button name="button" value="22" type="submit">22C</button></td> <td bgcolor="%s"><button name="button" value="23" type="submit">23C</button></td> <td bgcolor="%s"><button name="button" value="24" type="submit">24C</button></td> <td bgcolor="%s"><button name="button" value="25" type="submit">25C</button></td> <td bgcolor="%s"><button name="button" value="26" type="submit">26C</button></td> <td bgcolor="%s"> <button name="button" value="hotter" type="submit">HOTTER</button> </td> ''' % (tuple(buttonColours))
Chunk referenced in 15.7

This code is separated out because of the complexity of loading the colours of each of the demand buttons. Each button gets a graduated colour from blue through to red, except for the currently specified temperature, which is shown with a yellow background. Temperatures are rounded to the nearest integer in order to determine which button is so highlighted. Temperatures above and below the selectable range highlight the HOTTER and COOLER buttons respectively.

There is a slight glitch with the operation of this form, in that when no specific temperature button is selected (such as when a text value of temperature is entered, the first button is selected (which would normally be COOLER). (See <house get calling parameters >.) To avoid this, a dummy blank button (value="") is built in at the start of the table list, and when this blank value is recognized, the text value is used instead.

15.3 The HeatingModule module

This web page provides a user-friendly interace to setting the automatic heating on/off times, and the temperatures over the course of a day (this latter function not yet operational). The week is divided into 7 days, each with its own programme.

Each day can have up to 6 blocks of time, where the start time of the first block is midnight (00:00 hours), and the end time of the last block is the next midnight (24:00 hours). The end time of each block can be altered, and the number of blocks is determined by the block that has end time of 24:00.

On saving, if the last end time is earlier than 24:00, a new block is added (up to 6 blocks total). If six blocks are in use, and the last end time does not end at midnight, the program will be incomplete and the actual behaviour is not defined.

The desired temperature of each time block can be set independently.

"" 15.16 =
#! /usr/bin/python ## ********************************************************** ## * do NOT EDIT THIS FILE! * ## * Use $HOME/Computers/House/HouseMade.xlp instead * ## ********************************************************** ## ## 20141113:114917 1.0.0 ajh first version with number ## 20141113:114958 1.0.1 ajh elide start times if narrow column ## 20150722:164226 1.1.0 ajh copied from TimerModule and updated ## 20230705:175322 1.2.0 ajh re-engineered for HouseMade v4.3.1 ## import cgi,datetime,math,os,sys,re,time from HouseDefinitions import * DEBUG=False <Web: define the heatingData class 15.17> def heating(logMsg,remadr,args): #print("<P>Starting heating</P>") DEBUG=False active=False <HouseMade: collect date and time data 15.12> environ=os.environ if DEBUG: keys=list(environ.keys()) keys.sort() print("environ:<BR/>") for key in keys: print(" %s:%s<BR/>" % (key,environ[key])) if DEBUG and args: keys=list(args.keys()) print("<P>arguments</P>") lastKey='' for key in keys: if key[0:3]!=lastKey: #print("\n ",) lastKey=key[0:3] print(" %s:%s<BR/>" % (key,args[key]),) print("\n\n",end='') # put 2 newlines at end argdict={} for key in args.keys(): argdict[key]=args[key].value server=NEWPORT #print(f"<P>server={server}, remadr={remadr}</P>") clientIP=remadr OKlist=['','','','::1'] if clientIP in OKlist: clientOK=True else: clientOK=False logMsg("clientOK=%s (%s)" % (clientOK,clientIP)) #print(f"<P>clientOK={clientOK}</P>") sortargs=list(args.keys()) sortargs.sort() #for key in sortargs: # print(f"{key}={argdict[key]}<br/>") # create data structures and initialize #print("<P>loading heating data</P>") td=heatingData() # load previously saved data td.load('/home/ajh/Computers/House/heatProgram.dat') <Web: heating: collect parameters and update 15.18> <Web: heating: build widths for web page table 15.20> <Web: heating: build web page 15.21> if clientOK:'/home/ajh/Computers/House/heatProgram.dat') #print("--------") return out if __name__=='__main__': page1=heating(logMsg,'',\ {'temp-0-0':12,'start-0-0':'0000','end-0-0':'0542',\ 'temp-0-1':22,'start-0-1':'0542','end-0-1':'1123',\ 'temp-0-2':17,'start-0-2':'1123','end-0-2':'2133',\ 'temp-0-3':14,'start-0-3':'2133','end-0-3':'2400'}\ ) page2=heating(logMsg,'',\ {'temp-1-0':18,'start-1-0':'0000','end-1-0':'0800',\ 'temp-1-1':21,'start-1-1':'0800','end-1-1':'1215',\ 'temp-1-2':14,'start-1-2':'1215','end-1-2':'1700',\ 'temp-1-3':10,'start-1-3':'1700','end-1-3':'2400'}\ ) print(page1) print(page2)

15.3.1 Define the heatingData Class

<Web: define the heatingData class 15.17> =
class heatingData(): def __init__(self): self.days=['Sunday','Monday','Tuesday','Wednesday','Thursday','Friday','Saturday'] self.temp=[[ThermostatSetting for j in range(NTempBlocks)] for i in range(7)] self.start=[['0000' for j in range(NTempBlocks)] for i in range(7)] self.end=[['0000' for j in range(NTempBlocks)] for i in range(7)] self.width=[[10 for j in range(NTempBlocks)] for i in range(7)] self.colour=[['red' for i in range(NTempBlocks)] for j in range(7)] self.targett=11 def mins2Hours(self,m): return (m/60,m%60) def hours2Mins(self,h,m): return 60*h+m def load(self,filename='/home/ajh/Computers/House/heatProgram.dat'): f=open(filename,'r') for i in range(7): day=f.readline().strip() res=re.match('Day (\d)$',day) if res: rd=int( if rd!=i: print("Could not read data at day %s" % (i)) for j in range(NTempBlocks): block=f.readline().strip() res=re.match('(\d) (\d\d\d\d)-(\d\d\d\d):(\d\d)$',block) if res: n=int( if e=='0000': e='2400' t=int( self.temp[i][j]=t self.start[i][j]=s self.end[i][j]=e self.colour[i][j]=setColour(self.temp[i][j]) if n!=j: print("Error on block %d on day %d" % (j,i)) else: break; if block.strip()!='': blank=f.readline() line=f.readline() targ=int(line.strip()) self.targett=targ f.close() #print('heating data loaded') def save(self,filename='/home/ajh/Computers/House/heatProgram.dat'): f=open(filename,'w') for i in range(7): f.write("Day %d\n" % (i)) for j in range(NTempBlocks): s=self.start[i][j] sh=s[0:2]; sm=s[2:4] e=self.end[i][j] eh=e[0:2]; em=e[2:4] t=self.temp[i][j] #f.write("%1d %02d%02d-%02d%02d:%02d\n" % (j,sh,sm,eh,em,t)) f.write(f" {j:1} {s:4}-{e:4}:{t}\n") f.write("\n") pass f.write(f'{self.targett}\n') f.close() #print('heating data saved') def getcurrtemp(self): # isoweek is Mon-Sun, 1-7, but want 0-origin starting with Sun # Sun Mon Tue Wed Thu Fri Sat # iso: 7 1 2 3 4 5 6 # 0-org 0 1 2 3 4 5 6 nowstr=now.strftime("%H%M") weekday=now.isoweekday() % 7 todaystarts=self.start[weekday] todayends=self.end[weekday] todaytemps=self.temp[weekday] self.targett=10 for j in range(len(todaystarts)): if nowstr>todaystarts[j] and nowstr<todayends[j]: self.targett=todaytemps[j] break return self.targett def setcurrtemp(self,temp): # isoweek is Mon-Sun, 1-7, but want 0-origin starting with Sun # Sun Mon Tue Wed Thu Fri Sat # iso: 7 1 2 3 4 5 6 # 0-org 0 1 2 3 4 5 6 nowstr=now.strftime("%H%M") weekday=now.isoweekday() % 7 todaystarts=self.start[weekday] todayends=self.end[weekday] todaytemps=self.temp[weekday] self.targett=temp for j in range(len(todaystarts)): if nowstr>todaystarts[j] and nowstr<todayends[j]: todaytemps[j]=temp #print(f"<p>setting new temperature at ({weekday},{j}) to {temp}") #print(f"today's temperatures are (element {j} of) {todaytemps}</p>") break return self.targett def __str__(self): rtn='' for i in range(7): rtn+="\n Day %d: " % (i) for j in range(NTempBlocks): s=self.start[i][j] e=self.end[i][j] t=self.temp[i][j] w=self.width[i][j] rtn+=f" {j:2} {s:4}-{e:4}:{t:2} {w}" return rtn
Chunk referenced in 15.16

This class deals with all the logic needed to load and save the heating data, stored in a separate file. It handles conversion from external stored time data in hours:minutes format, converting it to internally stored minutes only (from the start of the day), and reconverting it back again on saving.

It also provides a few simple conversion routines for switching between the formats.

15.3.2 Collect Parameters and Update

<Web: heating: collect parameters and update 15.18> =
# collect parameters #print(f'clientOK={clientOK}') if clientOK: if args: active='True' keys=list(args.keys()) t='' for k in keys: <Web: heating: handle each argument 15.19> #print(f'<pre>td={td}</pre>')
Chunk referenced in 15.16

check that the arguments are non-empty, then process each argument according to its key k.

15.3.3 Web: heating: handle each argument

<Web: heating: handle each argument 15.19> =
#print(f"\nk={k}, arg[k]={argdict[k]}<br/>") if k=='button': arg=argdict[k] #print(f"Got {k} (button), args={arg}<br/>") if arg!='save': newtemp=int(arg) #print(f"button post sets new temperature to {newtemp}") td.setcurrtemp(newtemp) else: res=re.match('(button|temp|start|end|endmin)-(\d+)-(\d+)',k) if res: d=int( b=int( #print("got type=%s, day=%d, block=%d<br/>" % (t,d,b)) if t=='temp': tt=int(argdict[k]) #print(f"temp:{d},{b},{tt},{argdict[k]}<br/>") td.temp[d][b]=tt td.colour[d][b]=setColour(tt) #print(f"Colour change for new temp {tt} is {td.colour[d][b]}") # 'start' is never used if t=='start': tt=argdict[k] #print(f"start:{d},{b},{tt},{argdict[k]}") td.start[d][b]=tt #print('handled start arg<br/>') if t=='end': cur=td.end[d][b][2:4] tt=argdict[k] new=f"{tt:.2s}{cur:.2s}" #print(f'end:{d},{b},cur={cur},{tt},{argdict[k]}->{new}') td.end[d][b]=new #print('handled end arg<br/>') if t=='endmin': cur=td.end[d][b][0:2] tt=argdict[k] new=f"{cur:.2s}{tt:.2s}" #print(f'endmin:{d},{b},cur={cur},{tt},{argdict[k]}->{new}') td.end[d][b]=new #print('handled endmin arg<br/>') else: print(f'unknown argument key {t}, args {argdict[k]}<br/>')
Chunk referenced in 15.18

In the for loop to handle each http calling argument, each key is checked for validity, and the appropriate action taken.

15.3.4 Build Widths for Web Page Table

<Web: heating: build widths for web page table 15.20> =
# build widths for table for i in range(7): dayFinished=False for j in range(NTempBlocks): if j>0: # make unused blocks alternate in temperature if td.start[i][j]=='2400': # 2400 is midnight, hence unused if td.temp[i][j-1]==10: td.temp[i][j]=ThermostatSetting else: td.temp[i][j]=10 try: td.start[i][j]=td.end[i][j-1] except IndexError: print("index error in HeatingModule: i=%d, j=%d (start=%s, end=%s)" \ % (i,j,td.start,td.end)) if td.start[i][j]>td.end[i][j]: td.end[i][j]='2400' # compute width in minutes emh=td.end[i][j][0:2] ; emm=td.end[i][j][2:4] if emh and emm: em=60*int(emh)+int(emm) else: print(f"<P>Cannot find time in {emh},{emm} at end-{i}-{j}") sm=60*int(td.start[i][j][0:2])+int(td.start[i][j][2:4]) w=em-sm if w<0: w=0 if dayFinished: w=0 td.width[i][j]=math.trunc(100*w/1440.0) # percentage width, 1440=midnight endhours=td.end[i][j][0:2] if endhours==24 or endhours==0: dayFinished=True print("got day finished at day=%d, block=%d" % (i,j)) pass
Chunk referenced in 15.16

15.3.5 Build the web page

<Web: heating: build web page 15.21> =
# build web page #print(f"Now building web page, temps are {td}") redirect='' #if active: # or chookdooractive: # redirect='''<meta http-equiv="Refresh" content="10;URL='%s'>''' % (HServer) out="<HTML>\n<HEAD>\n" out+=redirect out+='<meta http-equiv="Pragma" content="no-cache">\n' out+='<TITLE>HeatingTimer</TITLE>\n' today=3 pretime=0.55 ; postime=100.0*(1.0-pretime) pretime=100.0*pretime if not clientOK: out += f"<P>Sorry, {clientIP}, you are not authorized to adjust this table</P>" else: out += '<form action="%s" method="post" name="heating">\n' % (WServer) out += ' <button name="button" value="save">Save</button>\n' out += ' <table border="1" width="100%" padding="0">\n' for i in range(7): if i==today: #out+=f"<tr width='100%'><td width='{pretime}%'/><td>X</td><td width='{postime}%'/></tr>" pass out += " <tr height='40px'>\n" if i==weekday: dayColour="#8f8" else: dayColour="#fff" out += " <td width='10%%' bgcolor='%s'>%s</td>\n" % (dayColour,td.days[i]) out += " <td><table width='100%' height='100%' border='0' padding='0' cellspacing='0'><tr>\n" for j in range(NTempBlocks): if td.width[i][j]==0: continue out += " <td bgcolor='%s' width='%d%%' height='35px' align='center'>\n" % (td.colour[i][j],td.width[i][j]) out += ' <select name="temp-%d-%d" id="temp-%d-%d" size="1">\n' % (i,j,i,j) for k in range(10,27): selected="" if k==td.temp[i][j]: selected="selected" out += ' <option value="%d" %s>%d</option>\n' % (k,selected,k) out += ' </select>\n' out += " </td>\n" out += " <td> </td>\n" out += " </tr>\n" out += " <tr>\n" for j in range(NTempBlocks): if td.width[i][j]==0: continue out += " <td bgcolor='%s' width='%d%%' height='35px' align='center'>\n" % (td.colour[i][j],td.width[i][j]) out += " <table border='0'>\n" stt=td.start[i][j] sh=int(stt[0:2]) ; sm=int(stt[2:4]) ; sm=5*((sm+2)//5) edt=td.end[i][j] eh=int(edt[0:2]) ; em=int(edt[2:4]) ; em=5*((em+2)//5) if td.width[i][j]>15: out += ' <tr><th>Start</th><th>End</th></tr>\n' out += ' <tr><td>%04s</td>\n' % (stt) else: out += ' <tr><th>End</th></tr>\n' out += ' <tr>\n' out += ' <td>\n' out += ' <select name="end-%d-%d" size="1">\n' % (i,j) for k in range(0,25): selected="" if k==eh: selected="selected" out += ' <option value="%02d" %s>%02d</option>\n' % (k,selected,k) out += ' </select>\n' out += ' <select name="endmin-%d-%d" size="1">\n' % (i,j) for k in range(0,12): selected="" if 5*k==em: selected="selected" out += ' <option value="%02d" %s>%02d</option>\n' % (5*k,selected,5*k) out += ' </select>\n' out += ' </td>\n' out += ' </tr>\n' out += ' <tr>\n' out += ' <td>\n' out += ' </td>\n' out += ' </tr>\n' out += " </table>\n" out += " </td>\n" out += " <td> </td>\n" out += " </tr></table></td>\n" out += " </tr>\n" out += " </table>\n" out += '</form>\n' out += '<A HREF="http://localhost/~ajh/cgi-bin/">back to house</A>\n'
Chunk referenced in 15.16

16. External Hardware

There are a number of additional elements to the HouseMade system, consisting of various subsystems and the wiring connecting them all. This section documents those elements.

16.1 The ChookDoor Controller

See separate documentation relating to the chook house door controller.

16.2 The Proving Circuitry Monitoring System


17. Test Programs

17.1 Check RPC Operation

The following short fragment of code is intended to check the operation of the RPC mechanisms on the Relay Server. It provides the user with an RPC object, which can be used to invoke the RPC interfaces. Several such interfaces are invoked as examples.

Usage is to import this code into an interpretive invocation of python, viz from testRPC import *.

"" 17.1 =
<edit warning 2.1> import xmlrpc.client import HouseDefinitions RelayServer=HouseDefinitions.RelayServer print("options are:") print(" RelayServer.getState()") print(" RelayServer.setState([0,0,...]) # 12-element vector of 0/1") print(" RelayServer.setBitOn(bitnumber) # bit number is an integer (0-11)") print(" RelayServer.setBitOff(bitnumber) # bit number is an integer (0-11)") print() #print(" RelayServer.getHeating()") #print(" RelayServer.setHeating(float,'on'/'off')") #print(" RelayServer.getSolar(n) # n is register number (32 is input amps)") #print(" RelayServer.getTemps()") #print(" RelayServer.getTank()") #print(" RelayServer.maxminTemp()") #print() #print("for example,") #print(" RelayServer.getHeating()=%s" % (RelayServer.getHeating())) #print(" RelayServer.getState()=%s" % (s.getState())) print

Note that the data logging operations are not currently available.

18. The Log Files

The log files for HouseMade.xlp have now been coalesced into one file: /home/ajh/logs/housemade/house.log. All others are obsolete.

19. Installing and Starting the HouseMade Software

19.1 Introduction

This has always been a somewhat fraught area of development since the earliest versions of this software. That is largely due to the variety of both hardware and software in use, and the various idiosyncracies involved. This section attempts to address these issues.

This is a list of all the processes that need starting:

  1. The Hardware subsystem software, involving a relay driver, a relay/chook door state request server HardwareServer, and the HardwareBone GPIO setup AJH-GPIO-Relay.dts.
  2. The House Data Logging Computer relay interface
  3. chook door (the need for this as a standalone is currently in question).

19.2 Details

19.2.1 Start the Hardware Server

The HardwareServer (aka the chook door server) provides an RPC interface to a) interrogate whether or not the chook door is closed, according to the proving microswitch, and b) drive the relay switching.

The proving circuits are independent of the actual closing and opening process, and provides a safety check that the door is actually closed, once the close command has been issued, and the door has had time to close. It runs continuously on a HardwareBone server (kerang), and can be started with the make call:

            make start-hardware

This make call just invokes the following script (after making sure that its code is up-to-date) on the HardwareBone (known by its network name kerang). The script can also be invoked directly on the kerang machine from the command line.

"" 19.1 =
#!/bin/bash LOGDIR='/home/ajh/logs/kerang' HOUSE='/home/ajh/Computers/House' BIN=${HOME}/bin # collect any previous instance ps aux | grep "" | grep -v grep | awk '{print $2}' >${LOGDIR}/hardwareServerPID # remove any previous instances if [ -f ${LOGDIR}/hardwareServerPID ] ; then for p in `cat ${LOGDIR}/hardwareServerPID` ; do kill -9 `head ${LOGDIR}/hardwareServerPID` done rm ${LOGDIR}/hardwareServerPID fi # start the new instance /home/ajh/binln/python /home/ajh/Computers/House/ >>~/logs/kerang/HardwareServer.log & # record the new instance ps aux | grep "" | grep -v grep | awk '{print $2}' >>${LOGDIR}/hardwareServerPID

If this doesn't work, or you need a more direct interface, then in a window on the kerang machine itself:

            kerang $ /home/ajh/Computers/House/
will run the server in that window. Note that as this program runs continuously, it should be started in a separate terminal window (which can then hidden from view but left running). This mode has the advantage that any output from the server can be seen immediately in that window, rather than having to examine the logfile (as required by the start Hardware methods above).

19.2.2 Relay Server

The RelayServer runs continously on the House Data Logging computer (currently set as terang. It can be started with a make call:

            make start-relayserver

If this doesn't work, or you need a more direct interface, then

            (any machine) $ ssh terang /home/ajh/Computers/House/
Since this program runs continuously, it should be started in a separate terminal window, which is then hidden from view, but left running.

20. The Cron Jobs

See the Cron Job page for details.

21. Validation and Maintenance Scripts

"" 21.1 =
#!/bin/bash case `hostname` in newport) remote=reuilly ;; reuilly) remote=newport ;; esac echo "Comparing remote ($remote) file $1 with local file of the same name" ssh $remote ls -l /home/ajh/Computers/House/$1 ls -l /home/ajh/Computers/House/$1 ssh $remote cat /home/ajh/Computers/House/$1 | diff - $1 if [ $? = 0 ] ; then echo "same" ; fi

This shell script compares the nominated generated file with the installed file in reuilly/newport. Useful to check that versions are aligned, and are up-to-date with local code.

"" 21.2 =
#! /bin/bash function isrunning() { pid=`ps aux | grep $prog | grep python3 | awk '{print $2}' | sed 'N;s/\n/ /;'` running=$? } AllPIDS='' prog='' isrunning AllPIDs="$AllPIDs $pid" prog='' isrunning AllPIDs="$AllPIDs $pid" prog='' isrunning AllPIDs="$AllPIDs $pid" prog='' isrunning AllPIDs="$AllPIDs $pid" echo $AllPIDs

A script to identify any process that is part of the HouseMade system and is currently running. It returns a list of process IDs.

"" 21.3 =
#!/bin/bash if [ "$HOST" = "reuilly" -o "$*" = "localhost" ] ; then ps h `/home/ajh/Computers/House/` else echo "HOUSEMADE RUNNING ON REUILLY" ssh reuilly /home/ajh/Computers/House/ fi

Show the HouseMade programs that are currently running. In normal usage, these should be:

There are two instances of the Event Server, as it spawns an additional thread.

"" 21.4 =
#!/bin/bash pids=`` ps $pids for pid in $pids ; do echo "kill $pid" kill $pid done

A script to terminate all processes associated with the HouseMade system.

"" 21.5 =
pids=`` nonhw=`ps h $pids | grep -v Hardware | mawk '{ print $1 }'` for p in $nonhw ; do echo "kill $p" kill -9 $p done

I found while testing that it was usually convenient to keep the HardwareServer running (especially as it lock up the RPC address until it timed-out), and that only the non-Hardware components needed restarting. Hence this script. Thus the usual development/debug cycle is

        (edit source code)
        make code
        (explore running system)
        (repeat from step 1) 
Note that the script collects all system process ids, drops the HardwareServer one, and then outputs just the non-hardware ones, which are killed.

"" 21.6 =
#! /bin/bash LOGS=$HOME/logs/housemade HOUSE=/home/ajh/Computers/House function isrunning() { ps aux | grep $prog | grep python3 >/dev/null running=$? } echo /home/ajh/bin/date prog='' isrunning if [ $running -eq 1 ] ; then echo "$prog is Not Running" (cd $HOUSE ; $prog; if [ $? = 1 ] ; then echo "wait for address" ; exit 1 ; fi) & echo "$prog restarted" sleep 1 else echo "$prog is Running" fi prog='' isrunning if [ $running -eq 1 ] ; then echo "$prog is Not Running" (cd $HOUSE ; $prog) & echo "$prog restarted" sleep 1 else echo "$prog is Running" fi prog='' isrunning if [ $running -eq 1 ] ; then echo "$prog is Not Running" (cd $HOUSE ; $prog) & echo "$prog restarted" sleep 2 else echo "$prog is Running" fi prog='' isrunning if [ $running -eq 1 ] ; then echo "$prog is Not Running" (cd $HOUSE ; $prog) & echo "$prog restarted" else echo "$prog is Running" fi

A script to check if all the relevant programs are running, and restart then if not.

22. Makefile

"Makefile" 22.1 =
# The two main protagonists: # HARDWARE is the dedicated house computer HARDWARE = reuilly # CENTRAL is the management system CENTRAL = reuilly # # NEWPORT is the desktop (used for documentation) NEWPORT = newport # SPENCER is a web client only SPENCER = spencer # HOME is home directory (common on all machines, default make) HOME = /home/ajh # HOUSE is this development directory relative to root HOUSE = $(HOME)/Computers/House # BIN is the main script repository (not used?) BIN = $(HOME)/bin # CGIBIN is the common gateway interface directory CGIBIN = $(HOME)/public_html/cgi-bin # RSYNC is the generic remote copy, with options RSYNC = /usr/bin/rsync -auv <Servers 22.2> <Modules 22.3> <Scripts 22.4> <cgi-bins 22.5> <Maintenance 22.6> <Executables 22.7> <Makefile: install all 22.8> <Makefile: install support components 22.9> <Makefile: install cgi-bins 22.10> # generic XLP making options include $(HOME)/etc/MakeXLP # creating code HouseMade.tangle: HouseMade.xlp xsltproc --xinclude $(XSLLIB)/litprog.xsl HouseMade.xlp >HouseMade.xml touch HouseMade.tangle # the main development option code: HouseMade.tangle executable if [ "$(HOST)" = "reuilly" -o "$*" = "localhost" ] ; then \ cp -p $(cgi-bins) $(CGIBIN)/ ;\ else \ rsync -auv $(cgi-bins) $(CENTRAL):$(CGIBIN)/ ;\ rsync -auv $(servers) $(modules) $(CENTRAL):$(HOUSE)/ ;\ fi touch code # the default documentation makes html: HouseMade.html pdf: HouseMade.pdf # past here is legacy default=HouseMade # expansion of the main installation makes CGIDIR=/home/ajh/public_html/cgi-bin $(CGIFILES) make-exec: chmod 755 $(EXECFILES) \ $(EXECFILES) install-%: HouseMade.tangle make-exec if [ "$(HOST)" = "$*" -o "$*" = "localhost" ] ; then \ cp -p $(CGIFILES) $(CGIDIR) ;\ else \ rsync -auv $(CGIFILES) $*:$(CGIDIR)/ ;\ rsync -auv $(HOUSEFILES) $*:$(HOUSE)/ ;\ fi touch install-$* debug: echo "[$(CENTRAL)]"

There are many things that might need to be "make"d. Here is a tentative list, bearing in mind that both the repository and delivery sites may vary over time, and hence hard-coding machine names needs to be avoided.

First of all, a classification of the system components:
Class Components
Maintenance config.txt* HouseMade.html HouseMade.tangle HouseMade.xlp HouseMade.xml
Makefile manifest Versions0-1Modifications Version2Modifications WebInterface.xlp
Note that executables are marked with an asterisk (*)

<Servers 22.2> =
Chunk referenced in 22.1
<Modules 22.3> = \ \
Chunk referenced in 22.1
<Scripts 22.4> = \ \
Chunk referenced in 22.1
<cgi-bins 22.5> =
Chunk referenced in 22.1
<Maintenance 22.6> =
maintenance=config.txt HouseMade.html HouseMade.tangle \ HouseMade.xlp HouseMade.xml Makefile manifest \ Versions0-1Modifications Version2Modifications
Chunk referenced in 22.1
<Executables 22.7> = \ \ \ \
Chunk referenced in 22.1

install all essential components <Makefile: install all 22.8> =
install:HouseMade.tangle install-modules install-cgis install-servers install-web touch install
Chunk referenced in 22.1
make house
install all support components <Makefile: install support components 22.9> =
install-modules: HouseMade.tangle
Chunk referenced in 22.1
make cgis
make executable, and install all cgi-bin components
<Makefile: install cgi-bins 22.10> =
executable: chmod 755 $(executables) install-cgis: install-cgis-newport install-cgis-reuilly cp -p $(cgi-bins) /home/ajh/public_html/cgi-bin # NEEDS TO BE FIXED! install-cgis-newport: HouseMade.tangle executable if [ `hostname` = 'newport' ] ; then \ cp -p $(cgi-bins) /home/ajh/public_html/cgi-bin/ ;\ else \ cp -p $(cgi-bins) newport:/home/ajh/public_html/cgi-bin/ ;\ fi install-cgis-reuilly: HouseMade.tangle executable if [ `hostname` = 'reuilly' ] ; then \ cp -p $(cgi-bins) /home/ajh/public_html/cgi-bin/ ;\ else \ cp -p $(cgi-bins) reuilly:/home/ajh/public_html/cgi-bin/ ;\ fi
Chunk referenced in 22.1
make servers
make executable, and install all server components
make web-components

23. Document History

20140108:175940 ajh 0.0 see file Version 0 Modifications for details
20150405:125114 ajh 1.0.0 see file Version 1 Modifications for details
20200715:133241 ajh 2.0.0 see file Version 2 Modifications for details
20201213:133903 ajh 3.0.0 merge of HouseMade 2.2.1 and EventServer 1.0.4, otherwise, little change
20201213:154451 ajh 3.0.1 migrated separate Makefile.xlp into this file
20201214:165523 ajh 3.0.2 Updated the History section.
20201221:131301 ajh 3.0.3 revisions in moving suite to Terang
20201229:133233 ajh 3.0.4 various tidy-ups of literate code
20210201:214221 ajh 3.0.5 bug fixes and Makefile fixes
20210416:174227 ajh 3.0.6 make time for added events to be 24:00 to avoid deletion conflict
20210509:104905 ajh 3.0.7 Refine HardwareServer to deliver 'opening' and 'closing' chookdoor status
20210511:153004 ajh 3.0.8 Revert HardwareServer - trying new approach (but not yet). Also clean up Makefile.
20210515:114111 ajh 3.0.9 Add 'None' when no previously scheduled events.
20210525:144450 ajh 3.1.0 revise much of the event handling
20220115:174744 ajh 3.1.1 Added 'NorthVegBed' control, moved RelayTable definitions to separate litcode chunk, and removed obsolete RelayTable entries
20220413:143129 ajh 3.1.2 change check call in HouseDefinitions to avoid using printEvents
20220414:085342 ajh 3.1.3 modified to use jeparit
20220414:165026 ajh 3.1.4 remove LOGDIR as a Python constant/xlp chunk
20220426:182654 ajh 3.1.5 add exception catching to RelayServer.setBitOn/setBitOff
20220504:105641 ajh 3.1.6 allow for gardensteps off event to be manually set, rather than automatically overriden
20220907:101729 ajh 3.1.7 changed references to jeparit to jeparit.local
20230109:082344 ajh 4.0.0 migrate to Raspberry Pi controller
20230112:150837 ajh 4.0.1 fixed the chook door moving logic in
20230115:173336 ajh 4.0.2 no change in functionality, but significant cleanup of the RelayServer code
20230116:130412 ajh 4.0.3 add Ring Main module, minor cleanups
20230128:093043 ajh 4.0.4 mainly documentation improvements, some minor code tweaks
20230128:180017 ajh 4.0.5 change off relay to cancel timer as well
20230213:170708 ajh 4.1.0 Significant changes to allow for day of the week specificity on events.
20230221:181524 ajh 4.1.1 allow multiple day entries in day field of events
20230320:142420 ajh 4.1.2 very minor change: change all references from House4 to House
20230328:140308 ajh 4.2.0 remove month parameter for events, make event day weekday
20230624:134449 ajh 4.3.0 migrate cpu intensive code to external routines (
20230705:175415 ajh 4.3.1 re-instated heating module
20230722:124156 ajh 4.3.2 adjustments to heating module to get working version
<current version 23.1> = 4.3.2
<current date 23.2> = 20230722:124156

24. Indices

24.1 Files

File Name Defined in 21.2 10.1 8.1 7.1 6.1 11.1 12.1 3.9 3.7 3.8 15.16 2.3 15.2
Makefile 22.1 5.19 4.1 5.1 2.6 13.1 14.1 21.1
config.txt 3.1 9.1, 9.2, 9.3 15.1 21.4 21.5 3.6 19.1 5.18 21.6 17.1 21.3

24.2 Chunks

Chunk Name Defined in Used in
ChookDoor: class ChookDoor 10.3 10.1
ChookDoor: main 10.15 10.1
ChookDoor: misc routines 10.2 10.1
Event Server: calling points 7.2, 7.3, 7.4, 7.5, 7.6, 7.7, 7.8, 7.9, 7.10, 7.11, 7.12, 7.13, 7.14, 7.15, 7.16 7.1
Event Server: main routine 7.18 7.1
Event Server: serverprocess routine 7.17 7.1
Event class: compare two events 6.3 6.2
Event class: definition 6.2 6.1
EventEditor: define get current events routine 9.5 9.1
EventEditor: define make edit page routine 9.7 9.1
EventEditor: define make home page routine 9.6 9.1
EventEditor: print instructions 9.4 9.1
EventList class: add event 6.5 6.4
EventList class: definition 6.4 6.1
EventList class: delete event 6.6 6.4
EventList class: load events 6.9 6.4
EventList class: nextEvent 6.8 6.4
EventList class: save events 6.10 6.4
EventList class: sort events 6.7 6.4
EventScheduler: collect ChookDoor times 8.2 8.1
EventScheduler: collect GardenSteps times 8.3 8.1
EventScheduler: main loop 8.4 8.1
Events: main routine for testing code 6.11 6.1
Executables 22.7 22.1
GardenRun 12.2 12.1
HouseDefinitions: general routines 2.8 2.3
HouseDefinitions: server connections and interfaces 2.7 2.3
HouseMade: Relay Control 15.8 15.7
HouseMade: check client connection 15.13 15.3
HouseMade: collect date and time data 15.12 15.3, 15.16
HouseMade: define the Generate Solar Data routine 15.10 15.2
HouseMade: define the Generate Tank Data section 15.11
HouseMade: define the Generate Weather Data routine 15.9
HouseMade: define the house interface 15.3 15.2
HouseMade: generate the web page content 15.14 15.3
HouseMade: get events information 15.6 15.3
HouseMade: get local information 15.4 15.3
HouseMade: get relay information 15.5 15.3
HouseMade: isDay definition 2.9 2.8
HouseMade: legacy code for 15.7 15.3
LOGDIR 2.2 2.3
Maintenance 22.6 22.1
Makefile: install all 22.8 22.1
Makefile: install cgi-bins 22.10 22.1
Makefile: install support components 22.9 22.1
Modules 22.3 22.1
RelayDriver: init method 4.2 4.1
RelayDriver: read method 4.4 4.1
RelayDriver: write method 4.3 4.1
RelayNameTable 2.5 2.3, 2.6
RelayServer: connect to the HardwareServer 5.2 5.1
Scripts 22.4 22.1
Servers 22.2 22.1
Web: define the heatingData class 15.17 15.16
Web: heating: build web page 15.21 15.16
Web: heating: build widths for web page table 15.20 15.16
Web: heating: collect parameters and update 15.18 15.16
Web: heating: handle each argument 15.19 15.18
cgi-bins 22.5 22.1
class ChookDoor: chookDoor 10.10 10.3
class ChookDoor: closeDoor 10.9 10.3
class ChookDoor: compute 10.6 10.3
class ChookDoor: doorState 10.11 10.3
class ChookDoor: handleEvent 10.12 10.3
class ChookDoor: init 10.4 10.3
class ChookDoor: load 10.5 10.3
class ChookDoor: openDoor 10.8 10.3
class ChookDoor: run 10.13 10.3
class ChookDoor: save 10.7 10.3
class ChookDoor: stop 10.14 10.3
current date 23.2
current version 23.1
gpioInputPins 3.4 3.6, 3.7
gpioPinsModA 3.2 3.6, 3.7
gpioPinsModB 3.3 3.6, 3.7
gpioSparePins 3.5
house make temperature panel 15.15 15.7
relayserver: countDown 5.17 5.1
relayserver: define getTank 5.12 5.1
relayserver: define the RPC-Server interface 5.4 5.1
relayserver: getSolar 5.15 5.1
relayserver: getState 5.5 5.1
relayserver: getTimer 5.13 5.1
relayserver: quiescent 5.6 5.1
relayserver: readDoor 5.7 5.1
relayserver: resetTimer 5.14 5.1
relayserver: setBit 5.9 5.1
relayserver: setBitOff 5.11 5.1
relayserver: setBitOn 5.10 5.1
relayserver: setState 5.8 5.1
relayserver: start 5.16 5.1
relayserver: strState 5.3 5.1
server definitions 2.4 2.3

24.3 Identifiers

Identifier Defined in Used in
EventList 6.4
HouseDefinitions 2.3 2.8, 2.8, 2.8, 2.8, 5.1, 5.19, 7.1, 8.1, 9.1, 10.1, 11.1, 12.1, 13.1, 14.1, 15.2, 15.2, 15.5, 15.5, 15.5, 15.5, 15.5, 15.5, 15.5, 15.5, 15.5, 15.5, 15.5, 15.5, 15.13, 15.16, 17.1, 17.1
eventsInfo 15.6 15.3, 15.14
heating 15.16
heatingsection 15.7 15.14
house 15.3
jobtime 15.12
relayStateStr 15.5 15.3, 15.14
relaycontrol 15.8 15.14
relaycontrol 15.8 15.14
solarsection 15.7 15.14