Kitz Forum
Chat => Tech Chat => Topic started by: sevenlayermuddle on November 17, 2020, 08:33:42 AM
-
I am interested in any encouragement or otherwise, relating to a device I’m thinking of actually constructing.
It would consist of a photocell, a microcontroller, and an LCD character display. All of these are conveniently to hand, residing in a drawer labelled “bits & bobs”
Placed on a window ledge, the photocell would monitor sunrise and sunset events. Based on the assumption that midday is halfway between these events the LCD would show a self-calibrating clock, tailored to my own personal time zone. My gut feeling is that, by taking a long term rolling average, it might become reasonably consistent.
Longer term, it would eventually notice the shortest/longest days of the year, and so make a guess at date as well as time. Averaged over enough years, this too might become increasingly accurate.
If I can figure out the maths it might even be able to tell my latitude, based on crunching the data from longest/shortest days. In that case I might even enhance it to have access to GMT, as the offset would allow full location.
Questions:
Is there any obvious reason why it would not work?
Even after settling to a long term average, would it always show a constant offset from GMT, or would that vary with the date (gut feeling says so)?
Am I reinventing the wheel, are such devices already cheap and easily available?
-
Cool idea, personally I always wanted to make a nixie clock (https://www.ebay.co.uk/itm/Nixie-Clock-Kit-For-IN-14-Nixie-Tubes-PV-Electronics-Quality/184305501538?hash=item2ae976c562:g:FE4AAOSwsO5b8D40), they are super cool
C
-
An excellent idea. `the longitude time zone offset would vary sinusoidally throughout the year, as the earth’s inclined rotational axis ‘rotates’ through the year from the viewpoint of the plane of the earth’s orbit compared with the position vector pointing to the sun. What ever processor you use, make sure that is has floating point and maths libraries for sin/cos/sqrt etc. You’ll need to do some trig possibly. The sunrise sunset thing is a really good idea. The clock won’t know it’s longitude and so it won’t know what the offset from GMT is, you’d have to configure it with the longitude somehow; could hardcode it in but you might not want to do that in case you ever move the clock or make additional ones.
I’m at latitude/longitude: 57.470456, -5.401124 according to https://www.doogal.co.uk/LatLong.php (https://www.doogal.co.uk/LatLong.php) which seems a nice tool. So 5.4 degrees west is what time difference now? Will have to do the trig based on the time of year.
Here’s yet another unsolicited and irrelevant plug for my personal love affair: https://dlang.org (https://dlang.org) which might help you on your journey (or delay you as you dive in). Anyway might be found very stimulating.
Will the interface to your photocell be an ADC? Or will it be a two-state on/off affair? I’m wondering about heavy clouds at sunset and dawn injecting noise into your calculations by delaying the times of sunset/sunrise, but the noise should level out to a constant positive error for both ?
I and others could try and help with the maths, although 40 years is a long time.
-
the photocell would monitor sunrise and sunset events. Based on the assumption that midday is halfway between these events
Your idea fails at the first hurdle!
London today: Sunrise 07:22, sunset 16:07 (see https://www.timeanddate.com/sun/)
If you could persuade all the relevant authorities to make the earths orbit around the sun circular would be a start to your project.
-
Thanks for the encouragement guys, I'm on the case.
What ever processor you use, make sure that is has floating point and maths libraries for sin/cos/sqrt etc.
Damn, I thought somebody might say that. I'm using a Microchip PIC, because I am familiar with them and have the dev tools, but I'd hoped to do it all in assembly simply because I enjoy it and the software is free. Microchip do offer a C Compiler, but there's licensed version and a (crippled) free version, not sure what exactly gets crippled. I'll need to find out if the free C compiler is up to the job....
Another problem with heavy maths libraries and high level languages might be the limited memory, the PIC I am playing with right now has just 512 bytes of RAM, and 7k bytes of program memory (Harvard architecture). I suppose as a last resort I could use a Pi or something, but I'd rather not.
Alternatively, I wonder if I could write a desktop App that provides the necessary maths, in the form of machine-written assembly lookup tables?
Re the sensor, I plan t use an LED as a photodiode simply because I haven't got any actual sensors to hand, but I have bags and bags of LEDs, looking for a purpose in life. Some PICs have on-chip operational amplifiers which offer a nice way, of interfacing to a photodiode(/LED), converting the photo current into a useful voltage scale. They also have ADCs, so you end up with a numeric representation of ambience. I can then play around with finding the right thresholds, add software filtering to ignore the odd cloud, etc. I've made all that work before, sensing ambience with LED and PIC, different project.
I also been thinking that, as an alternative to waiting for shortest/longest day, I ought to be able to make deductions about latitude based on rate of change of lighting levels combined with current hours of sunshine? Can't get my head around that - maybe I never will get my head around it, maybe I'm talking drivel, but it's on the "ideas" list. :D
Cool idea, personally I always wanted to make a nixie clock (https://www.ebay.co.uk/itm/Nixie-Clock-Kit-For-IN-14-Nixie-Tubes-PV-Electronics-Quality/184305501538?hash=item2ae976c562:g:FE4AAOSwsO5b8D40), they are super cool
That would be cool indeed. But for convenience of being portable to different window cills, I want my clock to self-contained and battery operated. It'll probably be about somewhere between cigarette box and paperback book in size. Getting a battery to drive Nixie tubes in such a portable device, to run uninterrupted for years or more, would be a whole different challenge. Next project, maybe? :-\
Your idea fails at the first hurdle!
London today: Sunrise 07:22, sunset 16:07 (see https://www.timeanddate.com/sun/)
If you could persuade all the relevant authorities to make the earths orbit around the sun circular would be a start to your project.
Thanks for the tip. If it simplifies the maths, I'll start writing the letters requesting the changes. :D
I'll keep the thread up to date. This all came about after I purchased the LCD displays, simply because they looked cute and were cheap. Having wired up an LCD to a PIC and got it to display "Hello World", my next challenge was to justify the cost and time I'd spent, by thinking of a problem that they might solve. Combined with my previous application of a PIC involving an ambient lighting sensor, the clock sprang to mind. ::)
-
I really wouldn’t use a PIC; you really need floating point and multiplication and division, although sin/cos/tan could be done with lookup tables they would eat up your ROM. And writing a ton of maths in asm isn’t very stimulating - it’s what C or D is meant for.
Thankfully someone has done all the maths for us: https://en.wikipedia.org/wiki/Sunrise_equation (https://en.wikipedia.org/wiki/Sunrise_equation)
-
You’ll have a job talking me out of the PIC, simply because I know them reasonably well. I also have drawers bursting with them, in various shapes and sizes.
Last PIC project required 32 bit multiply and divide that I wrote myself, which was fun, but I’d not expect any awards for it. There’s also a few more advanced maths sources floating around written by people orders of magnitude smarter than I am. Microchip even publish some of it in their application notes. But whether it has the required resolutions might be a big question...
A plus side of using a PIC is they can be configured to run on the tiniest of currents, yielding a sensible battery life. A good few years on two ‘C’ cells might be achievable, according to some earlier scribblings I composed. Running on a slow clock (say, 32kHz) is part of the trick for low power but for this device, it doesn’t have to be fast - it can spend all day doing the maths if it wants, as long as it’s done by the time it needs done again, next morning.
Of course, if I can get power consumption low enough there is also the option of being solar powered, maybe even on cloudy days, topping up batteries by day enough to see it through the night. The solar cell itself would then double as the light sensor, which makes that an appealing option.
-
If you know it well, then stick to it; I just meant that the code will be a bit boring, just a load of subroutine calls to maths library routines. I love writing assembler myself, but have to accept that nowadays on modern machine it has its place only in limited niches and it can’t be automatically optimised according to call site as modern C and D compilers can do.
I would have a problem here; I would need to correct for my altitude for very high precision results and can’t correct for the height of the mainland mountains to the east and the mountain Beinn nan Càrn to the west above the house; both delay sunrise and sunset by unpredictable amounts depending on how far south the angle to the rising sun is because of the changing height following the profile of the mountain(s).
-
Why a slow clock; doesn’t it have a sleep state? If it’s static internally then it can just do the work and then go to sleep until a timer or other event wakes it up. That’s how the Psion Organiser worked, which I worked on, had an 8 bit CMOS static Hitachi 6301 microcontroller (an enhanced 6801) with a clock that went to DC when it had no work to do and 0.92 MHz otherwise; it had a battery life of months. It had two sleep states, d.c. clock sleep and an ‘off’ which was waiting for a timer interrupt (or the ‘on’ button) to start the processor up.
-
The trouble with sleeping is, you have to wake up again and that may involve extra hardware, to raise the wakeup event. I’ve found that dynamically changing the oscillator frequency, slowing it right down instead of actually sleeping, is nearly as effective.
That said, you are probably right. I probably want extra hardware anyway in the form of a crystal-based external oscillator. These PICs have the option of internal oscillators which are factory calibrated, but still miles away from a proper crystal for accurate timekeeping. I think - need to confirm - there’s options that would then allow me to sleep for exact periods, leaving one of the internal clocks running with whilst asleep.
It won’t be sleeping for long periods as it’ll need to wake up once a second to update the hh:mm:ss on the display, but if it can sleep for 990mS out of every 1000mS, that is still obviously worthwhile.
I’m sulking right now because my “Hello World” has stopped working, and I’m darned if I can see what I did to break it. Reverting to yesterday’s source code works, but today’s does not, the screen just stays blank, yielding no clues. Yet single stepping with debugger seems to show all the expected register transitions. It’s a bug and I’ll find it but I do concede that, at times like this my ‘bug’, whatever it is, would probably be a lot easier to spot if I were working in C. >:(
-
That HD6301 had all the necessary hardware inside it to go to sleep and get woken up regularly; there was an (external iirc) counter that kept time while the machine was in a light sleep with clock at d.c., and when this overflowed then it would wake the processor and software would add n secs to a time variable in static RAM. In fact when that machine was off it woke up every so many seconds, and on the early models you could hear a faint click and maybe even see the display flicker when it came on, updated the time in software and then went straight back to sleep again. But many months battery life was not uncommon; later machines could only last ten hours or so (on two medium-sized NiCads).
Solar-powered would be really cool.
-
Using the LCD displays I purchased, with the PIC asleep and LCD displaying "Hello world", the overall contraption is getting by on just under 200µA. An alkaline AA can supply something like 2500mAh before dropping to 1V or so, so that would predict a useful battery life of ~500 days or so. These figures will change, as I'll probably use a boost converter for stable voltage, in which case a single 'C' cell might make more sense. But implication is that battery life won't be a show stopper - should be years, not weeks.
Bad news is this morning... RIP, prototype LCD number 1, I have just managed to kill it. :'(
Owing to a software bug, in an attempted sleep condition, I was driving the interface bus from the PIC whilst also setting the R/W signal so the LCD was driving the bus too. This creates a fight between the PIC and the LCD with a theoretical risk of damage to one or the other part which in this case seems to have instantly become a reality. :-[
While I'm very sad for the loss of that LCD, in time, I will probably get over it. It cost only £2 or £3 or so, and I've a few more. But worried now about how many more I might accidentally destroy if they're that darned fragile. Might be worth putting some resistors on the bus, but there's nothing to document what the actual current limits need to be. I'm beginning to remember why I chose a career in software, rather than hardware. :blush:
-
An interesting project to keep the brain active, good luck. :)
-
Indeed, a superb project.
-
An interesting project to keep the brain active, good luck. :)
Don’t raise your hopes re my brain activity, it might take more than this to yield any improvement, but thanks for encouragement.
Recent email order confirmations suggest that the nice man who drives the Parcelforce van might, tomorrow, be bringing me the remaining bits I need to make a ‘proof of concept’ first cut that can be left running undisturbed for a few days. It won’t do any geolocation stuff, but will simply try to tell me the time, or at least, tell me a time.
There is likely to be a lapse of updates while I implement that. Pressure from elsewhere suggests I may have other priorities too, like clearing up a small mountain of Autumn leaves, clearing gutters, etc.
Rest assured, even if the results from proof of concept are embarrassing and disastrous, I will confess in due course. :)
-
I don’t speak hardware, so I am full of awe.
-
A small preliminary experiment has yielded encouraging results.
Yesterday I rigged up a test device with existing hardware that has a PIC and photosensor, similar to the configuration I plan to use for the clock. I set a fairly arbitrary trigger threshold, at the low end of the scale, but comfortably above the diode’s noise floor. It was aimed out of a window that looks towards a wall and some trees, simply because that window was convenient.
Jelv’s excellent link to the Time & Date site gave me the predicted local sunrise/sunset times, so about 10 mins before official sunset I started watching.... the sensor triggered just about 45 seconds after “official” sunset. This morning it triggered a little early for sunrise, but only by about 5 minutes.
Considering there was no day to day averaging process, no calibration, and it wasn’t even pointing skywards, I’m pretty happy with that - maybe my clock could be accurate to within a couple of minutes. Possibly fluke results, but far exceeding my initial hopes for such a crude experiment. :)
-
Thankfully someone has done all the maths for us: https://en.wikipedia.org/wiki/Sunrise_equation (https://en.wikipedia.org/wiki/Sunrise_equation)
It’s just dawned on me that whilst I engaged Weaver in debate of PIC vs more advanced hardware I never actually thanked him for his input so far, including that link. Great stuff, maybe my existing long division assembly code won’t go to waste after all!
I am very nearly at a point where I have functional prototype hardware, ready to start SW development in earnest. It all depends whether the stuff I’ve done today works first time when a voltage is applied tomorrow, or vanishes with a flash and a bang in a cloud of smoke, or somewhere in between. :D
Weaver also offered assistance with the maths.
I and others could try and help with the maths, although 40 years is a long time.
A resounding ‘Yes please’, if I get as far as actually deploying any maths. Whilst I may have once qualified in electronics, and spent my career writing software, I have never remotely considered myself to be a mathematician. All input will be appreciated. :)
Photographic evidence of fully featured prototype with (initially spoofed) time display could be available very soon now, if anybody remains interested. :)
-
Photographic evidence of fully featured prototype with (initially spoofed) time display could be available very soon now, if anybody remains interested. :)
Cats and kittehs are always interested. :angel:
-
Cats and kittehs are always interested. :angel:
The display is spoofed so not really proof of anything other than perhaps, proof of my misguided dedication to the cause. But progress so far, development hardware...
The thing aimed skyward at the back, that looks like an LED, is in fact an LED. I will use it as a photodiode, but reserve the right to think of a reason it might want to light up. The four buttons have not got any specific purpose assigned but they seemed like a good idea. I have included a piezo sounder in the design but haven't yet got one.
Essentially I am thinking a device that has an LCD character display, four buttons, an LED, and has the ability to go 'beep', is bound to find a purpose. Even if it fails to improve upon GPS for navigational purposes.
Tomorrow, work starts in earnest on the software. :)
-
I’ve never seen PIC assembly language. I started out doing Z80 code, then Hitachi 6301, a small amount of MC68000 and VAX MACRO asm, then a good bit of 8086, and then switched to C for the next decade. Nowadays I write D, x86-64 asm, BASH occasionally and am dipping my toe into AWK and a lot of complex iOS Shortcuts programs for my iPad. I want to learn some ARM AAarch64 asm soon.
-
I’ve never seen PIC assembly language. I started out doing Z80 code, then Hitachi 6301, a small amount of MC68000 and VAX MACRO asm, then a good bit of 8086, and then switched to C for the next decade. Nowadays I write D, x86-64 asm, BASH occasionally and am dipping my toe into AWK and a lot of complex iOS Shortcuts programs for my iPad. I want to learn some ARM AAarch64 asm soon.
Fear not, when I referred to help with maths I was thinking more in terms of the underlying algorithms and equations, rather than with PIC assembly implementations.
PIC is an acquired taste. If interested, this is the device in my breadboarded prototype, the data sheet includes the instruction set, and bits of example assembly here and there. It’s quite old really, there are undoubtably newer options, but I know my way around that device family and it has all the peripherals I need for this project.
http://ww1.microchip.com/downloads/en/DeviceDoc/40001729C.pdf
-
No, we understand one another, I just meant maths equals maths, although I am extremely rusty. Many thanks for that link, just what I wanted.
-
I have just noticed, RS are now selling my chosen PIC16LF1709, for around 30p each, even for low volumes. Wow, for a device with Op Amps, ADC, DAC, PWMs and much more. That’s another reason I like PICs. ???
Meanwhile, progress continues. Nothing much to demonstrate but I have accurate timer code (+/-10ppm) working, based on a 32.768kHz watch crystal. :)
@Weaver (+others?), I feel I should warn in advance... If I get as far as building a real device, I might as well build a dozen or so, that’s how parts pricing works these days. So, I might be on the scrounge for folks at the other extremities of the country, to try it out for me, and report back.
Feel free to start thinking of reasons to decline, but your payment would be retained ownership of a gadget that you probably do not already own. :D
-
So, I might be on the scrounge for folks at the other extremities of the country, to try it out for me, and report back.
You know where you are located in the UK but, when considered from the viewpoint of other members, your location is an unknown. Hence it is somewhat difficult for members to decide whether to volunteer . . .
-
@7LM Count me in, provided Mrs Weaver approves, haven’t asked her yet because she’s asleep. It might not work properly here because of the mountains, particularly Beinn nan Càrn which is adjacent to the house (we are on the slope of it) to the west, which cuts off an hour or so of daylight in the evening, and the amount depends on the time of year; as the sun goes further south in Winter the time loss due to the mountain’s profile reduces, and in June the amount of loss is a maximum approximately.
-
You know where you are located in the UK but, when considered from the viewpoint of other members, your location is an unknown. Hence it is somewhat difficult for members to decide whether to volunteer . . .
South. But with a Scottish accent.
Latitude expressed as a precision towards the upper end of my expectations of accuracy for the device
Note that I don’t, as a rule, respond to PMs. If anybody would like to volunteer, best to do so in thread.
Note also that this is hypothetical, I may never succeed in producing a device suitable for widespread deployment. :'(
Admin - latitude figures removed at request of OP
-
I am at latitude/longitude: 57.2239, -5.8295 according to https://www.doogal.co.uk/LatLong.php (https://www.doogal.co.uk/LatLong.php)
-
It might not work properly here because of the mountains, particularly Beinn nan Càrn which is adjacent to the house (we are on the slope of it) to the west, which cuts off an hour or so of daylight in the evening, and the amount depends on the time of year; as the sun goes further south in Winter the time loss due to the mountain’s profile reduces, and in June the amount of loss is a maximum approximately.
That's the sort of challenge I'll need to think about solving.
The first thing I aim to do with the breadboarded version is simply get it to continuously display the actual output voltage of the sensor as measured by the ADC . That'll allow me to take some notes around sunset and sunrise, maybe sketch out some graphs and gain insight into how it is affected by weather, tree-shade, and the likes. I've also have a few different LEDs to try, each with different lens patterns, as well as some additional fresnel lenses I can stick over the LED to see if it helps.
Then, unless I give up at that point (which is possible), I can start to think about what kind of smoothing algorithms might work best, allowing genuine events to dominate.
-
I wonder if the sensor would only need to have an unobstructed view in a southerly direction? I.e. On the internal ledge of a south facing window. :-\
-
I wonder if the sensor would only need to have an unobstructed view in a southerly direction? I.e. On the internal ledge of a south facing window. :-\
That was my own first thought. But the very few experiments I have conducted so far, using a completely different device containing the same sensor, suggest that direction doesn’t matter that much, the photovoltaic current simply drops through the noise floor quite quickly (1-10 mins) after sunset. It’s Currently facing vaguely West, seeing mostly light reflected off a brick wall, and it triggers broadly symmetrically either side of sunset/sunrise.
I suspect that maybe our eyes are better than we realise at adapting to twilight, being well within our pupils’ range of automatic adjustment, making the onset appear more gradual than it is. That, at any rate, is what I am hoping will prove to be the case. :)
-
. . . the very few experiments I have conducted so far, using a completely different device containing the same sensor, suggest that direction doesn’t matter that much, the photovoltaic current simply drops through the noise floor quite quickly (1-10 mins) after sunset. It’s Currently facing vaguely West, seeing mostly light reflected off a brick wall, and it triggers broadly symmetrically either side of sunset/sunrise.
That is useful to know. So the mountainside looming over "The Weaving Shed", on Skye, should not be a problem.
I suspect that maybe our eyes are better than we realise at adapting to twilight, being well within our pupils’ range of automatic adjustment, making the onset appear more gradual than it is.
I, too, hold that same suspicion. :)
-
It might work, I’m not giving up. :graduate:
The device is now capable of measuring ambience via its LED, photo current converted to voltage with the PIC’s Op Amp and measured with the PIC’s ADC. As a temporary aid to development, it simply displays the A/D conversion result, as a decimal number, on the LCD. In other words, I’ve spent all this time creating what already exists... a digital light meter. :D
In pitch darkness, I get a reading of 2 which presumably is the background noise and diode noise, as amplified. Full scale ADC would read 1023. I didn’t get it working early enough in the day to confirm, but my aim was that mid day readings would pretty much max it out. Each unit represent a step of 2mV.
Here are a set of readings starting 40 mins before official sunset, from three different window sills, facing different directions. It was placed on the sill in turn, with sensor diode angled at about 45 deg skywards, capped with a fresnel lens.
T-40 29/23/20
T-30 16/13/11
T-20 12/10/8
T-15 11/9/8
T-10 8/7/6
T-5 7/6/5
Sunset 5/4/4
T+5 4/3/3
T+10 3/2/2
So with this configuration at least, today, a threshold of ‘4’ would have got it about right in all three locations. I can make the numbers bigger by changing the amplifier feedback resistor, which is probably worth trying. It might over saturate at mid day, but I don’t think that matters.
In two of the test locations, switching room lights on/off made no difference. In the third location room lights were a bit of a spoiler, adding about 8 to the reading at sunset. This needs further thought. :-\
-
Maths idea/question, for Weaver or anybody else....
I'm still not sure if I need trig functions. It depends whether there is any credible prospect of doing latitude location. But if I do need trig I'll need to find some shortcuts as, fool that I am, I am still determined to do it all in PIC assembly and if possible to see no name other than my own in code module headers.
Priority is to reduce memory footprint, including the footprint of big lookup tables as well as the footprint of large code libraries. It doesn't have to be fast, it will have all day (literally) to work out the answer.
One idea I have is to approximate the smooth Sinusoidal graph, that we all recognise, into a number of straight lines between points - thruppenny-bit style. These reference points would be accurately calculated and hardcoded at build time, and there'd not be many of them - say, 8 or 16, built into a small look up table.
When I want to approximate SIN(x) for some arbitrary value I simply find the points that surround my value, and scale it as a linear transition between these known points. I already have arbitrary precision binary multiply & divide working, so the scaling would hold no fears.
I have no idea whether that is a reasonable way of approximating Sin/Cos/Tan, or even a common way of doing so, or it is laughably flawed. ???
-
It’s what I would do. And linear interpolation is cheap, quadratic or cubic is more code and more time; and if you’re not too happy with the accuracy you can just add more points. You could also just do 0-45 degrees not 90 and use Pythagoras but that means having to calculate square roots which is awkward. I think you’re doing the right thing keeping it simple.
-
Many thanks for the fast response.
And linear interpolation is cheap,
Ah there’s a name for it. I knew it was vanishingly unlikely be be something I had newly ‘invented’. :D
I might start experimenting. My reference points don’t have to be evenly spaced of course, they can be more tightly packed at points where the graph is more ‘delicate’, ie top and bottom of a SIN curve.
I’m not afraid of square roots as they can be crudely iterated with multiply/divide. That’d be embarrassingly slow but nobody would ever know, as speed is just not an issue. I already have and use a very crude integer Log2 (Based on counting leading zeros), which might help to provide a first guess, for iterating a square root.
-
Indeed. Count leading zeros and then that gets you the log2, then you shift the result by half of that log2 when you’re done. That was my very first ever job, a 32-bit integer square root in Z80 code. It didn’t have to be very accurate, 1% very roughly but no exact choice was made regarding accuracy as all that mattered in this application was speed as it was called thousands of times. It was to calculate the vector that was normal to a surface and then the dot product of that normal with another vector. It was to calculate the illumination of light reflected from a rough surface (reflection in random directions) angled at some angle to a vector that was the direction of incident light from a source.
-
Interesting, Weaver.
My own biggest programming endeavour, since retirement in 2008, was to write a 3D OpenGL engine for an iOS App. So I learned a lot about vectors, normals and dot products and much more. Riveting stuff, for a guy who’d spent his career in comms protocols, it was an entirely different challenge.
My OpenGL engine was a lot of fun. Sadly Apple have deprecated OpenGL so I have to get around to rewriting my 3D engine in something else.. Not looking forwards to that at all, especially as I have now forgotten all the stuff I learned, about vectors, normals, etc. :(
-
In the remote chance that anybody remains interested....
Clock is nearing initial trial stages.
Crystal controlled system timers working, a scheduler rattling the cages of other system modules every 1.0 seconds. Lcd interface working reliably, interfaces that I have given names like “printChar”, “printDecimal24”, “printString”, “printHms” etc all work as the names imply. 8/24/32 bit maths (multiply/divide so far) behaving itself.
Ambience sensing works, though needs a smoothing filter.
Day/night clock setting logic has been implemented as a finite state machine. Crude for now, but easy to enhance when testing begins.
Next stage is...
1. Deployment on a window sill, and watch for a few days.
2. Concurrently, start to think about actual packaging, decide what batteries to use, find a decent looking plastic enclosure box. Maybe even get an initial PCB design submitted to the nice, amazingly cheap, Chinese fabricators. :)
But please tell me to stop if I’m boring you all, even if simply inferred by silence. :D
-
I'm still following your progress and, I suspect, so is Weaver. :)
-
I'm still following your progress and, I suspect, so is Weaver. :)
Thanks for the encouragement. :)
-
Not understanding a lot, but watching with interest.
-
I most certainly am, am in awe
-
I am wondering if any of us know much, or anything, about filtering algorithms.
I intend to filter/smooth the sensor diode with an algorithm I have used before, something like X = X - (X>>8) + newValue. To get the rolling average, you take X>>8.
It's a tiny memory footprint and it's fast, and does seem to work as a low pass filter. Taking a few tens/hundreds of samples per second, it succeeds in smoothing the signal from (say) 50Hz fluorescent lights. But Apart from being vaguely intuitive that it will work I don't pretend to understand the maths, I don't even know what it is called, it's just something I've seen suggested in online discussions. :-[
I will also want to filter my 'hours of daylight' variable, such that the value I use for clock setting is averaged over a few days. I wonder, is a similar algorithm appropriate? Maybe combined with a 'peak' detector, such that when I see two successive days with vastly different values, I tend to favour the latest sunset and earliest sunrise as being more likely to be truthful?
-
You can keep a rolling window of that last n values, when a value ages out of the oldest end of the window, we subtract that value from a total, when we add a new latest value in, we add it to the total. Suggest doing as you did and taking the total and subtracting the values
static size_t youngest_ix = 1;
static size_t oldest_ix = 0;
/* remove oldest from the total */
int oldest_val = circular_buffer[ oldest_ix ];
static int moving_total -= oldest_val;
oldest_ix = wrap_around( oldest_ix + 1 ); // size_t wrap_around (size_t x ) { return x >= circular_buf_size ? 0 : x; }
/* add in most recent value into the total */
int youngest_val = circular_buffer[ youngest_ix ];
moving_total += youngest_val;
youngest_ix = wrap_around( youngest_ix + 1 );
const int = 3; /* or anything you like */
weighted_moving_average = moving_total + ( youngest_val >> nshift ) - ( oldest_val >> nshift ) ;
This weighted moving average, when unweighted (ie nshift = 0 ) is called convolution (https://en.wikipedia.org/wiki/Convolution) in mathematics; it comes up in Fourier transforms when you consider a time domain response of a filter that is bandwidth-limited in the frequency domain, ie. limited to pass frequencies between some f1 ≤ f ≤ f2 the Fourier transform of a square function, one whose whose frequency response is = ( =1 ; f1 ≤ f ≤ f2; = 0 otherwise).
But there isn’t much mathematics in it as you can see; add up the last n entries sum [0..n-1] (where n = circular_buf_size ) and take that sum over entries 0..n-1, then take sum over entries 1..n, then 2..n+1 etc and this sequence of sums is the moving average. The size (width) ie no. of entries in the circular buffer is the crucial parameter that determines the amount of smoothing. The more entries in the buffer, then the more the smoothing effect. Here I’ve just incorporated your idea as an additional tuning technique to also help tweak the behaviour as you please. There are many variations on this that you could try, but this is one, which I don’t particularly like because of the computational cost unless you use shifts instead of division and multiplication and also there could be problems with overflow which are nasty, so I prefer the earlier expression, although it’s not as flexible.
/* An alternative technique; the weighting factor probably really ought to be a power of 2 to reduce the cost of the division */
const uint weighting_factor = 2; /* for example */
weighted_moving_average = (moving_total * weighting_factor + (youngest_val - oldest_val >> nshift )) / weighting_factor;
Need to watch out for overflow in the multiplications and in the additions particularly where the buffer complete sum is calculated.
Remember if you have to go this way the old trick that divisions can always be done as follows by suitable (( x * ((1.0/div) << n ) ) >> n ) where ((1.0/div) << n ) is an integer constant and the multiplication needs to be done in enhanced width. You might want to add on an offset value after the multiplication to adjust for the rounding error when doing the division represented by the >> n, worth doing all this with a spreadsheet, integer maths and lots of tweaking to get the best accuracy. Probably best avoided though where accuracy matters.
-
Many thanks, Weaver, much for me to think about there.
I had hoped to do deploy a working clock (no filtering) today before sunset, then wait to see what it showed in the morning. Code to do so was supposedly complete, just needed a bit of testing with the blinds drawn and a torch to simulate sunrise/sunset events. Unfortunately today has not gone well, too many gremlins, not yet debugged and I have run out of time - presence is required elsewhere for the rest of the afternoon. Ah well, that's software for you. ::)
-
If any of that doesn’t make sense then please do shout.
-
If any of that doesn’t make sense then please do shout.
Thanks Weaver.
I have now deployed Clock, but it will continue to display a count increasing from 00:00:00 (at power on) alternating with the word “Calibrating” until it encounters a sun transition, which will be tomorrow sunrise. It’ll then show some diagnostics. But nothing very exciting will happen until tomorrow sunset, when it might, just maybe, if it works, set the hh:mm:ss to a meaningful estimate.
In other words, since my prototype is unavailable for development, I should have all day tomorrow to consider your suggestions. My gut feeling is, I may well need to shout for help. :D
-
The circular buffer is the usual thing; you have a start index and an end index when wrap( last_index + 1 ) == wrap( start_index ) then the circular buffer is full and then the last_index chases the start_index for evermore. Could initialise the to start_index = 1 and last_index = 0 say; every time you increment it your wrap the value round to keep it inside the buffer’s index range. Could have 8 entries or 256 entries or whatever, then you only need a byte for an index type.
-
In my career, we frequently used what we often called “ring interfaces” for transfer between, say, a front end card and the host CPU. The read index chased the write index, which I think is similar to what you are suggesting.
As an aside that’s not relevant here as my PIC is single threaded and written in assembly... people would often scream “hardware bug!” or “compiler bug!” when they found corruptions in such a dual CPU interface. Despite careful code sequencing they’d sometimes find that an index had changed yet the corresponding value remained as it was. Nearly always the hardware engineers and compiler writers were off the hook, when one explored the lesser known topic of memory barriers. :)
-
Exactly; a ring buffer I would call it, often came up in i/o or protocol engineering. There are various ways of doing it.
-
Another point worth mentioning is that RAM on the PIC I’m using is split into disjointed “banks”, each bank separately addressable, and each bank containing 80 (decimal!) bytes.
For most purposes the disjointed banked RAM is a pain. But with carefully judged parameters the banks might actually lend themselves quite well to such circular buffering. :)
-
think about how much soothing you want and that determines the size of your ring buffer. It may be that quite a modest number of entries will be really effective.
What size/data type are the entries going to be? bytes/words/longs? signed or unsigned?
-
A thought - might be worth playing around with a simulation in a spreadsheet and/or a very short C program to get the numbers to your liking rather than have to fiddle around with machine code debugging both the code and the algorithm + maths all at the same time.
I was once writing a crude interpreted programming language to run on a Psion Organiser I which had a 0.92MHz Hitachi 6301 in it with 2k of RAM and no ROM at all! So it was 2k for everything code, data and user program code too. I first wrote part of it, the syntax checker, on a DEC VAX in Pascal to get the algorithms right and get all the bugs out of it and then after that I just machine coded it by hand very quickly. When everything was finished there was the fun of trying to get it all crammed into 2k.
-
think about how much soothing you want and that determines the size of your ring buffer. It may be that quite a modest number of entries will be really effective.
What size/data type are the entries going to be? bytes/words/longs? signed or unsigned?
I currently count things, length of days and of nights, in seconds. 24 hours in seconds is 0x15180 requiring a 17 bit width. I may eventually compromise in a count of 2-second intervals, which fits into a much more hardware-friendly 16 bits.
-
A thought - might be worth playing around with a simulation in a spreadsheet and/or a very short C program to get the numbers to your liking rather than have to fiddle around with machine code debugging both the code and the algorithm + maths all at the same time.
Also has the advantage of not eating into the prototype’s PIC flash endurance. Microchip say it’s 10k erase/write cycles which sounds a lot, but hitting ‘run’ on the debugger button every few minutes soon adds up.
-
You could store 17-bit values in 24 bits and just waste the last 7 bits, or could fully bit pack them; I suspect you might not need to have that many entries for your smoother’s ring buffer, fewer entries allow more spikes through, more entries provides powerful smoothing. Say you had 16 entries each 24-bit entries; that would easily fit into 80 bytes, but I don’t know you might want more entries than that. One other thing is you could make three parallel arrays one for each byte of your entry, low, mid, hi byte, and put 64 entries in the triple of parallel single-byte ring buffers and that would fit into three 80-byte banks. To bit pack to 17 bits exactly, means all that multiplication and masking and shifting - a real pain. Did an enormous amount of that kind of stuff with variable-width bit-packed data fields that straddled across byte boundaries, used for data compression.
-
At the moment I store my 17 bit vars in 24 bits of memory. And that preys upon my conscience.
But the PIC has 1024 bytes of SRAM, of which I have only used about 60 bytes so far.
The time may come when I need to optimise, but even if I give generously to the ring buffers, chances are it may not be necessary.
-
Oh dear, my first test launch is listing badly. :-[
When the lighting level hit my (arbitrary) threshold, about 5 mins after sunrise, it triggered the correct state transition. Clock then began counting seconds. Idea being, when it hit the same threshold this afternoon, it would calculate calculate Solar Time offset and synchronise the quartz controlled clock.
Unfortunately... a bug. I'd simulated the whole cycle by switching a desk lamp off/on, but I'd been impatient, and used 'days' that lasted only a minute or two. That all worked but when it encountered a real day my counter went skew whiff after 256 seconds.
Checking the source code it is a very obvious and trivial bug, exemplary of the perils of assembly language, an 'addwf' mnemonic that needs to be 'addwfc'. But I have to wait half a day before a chance to start another test. :(
Thinking further I'm reasoning that, in addition to estimating Solar Noon by sunrise/sunset, a very useful additional parameter will be to count seconds between sunrise transitions. That should give me the length of a solar day, and would be self compensating no matter which direction I am facing, or whether in the shade of mountains or buildings. And Solar Day length opens up all sorts of avenues for deductions and assumptions. ;)
-
Think about a really modest sized rig buffer, only say 16 entries, would that give you enough smoothing? Might be an idea to collect some real data first, by hand, with a clock and stored in a spreadsheet. That would give you something real to run your spreadsheet smoother or C smoother to work on and see how big the smoothing effects are. Ask yourself - what would the average of the last say 16 days be like?
-
Indeed I am now in data gathering mode. I’ve treated myself to a second prototype. It meant sacrificing a PIC as it needs to be soldered to an adapter but heck, they’re only about 30p each last time I looked. That means one device can now be semi-permanently deployed, gathering data that I can read from screen and simply note with pencil and paper each day, to get an idea of the trends.
One compromise with the averaging is, illustrated with an extreme example... It is now nearing the shortest day and if I were to average sunrises over the past 90 days it would be skewed numerically downwards because, averages aside, the sun actually rose earlier in Autumn. Similarly the sunsets would be skewed upwards. Taking the average of each would not yield a useful figure for today’s daylight seconds. It’d not even yield an average for day 45, as the changes over time are not linear.
With that in mind, I think the shorter the period of averaging the better, as the lesser the above errors will be. I’m sure mathematics can be used to correct these errors, but I’d rather just dodge them.
-
I think that’s entirely the right approach.
-
Bit of Eureka moment. The device survived its first full sunset/sunrise cycle and has successfully synchronised its display with the quartz reference.
It is telling me the time. :yay: :drink: :graduate:
Clock is about 11 minutes fast compared with my wristwatch, which is slightly disappointing as true solar time today should be just a couple of minutes 4 minutes ahead for my location. So a probable error of about 8-9 minutes. ???
edit: Corrected solar time adjustment figure.
-
It’s always nice to get to a point like that.
-
There's nothing like a major annoyance to start the day.
I decided that with progress made on the clock, it's high time I got my source and designs under version control.
I have my own subversion server on a Linux box, and svn client command line tools on the Mac OS are included with XCode, which Apple provide. Except they are not, they've been deprecated. I updated XCode a few weeks ago and have just discovered, svn command has evaporated. :o
Most common suggestion from Google hits is to install svn via homebrew, but I whilst I have unswerving trust in the good intentions of developers of such software the fact is it weakens my system's security, leaving me vulnerable to so-called 'supply chain' attacks. Same can be said of Linux of course, but the difference us that a 'pure' MacOS is pretty darned safe when shipped by Apple, it's a shame to break the seal, so as to speak. Not entirely happy, now! >:(
-
Could you build it yourself?
-
Could you build it yourself?
I'd rather not. But I think I have found a solution.
Since I have old versions of XCode installed, there are also old versions of svn lurking within directories you'd never think of looking... still seem to work - or at least, 'svn list' does as you'd expect.
-
SVN crisis averted, you may be interested to see what I can see.
On the 2nd line, '255' bottom left is the A/D conversion. ADC has 10 bit resolution but values above 100 or so are of no interest here so I cap it at 8 bits.
The middle number is the duration of daylight in seconds, currently incrementing. The last number is the duration of darkness, frozen until next sunset, telling me there were 56560 seconds between perceived sunset last night and sunrise this morning.
The ugly loops of black wire are just a scaffold to stop the LCD from slithering around. They could do with tidying, but I don't want to disturb it when it's running nicely.
The big blob of blue tack is a temporary shade for the sensor diode, giving it some immunity from room lighting.
I hope nobody looks too closely at the soldering on that SMD chip adapter. I'll try to be neater when making the final device(s). :blush:
[attachment deleted by admin]
-
With apologies for consecutive posts....
It has just triggered today's sunset. Based on today's daylight seconds, clock has resynchronised exactly 7 minutes and 24 seconds ahead of GMT. timeanddate.com predicts that for my location, the difference should be 4 minutes (they don't give seconds) so my clock is currently accurate to true Solar Time within about 3-4 minutes.
Better still as I watched it nudging down towards the trigger, I noticed a bug. I've set the transition threshold at '10 units of ADC', but on the way up it's testing for x>=10, on the way down it's testing x<10. That bug slightly delays the 'sunset' transition by a few minutes, perhaps just about the right amount bring it into exact agreement with timeanddata.com. :)
Beginners luck, fluke, call it what you like, but I'm reasonably happy so far. The Post Office speaking clock service could soon become completely redundant, once I enhance my device with Weaver's averaging suggestions. :D
-
@Weaver, ref your assistance with filtering & smoothing...
I found an interesting Application note from Microchip. It explains how to use the on-chip ADC filtering computation that's on some PICs.
My chosen PICs don't the ADC filter computation feature, just basic ADC. But the description of how the low pass filtering works is tantalisingly close to my existing diode smoothing...
http://ww1.microchip.com/downloads/en/AppNotes/AN2749-Using-PIC18F26K42-ADC2-00002749A.pdf
Low-Pass Filter (LPF) – Low-Pass Filter mode functions similar to Average mode, except that after an initial accumulation of samples based on the ADRPT setting, the module continues to accumulate samples and perform filtering operations indefinitely.
The only enhancement in that, beyond what I already do, is the initial averaging. Once there's too many samples for averaging, it drops into what I already do. I can easily amend my code to the initial averaging and then mode-switch.
Is there any great advantage in doing more than that? The sunrise and sunset variations due to cloud etc can be seen as equivalent to AC noise on a DC signal, so I'm thinking it's the same problem, with same solution?
I'm also thinking that the best parameter candidate for smoothing is the length of a solar day, Noon to Noon. That number changes fairly gradually so should respond well to smoothing. And moreover, by looking at absolute value (together with trend) I think I can attempt to compute the time of year whenever I like. This Wiki page has probably already been linked, but it talks about Solar Days...
https://en.wikipedia.org/wiki/Solar_time
-
It is indeed the same kind of idea.
-
I’m thinking there might be a way of trivialising the run-time maths for full precision Equation of Time.
Why don’t I simply have a look up table consisting of the incremental delta, in seconds, for EOT of each day in the year? That delta never exceeds 16 seconds per day which is nice, as the table can be expressed in 4 bit entries. It could even be further compressed with zero suppression but probably not worth the bother, memory’s not that tight. The sign of the delta changes four times a year, but that can be handled.
My table would also be slightly modified from the usual EOT as, when creating my table, I’d feed the EOT through the same smoothing filter my clock will use, thus compensating for the phase delay of my filter.
It wouldn’t be fast as each calculation, disregarding optimisations, would involve up to 365 16x4 bit additions. But who cares, it only needs to be done once a day.
Sound promising? Unless I awake tomorrow thinking it is a stupid idea, or am told so by Weaver or anybody else, I may start trying it out on a simple C program. :)
-
I think it is a good idea, and I agree I wouldn’t bother with the compression thing.
-
Among the pain of writing fairly sophisticated (by my standards) code in assembly, one small experiment looks worth pursuing.
If it is ever to be a useable indoors device, it must be usable with room lighting switched on. Early experiments had already shown that to be a challenge, requiring careful shielding of the sensor to avoid being dazzled by the room lights.
However, I vaguely remembered a snippet of information, that when used as a photodiode, an LED is sensitive to wavelengths equal to or shorter than the colour it emits. I'd been using red LEDs which would thus be sensitive to most visible light, whereas I think most indoor bulbs are weak at the blue end of the spectrum. So I bought some blue LEDs for comparison.
The blue LEDs still provide a strong signal from overcast daylight, but seem to be almost totally immune to room lighting from incandescent bulbs. A room lit by LED bulbs has a small effect, but probably at least an order of magnitude less interference than before. What remains to be seen is whether the blue LEDs are more affected by cloudy sunsets vs clear skies. I wait with interest to see. :-\
Only problem is, I began this project on the basis that
It would consist of a photocell, a microcontroller, and an LCD character display. All of these are conveniently to hand, residing in a drawer labelled “bits & bobs”
I'm now going ahead with a different PIC, a different LCD, and different photocell. Is anybody familiar with Trigger's broom? :D
-
The blue LEDs show promise. After a full sunrise/sunset cycle with blue LED, clock display synchronised just 12 seconds adrift from timeanddate.com prediction, for my location. And it seems completely immune to toggling on/off room lighting in my test location, even around sunset/sunrise. That accuracy is undoubtably a little bit of a fluke, but encouraging all the same. :graduate:
I’ll probably stop updating this thread for a while as development is likely to be suspended during Christmas festivities, and I’ve an awful lot of code(/bugs) to write to move it forwards. But if it ever does evolve from ‘breadboard prototype’ to ‘production candidate’, I might update again.
Thanks for all support & encouragement. :)
-
. . . if it ever does evolve from ‘breadboard prototype’ to ‘production candidate’, I might update again.
I hope so. It has been an interesting project development to follow.