Change Daynight to daemon

[virmaior]

Basically, rewrote some of the code related to the daynight management to make it work as a daemon.

This allows:

1. Intervals to be set at the second level instead of minute
2. Multiple methods for determining the whether to switch day night:

• previous behavior where high luminance / low luminance directly triggered made default and called "limit"
• "dlimit" same as limit but tries to prevent rapid switching by setting a max repeat count before cooling down
• "switch" - here the idea would be to change only if there's a big luminence change over the interval regardless of the ranges
• "file" determine state based on a file.

3. removes a buggyness associated with reading out of the crontab file to figure out the current interval (at least in my case, the basic idea is that from 8pm until 6am I just forced it to do night mode to avoid rapid switching before and only used daynight during the day time hours).

Also cleaned up the config html page by reorganizing things a bit more logically to what they do.

I switched dusk2dawn to use the file method for directing the daemon.

Hopefully there's something useful in this.

[themactep]

good job, but there is a problem. i was never a fan of a shell script looping in a blind fork. we tried that in openipc a few times and it always ends badly. we need either a proper standalone C-daemon with direct access to /proc, smart error handling and such, or we need daynight mode implemented on a streamer level. running with cron once a minute is a safe-ish tradeoff. i won't be pushing shell daemon further at least without a thourough testing. so i am going to compile these changes into an experimental firmware for a test camera sitting in a dark room with lights coming on and off at random intervals, we'll see how it affects the running system.

[virmaior]

That does make sense.
I mostly cribbed this from the /sbin/record code as a base but I was wondering whether that sort of loop was really considered "best practices" in bash world.

Doing it half with cron , half with a while loop might be a good compromise to avoid runaway situations, but depending on the use cases of others, 1 minute frequency of the cron job might be incompatible with what they're looking for.

For my own use case, it's mostly that I'm fine with day/night switching with a limiter during the day but at night, I have basically 3 cameras hitting the same cage and need them all to go to night mode at the same time or else I basically get purple footage from the IR light on the one still in color mode.

Writing a daemon in c is quite far from anything I've done in the last 20 years, so that'd be up to other contributors to do.

[themactep]

If the goal is to sync multiple cameras switching then looping with higher frequency won't be a good solution anyway.

What you could do is to make one camera a master monitoring the environment and triggering all cameras at once. Utilize ssh to run daynight <mode> on dependent cameras, or you can do it in a fancy way, sending the signal to a MQTT broker. We have mosquitto_pub in the firmware to send messages to the broker, but we also include mosquitto_sub for subsribing to topics.

You can even make all cameras dependant on an external sensor, say a rf433 photocell or an ESP module that would measure illumination and notify the broker of changes.

There are many creative ways of offloading the pressure from cameras and shifting it to other, more specialized devices.

[virmaior]

oh it's on the list of ideas.
This improvement was largely in response to
(1) other people pointing the current version was non-responsive (since it functions only on the minute level)
(2) adding the dlimit functionality to avoid minute-by-minute ping-ponging between day and night modes

after that using the file functionality was planning to see if there's a way to webhook this so that it does what you're suggesting.

would have to learn a bit more about mqtt to understand how to use it for this.

for my purposes, each squirrel's set of cameras would have a master and slave but the master could relay that to a rpi rather than directly.

[themactep]

as i already explained the issue with running gain reading faster than once a minute. we did not made it to run in a loop not because we are stupid and did not know how to do so, but because the idea is stupid. mode switching should be event-driven, not abusing the camera with an avalanche of nagging requests "are we there yet?". there are proper ways to implement it but i don't see anyone willing to work on that yet. let the existing delay be a stimulus for someone to step in and make it right. it does not bother me so it's on the low burner somewhere at the end of the list of improvements.

for dlimit, we have a set of two thresholds for swithing to day and to night mode. the difference between these two values provides hysteresis to avoid repetitive switching of modes on every run in a borderline conditions. no need to introduce another layer of complexity with dlimit. all you have to do is to adjust the thresholds to match your settings.

[virmaior]

For the priority of different items, obviously for each of us our priorities are things that matter to our use cases. And no one is required to work on any part they don't want to.

Why I think this should be worth considering is that this is one of two places where I was surprised to have less functionality than the Ingenic Reference firmware. On Wyzev3 and atom cams,

1. stock firmware constantly checks day/night and can change it less than minute increments
2. stock firmware local recording quality is higher

So for me these are two places where improving them makes the case for Thingino more compelling as a consumer-usable replacement.

Turning to the code, I get you're saying that bash loops have a problem and this was often a problem in openIPC. I can see how they could run away but am not sure I'm fully grasping the problem.

I agree that the state switches should be event driven but isn't that event a change in luminance? For the polling for the luminance, "nagging" suggests its a resource intensive check, but isn't the camera sensor constantly calculating this and storing it anyway?

For the point about the thresholds and dlimit, The standard limit code does not effectively prevent infinite looping. It is
two logical isolated conditionals:

(1) During state X, if L < A, change state to Y
(2) During state Y, if L > B, change state to X

Since state changes cause changes in L through filters, IR lights, and color/monochrome switching, these two checks are wholly independent. Infinite loops can occur, because there's no reason state X isn't beneath A and then state Y above B each time. (Someone separately mentioned this in discussion and I'd experienced it myself).

This can be addressed in three ways:

1. Fine-tune the values of A and B to minimize/prevent this
2. Recognize when it occurs and prevent it
3. Turn off daynight

In your version of fine-tuning, you're saying each user should go in and fine-tune the values.
D-limit is a version of 2.

Functionally, what I've done for my use case is: Only run daynight from 6am to 8pm. This slightly breaks the config-daynight cgi which tries to decipher a line from crontab and assumes it's going to be a number and not a range. (I also set the value for night -> day transition at about 100)

Rather than what I've provided, it seems like an ideal solution works by having the camera's core streamer (1) raise events when (a) there's either a sharp luminance change or (b) when threshold values are hit, and (2) tunes values automatically tune to avoid repeated switching.

[themactep]

i've sent you an invite to the new streamer repo which is at too early stage to publish. check files in modules/photosensing/ there. that is day/night mode done right. we'll be there sooner or later but not today :) join the development!

[virmaior]

i've sent you an invite to the new streamer repo which is at too early stage to publish. check files in modules/photosensing/ there. that is day/night mode done right. we'll be there sooner or later but not today :) join the development!

Okay started reading the docs.
Initial thoughts:

1. The new streamer is very impressive and the performance improvements look massive!
2. That's interesting to use counters to require it to meet the switch criterion multiple times in a row in a direction.

It will take me some time to grasp this, but I'll try to figure out if there's any way I can meaningfully contribute there.

So I should close this pull request as abandoned right?

[themactep]

i would let it hang at least as an ispiration. besides i did not say i am going to reject it. i said i am not a fan of the idea, but if people who need the frequent scanning pick the changes and test them thoroughly and report back that it suites them well, we could add it as an option, under a checkbox saying "yes, i am crazy enough to run it and won't complain".

[themactep]

@virmaior, check this out. only support t31 now. testing, ideas and prs are welcome.

[virmaior]

Reading through the daemon code, it looks like it will do everything I'm hoping for in terms of responsiveness and avoiding oscillation. I really like the hysteresis code and the multiple approaches to brightness.

I will try to get this on a camera ASAP and see how it does.

Not an issue with this code, but for the web interface, it seems important to separate the features that trigger day/night and the pieces included in day night. Current webpage doesn't really make clear how the two pieces are related (and as others have pointed out interacts a bit confusedly with dusk2dawn).

It seems like there's three separate pieces:

1. Daemon's performance at detecting day/night brightness changes and avoiding flipping back and forth.
2. Definition of what happens day/night (which IR LEDs turn etc)
3. When the daemon is operational (dusk2dawn, time windows)

For 1, this seems like it will be an awesome implementation.
For 2, just needs the web interface to be clearer on that front.
For 3, it seems like every user is going to have different needs.

[Oldsnakes]

Is that worth to try by embedding daynight function into the Prudynt-t? The VideoWorker has a loop(s) (with timestamps) already that can monitor the GetTotalGain (and rolling average) while fetching the stream packets. It also has the interface to WebUI already that can talk to the user settings. By adding GPIO controls it can turn on and off lights. The image_running_mode already there too. It also can corrordiate with motion to change light settings (for deterring).

The extra benifit may well be for the camera with multiple sensors(2 or 3, which seem rolling in lot laterly, do we capable doing that already?).

[themactep]

prudynt used in the master branch of thingino already has that feature.

[virmaior]

As far as I'm concerned this PR is obsolete. We've got a snazzy new way to manage day-night from themactep written in C.
it solves nearly all of the problems with the old (b)ash approach.

(I should qualify that I haven't actually been able to test it due yet, but the code looks great).

[Oldsnakes]

Great, this should solve the daemon issues I just run into.
(Last night I have the IRCUT bounce at 10 Hz rate that drives me off the bed. :| )

[themactep]

we run daynight once a minute with cron. it is impossible to get it switched 10 times a second (10Hz) with our recent solution.