To get a little further with the tmpo data, I decided to not directly move into the tmpo-daemon, but in a first proof-of-concept provide an extra daemon to handle query requests for FLM local tmpo data.
The result may be "admired" in https://github.com/gebhardm/flmlocal#querying-tmpo-data with the respective lua-code.
The idea is to handshake query requests and corresponding chart visualization directly on the fluksometer utilizing the existing local visualization capabilities, without any necessity to a web connection.
It works in the way that a query request is sent via MQTT to the FLM query deamon, relating tmpo files are retrieved and passed back as a related MQTT message.
This works fine, but is rather slow as sent back messages may have a high latency. So, looking forward to a discussion on how to increase the charting efficiency without an external database or "funny Python scripting".
Nice one Markus!
Just glanced over your script. I would not publish the replies with QoS=2. This will create lots of overhead, and thus latency. Futhermore, when publishing over MQTT, the unzipping will be done in JS, wich might cause additional latency (could you verify this?). With the HTTP API on flukso.net (and a proper encoding header), the unzipping will be taken care of by the browser itself.
I actually set the answer's QoS as 2 as I got tremendous latency with 0 and 1 (as well as double postings); but this is not in stone, of course ;-) - the query request is QoS 0, of course.
With respect to the zipped packages sent over by MQTT, this also is not in stone but a working variant, and as written, not optimal, even though HTTP handshaking from within a page I regard also as not really convenient. Work in progress, though.
As next step I will provide a chart with queried values on the FLM directly and then we may refactor all this to an architectural gem :-)
to be continued...
BTW: https://github.com/gebhardm/flmdisplay/tree/master/tmpo behaves just fine on the query replies; I actually experience no tangible latency difference in receiving with our without unzipping.
Success - My proof-of-concept to provide a complete FLM local visualization capability with near realtime gauges AND charting of locally stored TMPO data is available for download and trial; refer to https://github.com/gebhardm/flmlocal#querying-tmpo-data for details; the TMPO charting capability works with the queryd.lua script running and firmware v2.4.4 onwards.
Note: This is all MQTT based, so there might be some latency; also as tmpo blocks are visualized, I (currently) did not match query intervals sent and data retrieved; there is a "detailing option" - use mouse click on the chart to select a sub-interval - for drilling into the chart.
end-of-threadThe FLM local TMPO query now shows only the selected time interval, it is capable to dig into it by selecting a sub-interval by mouse-click, single graphs may be switched on/off, MQTT delivers a convenient speed and is able to show more data at once as I ever achieved with a Raspberry Pi/MySQL/websockets.
Only "minor" issue: As TMPO stores counter values only, thus time between, for example, reaching consumption of 1 Wh (for clamps), the differentiation for actual wattage values tends to "swing", thus I provided a rolling average on 5 continuous values to smooth the graph - but as this is a proof-of-concept (and to my convenience and needs ;-)) you may lay hands on what actually is shown in the chart...
Have fun.