In another forum post (https://www.flukso.net/content/first-time-install-problems) user "jgysenbergs" raised following question "Do you know if there exists a fancy web interface with nicely designed gauges?"
On this behalf (and on Bart's request :-)) I'd like to introduce this dedicated thread, reposting my reply for an open and fruitful discussion:
"Nicely designed gauges" are like "electronics in cases"; it takes an hour to solder a solution and weeks (if not years) to finally pack it into a nice box... Yesterday I actually started to get into the MQTT stuff again (the actual data collector runs smoothly on a RaspPi for some while) - I experimented with http://www.flotcharts.org/ but am not convinced yet; Jean-Claude Wippler (please continue your blog on www.jeelabs.org) proposes http://dygraphs.com/ - I am looking forward to continuing during winter vacation starting Friday... There "should" be some MQTT ready solutions that generically display readings, but I did not do any research yet.
I think, I will report progress in https://github.com/gebhardm/energyhacks/tree/master/RaspberryPi - there my "quickly hacked" python-to-html solution is available that has a lot of room for improvement...
All you hackers, tinkerers, nerds, and freaks out there - share your insights, as I am getting too old (means too lazy) to again and again and again program stuff that should be already feasibly solved in the 21st century ;-) (I know why I studied math and not computer science)
Best regards
Markus
Thank you very very much, I take my hat off to your patience and perseverance.
Just for the record, here's a collection of links, alas difficult to understand given my very limited knowledge.
http://mqtt.org/wiki/doku.php/tools
http://www.hivemq.com/demos/websocket-client/
https://github.com/fabaff/mqtt-panel
http://mqtt.io/
Many thanks for the link collection; I will definitly try the mqtt-panel that looks quite promising...
I would like to like to have an MQTT-to-websocket bridge added to the FLM, probably based on libwebsockets. This would allow any modern browser to access all messages published to the FLM's internal Mosquitto broker using the Paho Javascript MQTT client (which uses MQTT/Websocket). For rendering, you might take a look at nvd3.js. That web page contains a short discussion about the use of gauges. :) A pure syslog-like functionality with dynamic updates would already be very useful, I guess.
Hello,
I have created a small socket.io/highchart based proof of concept. It emits mqtt payloads, which highcharts picks up real time. Websockets work on iPhone|pad too, which is nice.
I'm no node developer but I was surprised it wasn't too difficult. Highchart is amazing, I recommend you check it out at http://www.highcharts.com/demo/dynamic-update (jsfiddle is nice!)
@icarus75 Would be nice to implement mqtt options, like JSON payloads encoding?
If any is interested I put this demo on github. (having trouble adding multi charts, would like help with this)
@icarus75: Maybe I am a dinosaur with respect to web programming (so brain in back, not head); but why the hack no native websocket where the data is produced in the first place? What I can image is having a single data provider (which is the MQTT server on the FLM) and all consumers shall be connecting to just this data provider (so web server able to be shut down...) I am just puzzled as there is a web frontend in the FLM that I naively would just reuse to also pipe out some graphics and thus same space in the flash...
Maybe you can educate me...
Thanks and regards
Markus - just looking into node.js w/ websockets as this is a c't example in the last issue (01/2014) and node.js is compilable on a RasPi...
Got it: https://www.flukso.net/files/presentations/flukso.20121026.architecture.pdf - slide 7 - decoupled architecture of data provider incl jeenode and the corresponding data consumers...
Laying not too much weight on "nicely designed" yet, I got a little further on node.js piping FLM data locally. Progress will be visible under https://github.com/gebhardm/energyhacks/tree/master/RaspberryPi/panel - Fabian Affolter's mqtt-panel gave the necessary kick...
So, what's here so far? serve_mqtt.js running under node.js (compiles fine as v0.10.25-release on a RasPi - take it from joyent/node) to receive MQTT messages from the FLM's broker and serving index.html on port 1080; so no extra web server required. Current work (yet to be pushed) includes rudimentarily automated detection of the sending sensors and their visualization in a proper html-page - prime directive at the moment: Keep It Simple and Stupid... (as I am not too deep into JavaScript programming yet)
Done - automatic sensor retrieval is working; interestingly the sensor gauge of the pulse PV meter is still showing 19W - again the issue with Wh unequal to W and the "guessing derivative" inbetween. Anyway, I am pleased ;-)
OK - the timestamp shows it "last PV pulse gauge sent 17:08:08" - so really everything is in order :-)
This is how it looks by now: http://energyhacks.wordpress.com/2014/02/06/mqtt-gauges-for-the-flm/ - still simple and stupid, but as the "big red button" fullfilling its purpose :-)
Any comments? Any contributors? Any alternatives?
Next step: Bring this thingy onto the FLM itself. Work continues...
As an additional comment on the script's IP addresses (thanks Jos for the hint): Make sure that following conditions are met:
In serve_mqtt.js line 21
needs to point to your FLM's LAN address. In index.html line 25
needs to point to the ID address of the computer where you run node (my iMac runs on 192.168.0.25, so this is the serverIP).
On running the server starter "./run.sh" (needs no sudo) a message "info - socket.io started" is shown. To check if everything is O.K. you can point your browser to the serverIP (sic!) above with port 3000 - there you'll see "Welcome to socket.io." - so socket.io is running - then point to port 1080 and see the FLM panel (JavaScript enabled!!)
Has someone a quick answer to not having to set the serverIP manually? There should be a system variable for that...
... and that leaves just addressing the FLM in the server script - it's like with mathematics, if you understand it, it becomes beautiful ...
I found a patch for lighttpd for websockets (mod_websocket) and compiled into an flm02 firmware so that the flukso is able to provide a websocket by itself without the need of another server for providing the websocket.
If there is interest, I can provide a package on github
Yessss, please, many thanks :-) (I also saw that openWRT is in preparation to provide the libwebsocket in its standard release coming up - big endian or little endian?)
I just pushed a lighttpd package to github which includes the patch to enable websockets.
https://github.com/pieterjm/lighttpd.git
Now https://github.com/gebhardm/energyhacks/tree/master/RaspberryPi/panel also works with an arbitrary number of sensors; you may get the FLM's sensor readings via its built-in MQTT broker, but also any other sensor reading if published to this broker as well. With mosquitto use, for example, following command
Here the payload is "timestamp, value, unit" - the sensor topic has to differ the sensor on second place, in the example "t1". I will make a Pollin AVRNetIO board now publishing data to this broker...
Bart, I love MQTT :-)
I tried it, but it gives an error:
Fluc@Toshiba:~$ mosquitto_sub -h 192.168.x.x -t /sensor//gauge -m
Error: Unknown option '-m'.
Another try:
Fluc@Toshiba:~$ mosquitto_pub -h 192.168.x.x -t /sensor/SENSOR-ID-HERE/gauge -m
Error: -m argument given but no message specified.
What dit i wrong in this command ?
Hi Fluc,
first post double slash indicates no sensor id provided and you used the subscribe variant of mosquitto (_sub); the topic must be named like in the second variant when publishing. You can use the subscribe as follows:
This should show all FLM published messages.
Second post misses indeed the message. Provide my example above including the single quotes in the beginning and end to indicate a string containing an array as payload.
The first entry in the array must be a valid timestamp...
HTH
Bust regards
Markus
For the records - continued work on AVRNetIO (Arduino with jcw's Ethercard) to be used as sensor entry point publishing additional values to the FLM's broker; sadly http://github.com/njh/EtherCardMQTTS implements an outdated protocol version (v1.2), so I have to update that first :-/
By the way, Fluc, does that mean my panel variant works for you?
No Markus, i only wanted to test this in the Ubuntu terminal as a subscribe variant of mosquitto (_sub).
mosquitto_sub -h -v -t /sensor/#
By the way, it seems that it's like the same output than following command gives:
mosquitto_sub -h -p 1883 -v -t /sensor/#
On the other hand, i have not yet a Rasp-PI or Arduino in house ;-)
-p 1883 is the default port, so you don't need to provide it. You neither need a RasPi nor an Arduino; just install node.js in your Ubuntu and see what happens :-)
For general MQTT pub/sub, please follow up on https://www.flukso.net/content/mqtt-mq-telemetry-transport and leave this one for the gauges...
Hi,
I started play with emonCMS. For now I am using online server (emoncms.org) but it is free and will install on Pi that could be broker and CMS server or I can run it on my NAS since it has http server (probably better to leave RPi only with other functions).
Since emoncms.org accepts json I did a quick and dirty shell script for parse mqtt from Flukso into json. This sounds weird because original flukso has REST so it would probably be easy to read REST and parse it into json to send to emoncms. However, I want to centralize all sensor information on a mqtt broker so, for this test I just subscribed to mqtt and parsed it into json for publish into emoncms.
Let's see what happens.
Note bene: I would add a "simple" to the "nicely designed"; IMHO the effort to set up an own emoncms infrastructure is too high - the FLM has an up and running MQTT broker and just lacks a local visualization; my using the Pi is just an intermediate stage. Midterm the same "simple and stupid" solution shall be put to the FLM as there is also already a web server running that currently "just" lacks a socket.io interfacing to push realtime information; the rest is done in the browser anyway... (w/o database, of course) - I still did not get the cross-compiler running to push my stuff to the FLM (time, o time, where are thou)
Nice work Markus! Pieter mentioned the lighttpd mqtt-to-websocket bridge at this weekend's EC1404. I'm a bit reluctant to add yet another web server just to enable this bridge. Writing a small standalone bridge, based on libwebsockets would be the "embedded" way to go. Once that's done, we could have an additional page on the local FLM web interface rendering the sensor gauges. Should prove to being a great debugging help at installation time as well.
PS: This EC1404 presentation might interest you, p.13 in particular. Lots more gauges will be needed in the near future. :)
Cheers
/Bart
Hey Bart, conforming to p.13 I actually also just publish additional sensor data to the /sensor/-topic of the FLM's broker (via Arduino from the outside - works well with net frequency, temperature, etc.) - they are instantly displayed in my panel adaptation that does not need sensor config (as it takes the /sensor/-topic, derives the SensorIDs and adds as many rows for display as necessary) - this part thus works; the missing link indeed is the websocket part on the FLM, that I see - as you - as inherent part of the current FLM webserver setup; my issue so far was cross-compilation onto the FLM target. As the bridge consists only of a topic forward this "should" be no witchcraft - sadly Pieter did not enlighten me on how to get the libwebsocket.a onto the FLM (I am sad, having not been in Leuven)... Actually I would spend another port for gauge display than the config port of the FLM...
So, what is to be done:
1. Get libwebsocket.a onto the FLM
2. Add a service that pipes the MQTT messages to the websocket
3. Add a panel page to the web server displaying the gauge information, using some .css and .js stuff for proper design; might be a separate web server instance serving another port than the current configuration port 80...
Currently I am still fiddling with my AVRNetIO to get it firmly running as a sensor server (the micros() function is rather "unreliable" for net frequency measurement); a RPi runs to serve the panel, so integration into the FLM (at least for me) has no hurry (even tough I'd like to understand openWRT cross-compiling).
Hi Gebhard,
Let me enlighten you. The webserver running on the Flukso is uhttpd. For another leightweight webserver , capable of running on the flukso: lighttpd, a patch is available that adds websockets support (https://github.com/nori0428/mod_websocket). What I did was create an openwrt package of a modified lighttpd that includes this patch (https://github.com/pieterjm/lighttpd) so that it can be included in a firmware build of the Flukso. As far as i can see the patch of lighttpd does not depend on libwebsockets.
Although this seems to work, I don't think this is the way to go for the Flukso. I just wanted to see if it could work. I completely agree with Bart that a single webserver (preferably uhttpd) is better.
Would it be possible to create patch for uhttpd? And how does libwebsockets fit in there? I'm not familiar with that library. Can it be integrated with another webserver, or does it use its own webserver?
Thanks Pieter; on a fine rainy day, I'll test that out; my expectation would be that the FLM's uhttpd webserver could still be used with an "underneath" binding to the websocket; but I#ll do some web research on this - kind regards, Markus (currently still working on a node.js based solution to display FLM triggered database content)