Cisco, Networking and Programming.
We had a few severe thunderstorms where I live and I the wind and rain caused damage to my electronics in my outdoor weather station.
The weather station is waterproof if there is just light rain, but strong winds will blow water into the Stevenson Screen damaging the electronics.
I have decided to seal off the electronics as much as possible – hopefully this new construction will be more robust.
The ThingSpeak Web Page does not allow you to pull historical data from your channel, plus simple things like converting Absolute Pressure to Mean Sea Level Pressure require running Matlab Scripts.
I decided to create a single page web app that had the features I required. I ended up with the following features:
The web app needs to request data from api.thingspeak.com which is a different domain to the domain where the web app is hosted. To do this I created a route in Ktor which accepts a JSON request and responds with the contents of the request.
post("/getJSON") {
val thisReq = call.receive()
val req = khttp.get(thisReq.url)
println(thisReq.url)
if (req.statusCode == 200) {
call.respond(req.jsonObject)
} else {
call.respond(json { "success" to "false" })
}
}
I understand this could be potentially abused as this is a effectively an anonymous router, but I am not worried just yet.
To access this in JavaScript:
var thingSpeakQuery = `https://api.thingspeak.com/channels/${feedID}/feeds/last.json`;
var getJSONThingSpeak = {
"url": thingSpeakQuery
};
$.ajax({
type: "POST",
url: "/getJSON",
data: JSON.stringify(getJSONThingSpeak),
contentType: "application/json; charset=utf-8",
dataType: "json",
success: function (element) {
// code here to handle data
},
failure: function (errMsg) {
alert(errMsg);
},
// catch internal server error, this happens when the server can't pass the JSON data,
// or the ThingSpeak ID is invalid
statusCode: {
500: function () {
console.log("Query returned no data.");
}
}
});
Here I am using an AJAX function to get the latest data from the specified ThingSpeakID.
The rest of the web is pretty much standard JavaScript and relies heavily on jQuery to update the web page. The JSON data structures get pretty complicated. In order to debug/test the code while I was writing it I made the returned JSON data a global variable and used Chromium’s Dev Tools to write the code.
For example here where are looping through all the Charts and adding the ThingSpeak data:
thisData.map.feeds.myArrayList.forEach(function (element) {
// loop over each data field in ThingSpeak data
for (var k = 0; k < chartList.length; k++) {
// store as a tmp string, as not to change with 'k'
var tmp = k + 1;
weatherCharts[k].data.datasets[0].data.push({
x: moment(element.map.created_at),
y: parseFloat(element.map["field" + tmp.toString()])
});
// snipped
The above would be very difficult to write by hand, or get working by trial and error. Being able to run the code in the browser and make things instantly change was actually a lot of fun.
JavaScript is fast becoming my favorite language, not due to it’s technical merits but due to the fact it allows we to be vendor agnostic. I can run my code on my FreeBSD/Linux/Android/Windows/Mac/iPhone hardware.
The coolest feature, is the auto update feature. It allows me to leave the browser window open on my computer and have a live stream graph of my weather station without having to continuously hit the refresh button!
It’s pretty simple, it just uses a JavaScript interval function:
setInterval(function () {
// This will be executed every 60 seconds
getLatestWeatherForWeatherCharts();
}, 60000);
I finally got some fused silica glass which is UV transmissible and built a rough housing for a VEML6075 UV sensor.
I will be able to get a UV A/B and UV Index reading. This is more interesting to me than rain as we don’t get a lot of rain where I live.
I wrote an app that uses various parameters to calculate the fishing rating for today.
It will automatically find the closest beach (enable location) and pull weather statistics from the nearest weather station.
This is another simple web app written in Vanilla JS.
I wrote a simple Content Management System using JQuery, Kotlin and Ktor. I did this to learn more about JavaScript, MySQL and the webstack before moving on to learn a web framework (React).
There was a fair bit of JS code, and several routes that had to be created, but it wasn’t that difficult and only took two days.
The current system runs on my website (opens3.net) and has a front end and backend that run as separate services.
You can view the source code here and here.
I designed a new Stevenson Screen that would be backwards compatibility with the last design.
I am still yet to paint it, but I hope to see more accurate measurements with this design.
That’s not a solar panel….
This is a solar panel!
165mm squared – 4.5 watts. Just what I needed.
I re-wrote my weather bot completely in Kotlin and split the functionality into three parts:
Both the front-ends are written using jQuery, Bootstrap, vanilla Javascript and Kotlin/ktor. They are both single page web apps and allow for all functions without page reload.
Here is the sign up page on my website:
Here is the admin interface:
The database functions are handled using data classes and SQL objects using a SQL DSL for Kotlin caused Exposed.
Below is an example of the object classes, data classes and a function required to return a list of user objects. This code is used to return a list of users to the admin interface.
The Kotlin SQL DSL makes the programming of very complicated data structures a breeze.
object users : Table() {
val id = integer("id").autoIncrement().primaryKey() // Column
val name = varchar("name", length = 150) // Column
val country = varchar("country", length = 150) // Column
val state = varchar("state", length = 150) // Column
val mobile = varchar("mobile", length = 50) // Column
val opt1 = varchar("opt1", length = 50).nullable() // Column
val opt2 = varchar("opt2", length = 50).nullable() // Column
}
object userkeywords : Table() {
val id = integer("id").autoIncrement().primaryKey() // Column
val uuid = integer("uuid").nullable()
val keywords = varchar("keywords", length = 150) // Column
}
data class completeUser(
val id: Int,
val name: String,
val country: String,
val state: String,
val mobile: String,
val keywords: MutableList,
val opt1: String,
val opt2: String
)
fun getAllUsers(): MutableList {
connectToDB()
var returnedUsers = mutableListOf()
transaction {
SchemaUtils.create(users, userkeywords)
val allKeywords = userkeywords.selectAll()
for (user in users.selectAll()) {
val thisUser = completeUser(
id = user[users.id],
name = user[users.name],
country = user[users.country],
state = user[users.state],
mobile = user[users.mobile],
opt1 = "NIL",
opt2 = "NIL",
keywords = mutableListOf()
)
println("${user[users.id]}: ${user[users.name]}")
for (k in allKeywords) {
if (k[userkeywords.uuid] == user[users.id]) {
val kw = k[userkeywords.keywords]
thisUser.keywords.add(kw)
}
}
returnedUsers.add(thisUser)
}
}
return returnedUsers
}
To send this data in JSON it is very easy.
First “install” the GSON feature:
install(ContentNegotiation) {
gson {
setDateFormat(DateFormat.LONG)
setPrettyPrinting()
}
}
Then add the route:
get("/getUsers") {
var theseUsers = getAllUsers()
call.respond(theseUsers)
}
Pretty braindead stuff really. Handling queries in JavaScript is also very easy (the following function adds all retrieved users to a drop down menu):
var getUserList = function() {
$.ajax({
type: "GET",
url: "/getUsers",
// The key needs to match your method's input parameter (case-sensitive).
contentType: "application/json; charset=utf-8",
dataType: "json",
success: function(data) {
$('#deleteUser').empty();
data.forEach(function(element) {
$('#deleteUser').append(`
${element.name} mobile: ${element.mobile}`);
});
},
failure: function(errMsg) {
alert(errMsg);
}
});
};
As for the back-end, I had to write the API in Kotlin to connect to the Telstra API. I had to use khttp because ktor client can not set custom headers at the moment.
Here is an example of how it works:
//provision a number
val response = khttp.post(
url = "https://tapi.telstra.com/v2/messages/provisioning/subscriptions",
headers = mapOf("content-type" to "application/json","authorization" to "Bearer ${token}" , "cache-control" to "no-cache"),
data = "{ \"activeDays\": 30 }")
// get/update API token
val payload = mapOf(
"grant_type" to "client_credentials",
"scope" to "NSMS",
"client_id" to "xxx",
"client_secret" to "xxx")
val response = khttp.post(
url = " https://tapi.telstra.com/v2/oauth/token",
headers = mapOf("Content-Type" to "application/x-www-form-urlencoded"),
data = payload)
// send a message
val response = khttp.post(
url = " https://tapi.telstra.com/v2/messages/sms",
headers = mapOf("content-type" to "application/json",
"authorization" to "Bearer ${token}"),
data = ourJson.toString())
I had to setup a dummy route in ktor and use WireShark to sniff the packets to figure out which headers needed to be set before I could finally figure out how the system worked.
Thanks to Telstra for the free 1000 messages per month and thanks to the folks at Jetbrains for all their awesome libraries and products.
I rebuilt my website using Kotlin, Ktor, Bootstrap and Vanilla JS. I am happy with the results so far.
I am still working my way through a book on Ecmascript 6 and have to admit that programming in modern JavaScript is quite fun. I am also happy with the Ktor web framework.
2019 is my year of web apps and I hope to create hundreds of them in the future. I feel that web apps are the future and enjoy the flexibility afforded by the web platform.
I used the great code here: https://github.com/HarringayMakerSpace/usb-esp
To add configuration via WiFi and OTA updates via ThingSpeak to my weather station, I also completely redesigned the base plate incorporating new components I have received from eBay. The new design allows the battery to be accessed as well as the reset switch on the Wemos D1 Mini.
I added the following parts:
I am no longer using the Wemos D1 Mini Battery Shield. See my GitHub repository for details, go to the my Thingiverse page to download the files, go to the ThingSpeak live feed to see live data.
Command Line Networking 