Weather station upgrade

That’s not a solar panel….

This is a solar panel!

165mm squared – 4.5 watts. Just what I needed.

Advertisements

Weather Bot goes live

I re-wrote my weather bot completely in Kotlin and split the functionality into three parts:

  1. The back-end downloads RSS feeds and compares them against the user database, it sends the alerts to user’s mobiles
  2. The front-end adds users and handles verification
  3. The front-end administration panel allows for addition, deletion of users.

Both the front-ends are written using jQuery, Bootstrap, vanilla Javascript and Kotlin/ktor. They are both single page web apps and allow for all functions without page reload.

Here is the sign up page on my website:

screencapture-opens3-net-main-html-2019-01-22-07_59_04

Here is the admin interface:

alerts

The database functions are handled using data classes and SQL objects using a SQL DSL for Kotlin caused Exposed.

Below is an example of the object classes, data classes and a function required to return a list of user objects. This code is used to return a list of users to the admin interface.

The Kotlin SQL DSL makes the programming of very complicated data structures a breeze.

object users : Table() {
    val id = integer("id").autoIncrement().primaryKey() // Column
    val name = varchar("name", length = 150) // Column
    val country = varchar("country", length = 150) // Column
    val state = varchar("state", length = 150) // Column
    val mobile = varchar("mobile", length = 50) // Column
    val opt1 = varchar("opt1", length = 50).nullable() // Column
    val opt2 = varchar("opt2", length = 50).nullable() // Column

}

object userkeywords : Table() {
    val id = integer("id").autoIncrement().primaryKey() // Column
    val uuid = integer("uuid").nullable()
    val keywords = varchar("keywords", length = 150) // Column
}

data class completeUser(
    val id: Int,
    val name: String,
    val country: String,
    val state: String,
    val mobile: String,
    val keywords: MutableList,
    val opt1: String,
    val opt2: String
)
fun getAllUsers(): MutableList {
    connectToDB()
    var returnedUsers = mutableListOf()
    transaction {
        SchemaUtils.create(users, userkeywords)
        val allKeywords = userkeywords.selectAll()
        for (user in users.selectAll()) {
            val thisUser = completeUser(
                id = user[users.id],
                name = user[users.name],
                country = user[users.country],
                state = user[users.state],
                mobile = user[users.mobile],
                opt1 = "NIL",
                opt2 = "NIL",
                keywords = mutableListOf()
            )
            println("${user[users.id]}: ${user[users.name]}")
            for (k in allKeywords) {
                if (k[userkeywords.uuid] == user[users.id]) {
                    val kw = k[userkeywords.keywords]
                    thisUser.keywords.add(kw)
                }
            }
            returnedUsers.add(thisUser)
        }

    }
    return returnedUsers
}

To send this data in JSON it is very easy.

First “install” the GSON feature:

install(ContentNegotiation) {
        gson {
            setDateFormat(DateFormat.LONG)
            setPrettyPrinting()
        }
    }

Then add the route:

get("/getUsers") {
    var theseUsers = getAllUsers()
    call.respond(theseUsers)
}

Pretty braindead stuff really. Handling queries in JavaScript is also very easy (the following function adds all retrieved users to a drop down menu):

var getUserList = function() {
  $.ajax({
    type: "GET",
    url: "/getUsers",
    // The key needs to match your method's input parameter (case-sensitive).
    contentType: "application/json; charset=utf-8",
    dataType: "json",
    success: function(data) {
      $('#deleteUser').empty();
      data.forEach(function(element) {
        $('#deleteUser').append(`
${element.name} mobile: ${element.mobile}`);
      });
    },
    failure: function(errMsg) {
      alert(errMsg);
    }
  });
};

As for the back-end, I had to write the API in Kotlin to connect to the Telstra API. I had to use khttp because ktor client can not set custom headers at the moment.

Here is an example of how it works:

//provision a number
val response = khttp.post(
      url = "https://tapi.telstra.com/v2/messages/provisioning/subscriptions",
      headers = mapOf("content-type" to "application/json","authorization" to "Bearer ${token}" , "cache-control" to "no-cache"),
            data = "{ \"activeDays\": 30 }")

// get/update API token
val payload = mapOf(
      "grant_type" to "client_credentials",
      "scope" to "NSMS",
      "client_id" to "xxx",
      "client_secret" to "xxx")

val response = khttp.post(
      url = " https://tapi.telstra.com/v2/oauth/token",
      headers = mapOf("Content-Type" to "application/x-www-form-urlencoded"),
      data = payload)

// send a message
val response = khttp.post(
      url = " https://tapi.telstra.com/v2/messages/sms",                  
      headers = mapOf("content-type" to "application/json",
      "authorization" to "Bearer ${token}"),
      data = ourJson.toString())
 

I had to setup a dummy route in ktor and use WireShark to sniff the packets to figure out which headers needed to be set before I could finally figure out how the system worked.

Thanks to Telstra for the free 1000 messages per month and thanks to the folks at Jetbrains for all their awesome libraries and products.

Web Apps with Kotlin+Ktor

I rebuilt my website using Kotlin, Ktor, Bootstrap and Vanilla JS. I am happy with the results so far.

I am still working my way through a book on Ecmascript 6 and have to admit that programming in modern JavaScript is quite fun. I am also happy with the Ktor web framework.

2019 is my year of web apps and I hope to create hundreds of them in the future. I feel that web apps are the future and enjoy the flexibility afforded by the web platform.

View the website here. Get the source code here.

Updated Weather Station with OTA and configuration via WiFi AP

I used the great code here: https://github.com/HarringayMakerSpace/usb-esp

To add configuration via WiFi and OTA updates via ThingSpeak to my weather station, I also completely redesigned the base plate incorporating new components I have received from eBay. The new design allows the battery to be accessed as well as the reset switch on the Wemos D1 Mini.

I added the following parts:

I am no longer using the Wemos D1 Mini Battery Shield. See my GitHub repository for details, go to the my Thingiverse page to download the files, go to the ThingSpeak live feed to see live data.

 

 

Creating decks programmatically Kotlin/Perl for Anki/AnkiDroid

I used Google’s Text To Speech API to create mp3 files for each of the top 6000 Korean words, I then created a csv file using Perl and imported them into Anki.

The CSV file is imported as a cloze deletion card with the audio files as the extra field.

The code is here: https://github.com/wilyarti/anki-translator

This year I am focusing on learning Kotlin/C/JS and this is the first Kotlin script/program. I am finding Kotlin an easy transition from my mostly Go/Perl background.

6.3 Configure, verify, and troubleshoot IPv4 and IPv6 access list for traffic filtering

IPv4 ACLs

6.3.a Standard

Screenshot_2018-12-09_06-18-36

Using the above topology, we will use standard ACLs to block (1-99) traffic. Using the implicit “deny” at the end of the ACL will reduce the amount of ACLs needed.

On R1:

interface GigabitEthernet1/0
 ip address 10.0.0.1 255.255.255.0
 ip access-group 1 out
 negotiation auto
!
interface GigabitEthernet2/0
 ip address 10.0.1.1 255.255.255.0
 ip access-group 2 out
 negotiation auto
!
access-list 1 permit 192.168.122.0 0.0.0.255 log
access-list 2 permit 10.0.2.0 0.0.0.255 log
!

To verify use pings, and extended pings:

BANKING> ping 10.0.0.2   
*10.0.1.1 icmp_seq=1 ttl=255 time=9.955 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.1.1 icmp_seq=2 ttl=255 time=3.232 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.1.1 icmp_seq=3 ttl=255 time=2.574 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.1.1 icmp_seq=4 ttl=255 time=2.354 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.1.1 icmp_seq=5 ttl=255 time=1.745 ms (ICMP type:3, code:13, Communication administratively prohibited)

DEVOPS> ping 10.0.0.2 
*10.0.2.1 icmp_seq=1 ttl=255 time=9.813 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.2.1 icmp_seq=2 ttl=255 time=2.622 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.2.1 icmp_seq=3 ttl=255 time=11.419 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.2.1 icmp_seq=4 ttl=255 time=10.533 ms (ICMP type:3, code:13, Communication administratively prohibited)
*10.0.2.1 icmp_seq=5 ttl=255 time=2.351 ms (ICMP type:3, code:13, Communication administratively prohibited)

router#ping
Protocol [ip]: 
Target IP address: 10.0.0.2
Repeat count [5]: 
Datagram size [100]: 
Timeout in seconds [2]: 
Extended commands [n]: y
Source address or interface: GigabitEthernet0/0
Type of service [0]: 
Set DF bit in IP header? [no]: 
Validate reply data? [no]: 
Data pattern [0xABCD]: 
Loose, Strict, Record, Timestamp, Verbose[none]: 
Sweep range of sizes [n]: 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
Packet sent with a source address of 192.168.122.180 
.!!!!
Success rate is 80 percent (4/5), round-trip min/avg/max = 4/266/1040 ms
router#

6.3.b Extended

Extended access list are in the range <100-199>.

Extended commands allow us to match on the destination address as well as the protocol.

<0-255>       An IP protocol number
  ahp           Authentication Header Protocol
  eigrp         Cisco's EIGRP routing protocol
  esp           Encapsulation Security Payload
  gre           Cisco's GRE tunneling
  icmp          Internet Control Message Protocol
  igmp          Internet Gateway Message Protocol
  ip            Any Internet Protocol
  ipinip        IP in IP tunneling
  nos           KA9Q NOS compatible IP over IP tunneling
  object-group  Service object group
  ospf          OSPF routing protocol
  pcp           Payload Compression Protocol
  pim           Protocol Independent Multicast
  tcp           Transmission Control Protocol
  udp           User Datagram Protocol

interface GigabitEthernet1/0
 ip address 10.0.0.1 255.255.255.0
 ip access-group 100 out
 negotiation auto
!
interface GigabitEthernet2/0
 ip address 10.0.1.1 255.255.255.0
 ip access-group 101 out
 negotiation auto
!
access-list 100 permit ip 192.168.122.0 0.0.0.255 host 10.0.0.2 log
access-list 101 permit ip 10.0.2.0 0.0.0.255 host 10.0.1.2 log

6.3.c Named

Using named ACLs is the same as the above, except individual sequences can be entered:

<1-2147483647>  Sequence Number
  default         Set a command to its defaults
  deny            Specify packets to reject
  dynamic         Specify a DYNAMIC list of PERMITs or DENYs
  evaluate        Evaluate an access list
  exit            Exit from access-list configuration mode
  no              Negate a command or set its defaults
  permit          Specify packets to forward
  remark          Access list entry comment
router(config)#ip access-list extended RULE1
router(config-ext-nacl)#1 permit ip 192.168.122.0 0.0.0.255 host 10.0.0.2 log

router(config)#ip access-list extended RULE2
router(config-ext-nacl)#1 permit ip 10.0.2.0 0.0.0.255 host 10.0.1.2 log

Then you can delete in individual sequence numbers:

router(config)#ip access-list extended RULE1
router(config-ext-nacl)#no ?
  <1-2147483647>  Sequence Number
  deny            Specify packets to reject
  dynamic         Specify a DYNAMIC list of PERMITs or DENYs
  evaluate        Evaluate an access list
  permit          Specify packets to forward
  remark          Access list entry comment

router(config-ext-nacl)#no 1

IPv6 ACLs

According to Cisco: “With IPv4, you can configure standard and extended numbered IP ACLs, named IP ACLs, and MAC ACLs. IPv6 supports only named ACLs.”

Here is the topology:

Screenshot_2018-12-09_06-18-36

Here are the ACLs:

interface GigabitEthernet0/0
 ip address dhcp
 duplex full
 speed 1000
 media-type gbic
 negotiation auto
 ipv6 address 2001:DB8:4::/48 eui-64
!
interface GigabitEthernet1/0
 no ip address
 negotiation auto
 ipv6 address 2001:DB8:1::/48 eui-64
 ipv6 traffic-filter RULE1 out
!
interface GigabitEthernet2/0
 no ip address
 negotiation auto
 ipv6 address 2001:DB8:2::/48 eui-64
 ipv6 traffic-filter RULE2 out
!
interface GigabitEthernet3/0
 no ip address
 negotiation auto
 ipv6 address 2001:DB8:3::/48 eui-64
!
ipv6 access-list RULE1
 permit ipv6 2001:DB8:4::/48 2001:DB8:1::/48 log
!
ipv6 access-list RULE2
 permit ipv6 2001:DB8:3::/48 2001:DB8:2::/48 log
!
control-plane

The steps to verify are as above. Adding the log keyword helps by showing when the ACL is triggered:

router#
*Dec 9 09:25:43.011: %IPV6_ACL-6-ACCESSLOGDP: list RULE2/10 permitted icmpv6 2001:DB8:3::1 -> 2001:DB8:2::1 (128/0), 1 packet

Ping commands will also return “prohibited” when attempting to reach ACL restricted subnets:

DEVOPS> ping 2001:db8:1::1

*2001:db8:3:0:c801:12ff:fe77:54 icmp6_seq=1 ttl=64 time=18.509 ms (ICMP type:1, code:1, Communication with destination administratively prohibited)
*2001:db8:3:0:c801:12ff:fe77:54 icmp6_seq=2 ttl=64 time=9.367 ms (ICMP type:1, code:1, Communication with destination administratively prohibited)
*2001:db8:3:0:c801:12ff:fe77:54 icmp6_seq=3 ttl=64 time=9.834 ms (ICMP type:1, code:1, Communication with destination administratively prohibited)
*2001:db8:3:0:c801:12ff:fe77:54 icmp6_seq=4 ttl=64 time=9.151 ms (ICMP type:1, code:1, Communication with destination administratively prohibited)
*2001:db8:3:0:c801:12ff:fe77:54 icmp6_seq=5 ttl=64 time=9.522 ms (ICMP type:1, code:1, Communication with destination administratively prohibited)