Skip to main content

IoT Hackathon Part IV : Using Web Services to send Sensordata

In the previous 3 posts, building towards the eProseed IoT Hackathon, I described how to setup your Raspberry Pi, and how to use the GrovePi sensors. The used example is a small weather-station that read temperature and humidity and shows the readings on a display. That is all very nice, however, the data remains local on the Raspberry Pi so there is nothing that we can do with this information from an 'enterprise' perspective. In this post I will explain how easy it is to send the data to whatever 'end point' by using a REST-JSON web-service.

The Database Tables
For this use case I decided I needed 2 tables. One to hold all my available sensors (yes, I know, I have only one) and one to store the measurements per sensor.


The Webservice

In order to store the data in the database tables I need to send it from the Raspberry Pi to the database. For that I use a simple REST/JSON webservice that has a POST method. I also implemented PUT, GET and DELETE methods, because they might come in handy later.

The Webserivce is described by the following WADL:
 <ns0:application xmlns:ns0="http://wadl.dev.java.net/2009/02">  
   <ns0:doc xmlns:ns1="http://jersey.java.net/" ns1:generatedBy="Jersey: 2.5.1 2014-01-02 13:43:00"/>  
   <ns0:doc xmlns:ns2="http://jersey.java.net/"  
        ns2:hint="This is simplified WADL with user and core resources only. To get full WADL with extended resources use the query parameter detail.  Link: http://yourhost.com:7101/IoTRestJsonService/resources/application.wadl?detail=true"/>  
   <ns0:grammars>  
     <ns0:include href="application.wadl/xsd0.xsd">  
       <ns0:doc title="Generated" xml:lang="en"/>  
     </ns0:include>  
   </ns0:grammars>  
   <ns0:resources base="http://yourhost.com:7101/IoTRestJsonService/resources/">  
     <ns0:resource path="iot">  
       <ns0:resource path="/sensordata">  
         <ns0:method id="createSensorData" name="POST">  
           <ns0:request>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:request>  
         </ns0:method>  
         <ns0:method id="updateSensorData" name="PUT">  
           <ns0:request>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:request>  
         </ns0:method>  
         <ns0:method id="findAllSensorData" name="GET">  
           <ns0:response>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:response>  
         </ns0:method>  
       </ns0:resource>  
       <ns0:resource path="/sensordata/{id}">  
         <ns0:param xmlns:xsd="http://www.w3.org/2001/XMLSchema" name="id" style="template" type="xsd:int"/>  
         <ns0:method id="deleteSensorData" name="DELETE"/>  
         <ns0:method id="getSensorDataById" name="GET">  
           <ns0:response>  
             <ns0:representation element="iotSensorData" mediaType="application/json"/>  
           </ns0:response>  
         </ns0:method>  
       </ns0:resource>  
     </ns0:resource>  
   </ns0:resources>  
 </ns0:application>  

The service can be tested from any REST client. You can use 'Test Webserice' from within JDeveloper, or use a tool such as postman to POST a test message.
Calling Webservices from Python

Database is in place, Webservice is up and running, so the only remaining thing is to call the service from my Python code. By now it starting to get pretty obvious that this also cannot be very difficult with Python. And indeed, it is very simple again. Python comes with two libraries that can be used for this purpose: The request library and the json library. When you use those two, it takes only a couple of minutes to implement the webservice call. The code sample below shows you how to add the import of the two libraries and then how to construct the request; The request of course needs a resource URL in our case the relevant part is'iot/sensordata'. We also need to tell the request that is a json type request. This is part of the header. Finally we need to construct our data string that contains the measurement data. Here we can construct the exact JSON string that is expected by the service. The data that is sent in the request is simply dumped as JSON string, by calling "json.dumps()". This is the exact same string as can be seen from the Postman screenshot, whit other data ofcourse.

Note that I use sensorid =1 because this is the unique identifier of this sensor. To be 100% dynamic we could also use the ip adres of the Raspberry Pi as sensorid. We could call out to http://httpbin.org/ip to get the ip adres. For now I keep it simple and stick with the hardcoded value of 1.

 # import for the ws  
 import requests  
 import json  
 # end import for the ws  

 # here we prepare and call the ws  
  url = "http://yourhost.com:7101/IoTRestJsonService/resources/iot/sensordata"  
  headers = {'Content-Type':'application/json'}  
  sensordata = {"dataDate":i.isoformat(),"dataType":"T", "dataValue":t,"dataValueNumber":t,"sensorid":"1" }  
  r=requests.post(url, data=json.dumps(sensordata),headers=headers)  
Finally I share the complete code of the Python script with you so you can see and learn how to do this.
 # grovepi_lcd_dht.py  
 #  
 # This is an project for using the Grove LCD Display and the Grove DHT Sensor from the GrovePi starter kit  
 #   
 # In this project, the Temperature and humidity from the DHT sensor is printed on the DHT sensor  
 from grovepi import *  
 from grove_rgb_lcd import *  
 import logging  
 import datetime  
 # for the ws  
 import requests  
 import json  
 # end for the ws  
 dht_sensor_port = 7          # Connect the DHt sensor to port 7  
 # lets log to file  
 logger = logging.getLogger('weather.logger')  
 logger.setLevel('DEBUG')  
 file_log_handler = logging.FileHandler('/home/pi/Desktop/Lucs_projects/weatherstation/weather.log')  
 logger.addHandler(file_log_handler)  
 stderr_log_handler = logging.StreamHandler()  
 logger.addHandler(stderr_log_handler)  
 formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')  
 file_log_handler.setFormatter(formatter)  
 stderr_log_handler.setFormatter(formatter)  
 while True:  
      try:  
           [ temp,hum ] = dht(dht_sensor_port,0)          #Get the temperature and Humidity from the DHT sensor  
           t = '{0:.2f}'.format(temp)  
           h = '{0:.2f}'.format(hum)  
           i = datetime.datetime.now()  
           logger.info("temp ="+ t + "C\thumidity ="+ h + "%")   
           setRGB(0,128,64)  
           setRGB(0,255,0)  
           setText("Temp:" + t + "C   " + "Humidity:" + h + "%")  
         # here we prepare and call the ws  
           url = "http://yourhost.com:7101/IoTRestJsonService/resources/iot/sensordata"  
           headers = {'Content-Type':'application/json'}  
         #prep the temp message  
           sensordata = {"dataDate":i.isoformat(), "dataType":"T", "dataValue":t,"dataValueNumber":t,"sensorid":"1"}  
           r=requests.post(url, data=json.dumps(sensordata),headers=headers)  
         #prep the humidity message  
           sensordata = {"dataDate":i.isoformat(), "dataType":"H", "dataValue":h,"dataValueNumber":h,"sensorid":"1"}  
           r=requests.post(url, data=json.dumps(sensordata),headers=headers)  
         # end ws call  
            time.sleep(60)                 
      except (IOError,TypeError) as e:  
           logger.error('Error')  
           logger.error(r)  
Note that, almost at the end of the script, I use a time.sleep(60). This actually puts the script to sleep for one minute. This means, that every minute the new data is measured and sent to the webservice and saved in the database. It is very simple to build an ADF Page to display the data in some graphs. That is beyond the scope of this blogpost, however, here is an image of the result.

Note
The core purpose of this post is to describe how to use webservices from python, to forward sensor data to a (cloud) server. In a later post I describe how to use the better suited MQTT protocol. An example of this is CloudMQTT. CloudMQTT are managed Mosquitto servers in the cloud.Mosquitto implements the MQ Telemetry Transport protocol, MQTT, which provides lightweight methods of carrying out messaging using a publish/subscribe message queueing model.

Resources
1) RESTful Web Service in JDeveloper 12c
2) Calling REST/JSON from Python

Comments

Popular posts from this blog

ADF 12.1.3 : Implementing Default Table Filter Values

In one of my projects I ran into a requirement where the end user needs to be presented with default values in the table filters. This sounds like it is a common requirement, which is easy to implement. However it proved to be not so common, as it is not in the documentation nor are there any Blogpost to be found that talk about this feature. In this blogpost I describe how to implement this. The Use Case Explained Users of the application would typically enter today's date in a table filter in order to get all data that is valid for today. They do this each and every time. In order to facilitate them I want to have the table filter pre-filled with today's date (at the moment of writing July 31st 2015). So whenever the page is displayed, it should display 'today' in the table filter and execute the query accordingly. The problem is to get the value in the filter without the user typing it. Lets first take a look at how the ADF Search and Filters are implemented by

How to: Adding Speech to Oracle Digital Assistant; Talk to me Goose

At Oracle Code One in October, and also on DOAG in Nurnberg Germany in November I presented on how to go beyond your regular chatbot. This presentation contained a part on exposing your Oracle Digital Assistant over Alexa and also a part on face recognition. I finally found the time to blog about it. In this blogpost I will share details of the Alexa implementation in this solution. Typically there are 3 area's of interest which I will explain. Webhook Code to enable communication between Alexa and Oracle Digital Assistant Alexa Digital Assistant (DA) Explaining the Webhook Code The overall setup contains of Alexa, a NodeJS webhook and an Oracle Digital Assistant. The webhook code will be responsible for receiving and transforming the JSON payload from the Alexa request. The transformed will be sent to a webhook configured on Oracle DA. The DA will send its response back to the webhook, which will transform into a format that can be used by an Alexa device. To code

ADF 11g Quicky 3 : Adding Error, Info and Warning messages

How can we add a message programatically ? Last week I got this question for the second time in a months time. I decided to write a short blogpost on how this works. Adding messages is very easy, you just need to know how it works. You can add a message to your faces context by creating a new FacesMessage. Set the severity (ERROR, WARNING, INFO or FATAL ), set the message text, and if nessecary a message detail. The fragment below shows the code for an ERROR message. 1: public void setMessagesErr(ActionEvent actionEvent) { 2: String msg = "This is a message"; 3: AdfFacesContext adfFacesContext = null; 4: adfFacesContext = AdfFacesContext.getCurrentInstance(); 5: FacesContext ctx = FacesContext.getCurrentInstance(); 6: FacesMessage fm = 7: new FacesMessage(FacesMessage.SEVERITY_ERROR, msg, ""); 8: ctx.addMessage(null, fm); 9: } I created a simple page with a couple of buttons to show the result of setting the message. When the but