1.0 Introduction
So far we have learned MQTT in Lab-46 and AMQP protocol in Lab-53, its time to learn another IoT protocol called CoAP (Constrained Application Protocol).
CoAP is defined in RFC 7252. CoAP was developed for devices which are resource constrained and don’t have large memory or processing capability. The protocol is designed for machine to-machine (M2M) applications such as smart energy and building automation.
CoAP is very similar to HTTP protocol, it uses same methods as HTTP like GET, POST, PUT and DELETE. It uses same URI resource concept as HTTP and client server model. But that’s where similarity ends. Below are some points where CoAP differ from HTTP protocol
- CoAP uses UDP as transport protocol instead of TCP in HTTP
- Asynchronous message exchange, instead of synchronous message transfer in HTTP
- Capability to work as pub sub protocol
- CoAP URI starts with coap:// or coaps://
CoAP default port is 5683 and CoAP with DTLS default port is 5684
2.0 CoAP message header
CoAP uses fixed length binary header of 4 Bytes. CoAP messages are encoded in a simple binary format. Each message contains a Message ID. Message ID used to detect duplicate packets and for optional reliability. Reliability is provided by marking a message as Confirmable (CON). A Confirmable message is re-transmitted using a default timeout and exponential back-off between re-transmissions, until the recipient sends an Acknowledgement message (ACK) with the same Message ID
Version (Ver)
2-bit unsigned integer. Indicates the CoAP version number. It MUST be set to 1 (01 binary). Other values are reserved for future versions. Messages with unknown version numbers MUST be silently ignored.
Type (T)
2-bit unsigned integer. Indicates if this message is of type Confirmable (0), Non-confirmable (1), Acknowledgement (2), or Reset (3)
Token Length (TKL)
4-bit unsigned integer. Indicates the length of the variable-length Token field (0-8 bytes). Lengths 9-15 are reserved, MUST NOT be sent, and MUST be processed as a message format error
Code
8-bit unsigned integer, split into a 3-bit class (most significant bits) and a 5-bit detail (least significant bits), documented as “c.dd” where “c” is a digit from 0 to 7 for the 3-bit subfield and “dd” are two digits from 00 to 31 for the 5-bit subfield. The class can indicate a request (0), a success response (2), a client error response (4), or a server error response (5). (All other class values are reserved).
As a special case, Code 0.00 indicates an Empty message
Response code definition
Message ID
16-bit unsigned integer in network byte order. Used to detect message duplication and to match messages of type Acknowledgement/Reset to messages of type Confirmable/Non confirmable
Token
The Token is used to match a response with a request. The token value is a sequence of 0 to 8 bytes. (Note that every message carries a token, even if it is of zero length.) Every request carries a client-generated token that the server MUST echo (without modification) in any resulting response
Options
Both requests and responses may include a list of one or more options. For example, the URI in a request is transported in several options, and metadata that would be carried in an HTTP header in HTTP is supplied as options as well
An Option is identified by an option number. Options can be Critical, Unsafe, NoCacheKey and Repeatable
Critical: An option that would need to be understood by the endpoint ultimately receiving the message in order to properly process the message
Unsafe: An option that would need to be understood by a proxy receiving the message in order to safely forward the message
NoCacheKey:
Repeatable: The definition of some options specifies that those options are repeatable. An option that is repeatable MAY be included one or more times in a message. An option that is not repeatable MUST NOT be included more than once in a message
Below is the definition of commonly used Options
Uri-Host, Uri-Port, Uri-Path, and Uri-Query
The Uri-Host, Uri-Port, Uri-Path, and Uri-Query Options are used to specify the target resource of a request to a CoAP server.
- the Uri-Host Option specifies the Internet host of the resource being requested. The default value of the Uri-Host Option is the IP literal representing the destination IP address of the request message
- the Uri-Port Option specifies the transport-layer port number of the resource,
- each Uri-Path Option specifies one segment of the absolute path to the resource, and
- each Uri-Query Option specifies one argument parameterizing the resource
Example: Below example show how uri constructed from Options Uri-Path,Uir-Port & Uri-Host. Destination IP and port number are used from IP header and UDP header respectively
Input: Destination IP Address = [2001:db8::2:1] Destination UDP Port = 5683 Uri-Host = "example.net" Uri-Path = ".well-known" Uri-Path = "core" Output: coap://example.net/.well-known/core
Content-format:
The Content-Format Option indicates the representation format of the message payload.
Max-Age
The Max-Age Option indicates the maximum time a response may be cached before it is considered not fresh. It is used by the proxy server
Accept
The CoAP Accept option can be used to indicate which Content-Format is acceptable to the client. The representation format is given as a numeric Content-Format identifier that is defined in the “CoAP Content-Formats” registry. If no Accept option is given, the client does not express a preference (thus no default value is assumed)
3.0 CoAP URI
CoAP uses the “coap” and “coaps” URI schemes for identifying CoAP resources and providing a means of locating the resource. Resources are organized hierarchically and governed by a potential CoAP origin server listening for CoAP requests (“coap”) or DTLS-secured CoAP requests (“coaps”) on a given UDP port
coap-URI = “coap:” “//” host [ “:” port ] path-abempty [ “?” query ]
URI for secured CoAP. Default port for secured CoAP is 5684
coaps-URI = “coaps:” “//” host [ “:” port ] path-abempty [ “?” query ]
4.0 CoAP message types
Message types are defined by 2 bit type field in header. CoAP defines four messages (Confirmable, Non-confirmable, Reset, Acknowledgement ) and two responses (Piggybacked response and Separate response)
Confirmable Message
Some messages require an acknowledgement. These messages are called “Confirmable”. When no packets are lost, each Confirmable message elicits exactly one return message of type Acknowledgement or type Reset.
If acknowledgement is not received a Confirmable message is re-transmitted using a default timeout and exponential back-off between re-transmissions, until the recipient sends an Acknowledgement message (ACK)
Non-confirmable Message
Some messages do not require an acknowledgement. This is particularly true for messages that are repeated regularly for application requirements, such as repeated readings from a sensor
Acknowledgement Message
An Acknowledgement message acknowledges that a specific Confirmable message arrived. By itself, an Acknowledgement message does not indicate success or failure of any request encapsulated in the Confirmable message, but the Acknowledgement message may also carry a Piggybacked Response.
Reset Message
A Reset message indicates that a specific message (Confirmable or Non-confirmable) was received, but some context is missing to properly process it. This condition is usually caused when the receiving node has rebooted and has forgotten some state that would be required to interpret the message. Provoking a Reset message (e.g., by sending an Empty Confirmable message) is also useful as an inexpensive check of the liveness of an endpoint (“CoAP ping”).
Piggybacked Response
A piggybacked Response is included right in a CoAP Acknowledgement (ACK) message that is sent to acknowledge receipt of the Request for this Response.
In this example response to client request of temperature is piggybacked into ACK message
Separate Response
When a Confirmable message carrying a request is acknowledged with an Empty message (e.g., because the server doesn’t have the answer right away), a Separate Response is sent in a separate message exchange
In this example server can’t respond to client request right away so ACK is generated without response piggybacked. A separate response message is sent after some time. The message ID is different in separate response but token is same as original request
5.0 Resource Discovery
CoAP allows client to discover resource in server. This is accomplished by sending a GET request to server at well known URI-path (/.well-known/core). Upon receiving the GET request server response with all resources it currently serving
In our example server we have two resources /temp & /humidity. Client sends GET with Uri-path = /.well-known/core
Server respond with resources /temp & /humidity
6.0 Observing resources in CoAP
There is an extension to core CoAP protocol (RFC 7641) which provides CoAP subscribe and notification service. With this extension a CoAP client can subscribe to a resource in server. Whenever the state of resource changes server sends the notification to client.
Figure 2 below shows an example of a CoAP client registering its interest in a resource (/temperature). It sends a GET request with OBSERVE Option. The server respond with current state of resource and add client to list of observer. After that server sends two notification upon changes to the resource state. The notification message from server contains same Token id as GET. Both the registration request and the notifications are identified as such by the presence of the Observe Option
Observe Option:
The Observe Option has the following properties. Its meaning depends on whether it is included in a GET request or in a response
When included in a GET request, the Observe Option extends the GET method so it does not only retrieve a current representation of the target resource, but also requests the server to add or remove an entry in the list of observers of the resource depending on the option value. The list entry consists of the client endpoint and the token specified by the client in the request. Possible Observer Option values are:
0 (register) – adds the entry to the list, if not present;
1 (deregister) – removes the entry from the list, if present.
When included in a response, the Observe Option identifies the message as a notification. This implies that a matching entry exists in the list of observers and that the server will notify the client of changes to the resource state
Notifications are additional responses sent by the server in reply to the single extended GET request that created the registration. Each notification includes the token specified by the client in the request. The only difference between a notification and a normal response is the presence of the Observe Option.
A notification can be confirmable or non-confirmable, i.e., it can be sent in a confirmable or a non-confirmable message.
If the Observe Option in a GET request is set to 1 (deregister), then the server MUST remove any existing entry with a matching endpoint/ token pair from the list of observers and process the GET request as usual. The resulting response MUST NOT include an Observe Option.
Because messages can get reordered, the client needs a way to determine if a notification arrived later than a newer notification. For this purpose, the server MUST set the value of the Observe Option of each notification it sends to the 24 least significant bits of a strictly increasing sequence number. The sequence number MAY start at any value and MUST NOT increase so fast that it increases by more than 2^23 within less than 256 seconds
Below Wireshark capture of GET method with Observe Option. In this case client wants to subscribe to resource /temp
Demo
We will use CoAP client and server python library from this link. I am using Ubuntu 16.04 on my Windows 10 Virtual box VM
Pre-requisite:
- Ubuntu 16.04 VM
- Start Wireshark and monitor packets on lookback interface. You can apply filter ‘coap’
- Install coap python library
$sudo apt-get install pip $pip install CoAPthon
We are adding two resources in the server, /temp and /humidity
self.add_resource('temp/', TempSensor())
self.add_resource('humidity/', HumiditySensor())
Copy and paste below code in file coapserver.py. This will be our CoAP server
from coapthon.server.coap import CoAP
from coapthon.resources.resource import Resource
class TempSensor(Resource):
def __init__(self, name="TempSensor", coap_server=None):
super(TempSensor, self).__init__(name, coap_server, visible=True,
observable=True, allow_children=True)
self.payload = "{Temp: 60}"
def render_GET(self, request):
return self
def render_PUT(self, request):
self.payload = request.payload
return self
def render_POST(self, request):
res = TempSensor()
res.location_query = request.uri_query
res.payload = request.payload
return res
def render_DELETE(self, request):
return True
class HumiditySensor(Resource):
def __init__(self, name="HumiditySensor", coap_server=None):
super(HumiditySensor, self).__init__(name, coap_server, visible=True,
observable=True, allow_children=True)
self.payload = "{Humidity: 80}"
def render_GET(self, request):
return self
def render_PUT(self, request):
self.payload = request.payload
return self
def render_POST(self, request):
res = HumiditySensor()
res.location_query = request.uri_query
res.payload = request.payload
return res
def render_DELETE(self, request):
return True
class CoAPServer(CoAP):
def __init__(self, host, port):
CoAP.__init__(self, (host, port))
self.add_resource('temp/', TempSensor())
self.add_resource('humidity/', HumiditySensor())
def main():
server = CoAPServer("0.0.0.0", 5683)
try:
server.listen(10)
except KeyboardInterrupt:
print "Server Shutdown"
server.close()
print "Exiting..."
if __name__ == '__main__':
main()
|
Copy and paste below code in file coapclient.py. This will simulate a CoAP client
#!/usr/bin/env python
import getopt
import socket
import sys
from coapthon.client.helperclient import HelperClient
from coapthon.utils import parse_uri
__author__ = 'Giacomo Tanganelli'
client = None
def usage(): # pragma: no cover
print "Command:\tcoapclient.py -o -p [-P]"
print "Options:"
print "\t-o, --operation=\tGET|PUT|POST|DELETE|DISCOVER|OBSERVE"
print "\t-p, --path=\t\t\tPath of the request"
print "\t-P, --payload=\t\tPayload of the request"
print "\t-f, --payload-file=\t\tFile with payload of the request"
def client_callback(response):
print "Callback"
def client_callback_observe(response): # pragma: no cover
global client
print "Callback_observe"
check = True
while check:
chosen = raw_input("Stop observing? [y/N]: ")
if chosen != "" and not (chosen == "n" or chosen == "N" or chosen == "y" or chosen == "Y"):
print "Unrecognized choose."
continue
elif chosen == "y" or chosen == "Y":
while True:
rst = raw_input("Send RST message? [Y/n]: ")
if rst != "" and not (rst == "n" or rst == "N" or rst == "y" or rst == "Y"):
print "Unrecognized choose."
continue
elif rst == "" or rst == "y" or rst == "Y":
client.cancel_observing(response, True)
else:
client.cancel_observing(response, False)
check = False
break
else:
break
def main(): # pragma: no cover
global client
op = None
path = None
payload = None
try:
opts, args = getopt.getopt(sys.argv[1:], "ho:p:P:f:", ["help", "operation=", "path=", "payload=",
"payload_file="])
except getopt.GetoptError as err:
# print help information and exit:
print str(err) # will print something like "option -a not recognized"
usage()
sys.exit(2)
for o, a in opts:
if o in ("-o", "--operation"):
op = a
elif o in ("-p", "--path"):
path = a
elif o in ("-P", "--payload"):
payload = a
elif o in ("-f", "--payload-file"):
with open(a, 'r') as f:
payload = f.read()
elif o in ("-h", "--help"):
usage()
sys.exit()
else:
usage()
sys.exit(2)
if op is None:
print "Operation must be specified"
usage()
sys.exit(2)
if path is None:
print "Path must be specified"
usage()
sys.exit(2)
if not path.startswith("coap://"):
print "Path must be conform to coap://host[:port]/path"
usage()
sys.exit(2)
host, port, path = parse_uri(path)
try:
tmp = socket.gethostbyname(host)
host = tmp
except socket.gaierror:
pass
client = HelperClient(server=(host, port))
if op == "GET":
if path is None:
print "Path cannot be empty for a GET request"
usage()
sys.exit(2)
response = client.get(path)
print response.pretty_print()
client.stop()
elif op == "OBSERVE":
if path is None:
print "Path cannot be empty for a GET request"
usage()
sys.exit(2)
client.observe(path, client_callback_observe)
elif op == "DELETE":
if path is None:
print "Path cannot be empty for a DELETE request"
usage()
sys.exit(2)
response = client.delete(path)
print response.pretty_print()
client.stop()
elif op == "POST":
if path is None:
print "Path cannot be empty for a POST request"
usage()
sys.exit(2)
if payload is None:
print "Payload cannot be empty for a POST request"
usage()
sys.exit(2)
response = client.post(path, payload)
print response.pretty_print()
client.stop()
elif op == "PUT":
if path is None:
print "Path cannot be empty for a PUT request"
usage()
sys.exit(2)
if payload is None:
print "Payload cannot be empty for a PUT request"
usage()
sys.exit(2)
response = client.put(path, payload)
print response.pretty_print()
client.stop()
elif op == "DISCOVER":
response = client.discover()
print response.pretty_print()
client.stop()
else:
print "Operation not recognized"
usage()
sys.exit(2)
if __name__ == '__main__': # pragma: no cover
main()
In one terminal execute coapserver.py script
$python ./coapserver.py
and in another terminal execute coapclient.py. Below command will GET resource /temp from server. As can be seen server piggybacked payload {Temp: 60} with Ack frame
./coapclient.py -o GET -p coap://127.0.0.1:5683/temp Source: ('127.0.0.1', 5683) Destination: None Type: ACK MID: 29531 Code: CONTENT Token: None Payload: {Temp: 60}
Below command for OBSERVE option. Here we are subscribing to resource /temp
./coapclient.py -o OBSERVE -p coap://127.0.0.1:5683/temp
To discover resources on server. Try below command. As can be seen in response server returns two resources /temp and /humidity
./coapclient.py -o DISCOVER -p coap://127.0.0.1:5683/ Source: ('127.0.0.1', 5683) Destination: None Type: ACK MID: 41102 Code: CONTENT Token: None Content-Type: 40 Payload: </temp>;obs,</humidity>;obs,
Below example of POST
divine@divine-VirtualBox:~/coap$ ./coapclient.py -o POST -p coap://127.0.0.1:5683/temp -P {temp:100} Source: ('127.0.0.1', 5683) Destination: None Type: ACK MID: 53281 Code: CREATED Token: dn Location-Path: temp Payload: None divine@divine-VirtualBox:~/coap$ ./coapclient.py -o GET -p coap://127.0.0.1:5683/temp Source: ('127.0.0.1', 5683) Destination: None Type: ACK MID: 60262 Code: CONTENT Token: None Payload: {temp:100}
Below example of DELETE
divine@divine-VirtualBox:~/coap$ ./coapclient.py -o DELETE -p coap://127.0.0.1:5683/temp Source: ('127.0.0.1', 5683) Destination: None Type: ACK MID: 39338 Code: DELETE Token: None Payload: None divine@divine-VirtualBox:~/coap$ ./coapclient.py -o GET -p coap://127.0.0.1:5683/temp Source: ('127.0.0.1', 5683) Destination: None Type: ACK MID: 55286 Code: NOT_FOUND Token: None Payload: None
Welcome to cloudNet 