Lab-51: Azure IoT hub, routes and endpoints

This lab is the continuation of  Lab-50  In Lab-50 we learned how to setup Azure IoT hub, simulate  IoT device and read device to cloud messages using web app and device explorer/iothub-explorer. In this lab we will learn how to read D2C and send C2D messages programmatically using Visual Studio 2017

Before we proceed it will be good to understand few concepts in IoT hub, mainly Endpoints and Routes

azure_iot_19

Endpoints:

There are two type of endpoints in IoT hub 1) Built-in endpoints and 2) custom end-points. Think of endpoints as topics or queue in AMQP, MQTT protocols.

azure_iot_38

  1. Built-in Endpoints

For each device in the identity registry, IoT Hub exposes a set of built-in endpoints, these endpoints can  be device facing or service facing

Device side endpoints

When device sending message to cloud it uses this endpoint. If you are using Azure client SDK you don’t need to worry about setting this endpoint, SDK does it for you.

/devices/{device_id}/messages/events

Device receives C2D messages on this endpoint

/device/{device_id}/messages/devicebound

Service endpoints

Each IoT hub exposes a set of endpoints for your solution back end to communicate with your devices. With one exception, these endpoints are only exposed using the AMQP protocol.

D2C endpoint. This endpoint is compatible with Azure Event Hubs. A back-end service can use it to read device-to-cloud messages sent by your devices.

/messages/events

C2D endpoints enable your solution back end to send reliable cloud-to-device messages, and to receive the corresponding acknowledgments.

/messages/devicebound

2. Custom Endpoints

Custom endpoints mainly deal with service endpoints and D2C messages. Instead of using built-in service endpoint (/messages/events) you can create custom endpoints. Custom endpoints can be on even hub, service bus queue or service bus topics

Azure IoT hub provides following type of custom endpoints each cater different requirement

  • Event Hubs
    • Event Hubs is a scalable event processing service that ingests and processes large volumes of events, with low latency and high reliability. Events in even hub can be retained for one to seven days. Events can be played back again. You can read more about event hub here
  • Service Bus Queues. Service bus messaging contains:
    • Queues, which allow one-directional communication. Each queue acts as an intermediary (sometimes called a broker) that stores sent messages until they are received. Each message is received by a single recipient
    • Service Bus Topics. Topics, which provide one-directional communication using subscriptions-a single topic can have multiple subscriptions. Like a queue, a topic acts as a broker, but each subscription can optionally use a filter to receive only messages that match specific criteria

 

Routes

Routes routes D2C messages to service endpoints in IoT hub. If there are no routes configured messages are routed to default service endpoint (/messages/events). Messages can be routed to custom endpoints also, check this link for routing rules

azure_iot_39

 

As seen in the above picture messages received by IoT hub is checked against route, if there are no route configured or none matches the rule, message routes to default endpoint (messages/events) otherwise message routed to respective endpoints

Pre-requisite

  1. Microsoft Visual Studio 2017. Link to install visual Studio here
  2. Account in Azure IoT hub
  3. Ubuntu 16.04 to simulate IoT device

Procedure

In this lab we will try these exercises

  1. Read D2C messages using built-in endpoint (/messages/events)
  2. Send C2D messages using built-in endpoint (/messages/devicebound)
  3. Read D2C messages using service bus queue
  4. Send C2D messages using service bus queue

Lot’s to do so let’s get started

1. Reading D2C messages using built-in endpoint (messages/events)

D2C messages are routed to default service endpoint (messages/events) if there is no route created in IoT hub. messages/events is Event hub compatible endpoint. We are not creating any route in this exercise so we can use this service endpoint to read D2C messages. This is the built-in endpoint in IoT hub so no need to create one.

You can find this endpoint under IoT Hub -> Endpoints

azure_iot_18

In Visual Studio, add a Visual C# Windows Classic Desktop project to the current solution, by using the Console App (.NET Framework) project template. Name the project readD2CMessage

azure_iot_26

In Solution Explorer, right-click the readD2CMessage project, and then click Manage NuGet Packages. This operation displays the NuGet Package Manager window.

Browse for WindowsAzure.ServiceBus, click Install, and accept the terms of use. This operation downloads, installs, and adds a reference to the Azure Service Bus, with all its dependencies.

azure_iot_27

Add the following using statements at the top of the Program.cs file:

using Microsoft.ServiceBus.Messaging;

Add your IoT connection string and default built-in service endpoint (/messages/events)

static string connectionString = "<your IoT hub connection string>";        
static string iotHubD2cEndpoint = "messages/events";

complete code looks like this

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.ServiceBus.Messaging;
using System.Threading;
using System.IO;

namespace ConsoleApp3
{
   class Program
   {
       static void Main(string[] args)
       {
           Console.WriteLine("Receive messages. Ctrl-C to exit.\n");
           eventHubClient = EventHubClient.CreateFromConnectionString(connectionString, iotHubD2cEndpoint);
         var d2cPartitions = eventHubClient.GetRuntimeInformation().PartitionIds;
           CancellationTokenSource cts = new CancellationTokenSource();
           System.Console.CancelKeyPress += (s, e) =>
           {
               e.Cancel = true;
               cts.Cancel();
               Console.WriteLine("Exiting...");
           };
           var tasks = new List<Task>();
           foreach (string partition in d2cPartitions)
           {
               tasks.Add(ReceiveMessagesFromDeviceAsync(partition, cts.Token));
           }
           Task.WaitAll(tasks.ToArray());
       }
       static string connectionString = "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxx=";
       static string iotHubD2cEndpoint = "messages/events";
       static EventHubClient eventHubClient;
       private static async Task ReceiveMessagesFromDeviceAsync(string partition, CancellationToken ct)
       {
           var eventHubReceiver = eventHubClient.GetDefaultConsumerGroup().CreateReceiver(partition, DateTime.UtcNow);
           while (true)
           {
               if (ct.IsCancellationRequested) break;
               EventData eventData = await eventHubReceiver.ReceiveAsync();
               if (eventData == null) continue;
               string data = Encoding.UTF8.GetString(eventData.GetBytes());
               var prop = (string)eventData.Properties["temperatureAlert"];
		Console.WriteLine("Message received. Partition: {0} Data: '{1}' Property: '{2}'", partition, data, prop);
           }
       }
   }
}

Now you are ready to run the applications.

Press F5 to start the console app. The App display D2C messages.

Start IoT device simulation program in your VM as we did in Lab-50.

You will see D2C messages on Console

azure_iot_21

2. Sending C2D messages using built-in endpoint (/messages/devicebound)

In Visual Studio, add a Visual C# Windows Classic Desktop project to the current solution, by using the Console App (.NET Framework) project template. Name the project sendC2DMessage

azure_iot_22

In Solution Explorer, right-click the sendC2DMessage project, and then click Manage NuGet Packages. This operation displays the NuGet Package Manager window.

Browse for Microsoft.Azure.Devices, click Install, and accept the terms of use. This operation downloads, installs, and adds a reference to the Azure Devices, with all its dependencies.

azure_iot_23

azure_iot_24

Add the following using statements at the top of the Program.cs file:

using Microsoft.Azure.Devices;

Add IoT hub connection string

static string connectionString = "<your IoT hub connection string>";

Add your device ID, in my case it is myIotDevice

await serviceClient.SendAsync("myIotDevice", commandMessage);

complete code looks like this

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.Devices;

namespace sendC2DMessage
{
   class Program
   {
       static ServiceClient serviceClient;
       static string connectionString = "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxxxxxxxx=";
       static void Main(string[] args)
       {
           Console.WriteLine("Send Cloud-to-Device message\n");
           serviceClient = ServiceClient.CreateFromConnectionString(connectionString);
           Console.WriteLine("Press any key to send a C2D message.");
           Console.ReadLine();
           sendC2DMessage().Wait();
           Console.ReadLine();
       }
       private async static Task sendC2DMessage()
       {
           var commandMessage = new Message(Encoding.ASCII.GetBytes("This is Cloud to Device message.."));
            await serviceClient.SendAsync("myIotDevice", commandMessage);
       }
   }
}

Now you are ready to run the applications.

Press F5 to start the console app.

Start IoT device simulation program in your VM as we did in Lab-50

Every time you hit enter on console app it will send C2D message “This is Cloud to Device message..” which will be visible in IoT device

3. Read D2C message using service bus queue endpoint

In this exercise we will route D2C messages to custom endpoint which is service bus queue and the read messages from endpoint

  1. Create service bus resource, New -> Enterprise Integration -> Service Bus

azure_iot_28

2.  Give service bus resource a name and attach to existing resource group

azure_iot_29

3. Note down service bus queue connection string, All resources -> <service bus> -> Shared access policies -> RootManagerSharedAccessKey -> Primary Connection String

azure_iot_30

4. Create a queue inside service bus resource, <service bus> -> Queues. Give queue name and click Create

azure_iot_31

5. Create an endpoint in IoT hub, click on All Resource -> IoT hub -> Endpoint -> Add. Select endpoint type as ‘Service Bus Queue’,  select Service Bus name in Service Bus namespace and finally select queue name

azure_iot_32

In Visual Studio, add a Visual C# Windows Classic Desktop project to the current solution, by using the Console App (.NET Framework) project template. Name the project receiveD2CMessageFromQueue

In Solution Explorer, right-click the receiveD2CMessageFromQueue project, and then click Manage NuGet Packages. This operation displays the NuGet Package Manager window.

Browse for WindowsAzure.Servicebus, click Install, and accept the terms of use. This operation downloads, installs, and adds a reference to the Azure Devices, with all its dependencies.

Add the following using statements at the top of the Program.cs file:

using System.IO;
using Microsoft.ServiceBus.Messaging;

Add your service bus queue connection string saved in earlier step. Provide your queue name

const string ServiceBusConnectionString = "<your service bus queue connection string>";      
const string QueueName = "<your queue name>";

Complete code looks like this

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.IO;
using Microsoft.ServiceBus.Messaging;

namespace readD2CMessageFromQueue
{
  class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("Receive critical messages. Ctrl-C to exit.\n");
            var connectionString = "Endpoint=sb://divinesvcbus.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=xxxxxx=";
            var queueName = "divineQueue";
            var client = QueueClient.CreateFromConnectionString(connectionString, queueName);
            client.OnMessage(message =>
            {
                Stream stream = message.GetBody<Stream>();
                StreamReader reader = new StreamReader(stream, Encoding.ASCII);
                string s = reader.ReadToEnd();
                Console.WriteLine(String.Format("Message body: {0}", s));
            });

           Console.ReadLine();
        }
    }
}

Now you are ready to run the applications.

Press F5 to start the console app.

Start IoT device simulation program in your VM as we did in Lab-50

You will see D2C messages on console

azure_iot_33

Let’s tweak the routing rule to route messages to queue endpoint only when temperatureAlert = true. The logic in simulated device (Ubuntu VM) is to send temperatureAlert = true whenever temp > 28. More on creating routing query here

Edit route  to filter based on temperatureAlert = true. All Resources -> IoT hub -> Route and add following in query temperatureAlert = ‘true’

azure_iot_34

Start console app and simulated device. Now you should see only messages with temperature > 28. This mean our routing rule is working

azure_iot_35

 

Lab-50: Azure IoT Hub

In this lab I will show how to setup Azure  IoT hub, setup IoT device and send telemetry from device to cloud.  We will use Ubuntu 16.04 VM to simulate IoT device and connect it to Azure IoT hub. We will use free Azure cloud account.

You can read more about Azure IoT hub here

azure_iot_16

Devices can be connected directly or indirectly via a gateway, and both may implement edge intelligence with different levels of processing capabilities. A cloud gateway provides endpoints for device connectivity and facilitates bidirectional communication with the backend system.
The back end comprises multiple components to provide device registration and discovery, data collection, transformation, and analytics, as well as business logic and visualizations.

Prerequisite:

  • Azure cloud account. I have setup a free subscription account. Follow instruction in this link to setup free Azure account
  • A VM with Ubuntu 16.04  and Python 2.7 to simulate IoT device
  • A Windows 10 laptop

Procedure:

We will follow these steps:

  1. Build device SDK for Python in Ubuntu VM
  2. Create device identity in Azure IoT hub
  3. Download and execute Python script to simulate temp + humidity sensor
  4. Create Web App to monitor device to cloud messages

Lots to do so let’s get started

Step-1: Build device SDK for Python in Ubuntu VM

I am using Ubuntu 16.04 VM to simulate IoT device. In this step we will download IoT device SDK from git and build it in VM. This SDK will be consumed by our temp+humidity sensor application in step-3. There are two ways to setup SDK as outline in this link

  1. Install the Python modules using PyPI wheels from PyPI.  This procedure didn’t work for me I get this error while running script which I couldn’t resolve

azure_iot_7

2. Build the Azure IoT Hub SDKs for Python on Linux. This procedure worked for me. I have Python 2.7 installed in Ubuntu 16.04 VM

#Clone the Azure IoT Python SDK Repository
$git clone --recursive https://github.com/Azure/azure-iot-sdk-python.git 

#For Ubuntu, you can use apt-get to install the right packages:
$sudo apt-get update
$sudo apt-get install -y git cmake build-essential curl libcurl4-openssl-dev libssl-dev uuid-dev

#Verify that CMake is at least version 2.8.12:
$cmake --version

#Verify that gcc is at least version 4.4.7:
$gcc --version

#Clone the Azure IoT Python SDK Repository
$git clone --recursive https://github.com/Azure/azure-iot-sdk-python.git

#Open a shell and navigate to the folder build_all/linux in your local copy of the repository

Run the ./setup.sh script to install the prerequisite packages and the dependent libraries
Run the ./build.sh script.

After a successful build, the iothub_client.so Python extension module is copied to the device/samples and service/samples folders.

Step-2: Create device identity in Azure IoT hub

Before your IoT device talk to IoT hub it needs to be registered. Device needs to be assigned a unique ID. A connection string will be auto-generated which will be used by device to connect to IoT hub.

To register device into IoT hub

  • Login to Azure portal (https://portal.azure.com)
  • Click on All resources -> Your IoT hub -> IoT Devices -> Add. Give unique ID to device and click on Save button

azure_iot_8

Below two steps are to find device and IoT hub connection string. In Azure communication to device or cloud happens using connection string

How to find IoT hub connection string

  • Login to Azure portal (https://portal.azure.com)
  • Click on All Resources -> IoT hub -> Shared access policies -> iothubowner .

Your IoT hub connection string will be located under Connection string – Primary key. Copy and paste this string in a notepad we will need it in later steps

azure_iot_10

How to find device connection string

  • Login to Azure portal (https://portal.azure.com)
  • Click on All Resources -> IoT hub -> IoT Devices -> IoT device ID. Connection string will be under ‘Connection string – Primary key’. Copy and paste this string to notepad we will need it in later steps

azure_iot_4

Step-3: Execute Python script to simulate temp + humidity sensor

As part of Step-1 an application script to simulate temp + humidity sensor is also installed in your VM. Login to your Ubuntu VM. Go to directory /home/<username>/azure-iot-sdk-python/device/samples. Open  script ‘iothub_client_sample.py’  and make below two changes:

  1. PROTOCOL = IotHubTransportProvider.AMQP
  2. CONNECTION_STRING = <your device connection string>

Note: device connection string was identified in Step-2

azure_iot_5

We will be using AMQP protocol. Azure IoT hub allows devices to use following protocols: MQTT, MQTT over WebSockets, AMQP, AMQP over WebSockets and  HTTP. This link contains supported protocols and port numbers

 

Execute Python script:

$python iothub_client_sample.py

Monitor device to cloud messages using device explorer

  1. Download device explorer from here Downloads . Device explorer allows you to monitor device to cloud messages, send message to device and also register device to IoT hub
  2. Enter your IoT hub connection string and click Update

azure_iot_9

3. Under Data select Device ID and click on Monitorazure_iot_6

Azure IoT web client

You can also monitor device to cloud messages and send messages to device using Azure IoT web client. Enter this url in your browser https://azure-iot.github.io/#/monitor?_k=bk33tk. Enter IoT hub connection string and click Monitor

azure_iot_12

Step-4: Create Web App to monitor device to cloud messages

Follow instructions to deploy web app, link. Once web app installed you can visualize temperature and humidity data in graph

Note: If you are new to Git on Windows you can download Git bash tool from here to run Git commands in Windows

azure_iot_13

 

iothub-explorer

iothub-explorer is a command line tool. It is a very versatile tool to create device identity in iot hub, simulate IoT device, send D2C messages or C2D messages and monitor D2C or C2D messages

Install iothub-explorer

I have installed it in my Ubuntu 16.04 VM

$sudo apt-get install npm
$sudo apt-get install nodejs-legacy
$sudo npm install -g iothub-explorer
$iothub-explorer -V
1.1.20

Monitor D2C messages

Open two terminals in Ubuntu VM in one terminal execute temp + humidity sensor script (Step-3) and in second terminal execute iothub-explorer to monitor D2C messages

$iothub-explorer monitor-events <device-id> –login <iot hub connection string>

$iothub-explorer monitor-events myIotDevice --login "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxx="

Monitoring events from device myIotDevice...
==== From: myIotDevice ====
{
  "deviceId": "myPythonDevice",
  "windSpeed": 15.18,
  "temperature": 26.57,
  "humidity": 71.8
}
---- application properties ----
{
  "temperatureAlert": "false"
}

Create device identity in iot hub

$iothub-explorer create <device-id> –login <iot hub connection string>

$iothub-explorer create myIotDevice_1 --login "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxx="

Monitor D2C messages with simulated device in iothub-explorer

Open two terminals in one terminal simulate iot device and send D2C messages to and in second terminal monitor D2C messages

Simulate iot device and send messages to iot hub

$ iothub-explorer simulate-device myIotDevice --login "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxx=" --send "Hello Cloud!"
Message #0 sent successfully
Message #1 sent successfully
Message #2 sent successfully
Message #3 sent successfully
Message #4 sent successfully
Message #5 sent successfully

Monitor D2C messages in second terminal

$ iothub-explorer monitor-events myIotDevice --login "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxx="

Monitoring events from device myIotDevice...
==== From: myIotDevice ====
Hello Cloud!
====================
==== From: myIotDevice ====
Hello Cloud!
====================
==== From: myIotDevice ====
Hello Cloud!
====================
==== From: myIotDevice ====
Hello Cloud!
====================
==== From: myIotDevice ====
Hello Cloud!
====================
==== From: myIotDevice ====
Hello Cloud!
====================

 

Send C2D messages and monitor using iothub-explorer

Open two terminals in one terminal send C2D messages and in second terminal monitor messages

Send C2D messages

$iothub-explorer send myIotDevice --login "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxx" "Hello Iot Device!"
Message sent with id: d15b5000-d20d-46e9-ac11-1de7861ef3ea

Monitor C2D messages in second terminal

$ iothub-explorer simulate-device myIotDevice --login "HostName=myIoT-Hub.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxx=" --receive

==================
Message received:
Hello Iot Device!
==================

 

$ iothub-explorer help

  Usage: iothub-explorer [options] <command> [command-options] [command-args]


  Options:

    -V, --version  output the version number
    -h, --help     output usage information


  Commands:

    login                                                                          start a session on your IoT hub
    logout                                                                         terminate the current session on your IoT hub
    list                                                                           list the device identities currently in your IoT hub device registry
    create <device-id|device-json>                                                 create a device identity in your IoT hub device registry
    delete <device-id>                                                             delete a device identity from your IoT hub device registry
    get <device-id>                                                                get a device identity from your IoT hub device registry
    import-devices                                                                 import device identities in bulk: local file -> Azure blob storage -> IoT hub
    export-devices                                                                 export device identities in bulk: IoT hub -> Azure blob storage -> local file
    send <device-id> <message>                                                     send a message to the device (cloud-to-device/C2D)
    monitor-feedback                                                               monitor feedback sent by devices to acknowledge cloud-to-device (C2D) messages
    monitor-events [device-id]                                                     listen to events coming from devices (or one in particular)
    monitor-uploads                                                                monitor the file upload notifications endpoint
    monitor-ops                                                                    listen to the operations monitoring endpoint of your IoT hub instance
    sas-token <device-id>                                                          generate a SAS Token for the given device
    simulate-device <device-id>                                                    simulate a device with the specified id
    get-twin <device-id>                                                           get the twin of a device
    update-twin <device-id> <twin-json>                                            update the twin of a device and return it.
    query-twin <sql-query>                                                         get twin data matching the sql-query argument
    query-job [job-type] [job-status]                                              get scheduled job data matching the sql-query argument
    device-method <device-id> <method-name> [method-payload] [timeout-in-seconds]  executes a device method on the specified device
    help [cmd]

 

Individual command help

$ iothub-explorer help simulate-device

  Usage: iothub-explorer-simulate-device [options]

  Simulate a device.


  Options:

    -V, --version                                          output the version number
    --device-connection-string <device-connection-string>  connection string to use for the device
    -l, --login <iothub-connection-string>                 use the connection string provided as argument to use to authenticate with your IoT Hub instance
    --protocol <amqp|amqp-ws|http|mqtt|mqtt-ws>            protocol used to send and receive messages (defaults to amqp)
    --send [message]                                       send a test message as a device. If the message is not specified, a default message will be used
    --send-interval <interval-in-milliseconds>             interval to use between each message being sent (defaults to 1000ms)
    --send-count <message-count>                           number of messages to send
    --receive                                              Receive cloud-to-device (C2D) messages as a device
    -v, --verbose                                          shows all the information contained in the message received, including annotations and properties
    --receive-count <message-count>                        number of C2D messages to receive
    --settle <complete|abandon|reject>                     indicate how the received C2D messages should be settled (defaults to 'complete')
    --upload-file <file-path>                              upload a file from the simulated device
    -h, --help