Dynamics Edge offers Microsoft Windows Azure IoT Hub MQTT Training Macon, Georgia USA in San Jose, California USA and more.
How to Communicate with your IoT hub using the MQTT protocol (source)
Message Queuing Telemetry Transport or MQTT (Message Queuing Telemetry Transport) is actually an ISO standard (ISO prefix number 20922). it is a pubsub design pattern or publish-subscribe-based messaging protocol used within Azure IoT or Internet of Things. This MQTT works on top of the TCP/IP protocol and is used for things like Azure Telemetry and we have custom Azure training courses on all this.
Azure IoT Hub Training 2019 year-round by Dynamics Edge enables you to learn how to make devices communicate with the IoT Hub device endpoints using one of:
MQTT v3.1.1 over WebSocket on port 443.
MQTT v3.1.1 on port 8883.
Azure IoT Hub is not actually an MQTT broker that has all features in it, and currently does not support all the full behaviors specified in the MQTT v3.1.1 standard spec. This free Windows Azure IoT Training for MQTT article describes how devices can use supported MQTT behaviors to communicate with IoT Hub, more info can be found by contacting Dynamics Edge for a custom course.
Some of the features mentioned in this free Azure Training article, such as and including device twins, cloud-to-device messaging, device management, and more, might only be available in the standard tier of IoT hub on Azure. For more relevant information about the basic and standard IoT Hub tiers, contact Dynamics Edge, the Microsoft partner you can trust.
Note that all device communication with IoT Hub must absolutely be secured using TLS/SSL protocol. So therefore, Azure IoT Hub’s support is not enabled on purpose for non-secure plaintext connections over port 1883 on Azure.
How to start connecting to IoT Hub.
In most cases, any kind of device can use the MQTT protocol to connect to an IoT hub using one of the following:
Liraries in the Azure IoT SDKs.
MQTT protocol directly.
How to start using the device SDKs
The relevant device SDKs that support the MQTT protocol are available for Node.js, Python, C, Java and C-Sharp or C# . The related device SDKs use the known standard IoT Hub connection string, the purpose of which is to establish a connection to an IoT hub on Azure. For starting utilization of the MQTT protocol, what’s necessary is that the client protocol parameter must actually be set to MQTT in the proper location. The default setting is that the device SDKs actually connect to an Azure IoT Hub with the CleanSession flag already pre-set to zero ( 0 ) and use a value for QoS of one ( 1 ) in relation to the actual message exchange with the Microsoft Azure IoT hub platform. At the time that a device is connected to an IoT hub, the thing is that the relevant device SDKs provide exposed methods that actually enable the device to exchange messages with an IoT hub on Azure.
Related Language. Protocol parameter reference.
C# TransportType.Mqtt.
Python IoTHubTransportProvider.MQTT.
Java IotHubClientProtocol.MQTT.
C MQTT_Protocol.
Migrating a device app from AMQP to MQTT.
Node.js azure-iot-device-mqtt.
In the case that you are really using the device SDKs, you should actually know that making the switch from AMQP to MQTT protocol has the requirement of modifying what’s known as the protocol parameter that is specified in the client initialization.
Know that AMQP returns errors for some conditions. In the case of the other one. MQTT terminates the connection currently in session. Any related exception handling flows of logic could in some cases need modifications to handle all of this.
Know that MQTT does not currently support the ‘reject’ operations when in the process of receiving cloud-to-device messages in relation to Azure IoT hub.
In the case that your back-end app needs to receive a response from the device app, please consider using direct methods when using Azure IoT MQTT.
Using the Azure MQTT protocol directly.
If you see that a device cannot use the device SDKs, please note that it can still connect to the public device endpoints using the MQTT protocol on port 8883. In the CONNECT packet the device should use the following values:
For the ClientId field, use the deviceId.
For the Username field, use {iothubhostname}/{device_id}/api-version=2016-11-14, where {iothubhostname} is the full CName of the IoT hub.
For example, if the name of your IoT hub is contoso.azure-devices.net and if the name of your device is MyDevice01, the full Username field should contain:
contoso.azure-devices.net/MyDevice01/api-version=2016-11-14
For the Password field, use a SAS token. The format of the SAS token is the same as for both the HTTPS and AMQP protocols:
SharedAccessSignature sig={signature-string}&se={expiry}&sr={URL-encoded-resourceURI}
Note
If you use X.509 certificate authentication, SAS token passwords are not required. For more information, see Set up X.509 security in your Azure IoT Hub
For more information about how to generate SAS tokens, see the device section of Using IoT Hub security tokens.
When testing, you can also use the cross-platform Azure IoT Toolkit extension for Visual Studio Code or the Device Explorer tool to quickly generate a SAS token that you can copy and paste into your own code:
For Azure IoT Toolkit:
Expand the AZURE IOT HUB DEVICES tab in the bottom left corner of Visual Studio Code.
Right-click your device and select Generate SAS Token for Device.
Set expiration time and press ‘Enter’.
The SAS token is created and copied to clipboard.
For Device Explorer:
Go to the Management tab in Device Explorer.
Click SAS Token (top right).
On SASTokenForm, select your device in the DeviceID drop down. Set your TTL.
Click Generate to create your token.
The SAS token that’s generated has the following structure:
HostName={your hub name}.azure-devices.net;DeviceId=javadevice;SharedAccessSignature=SharedAccessSignature sr={your hub name}.azure-devices.net%2Fdevices%2FMyDevice01%2Fapi-version%3D2016-11-14&sig=vSgHBMUG…..Ntg%3d&se=1456481802
The part of this token to use as the Password field to connect using MQTT is:
SharedAccessSignature sr={your hub name}.azure-devices.net%2Fdevices%2FMyDevice01%2Fapi-version%3D2016-11-14&sig=vSgHBMUG…..Ntg%3d&se=1456481802
For MQTT connect and disconnect packets, IoT Hub issues an event on the Operations Monitoring channel. This event has additional information that can help you to troubleshoot connectivity issues.
The device app can specify a Will message in the CONNECT packet. The device app should use devices/{device_id}/messages/events/ or devices/{device_id}/messages/events/{property_bag} as the Will topic name to define Will messages to be forwarded as a telemetry message. In this case, if the network connection is closed, but a DISCONNECT packet was not previously received from the device, then IoT Hub sends the Will message supplied in the CONNECT packet to the telemetry channel. The telemetry channel can be either the default Events endpoint or a custom endpoint defined by IoT Hub routing. The message has the iothub-MessageType property with a value of Will assigned to it.
TLS/SSL configuration
To use the MQTT protocol directly, your client must connect over TLS/SSL. Attempts to skip this step fail with connection errors.
In order to establish a TLS connection, you may need to download and reference the DigiCert Baltimore Root Certificate. This certificate is the one that Azure uses to secure the connection. You can find this certificate in the Azure-iot-sdk-c repository. More information about these certificates can be found on Digicert’s website.
An example of how to implement this using the Python version of the Paho MQTT library by the Eclipse Foundation might look like the following.
First, install the Paho library from your command-line environment:
cmd/sh
Copy
pip install paho-mqtt
Then, implement the client in a Python script. Replace the placeholders as follows:
is the path to a local file that contains the DigiCert Baltimore Root certificate. You can create this file by copying the certificate information from certs.c in the Azure IoT SDK for C. Include the lines —–BEGIN CERTIFICATE—– and —–END CERTIFICATE—–, remove the ” marks at the beginning and end of every line, and remove the \r\n characters at the end of every line.
is the ID of a device you added to your IoT hub.
is a SAS token for the device created as described previously in this article.
the name of your IoT hub.
Python
Copy
from paho.mqtt import client as mqtt
import ssl
path_to_root_cert = “”
device_id = “”
sas_token = “”
iot_hub_name = “”
def on_connect(client, userdata, flags, rc):
print (“Device connected with result code: ” + str(rc))
def on_disconnect(client, userdata, rc):
print (“Device disconnected with result code: ” + str(rc))
def on_publish(client, userdata, mid):
print (“Device sent message”)
client = mqtt.Client(client_id=device_id, protocol=mqtt.MQTTv311)
client.on_connect = on_connect
client.on_disconnect = on_disconnect
client.on_publish = on_publish
client.username_pw_set(username=iot_hub_name+”.azure-devices.net/” + device_id, password=sas_token)
client.tls_set(ca_certs=path_to_root_cert, certfile=None, keyfile=None, cert_reqs=ssl.CERT_REQUIRED, tls_version=ssl.PROTOCOL_TLSv1, ciphers=None)
client.tls_insecure_set(False)
client.connect(iot_hub_name+”.azure-devices.net”, port=8883)
client.publish(“devices/” + device_id + “/messages/events/”, “{id=123}”, qos=1)
client.loop_forever()
Sending device-to-cloud messages
After making a successful connection, a device can send messages to IoT Hub using devices/{device_id}/messages/events/ or devices/{device_id}/messages/events/{property_bag} as a Topic Name. The {property_bag} element enables the device to send messages with additional properties in a url-encoded format. For example:
text
Copy
RFC 2396-encoded()=RFC 2396-encoded()&RFC 2396-encoded()=RFC 2396-encoded()…
Note
This {property_bag} element uses the same encoding as for query strings in the HTTPS protocol.
The following is a list of IoT Hub implementation-specific behaviors:
IoT Hub does not support QoS 2 messages. If a device app publishes a message with QoS 2, IoT Hub closes the network connection.
IoT Hub does not persist Retain messages. If a device sends a message with the RETAIN flag set to 1, IoT Hub adds the x-opt-retain application property to the message. In this case, instead of persisting the retain message, IoT Hub passes it to the backend app.
IoT Hub only supports one active MQTT connection per device. Any new MQTT connection on behalf of the same device ID causes IoT Hub to drop the existing connection.
For more information, see Messaging developer’s guide.
Receiving cloud-to-device messages
To receive messages from IoT Hub, a device should subscribe using devices/{device_id}/messages/devicebound/# as a Topic Filter. The multi-level wildcard # in the Topic Filter is used only to allow the device to receive additional properties in the topic name. IoT Hub does not allow the usage of the # or ? wildcards for filtering of subtopics. Since IoT Hub is not a general-purpose pub-sub messaging broker, it only supports the documented topic names and topic filters.
The device does not receive any messages from IoT Hub, until it has successfully subscribed to its device-specific endpoint, represented by the devices/{device_id}/messages/devicebound/# topic filter. After a subscription has been established, the device receives cloud-to-device messages that were sent to it after the time of the subscription. If the device connects with CleanSession flag set to 0, the subscription is persisted across different sessions. In this case, the next time the device connects with CleanSession 0 it receives any outstanding messages sent to it while disconnected. If the device uses CleanSession flag set to 1 though, it does not receive any messages from IoT Hub until it subscribes to its device-endpoint.
IoT Hub delivers messages with the Topic Name devices/{device_id}/messages/devicebound/, or devices/{device_id}/messages/devicebound/{property_bag} when there are message properties. {property_bag} contains url-encoded key/value pairs of message properties. Only application properties and user-settable system properties (such as messageId or correlationId) are included in the property bag. System property names have the prefix $, application properties use the original property name with no prefix.
When a device app subscribes to a topic with QoS 2, IoT Hub grants maximum QoS level 1 in the SUBACK packet. After that, IoT Hub delivers messages to the device using QoS 1.
Retrieving a device twin’s properties
First, a device subscribes to $iothub/twin/res/#, to receive the operation’s responses. Then, it sends an empty message to topic $iothub/twin/GET/?$rid={request id}, with a populated value for request ID. The service then sends a response message containing the device twin data on topic $iothub/twin/res/{status}/?$rid={request id}, using the same request ID as the request.
Request ID can be any valid value for a message property value, as per IoT Hub messaging developer’s guide, and status is validated as an integer.
The response body contains the properties section of the device twin. The following snippet shows the body of the identity registry entry limited to the “properties” member, for example:
JSON
Copy
{
“properties”: {
“desired”: {
“telemetrySendFrequency”: “5m”,
“$version”: 12
},
“reported”: {
“telemetrySendFrequency”: “5m”,
“batteryLevel”: 55,
“$version”: 123
}
}
}
The possible status codes are:
Status Description
200 Success
429 Too many requests (throttled), as per IoT Hub throttling
5** Server errors
For more information, see Device twins developer’s guide.
Update device twin’s reported properties
To update reported properties, the device issues a request to IoT Hub via a publication over a designated MQTT topic. After processing the request, IoT Hub responds the success or failure status of the update operation via a publication to another topic. This topic can be subscribed by the device in order to notify it about the result of its twin update request. To implment this type of request/response interaction in MQTT, we leverage the notion of request id ($rid) provided initially by the device in its update request. This request id is also included in the response from IoT Hub to allow the device to correlate the response to its particular earlier request.
The following sequence describes how a device updates the reported properties in the device twin in IoT Hub:
A device must first subscribe to the $iothub/twin/res/# topic to receive the operation’s responses from IoT Hub.
A device sends a message that contains the device twin update to the $iothub/twin/PATCH/properties/reported/?$rid={request id} topic. This message includes a request ID value.
The service then sends a response message that contains the new ETag value for the reported properties collection on topic $iothub/twin/res/{status}/?$rid={request id}. This response message uses the same request ID as the request.
The request message body contains a JSON document, that contains new values for reported properties. Each member in the JSON document updates or add the corresponding member in the device twin’s document. A member set to null, deletes the member from the containing object. For example:
JSON
Copy
{
“telemetrySendFrequency”: “35m”,
“batteryLevel”: 60
}
The possible status codes are:
Status Description
200 Success
400 Bad Request. Malformed JSON
429 Too many requests (throttled), as per IoT Hub throttling
5** Server errors
The python code snippet below, demonstrates the twin reported properties update process over MQTT (using Paho MQTT client):
Python
Copy
from paho.mqtt import client as mqtt
Starting point on using Azure IoT Hub for client authentication
client.subscribe(“$iothub/twin/res/#”)
rid = “1”
twin_reported_property_patch = “{\”firmware_version\”: \”v1.1\”}”
client.publish(“$iothub/twin/PATCH/properties/reported/?$rid=” + rid, twin_reported_property_patch, qos=0)
Upon success of twin reported properties update operation above, the publication message from IoT Hub will have the following topic: $iothub/twin/res/204/?$rid=1&$version=6, where 204 is the status code indicating success, $rid=1 corresponds to the request ID provided by the device in the code, and $version corresponds to the version of reported properties section of device twins after the update.
For more information, see Device twins developer’s guide.
Receiving desired properties update notifications
When a device is connected, IoT Hub sends notifications to the topic $iothub/twin/PATCH/properties/desired/?$version={new version}, which contain the content of the update performed by the solution back end. For example:
JSON
Copy
{
“telemetrySendFrequency”: “5m”,
“route”: null
}
As for property updates, null values means that the JSON object member is being deleted.
Important
IoT Hub generates change notifications only when devices are connected. Make sure to implement the device reconnection flow to keep the desired properties synchronized between IoT Hub and the device app.
For more information, see Device twins developer’s guide.
Respond to a direct method
First, a device has to subscribe to $iothub/methods/POST/#. IoT Hub sends method requests to the topic $iothub/methods/POST/{method name}/?$rid={request id}, with either a valid JSON or an empty body.
To respond, the device sends a message with a valid JSON or empty body to the topic $iothub/methods/res/{status}/?$rid={request id}. In this message, the request ID must match the one in the request message, and status must be an integer.
For more information, see Direct method developer’s guide.
Additional considerations
As a final consideration, if you need to customize the MQTT protocol behavior on the cloud side, you should review the Azure IoT protocol gateway. This software enables you to deploy a high-performance custom protocol gateway that interfaces directly with IoT Hub. The Azure IoT protocol gateway enables you to customize the device protocol to accommodate brownfield MQTT deployments or other custom protocols. This approach does require, however, that you run and operate a custom protocol gateway.
