pfod Link Design

by Matthew Ford 12^th August 2018 (original 20^th July 2018)
© Forward Computing and Control Pty. Ltd. NSW Australia
All rights reserved.

Design Considerations for pfod Links

Introduction

pfod (Protocol For Operation Discovery) specifies the logical connection of a pfodClient (usually pfodApp running an Android mobile) to a pfodServer, usually the pfodDevice. The pfodSpecification.pdf defines the the format of the messages that are allowed between the pfodClient and pfodServer. However the 'physical' layer of the connection sometimes has additional requirements. These additional link requirements are discussed here for a variety of types of connections.

This is for instructional information only, the free pfodDesigner Android app will generate all the Arduino code you need to implement connections via Bluetooth, BLE (Bluetooth Low Energy), WiFi/Ethernet (TCP/IP), Mobile SMS and LoRa/Radio.

Connection Types

The types of 'physical' connection layers considered here are:- Bluetooth, BLE (Bluetooth Low Energy), WiFi/Ethernet (TCP/IP), Mobile SMS and LoRa/Radio.

Conditions Imposed by the pfodSpecification

The pfodSpecification.pdf imposes a number of conditions on the logical operation of the connection.

The pfodClient controls the opening and closing of connections. Only the pfodClient (e.g. pfodApp) can initiate a connection to the pfodServer and only the pfodClient can initiate a close of the current connection.
There is only ever one connection to the pfodServer at a time.
The pfodClient sends one command at a time and the pfodServer must reply with a response to that command before another command can be sent. Command and responses are enclosed in { } and are send in UTF-8 encoding.
If the pfodClient does not receive a response from the pfodServer to the pfodClient's command within the responseTimeout, then the pfodClient considers the connection to have failed and will attempt to establish a new connection. The connectionTimeout varies depending on the type of connection.
The pfodServer can send RawData at any time. RawData is any data outside the { } that enclose responses. Typically measurements are sent at regular intervals as RawData for logging and plotting by the pfodApp (pfodClient).
Note 1: Since the pfodServer cannot initiate connections, the pfodClient must already be connected to the pfodServer, before RawData can be received by the pfodClient.
Note 2: RawData is NOT transmitted reliably. It can be truncated or lost completely if connection is broken.
The CloseConnection command, {!}, is sent by the pfodClient when it is shutting down. No response is expected from the pfodServer. The pfodClient will need to open a new connection to send another command. Because no response is expected, delivery of this command is NOT reliable. That is the pfodServer may not receive it at all and the pfodClient will not detect that is has not been delivered.
The CloseConnection response, {! … }, is can be sent by the pfodServer in response to any pfodClient to indicate to the pfodClient that no further commands are need to be sent. The pfodServer leaves the connection open after sending this response.

In summary, the pfodClient needs to maintain a response timer which is started when a command is sent and stopped when the response if received. If the response timer times out ( responseTimeout ), then the pfodClient considers the connection has failed and, after cleaning up the connection, attempts to open a new connect. Any pending commands on the client side are dropped and the new connection starts from scratch, usually by sending the MainMenu command, {.}, to request the main menu from the pfodServer.

The response timer is started when the opening { of the command is sent by the pfodClient and stopped when the last byte of the pfodServer's response received, i.e. either the closing } or the end of the server's response hash if 128bit security is enabled.

Conditions Imposed by the 'physical' connection layer

The conditions imposed by the 'physical' connection layer vary depending on the type of connection.

Bluetooth and BLE – The pfodClient can open and close the connection, but in general does not know if the pfodServer is still in range. There is no special handling for these connection over and above the responseTimeout. The pfodApp does not provide for specifying a security password (see Secure Challenge and Response) for these types of connections.

Some BLE board libraries have a bug that deliveries the same BLE msg twice in quick succession. To detect these duplicate messages, pfodApp adds a msg seq character immediately before the opening { of the command and the pfodParser drops duplicate messages. Unfortunately this design only works with commands of less than 20 bytes. Fortunately most pfodCommands are less than 20 bytes. The exceptions are multi-selection lists commands and text input commands.

WiFi/Ethernet (TCP/IP) – WiFi and Ethernet connections require special handling because the connection can end being 'half closed', which happen went the client just disappears due to a bad WiFi connection, power loss at the router or forced shut down of the client. See Detection of Half-Open (Dropped) TCP/IP Socket Connections (local copy) for more details. To avoid this situation an IdleTimeout is added to the pfodServer code and KeepAlives added to the pfodClient. While the link is operational, the pfodClient sends commands at most every IdleTimeout secs. If there is no user command to send, then the pfodClient sends an empty command, { }, as a keep alive. On the pfodServer side, if there is no command is received for IdleTimeout secs then the server assumes the link is broken and closes its end of the TCP/IP connection.

Another characteristic of TCP/IP connections is that if the pfodClient is slow to consume the RawData, the RawData can build up in the physical link's buffers and still be there after the pfodClient has closed. So on opening a new connection, the pfodApp first reads and discards any buffered data before sending it first command.

pfodApp provides for specifying a security password for WiFi connections (see Secure Challenge and Response).

Mobile SMS – SMS messages are small (160 bytes), can be very slow and may be delivered out of order or not be delivered at all. An SMS connection needs special handling to allow for this. Each pfod command/response is broken up into SMS sized messages and the UTF-8 bytes encode into subset of ASCII that is supported by SMS. To allow for out-of-order and/or delayed delivery of SMS messages, each SMS message contains a connection number and a message sequence no of that message in that connection. This allows the pfodClient and pfodServer to reorder the messages and detect duplicates. Each new connection initiated by a pfodClient, increments the connection number. The first SMS message of each new connection always has 0 as its message seq no.

To allow for lost SMS messages, if the pfodClient does not receive a response to a command with in the responseTimeout (typically 3 minutes for SMS connections), it tries to resending the all the SMS messages that make up that command up to 5 times before the pfodClient decides the link has failed. On the pfodServer side, if it sees a duplicate command (by connection number and msg seq number) then is just resends the previous response already sent for that command. If the pfodClient receives both responses it just discards the second duplicate one.

To enforce a single connection at a time to the pfodServer, connections are defined by the sending phone number together with the connection number. Once a connection has been set-up, the pfodServer ignores SMS msgs from other phone numbers until, either the pfodServer receives a CloseConnection command, {!}, or the IdleTimeout timer expires (typically 10mins for SMS connections). Because the data transfer rate of SMS connections is very low, KeepAlive commands are not sent by the pfodClient, instead on the pfodServer side, when the IdleTimeout timer expires, the current connection is not closed by just marked idle. The pfodServer will continue to accept an SMS message, with the next msg seq no, from an idle connection making it active again. But once a connection is in the idle state a new connection from a different phone number (or indeed from the previous phone number with a incremented connection number) will be accepted by the pfodServer and the previous connection closed.

pfodApp provides for specifying a security password for SMS connections (see Secure Challenge and Response). If the password security fails, the pfodServer, marks that connection as closed and drop any further SMS msgs from that phone number until the connection number is incremented and the msg seq no is 0, i.e. a new connection attempt.

LoRa/Radio – These connections typically have smal messages sizes (say 255 bytes) but much faster delivery than SMS. Due to interference or range/signal strength, the messages may not be delivered at all. Again the pfod commands and responses are broken up into smaller radio messages, each with its own message seq no, and re-assembled at the receiving side.

Each radio msg contains in its header, the source address (0 to 254), the destination address (0 to 254), the msg seq no (0 to 255), and the Acked msg seq no.

Radio libraries (i.e. RadioHead) commonly use the 255 address as a broadcast destination address. pfodRadio links do not support broadcast and so do not support using 255 as source or destination addresses on pfodRadio links.

Msg seq nos start from 0 for a new connection from the pfodClient and from 1 for pfodServer responses and wrap around from 255 to 1 for both pfodClient and pfodServer. Every radio msg includes an Acked msg seq no, being the msg seq no of the last radio msg received.

Because delivery of acks, if it happens, is quick (less than a few mS), in this design each radio message must be Acked by the receiver before the next radio message is sent. If an ACK is not received within the AckTimeout, the radio msg is reset, up to 5 time , with a random delay of between AckTimeout and 2*ActTimeout. If the receiver has already seen this resent msg (i.e. the ack got lost), then is just resends the last radio message that is sent, which will include the ack the sender is waiting for. After 5 attempts, the link will be marked as failed. For pfodServers a link failure pushes EOF (0xff) into the receive buffer. The 0xff (EOF) tells the pfodSecurity class to close the connection and clear out any partially parsed command and instead parse an injected CloseConnection command {!} to pass back up to pfodDevice.

Once a connection has been established from the pfodClient to the pfodSever, the pfodServer will not accept any radio message from another address until either it has processed a CloseConnection command, {!} or the IdleTimout (typically 10sec) has expired without another command being received. In either case the pfodServer marks the current connection as closed. The pfodClient uses KeepAlives commands (typically 5sec apart) to keep the connection open as long as it needs to.

pfodApp provides for specifying a security password for LoRa/Radio connections (see Secure Challenge and Response). If the password security fails, the pfodServer, marks that connection as closed and drop any further radio msgs from that address until the there is a new connection attempt with msg seq no 0.

pfod Bridges – Bridges between different types of physical connections, eg WiFi to LoRa/Radio, require a couple of special settings. The example used here is a pfodApp (Android app) connecting to a WiFi Server Bridge which in turn is connected via a serial UART connection to an LoRa/Radio pfodClient and then via radio to the destination pfodDevice (pfodServer)

i) The LoRa/Radio pfodClient must be configured to not send keep alives commands because the responses will be forwarded all the way back to the pfodApp which did not send the command and will interfere with the real command responses. The pfodApp provides the keepAlive messages for both part of the links. The WiFi Server Bridge forwards on the the radio link.

ii) The LoRa/Radio pfodClient must have an idle timeout set so that if the WiFi (or serial) link breaks, the pfodClient can cleanly close and cleanly restart the radio connection later. The pfodDevice (pfodServer) at the other end of the radio link also has the usual an idle time out setting for pfodServers, so it will timeout and be ready to accept the new connection.
By default, the pfodClient automatically configures an idleTimeout when the keepAlive is set to 0, i.e. no keep alives. However setIdleTimeout(secs) can be called to change the default setting if it does not match the setting used in the pfodServer. The default idleTimeout is picked up from the pfod_Base class (in this case the pfodRadio class) so by default both the pfodClient and pfodServer will use the same idleTimeout unless you set one yourself.

pfodParser Library Classes

The above connections are supported by three main classes in the pfodParser library. :- pfodParser, pfodSecurity and pfodSecurityClient.

pfodServers (pfodDevices) use the pfodParser and pfodSecurity classes. pfodClients are usually the Android pfodApp , however Android mobiles do not support LoRa/Radio connections so a pfodSecurityClient class is provided to implement pfodClients for LoRa/Radio connections. These LoRa/Radio pfodClients can then be used to build bridges from LoRa/Radio to WiFi or Bluetooth connections.

All three classes, pfodParser, pfodSecurity and pfodSecurityClient, take a Arduino Stream based object as an argument to their connect(..) method. pfodSecurity and pfodSecurityClient can optionally have a password specified to enable the 128bit challenge and response security.

pfodParser is used for Serial connections and also by Bluetooth and BLE devices. The pfodParser reads from and writes to the serial connection provided. For BLE boards, the serial connection is usually uses a NORDIC UART Service, but pfodApp also supports a number other manufacturers' serial connections. There is 'standard' BLE serial service defined in the BLE spec. The pfodParser does not provide an IdleTimeout timer, so the sketch code must provide one if wanted. Alternatively the pfodParser can be replaced with the pfodSecurity class, using an empty password to disable the security, and call setIdleTimeout(sec) to set the IdleTimeout timer (defaults to 10 secs if not explicitly set)

The pfodSecurity class is used for WiFi and connects to the WiFi TCP/IP socket (wrapped in an Arduino Stream) when a TCP connection is made to the device. For SMS and LoRa/Radio connections, the pfodSecurity connects to a pfod_Base derived class which provides the message seq no handling and resends. These base classes include a suggested default for the IdleTimeout which will initialize the IdleTimeout setting in pfodSecurity. This default can be overridden by calling pfodSecurity's setIdleTimeout(secs) method.

The pfod_Base method, getDefaultTimeOut() returns the default value for the idleTimeout (in seconds). For the SMS connection this returns, 600 secs.(i.e. 10 mins). For the LoRa/Radio connection this returns 10 secs.

pfodSecuityClient is used for the client side of LoRa/Radio connections. Again it connects to a pfod_Base derived class to provide the message seq no handling and resends for the client as well as handling the client side of the pfodSecurity challenge and response security. The KeepAlive interval for pfodSecuityClient defaults to 0.5 * IdleTimeout default. This default can be overridden by calling pfodSecurityClient's setKeepAliveInterval(secs)

For SMS boards based on the SIM5320 chipset, the pfod_Base derived class is pfodSMS_SIM5320.

For LoRa/Radio boards, the pfod_Base derived class is pfodRadio. The pfodRadio class in turn depends on a low level chip driver library for that particular board, in this case the RadioHead library. The low level chip driver class needs to be wrapped in a pfodRadioDriver derived class to be used by pfodRadio. This pfodRadioDriver derived class is defined in the Arduino sketch so that pfodParser library code does not depend on the particular low level drive library used and so that the pfodParser library will compile without the low level driver library being present on the compile path.

Android^TM is a trademark of Google Inc. For use of the Arduino name see http://arduino.cc/en/Main/FAQ

The General Purpose Android/Arduino Control App.
pfodDevice™ and pfodApp™ are trade marks of Forward Computing and Control Pty. Ltd.