Home | pfodApps/pfodDevices | WebStringTemplates | Java/J2EE | Unix | Torches | Superannuation | | About Us
 

Forward Logo (image)      

Arduino to Arduino
via Serial

by Matthew Ford 19th July 2021 (original 22nd April 2021)
© Forward Computing and Control Pty. Ltd. NSW Australia
All rights reserved.

How to send text between
Arduino boards via Serial

Introduction (still under construction)

This is a common question on the Arduino Forum. How to connect two Arduino boards via Serial to send/received text. One particular use case is when you have programmed your data collection system on an UNO or Mega2560 and then want to publish the results on a web page using an ESP8266 or ESP32. In another case you may want to POST the data to a server or GET results from the internet to return them to the Arduino. First a simple println()/readUntil('\n') sketch is presented. If that does not work for you then the SerialComs software works well with SoftwareSerial which cannot send/receive at the same time, it automatically connects and re-connects as necessary, has a checksum to detect transmission errors and can tolerant of long delay()'s in either side's loop() code. While the data sent is usually text (utf-8), SerialComs will handle any byte data that does not include the bytes 0x00 ('\0') or 0x13 (XON)

SerialComs is included in the SafeString library. SafeString is available via the Arduino Library Manager (recommended) or for manual installation from a zip file.

This tutorial includes both code and circuit diagrams to connect the two boards together, including 5V to 3V3 boards

The main example uses a Mega2560 to measure Input/Output volts and current, calculate the powers (watts) and then transfers those values, in CSV format, to an ESP8266 web server, for display on a web page. The sampling and web page code is courtesy of jelka_bisa. . An alternative example using JSON is also included.

For an alternative coms library see PowerBroker2's SerialTransfer and ArduSerial.

Tutorial Outline

CSV println( )/readUntil( ) Serial Transfers
Quick Start - SerialComs

Parsing the textReceived
Message Transfer Failures – How to fault find and fix – The importance of the serial RX buffer size
    No Output
    Odd Characters
    Error needs capacity
    Input length exceeds capacity
    textReceived timeout or CheckSum failed
Use Cases
    Displaying Mega2560 Data on a web page hosted by the ESP8266


Also see Arduino For Beginners – Next Steps
Taming Arduino Strings
How to write Timers and Delays in Arduino
SafeString Processing for Beginners
Simple Arduino Libraries for Beginners
Simple Multi-tasking in Arduino
Arduino Serial I/O for the Real World

CSV println( )/readUntil( ) Serial Transfers

The simplest way to send data via serial is to use CSV (comma separated values) and parse them on the other side. See the Pros and Cons at the end of this section.

Using this connection between an Arduino Mega2560 and an ESP8266 (Adafruit HUZZAH), the sending sketch on the Mega2560 is csvMega2560Serial1Sender.ino that just uses print to send the CSV to the hardware Serial1.

void loop() {
  delay(1000); // don't use delays in your code !!
  Serial.println(F("sending CSV"));
  Serial1.print(longitudeOut, 6);
  Serial1.print(',');
  Serial1.print(latitudeOut);
  Serial1.print('\n'); //use to only send \n
  //Serial1.println(); // actually sends \r\n
}

On the ESP8266 side a SoftwareSerial connection is used. The receiving code is csvESP8266SoftwareParser.ino This uses the SafeString library's readUntil( ) non-blocking read method and then uses the firstToken( )/nextToken( ) to pull out the fields from the CSV and toFloat( ) convert them back to floats. The SafeString toFloat( )/toInt( ) etc methods perform strict checking of the text and return false if it is not a number. atof( ) and String.toFloat() just return 0.0 on errors.

bool decodeCSV(SafeString &sfCSV, float &longitude, float &latitude) {
  if (!sfCSV.endsWith('\n')) {
    // input filled up but no \n
    Serial.print(F(" missing \\n terminator ")); Serial.println(sfCSV);
    return false;
  }
  Serial.print("sfCSV: "); Serial.print(sfCSV);

  float lng = 0, lat = 0;
  cSF(field, 20); // to hold numbers
  sfCSV.firstToken(field, ',', true); // return empty fields to detect missing data
  if (!field.toFloat(lng)) { // ignores leading and trailing whitespace
    Serial.print(F("longitude not a valid float '")); Serial.print(field); Serial.println("'");
    return false;
  }
  sfCSV.nextToken(field, ',', true);
  if (!field.toFloat(lat)) { // ignores leading and trailing whitespace
    Serial.print(F("latitude not a valid float '")); Serial.print(field); Serial.println("'");
    return false;
  }
  // else check no extra fields
  if (!sfCSV.isEmpty()) {
    Serial.print(F("More than two fields. Remaining data:")); Serial.print(sfCSV);
    return false;
  }
  // else all OK update returns
  longitude = lng;
  latitude = lat;
  return true;
}


void loop() {
  if (sfInput.readUntil(softSerial, '\n')) {
    // returns true if found \n OR reached sfInput limit
    // if \n found it is returned.
    if (decodeCSV(sfInput, longitudeIn, latitudeIn)) {
      // got new valid inputs update
      Serial.print(F(" new longitude:")); Serial.print(longitudeIn); Serial.print(F(" new latitude:")); Serial.print(latitudeIn); Serial.println();
    } else {
      // error in CSV nothing updated
    }
    sfInput.clear(); // for next line
  }
}

Pros: – The sketches are simple.
Cons: – May not work well for Software Serial connections sending in both directions. Limited error checking. Sending not throttled by the receiver, so can over run the RX buffer if the receiver is slow even if the entire message fits in the buffer.

If the message is truncated, try sending at a much slower baud rate (say 300baud) OR sending smaller messages (<63bytes each) and add a loopTimer.check(Serial) to the loop( ) to see how slow your loop() code is running.

Quick Start - SerialComs

If the simple sketch above does not work, you can use the SerialComs class contained in the SafeString library. SerialComs adds a checksum to each message and synchronizes sends/receives so a slow receiver is not overrun with data.

Here is a complete sketch. A similar sketch is used on both sides. coms.connect(..) connects to the serial connection each side is using. Only one side is set as the controller using coms.setAsController(). The controller is in charge of making the connection and re-connecting if the connection times out (about 5sec). See the Use Cases below for example sketches

#include <SoftwareSerial.h>
#include "SerialComs.h"
SoftwareSerial softSerial(10, 9); // RX, TX  (works for both Uno and Mega2560, but on Mega2560 hardware serial1 is preferred.
SerialComs coms; // default send/receive size 60

void setup() {
  softSerial.begin(9600); // not too fast
  SafeString::setOutput(Serial); // enable error msgs to be sent to Serial
  //  coms.setAsController(); // always set one side (and only one side) as the controller
  // The slowest loop code should be set at the controller, usually the web side
  if (!coms.connect(softSerial)) { // always check the return
      Serial.println(F("OutOfMemory"));
  }
  // coms.textToSend = F("Started"); // optionally send a started message to the other side.
}

void loop() {
  coms.sendAndReceive(); // always first clears textReceived and then sets any new text when complete response received

  if (!coms.textReceived.isEmpty()) { // got a new complete response
    Serial.print(F(" Received Data '")); Serial.print(coms.textReceived); Serial.println("'");
  }

  if (coms.textToSend.isEmpty() ) { // last msg has been sent can set up another one
    coms.textToSend.print(F("SoftwareSerial at ")); // can print() to the SafeString textToSend 
    coms.textToSend.print(millis()/1000.0, 2); // secs since start
    coms.textToSend.print(F("s"));
  }
}

To send data you put in the SafeString coms.textToSend usually by using the print() methods but you can also use the = and += operators to add text. The coms.sendAndRecieve() method will send the text after the other side as finished sending its text. Once the textToSend has been sent it is cleared and coms.textToSend isEmpty() becomes true. So your code can test this to determine when to load the next message to be sent.

The coms.textRecieved is filled with text received from the other side. Send and receive both have a check sum added. If the received check sum is not correct the entire message is discarded. Each time coms.sendAndReceive() is called, it starts by clearing the coms.textRecieved SafeString. If a message has been received it will be returned in coms.textRecieved. So you should test !coms.textReceived.isEmpty() to see if there is a new message. If there is a new message, your code needs to process it or save it then because coms.textRecieved will be cleared when coms.sendAndReceive() is called next loop().

This code is non-blocking, so the loop() will run as fast at it can. If the textToSend is longer the the serial TX buffer size, then the loop will pause in coms.sendAndRecive() until the entire textToSend is output to the serial connecton. For receive, the Serial software will fill the Serial RX buffer with the incoming message and each time coms.sendAndReceive() is called, that RX buffer will be quickly emptied into an internal SerialComs buffer and the loop allowed to continue. Once the last of the incoming message arrives and is read into the internal buffer the check sum is verified and the message (less the check sum and terminating XON) is transferred to coms.textReceived and !coms.textReceived.isEmpty() become true and the message should be processed by the loop() code.

Parsing the textReceived

Sending the message as a CSV is a common and convenient method. You can also use JSON for which you can use the ArduinoJson library and others to decode the message, but JSON message are typically more the twice as long as CSV (see textReceived timeout below)

To parse a CSV coms.textReceived message you can use the SafeString firstToken( )/nextToken( ) and toInt( ) and toFloat( ) methods as in the previous example. e.g.

#include "SerialComs.h"
SerialComs coms;

float temperature;
int humidity; 
cSF(label,20); // label for these readings

void setup() {
  Serial.begin(115200);
  SafeString::setOutput(Serial); // enable error messages and debugging
  Serial1.begin(9600); // the coms serial
  if (!coms.connect(Serial1)) {
      Serial.println(F("Out of memory"));
  }
}

void loop() {
  coms.sendAndReceive(); // must do this every loop

  if (!coms.textReceived.isEmpty()) { // got some data e.g  Bedroom,27.5,60 
    bool dataError = false;
    coms.textReceived.firstToken(label, ',',true); // pickup label Bedroom true => return empty fields   
    cSF(token, 20); // temp SafeString large enough for longest float
    coms.textReceived.nextToken(token, ','); // true => return empty fields   
    if (!token.toFloat(temperature)) {
      dataError = true; // conversion failed not a valid float
    }
    coms.textReceived.nextToken(token, ',',true); // true => return empty fields   
    if (!token.toInt(humidity)) {
      dataError = true;// conversion failed not a valid float
    }
    if ((!dataError) && (!label)) { // no data error or error getting label
      Serial.print(label); Serial.print(" ");
      Serial.print(temperature); Serial.print("C ");
      Serial.print(humidity); Serial.print("%");
      Serial.println();
    } else {
      Serial.println(F("Invalid data received"));
    }
  }
}

Message Transfer Failures – How to fault find and fix – The importance of the serial RX buffer size

What to check when message transfers fail. First set SafeString::setOutput(Serial); to enable debug and error messages for SafeString and SerialComs
If the message transfers are failing, first check the circuit connections and baud rate settings on both sides. Also check the SerialCom com(sendSize, receiveSize); on both sides. The sendSize on one side must match the receiveSize on the other and visa versa. Other faults are:-

No Output

If you see no error output after starting up except

Prompt other side to connect

on the controller side. Then check the circuit wiring. Make sure TX ↔ RX and RX ↔ TX and that if you are connecting a 5V board (Uno/Mega2560/Nano) to a 3V3 board (ESP8266/ESP32/Adafruit nRF52 etc) that you have the resistor divider circuit installed.

Odd Characters

If you see error messages like...

textReceived timeout without receiving terminating XON (0x13)
  textReceived cap:63 len:1 '⸮'

Then it is most likely that the baud rates do not match between the two board's serial connections. Check the code!!

Error needs capacity ...

If you see an error message like

Error: textToSend.print() needs capacity of 69
        Input arg was F(" a very long text msgs")

Then you are trying to send a message that is longer the then the SerialComs sendSize. Use the SerialCom com(sendSize, receiveSize); constructor to increase the sendSize to accommodate the message you want to send. The other side's receiveSize needs to match this side's sendSize and visa versa

Input length exceeds capacity

If you see an error message like

!! Error: receiveBuffer -- input length exceeds capacity 
        receiveBuffer cap:63 len:63 '+11.924, -4.082,-48.670, +1.574, -9.409,-14.807 a very long tex'
!! Input exceeded buffer size. Skipping Input upto next delimiter.

Then the receiveSize is less then the other side's sendSize. Make the receiveSize match the other side's sendSize and visa versa

textReceived timeout or CheckSum failed

If you see an error message like

textReceived timeout without receiving terminating XON (0x13)
  textReceived cap:203 len:63 ' +4.290, -9.260,-39.728, +0.811,-17.669,-14.335 a very long tex'
OR
Received '+12.119, -3.421,-41.462, +1.735, -8.089,-14.035 a very long text msgs a very long g text msgs a very long text msgs a very long text msgs'
 CheckSum failed --  CheckSum Hex received '8D' calculated '28'

where the text received is recognizable but truncated. Then the loop() code is taking so long to run that the serial RX buffer is overflowing and loosing the end of the message. Electrical noise on the serial wires can also cause the CheckSum to fail.

If the loop() runs fast enough so that coms.sendAndReceive() is called before the serial RX buffer fills then even very long messages can be handled just by defining large message sizes for SerialComs coms(sendSize, receiveSize). However sketches often have delays in them. These can be due to handling internet POST/GETS or due to delays the are build into third part sensor libraries or sometimes there are odd delay( ) statements in the sketch (these should be removed!!). In those cases if the send/receive message size + 3 (2 checksum + XON) exceeds the default serial RX buffer size then parts of the message can be lost and the check sum will fail and the messages will not be received.

The statement SerialCom com; sets the sendSize and receiveSize to a default 60 chars which fit in almost every board's RX buffer.
Hardware Serial RX buffer sizes for some common boards:-
Uno/Mega2560/Nano – 64 bytes (63 usable) set in HardwareSerial.h
ESP8266/ESP32 – 256 bytes can be changed with setRxBufferSize( )
Adafruit nRF52 BLE – 64 bytes (63 usable)

Arduino Software Serial library defaults to 64 bytes (63 usable) fixed in the Uno/Mega2560/Nano. On ESP8266/ESP32 the 64 byte default can be changed in the constructor.

Adding a loopTimer

First thing to do in this case is remove any delay( ) statements you have added to your loop( ) code. If that does not fix the issue then add a loopTimer (also in the SafeString library) to see how long the loop() is taking to run. i.e.

#include "SerialComs.h"
#include "loopTimer.h"
 . .  .
void loop() {
  loopTimer.check(Serial);
 . . . 

You will see output like

loop uS Latency
 5sec max:304600 avg:164921
 sofar max:304600 avg:164921 max - prt:1784

which says on average the loop() is taking 164mS to run and sometimes it takes upto 304mS to run. The prt:1784 indicates it is taking 1.8mS to print the loop Latency message. That print time is not included in the loop() times.

In this case you can try the following:-
1) As noted above, remove any delay() statements in your loop. This is always the first thing to do
2) Look for long print statements, like the error msgs above, that will delay the loop when the Serial TX buffer fills up waiting for space to send the rest of the print output. Increase the baud rate of your debug Serial. You can also add a
BufferedOutput (also in the SafeString library) to avoid these long prints from blocking your loop code. See Arduino Serial I/O for the Real World
3) Reduce the baud rate of the serial connection between the two boards to slow down how quickly the RX buffer fills. Say 4800, 1200 or even 300 baud.
4) Reduce the size of the message so that it completely fits in the RX buffer size. That is 60 chars for receiving on UNO/Mega2560/Nano (AVR) and 250 for receiving on hardware serial on ESP32/ESP8266.
5) Increase the size of the serial RX buffer to hold the entire message. This is easy to do for ESP Software Serial, but UNO/Mega2560/Nano (AVR) it requires editing the board's code files.

If fixes 1), 2) and 3) don't solve the problem and you are not receiving on an ESP board, then breaking the message up into smaller parts is probably easiest. Adding a counter field to the front of each message to allow them to be re-combined at the receiver.

Notes about Software Serial

A number of the examples in this tutorial use Software Serial so that debug message can be viewed via the Arduino IDE monitor. However where it is available, you should always use hardware Serial or Serial1 or Serial2. However on Uno and ESP8266 (ESP-12 based boards) where you want to use the USB Serial connection for debugging messages you need to use Software Serial to connect the two boards.

The ESP8266 Software Serial support full duplex send and receive, except at high baud rates (115200+) provided there are no other interrupts happening.

On AVR boards (Uno/Mega2560/Nano etc), the Arduino SoftSerial disables all interrupts when sending a byte which interferes with receiving data. The SoftwareSerial library has the following known limitations: -
If using multiple software serial ports, only one can receive data at a time.
Not all pins on the Mega and Mega 2560 support change interrupts, so only the following can be used for RX: 10, 11, 12, 13, 14, 15, 50, 51, 52, 53, A8 (62), A9 (63), A10 (64), A11 (65), A12 (66), A13 (67), A14 (68), A15 (69).
Not all pins on the Leonardo and Micro support change interrupts, so only the following can be used for RX: 8, 9, 10, 11, 14 (MISO), 15 (SCK), 16 (MOSI).
On Arduino or Genuino 101 the current maximum RX speed is 57600bps
On Arduino or Genuino 101 RX doesn't work on Pin 13

There are other alternative SoftSerial libraries AltSoftSerial (RX buffersize 80 bytes) for example that have better performance then the standard Arduino SoftwSerial library, but with more restrictions on pins etc.

Use Cases

Displaying Mega2560 Data on a web page hosted by the ESP8266

The first example is courtesy of jelka_bisa, simplified to use CSV to transfer the data. The circuit is shown below (pdf verison). A JSON version of the code is also provided.

The Mega2560 has 4 hardware serial ports Serial (USB), Serial1 Serial2 and Serial3. So the Serial1 (TX 18, RX19) will be used connect to the Adafruit Feather ESP8266 board (or WeMos D1 or similar). On the Adafruit Feather ESP8266, a SoftSerial connection will be setup on Pins 12 (TX) and 13(RX) so that the USB (Serial) can be used for debugging.

The CSV data is small so the default 60 char send/receive SerialComs can be used and default SoftSerial (64byte Rx buffer) on ESP8266. The code for the Mega2560 is in Mega2560CSVDataToESP8266.ino and the code for the ESP8266 and the web page is in ESP8266CSVWebpage_fromMega2560.zip

Running the Code

To run the code, connect the boards as shown above and then open two (2) Arduino IDE instances (i.e. open the Arduino IDE twice). Install the SafeString V4.1.4+ from the Arduino library manager. Then in one instance load the Mega2560CSVDataToESP8266.ino sketch and program the Mega2560, in the other instance load the two files from the ESP8266CSVWebpage_fromMega2560.zip . Edit the ssid and password to match your WiFi network and program the ESP8266.

On the ESP8266 IDE monitor you will see the connection message and the IP address that has been assigned by the router (using DHCP).

Jelka Bisa MPPT Project V, I and P measurements
ESP8266 Setup finished.
Connecting network ...... done
Successfully connected to : . . .  
IP address: 10.1.1.69
HTTP server started

Then open your web browser and connect to the IP address, e.g. http://10.1.1.69 to display the web page of the measurements. Without any sensors connected to the Mega2560 ADC inputs you will just see values the vary with the noise.

Code Outline

The Mega2560 sketch is Mega2560CSVDataToESP8266.ino The main parts of the Mega2560 code are:-

// Mega2560 Serial1 CSV to ESP2866
#include "SerialComs.h"
#define toESP Serial1

// default send/receive length 60 chars
SerialComs coms;

void setup() {
  SafeString::setOutput(Serial); // enable error msgs and debug
  toESP.begin(9600);   // Initialize the "link" serial port

  if (!coms.connect(toESP)) {
    while (1) {
      Serial.println(F("Out of memory"));
      delay(3000);
    }
  }
  Serial.println(F("Mega2560 Setup finished."));
  Serial.println(F("waiting for connection."));
}

// . .  .

void loop() {
  coms.sendAndReceive(); // must do this every loop
  // .. read the ADC inputs and calculate the Power 

  if (coms.textToSend.isEmpty()) { // last msg sent write new CSV msg
    coms.textToSend.print(Vin, 3, 7, true); coms.textToSend.print(',');
    coms.textToSend.print(Iin, 3, 7, true); coms.textToSend.print(',');
    coms.textToSend.print(Pin, 3, 7, true); coms.textToSend.print(',');
    coms.textToSend.print(Vout, 3, 7, true); coms.textToSend.print(',');
    coms.textToSend.print(Iout, 3, 7, true); coms.textToSend.print(',');
    coms.textToSend.print(Pout, 3, 7, true);
    // Serial.println(coms.textToSend); //debugging
  }
}

The float values Vin etc are formatted using the SafeString print(value, decimals, width, forceSign) methods to be output with 3 decimals points and a fixed width of 7 with a + sign forced if positive. If the value is too large the number of decimals is reduces to keep the width fixed otherwise the output is padded with blanks to the width.

This sketch does not expect any data to come back from the ESP8266 so the coms.textReceived is not used on the Mega2560 side

The ESP8266 sketch and webpage is in ESP8266CSVWebpage_fromMega2560.zip The main parts of the sketch are:-

#include <ESP8266WebServer.h>
#include <ESP8266WiFi.h>
#include <WiFiClient.h>
#include "SerialComs.h"
#include "SoftwareSerial.h"
// include the web page html
#include "PageIndex.h"

const int RX_pin = 13; // for ESP8266 use 13  D7 on wemos-d1-esp8266
const int TX_pin = 12; // for ESP8266 use 12  D6  on wemos-d1-esp8266
SoftwareSerial toESP(RX_pin, TX_pin);
SerialComs coms; // sendLineLength, receiveLineLength default 60 char

cSF(sfData, 100); // data to send to webpage

// webserver code and methods ...

void setup() {
  Serial.begin(115200);
  SafeString::setOutput(Serial); // enable error messages and debugging

  toESP.begin(9600); // use previous rxPin, txPin and set 256 RX buffer
  coms.setAsController(); // always set one side (and only one side) as the controller
  // The slowest loop code should be set at the controller, usually the web side

  if (!coms.connect(toESP)) {
    while (1) {
      Serial.println(F("Out of memory"));
    }
  }
  // start webserver . . .
}

void loop() {
  coms.sendAndReceive(); // must do this every loop

  if (!coms.textReceived.isEmpty()) { // got some data
    sfData = coms.textReceived; // save the data for the webpage display
    // NOTE coms.textReceived is cleared at the beginning of coms.sendAndReceive()
    Serial.println(F("  Vin  ,   Iin ,  Pin  , Vout  , Iout  , Pout"));
    Serial.println(sfData);
  }
  server.handleClient(); // handle webpages
}

In the ESP8266 code the data arrives in the com.textReceived SafeString which is reset each loop when coms.sendAndReceived() is called. So the data has to be saved in another SafeString, sfData, for sending to the web page. Nothing is sent back to the Mega2560 so coms.textToSend is not used on th ESP8266 side.

A JSON version of the project

The sketches Mega2560JsonToESP8266.ino and ESP8266JsonWebpage_fromMega2560.zip contain the JSON versions. Apart form using ArduinoJson library to create and parse the text, the main difference is that the ESP8266 Software Serial setup allocates a 256byte RX buffer so that the entire JSON message can fit in the buffer and the ESP8266 SerialComs is defined with a send size of 10 and a receive size of 250 to handle the JSON text.

SerialComs coms(10, 250); // send 10 (nothing sent), receive 250 chars on the ESP8266 side
. . .
 toESP.begin(9600, SWSERIAL_8N1, -1, -1, false, 256); // use previous rxPin, txPin and set 256 RX buffer

On the Mega2560 send the SerialComs is defined with the complementary values, ie. send 250 and receive 10

SerialComs coms(250, 10);  // 250 sendlength, 10 receive (nothing received)
// sendLineLength must be > json string
// but must be < 250 so whole line fits in Serial RX buffer

Alternative JSON libraries

Alternative JSON librarySome users have reported problems using the ArduinoJson library not releasing memory. The current example failed if the Json object was create in the loop() code instead of as a global. The code here runs reliably but if you run into problems you could look at an alternative Json libraries like jRead and it companion jWrite which work on a fixed buffer.

Conclusion

The sketches and programs presented here allow for reliable transfer of lines of text data between Arduino and Arduino via a serial (uart) connection. The send/receive is synchronized so that SoftwareSerial connections can be used and so that a slow consumer will automatically throttle the sender. If the message completely fits within the serial RX buffer then messages are not lost even if the loop() code has long delays in it.


For use of the Arduino name see http://arduino.cc/en/Main/FAQ


Forward home page link (image)

Contact Forward Computing and Control by
©Copyright 1996-2020 Forward Computing and Control Pty. Ltd. ACN 003 669 994