Home
| pfodApps/pfodDevices
| WebStringTemplates
| Java/J2EE
| Unix
| Torches
| Superannuation
|
| About
Us
|
Arduino to Arduino
|
by Matthew Ford 19th July 2021 (original 22nd April 2021)
©
Forward Computing and Control Pty. Ltd. NSW Australia
All rights
reserved.
This is a common question on the Arduino Forum. How to connect two Arduino boards via Serial to send/received text. One particular use case is when you have programmed your data collection system on an UNO or Mega2560 and then want to publish the results on a web page using an ESP8266 or ESP32. In another case you may want to POST the data to a server or GET results from the internet to return them to the Arduino. First a simple println()/readUntil('\n') sketch is presented. If that does not work for you then the SerialComs software works well with SoftwareSerial which cannot send/receive at the same time, it automatically connects and re-connects as necessary, has a checksum to detect transmission errors and can tolerant of long delay()'s in either side's loop() code. While the data sent is usually text (utf-8), SerialComs will handle any byte data that does not include the bytes 0x00 ('\0') or 0x13 (XON)
SerialComs is included in the SafeString library. SafeString is available via the Arduino Library Manager (recommended) or for manual installation from a zip file.
This tutorial includes both code and circuit diagrams to connect the two boards together, including 5V to 3V3 boards
The main example uses a Mega2560 to measure Input/Output volts and current, calculate the powers (watts) and then transfers those values, in CSV format, to an ESP8266 web server, for display on a web page. The sampling and web page code is courtesy of jelka_bisa. . An alternative example using JSON is also included.
For an alternative coms library see PowerBroker2's SerialTransfer and ArduSerial.
CSV println(
)/readUntil( ) Serial Transfers
Quick Start -
SerialComs
Parsing
the textReceived
Message
Transfer Failures – How to fault find and fix – The
importance of the serial RX buffer size
No
Output
Odd
Characters
Error
needs capacity
Input
length exceeds capacity
textReceived
timeout or CheckSum failed
Use
Cases
Displaying
Mega2560 Data on a web page hosted by the ESP8266
Also see Arduino For
Beginners – Next Steps
Taming
Arduino Strings
How
to write Timers and Delays in Arduino
SafeString
Processing for Beginners
Simple
Arduino Libraries for Beginners
Simple
Multi-tasking in Arduino
Arduino
Serial I/O for the Real World
The simplest way to send data via serial is to use CSV (comma separated values) and parse them on the other side. See the Pros and Cons at the end of this section.
Using this connection between an Arduino Mega2560 and an ESP8266 (Adafruit HUZZAH), the sending sketch on the Mega2560 is csvMega2560Serial1Sender.ino that just uses print to send the CSV to the hardware Serial1.
void loop() { delay(1000); // don't use delays in your code !! Serial.println(F("sending CSV")); Serial1.print(longitudeOut, 6); Serial1.print(','); Serial1.print(latitudeOut); Serial1.print('\n'); //use to only send \n //Serial1.println(); // actually sends \r\n }
On the ESP8266 side a SoftwareSerial connection is used. The receiving code is csvESP8266SoftwareParser.ino This uses the SafeString library's readUntil( ) non-blocking read method and then uses the firstToken( )/nextToken( ) to pull out the fields from the CSV and toFloat( ) convert them back to floats. The SafeString toFloat( )/toInt( ) etc methods perform strict checking of the text and return false if it is not a number. atof( ) and String.toFloat() just return 0.0 on errors.
bool decodeCSV(SafeString &sfCSV, float &longitude, float &latitude) { if (!sfCSV.endsWith('\n')) { // input filled up but no \n Serial.print(F(" missing \\n terminator ")); Serial.println(sfCSV); return false; } Serial.print("sfCSV: "); Serial.print(sfCSV); float lng = 0, lat = 0; cSF(field, 20); // to hold numbers sfCSV.firstToken(field, ',', true); // return empty fields to detect missing data if (!field.toFloat(lng)) { // ignores leading and trailing whitespace Serial.print(F("longitude not a valid float '")); Serial.print(field); Serial.println("'"); return false; } sfCSV.nextToken(field, ',', true); if (!field.toFloat(lat)) { // ignores leading and trailing whitespace Serial.print(F("latitude not a valid float '")); Serial.print(field); Serial.println("'"); return false; } // else check no extra fields if (!sfCSV.isEmpty()) { Serial.print(F("More than two fields. Remaining data:")); Serial.print(sfCSV); return false; } // else all OK update returns longitude = lng; latitude = lat; return true; } void loop() { if (sfInput.readUntil(softSerial, '\n')) { // returns true if found \n OR reached sfInput limit // if \n found it is returned. if (decodeCSV(sfInput, longitudeIn, latitudeIn)) { // got new valid inputs update Serial.print(F(" new longitude:")); Serial.print(longitudeIn); Serial.print(F(" new latitude:")); Serial.print(latitudeIn); Serial.println(); } else { // error in CSV nothing updated } sfInput.clear(); // for next line } }
Pros: – The sketches are simple.
Cons: –
May not work well for Software Serial connections sending in both
directions. Limited error checking. Sending not throttled by the
receiver, so can over run the RX buffer if the receiver is slow even
if the entire message fits in the buffer.
If the message is truncated, try sending at a much slower baud rate (say 300baud) OR sending smaller messages (<63bytes each) and add a loopTimer.check(Serial) to the loop( ) to see how slow your loop() code is running.
If the simple sketch above does not work, you can use the SerialComs class contained in the SafeString library. SerialComs adds a checksum to each message and synchronizes sends/receives so a slow receiver is not overrun with data.
Here is a complete sketch. A similar sketch is used on both sides. coms.connect(..) connects to the serial connection each side is using. Only one side is set as the controller using coms.setAsController(). The controller is in charge of making the connection and re-connecting if the connection times out (about 5sec). See the Use Cases below for example sketches
#include <SoftwareSerial.h> #include "SerialComs.h" SoftwareSerial softSerial(10, 9); // RX, TX (works for both Uno and Mega2560, but on Mega2560 hardware serial1 is preferred. SerialComs coms; // default send/receive size 60 void setup() { softSerial.begin(9600); // not too fast SafeString::setOutput(Serial); // enable error msgs to be sent to Serial // coms.setAsController(); // always set one side (and only one side) as the controller // The slowest loop code should be set at the controller, usually the web side if (!coms.connect(softSerial)) { // always check the return Serial.println(F("OutOfMemory")); } // coms.textToSend = F("Started"); // optionally send a started message to the other side. } void loop() { coms.sendAndReceive(); // always first clears textReceived and then sets any new text when complete response received if (!coms.textReceived.isEmpty()) { // got a new complete response Serial.print(F(" Received Data '")); Serial.print(coms.textReceived); Serial.println("'"); } if (coms.textToSend.isEmpty() ) { // last msg has been sent can set up another one coms.textToSend.print(F("SoftwareSerial at ")); // can print() to the SafeString textToSend coms.textToSend.print(millis()/1000.0, 2); // secs since start coms.textToSend.print(F("s")); } }
To send data you put in the SafeString coms.textToSend usually by using the print() methods but you can also use the = and += operators to add text. The coms.sendAndRecieve() method will send the text after the other side as finished sending its text. Once the textToSend has been sent it is cleared and coms.textToSend isEmpty() becomes true. So your code can test this to determine when to load the next message to be sent.
The coms.textRecieved is filled with text received from the other side. Send and receive both have a check sum added. If the received check sum is not correct the entire message is discarded. Each time coms.sendAndReceive() is called, it starts by clearing the coms.textRecieved SafeString. If a message has been received it will be returned in coms.textRecieved. So you should test !coms.textReceived.isEmpty() to see if there is a new message. If there is a new message, your code needs to process it or save it then because coms.textRecieved will be cleared when coms.sendAndReceive() is called next loop().
This code is non-blocking, so the loop() will run as fast at it can. If the textToSend is longer the the serial TX buffer size, then the loop will pause in coms.sendAndRecive() until the entire textToSend is output to the serial connecton. For receive, the Serial software will fill the Serial RX buffer with the incoming message and each time coms.sendAndReceive() is called, that RX buffer will be quickly emptied into an internal SerialComs buffer and the loop allowed to continue. Once the last of the incoming message arrives and is read into the internal buffer the check sum is verified and the message (less the check sum and terminating XON) is transferred to coms.textReceived and !coms.textReceived.isEmpty() become true and the message should be processed by the loop() code.
Sending the message as a CSV is a common and convenient method. You can also use JSON for which you can use the ArduinoJson library and others to decode the message, but JSON message are typically more the twice as long as CSV (see textReceived timeout below)
To parse a CSV coms.textReceived message you can use the SafeString firstToken( )/nextToken( ) and toInt( ) and toFloat( ) methods as in the previous example. e.g.
#include "SerialComs.h" SerialComs coms; float temperature; int humidity; cSF(label,20); // label for these readings void setup() { Serial.begin(115200); SafeString::setOutput(Serial); // enable error messages and debugging Serial1.begin(9600); // the coms serial if (!coms.connect(Serial1)) { Serial.println(F("Out of memory")); } } void loop() { coms.sendAndReceive(); // must do this every loop if (!coms.textReceived.isEmpty()) { // got some data e.g Bedroom,27.5,60 bool dataError = false; coms.textReceived.firstToken(label, ',',true); // pickup label Bedroom true => return empty fields cSF(token, 20); // temp SafeString large enough for longest float coms.textReceived.nextToken(token, ','); // true => return empty fields if (!token.toFloat(temperature)) { dataError = true; // conversion failed not a valid float } coms.textReceived.nextToken(token, ',',true); // true => return empty fields if (!token.toInt(humidity)) { dataError = true;// conversion failed not a valid float } if ((!dataError) && (!label)) { // no data error or error getting label Serial.print(label); Serial.print(" "); Serial.print(temperature); Serial.print("C "); Serial.print(humidity); Serial.print("%"); Serial.println(); } else { Serial.println(F("Invalid data received")); } } }
What to check when
message transfers fail. First set SafeString::setOutput(Serial);
to enable debug and error messages for SafeString and SerialComs
If
the message transfers are failing, first check the circuit
connections and baud rate settings on both sides. Also check the
SerialCom com(sendSize, receiveSize);
on both sides. The sendSize on one side must match the receiveSize on
the other and visa versa. Other faults are:-
If you see no error output after starting up except
Prompt other side to connect
on the controller side. Then check the circuit wiring. Make sure TX ↔ RX and RX ↔ TX and that if you are connecting a 5V board (Uno/Mega2560/Nano) to a 3V3 board (ESP8266/ESP32/Adafruit nRF52 etc) that you have the resistor divider circuit installed.
If you see error messages like...
textReceived timeout without receiving terminating XON (0x13)
textReceived cap:63 len:1 '⸮'
Then it is most likely that the baud rates do not match between the two board's serial connections. Check the code!!
If you see an error message like
Error: textToSend.print() needs capacity of 69
Input arg was F(" a very long text msgs")
Then you are trying to send a message that is longer the then the SerialComs sendSize. Use the SerialCom com(sendSize, receiveSize); constructor to increase the sendSize to accommodate the message you want to send. The other side's receiveSize needs to match this side's sendSize and visa versa
If you see an error message like
!! Error: receiveBuffer -- input length exceeds capacity receiveBuffer cap:63 len:63 '+11.924, -4.082,-48.670, +1.574, -9.409,-14.807 a very long tex' !! Input exceeded buffer size. Skipping Input upto next delimiter.
Then the receiveSize is less then the other side's sendSize. Make the receiveSize match the other side's sendSize and visa versa
If you see an error message like
textReceived timeout without receiving terminating XON (0x13) textReceived cap:203 len:63 ' +4.290, -9.260,-39.728, +0.811,-17.669,-14.335 a very long tex' OR Received '+12.119, -3.421,-41.462, +1.735, -8.089,-14.035 a very long text msgs a very long g text msgs a very long text msgs a very long text msgs' CheckSum failed -- CheckSum Hex received '8D' calculated '28'
where the text received is recognizable but truncated. Then the loop() code is taking so long to run that the serial RX buffer is overflowing and loosing the end of the message. Electrical noise on the serial wires can also cause the CheckSum to fail.
If the loop() runs fast enough so that coms.sendAndReceive() is called before the serial RX buffer fills then even very long messages can be handled just by defining large message sizes for SerialComs coms(sendSize, receiveSize). However sketches often have delays in them. These can be due to handling internet POST/GETS or due to delays the are build into third part sensor libraries or sometimes there are odd delay( ) statements in the sketch (these should be removed!!). In those cases if the send/receive message size + 3 (2 checksum + XON) exceeds the default serial RX buffer size then parts of the message can be lost and the check sum will fail and the messages will not be received.
The statement
SerialCom com; sets the sendSize and receiveSize
to a default 60 chars which fit in almost every board's RX buffer.
Hardware Serial RX buffer sizes for some common
boards:-
Uno/Mega2560/Nano – 64 bytes (63 usable) set in
HardwareSerial.h
ESP8266/ESP32 – 256 bytes can be changed
with setRxBufferSize( )
Adafruit nRF52 BLE – 64 bytes (63
usable)
Arduino Software Serial library defaults to 64 bytes (63 usable) fixed in the Uno/Mega2560/Nano. On ESP8266/ESP32 the 64 byte default can be changed in the constructor.
First thing to do in this case is remove any delay( ) statements you have added to your loop( ) code. If that does not fix the issue then add a loopTimer (also in the SafeString library) to see how long the loop() is taking to run. i.e.
#include "SerialComs.h" #include "loopTimer.h" . . . void loop() { loopTimer.check(Serial); . . .
You will see output like
loop uS Latency 5sec max:304600 avg:164921 sofar max:304600 avg:164921 max - prt:1784
which says on average the loop() is taking 164mS to run and sometimes it takes upto 304mS to run. The prt:1784 indicates it is taking 1.8mS to print the loop Latency message. That print time is not included in the loop() times.
In
this case you can try the following:-
1) As noted above, remove
any delay() statements in your loop. This is always the first thing
to do
2) Look for long print statements, like the error msgs
above, that will delay the loop when the Serial TX buffer fills up
waiting for space to send the rest of the print output. Increase the
baud rate of your debug Serial. You can also add a BufferedOutput
(also
in the SafeString library) to avoid these long prints from blocking
your loop code. See Arduino
Serial I/O for the Real World
3) Reduce the baud rate of the
serial connection between the two boards to slow down how quickly the
RX buffer fills. Say 4800, 1200 or even 300 baud.
4) Reduce the
size of the message so that it completely fits in the RX buffer size.
That is 60 chars for receiving on UNO/Mega2560/Nano (AVR) and 250 for
receiving on hardware serial on ESP32/ESP8266.
5) Increase the
size of the serial RX buffer to hold the entire message. This is easy
to do for ESP Software Serial, but UNO/Mega2560/Nano (AVR) it
requires editing the board's code files.
If fixes 1), 2) and 3) don't solve the problem and you are not receiving on an ESP board, then breaking the message up into smaller parts is probably easiest. Adding a counter field to the front of each message to allow them to be re-combined at the receiver.
A number of the examples in this tutorial use Software Serial so that debug message can be viewed via the Arduino IDE monitor. However where it is available, you should always use hardware Serial or Serial1 or Serial2. However on Uno and ESP8266 (ESP-12 based boards) where you want to use the USB Serial connection for debugging messages you need to use Software Serial to connect the two boards.
The ESP8266 Software Serial support full duplex send and receive, except at high baud rates (115200+) provided there are no other interrupts happening.
On AVR boards (Uno/Mega2560/Nano etc), the
Arduino SoftSerial disables all interrupts when sending a byte which
interferes with receiving data. The SoftwareSerial
library has the following known limitations: -
If using
multiple software serial ports, only one can receive data at a
time.
Not all pins on the Mega and Mega 2560 support change
interrupts, so only the following can be used for RX: 10, 11, 12, 13,
14, 15, 50, 51, 52, 53, A8 (62), A9 (63), A10 (64), A11 (65), A12
(66), A13 (67), A14 (68), A15 (69).
Not all pins on the Leonardo
and Micro support change interrupts, so only the following can be
used for RX: 8, 9, 10, 11, 14 (MISO), 15 (SCK), 16 (MOSI).
On
Arduino or Genuino 101 the current maximum RX speed is 57600bps
On
Arduino or Genuino 101 RX doesn't work on Pin 13
There are other alternative SoftSerial libraries AltSoftSerial (RX buffersize 80 bytes) for example that have better performance then the standard Arduino SoftwSerial library, but with more restrictions on pins etc.
The first example is courtesy of jelka_bisa, simplified to use CSV to transfer the data. The circuit is shown below (pdf verison). A JSON version of the code is also provided.
The Mega2560 has 4 hardware serial ports Serial (USB), Serial1 Serial2 and Serial3. So the Serial1 (TX 18, RX19) will be used connect to the Adafruit Feather ESP8266 board (or WeMos D1 or similar). On the Adafruit Feather ESP8266, a SoftSerial connection will be setup on Pins 12 (TX) and 13(RX) so that the USB (Serial) can be used for debugging.
The CSV data is small so the default 60 char send/receive SerialComs can be used and default SoftSerial (64byte Rx buffer) on ESP8266. The code for the Mega2560 is in Mega2560CSVDataToESP8266.ino and the code for the ESP8266 and the web page is in ESP8266CSVWebpage_fromMega2560.zip
To run the code, connect the boards as shown above and then open two (2) Arduino IDE instances (i.e. open the Arduino IDE twice). Install the SafeString V4.1.4+ from the Arduino library manager. Then in one instance load the Mega2560CSVDataToESP8266.ino sketch and program the Mega2560, in the other instance load the two files from the ESP8266CSVWebpage_fromMega2560.zip . Edit the ssid and password to match your WiFi network and program the ESP8266.
On the ESP8266 IDE monitor you will see the connection message and the IP address that has been assigned by the router (using DHCP).
Jelka Bisa MPPT Project V, I and P measurements ESP8266 Setup finished. Connecting network ...... done Successfully connected to : . . . IP address: 10.1.1.69 HTTP server started
Then open your web browser and connect to the IP address, e.g. http://10.1.1.69 to display the web page of the measurements. Without any sensors connected to the Mega2560 ADC inputs you will just see values the vary with the noise.
The Mega2560 sketch is Mega2560CSVDataToESP8266.ino The main parts of the Mega2560 code are:-
// Mega2560 Serial1 CSV to ESP2866 #include "SerialComs.h" #define toESP Serial1 // default send/receive length 60 chars SerialComs coms; void setup() { SafeString::setOutput(Serial); // enable error msgs and debug toESP.begin(9600); // Initialize the "link" serial port if (!coms.connect(toESP)) { while (1) { Serial.println(F("Out of memory")); delay(3000); } } Serial.println(F("Mega2560 Setup finished.")); Serial.println(F("waiting for connection.")); } // . . . void loop() { coms.sendAndReceive(); // must do this every loop // .. read the ADC inputs and calculate the Power if (coms.textToSend.isEmpty()) { // last msg sent write new CSV msg coms.textToSend.print(Vin, 3, 7, true); coms.textToSend.print(','); coms.textToSend.print(Iin, 3, 7, true); coms.textToSend.print(','); coms.textToSend.print(Pin, 3, 7, true); coms.textToSend.print(','); coms.textToSend.print(Vout, 3, 7, true); coms.textToSend.print(','); coms.textToSend.print(Iout, 3, 7, true); coms.textToSend.print(','); coms.textToSend.print(Pout, 3, 7, true); // Serial.println(coms.textToSend); //debugging } }
The float values Vin etc are formatted using the SafeString print(value, decimals, width, forceSign) methods to be output with 3 decimals points and a fixed width of 7 with a + sign forced if positive. If the value is too large the number of decimals is reduces to keep the width fixed otherwise the output is padded with blanks to the width.
This sketch does not expect any data to come back from the ESP8266 so the coms.textReceived is not used on the Mega2560 side
The ESP8266 sketch and webpage is in ESP8266CSVWebpage_fromMega2560.zip The main parts of the sketch are:-
#include <ESP8266WebServer.h> #include <ESP8266WiFi.h> #include <WiFiClient.h> #include "SerialComs.h" #include "SoftwareSerial.h" // include the web page html #include "PageIndex.h" const int RX_pin = 13; // for ESP8266 use 13 D7 on wemos-d1-esp8266 const int TX_pin = 12; // for ESP8266 use 12 D6 on wemos-d1-esp8266 SoftwareSerial toESP(RX_pin, TX_pin); SerialComs coms; // sendLineLength, receiveLineLength default 60 char cSF(sfData, 100); // data to send to webpage // webserver code and methods ... void setup() { Serial.begin(115200); SafeString::setOutput(Serial); // enable error messages and debugging toESP.begin(9600); // use previous rxPin, txPin and set 256 RX buffer coms.setAsController(); // always set one side (and only one side) as the controller // The slowest loop code should be set at the controller, usually the web side if (!coms.connect(toESP)) { while (1) { Serial.println(F("Out of memory")); } } // start webserver . . . } void loop() { coms.sendAndReceive(); // must do this every loop if (!coms.textReceived.isEmpty()) { // got some data sfData = coms.textReceived; // save the data for the webpage display // NOTE coms.textReceived is cleared at the beginning of coms.sendAndReceive() Serial.println(F(" Vin , Iin , Pin , Vout , Iout , Pout")); Serial.println(sfData); } server.handleClient(); // handle webpages }
In the ESP8266 code the data arrives in the com.textReceived SafeString which is reset each loop when coms.sendAndReceived() is called. So the data has to be saved in another SafeString, sfData, for sending to the web page. Nothing is sent back to the Mega2560 so coms.textToSend is not used on th ESP8266 side.
The sketches Mega2560JsonToESP8266.ino and ESP8266JsonWebpage_fromMega2560.zip contain the JSON versions. Apart form using ArduinoJson library to create and parse the text, the main difference is that the ESP8266 Software Serial setup allocates a 256byte RX buffer so that the entire JSON message can fit in the buffer and the ESP8266 SerialComs is defined with a send size of 10 and a receive size of 250 to handle the JSON text.
SerialComs coms(10, 250); // send 10 (nothing sent), receive 250 chars on the ESP8266 side . . . toESP.begin(9600, SWSERIAL_8N1, -1, -1, false, 256); // use previous rxPin, txPin and set 256 RX buffer
On the Mega2560 send the SerialComs is defined with the complementary values, ie. send 250 and receive 10
SerialComs coms(250, 10); // 250 sendlength, 10 receive (nothing received) // sendLineLength must be > json string // but must be < 250 so whole line fits in Serial RX buffer
Alternative JSON library – Some users have reported problems using the ArduinoJson library not releasing memory. The current example failed if the Json object was create in the loop() code instead of as a global. The code here runs reliably but if you run into problems you could look at an alternative Json libraries like jRead and it companion jWrite which work on a fixed buffer.
The sketches and programs presented here allow for reliable transfer of lines of text data between Arduino and Arduino via a serial (uart) connection. The send/receive is synchronized so that SoftwareSerial connections can be used and so that a slow consumer will automatically throttle the sender. If the message completely fits within the serial RX buffer then messages are not lost even if the loop() code has long delays in it.
For use of the Arduino name see http://arduino.cc/en/Main/FAQ
Contact Forward Computing and Control by
©Copyright 1996-2020 Forward Computing and Control Pty. Ltd.
ACN 003 669 994