Arduino Serial -- Reading/Parsing
A Code Collection

by Matthew Ford 7th July 2021 (original 12th March 2021)
© Forward Computing and Control Pty. Ltd. NSW Australia
All rights reserved.

A collection of Arduino Sketches
for Reading and Parsing Serial input

Note: this is a partially completed work in progress, so expect corrections/additions

Arduino to Arduino via Serial – robust with circuit diagrams for various boards (work in progress)
ReadOneChar – non-blocking
readStringUntil – blocks while reading input line otherwise non-blocking
Read a line – non-blocking, with line limit, with limit and timeout
Flushing the Input – blocking and non-blocking
SafeStringReader – Read a line with line length limit and timeout and flushing, non-blocking
Number Conversions – Arduino String to int/long, SafeString to int/long float/double, from hex,binary,octal
Parsing Input – Read an Int, non-blocking

Introduction

These sketches are solutions for questions that commonly come up on the Arduino Forum. The sketches cover reading and parsing text input and writing delays and timers. The sketches use the robust Arduino Strings and SafeString library. They don't use any low-level c-string methods or char[] manipulations that are so prone to programming errors and are difficult to fault find.

The reading/parsing sketches are for text data and are not suitable for handling binary data. That is data that have 0x00 values embedded in the data.

Not all the sketches are 'perfect'. The pros and cons are listed for each. The important parts of the sketch are shown here with a link to a full working sketch. Links are also provide to further detailed information where appropriate.

Also see Arduino For Beginners – Next Steps
Taming Arduino Strings
How to write Timers and Delays in Arduino
SafeString Processing for Beginners
Simple Arduino Libraries for Beginners
Simple Multi-tasking in Arduino
Arduino Serial I/O for the Real World

Definitions

Non-blocking/Blocking: This code does not delay the running of the rest of the loop() code. Non-blocking is good. Blocking is not so good, but is sometimes acceptable for very simplistic sketches.
Robust/Fragile: This code handles less then perfect input data. For example “a” entered for an integer number should be rejected by Robust code. Fragile code only works for 'correct' data. Robust is good. Fragile is bad.

Pros/Cons: Each Solution has a list of Pros and Cons to help you decide if it is suitable for your application.

Arduino to Arduino/PC via Serial

For Arduino to Arduino/PC Serial connections to send and receive data for UNO SoftwareSerial, Mega2560, ESP8266, EPP32 and python.

Reading and Parsing Text Input

ReadOneChar - non-blocking

At the lowest level Arduino Serial reads a character at a time using Serial.read() which returns -1 if there is no char to be read, readOneChar.ino

void loop() {
  int c = Serial.read(); // note read() returns an int
  if (c != -1) { // read() return -1 if there is nothing to be read. 
    // got a char handle it
    Serial.println((char)c); // need to cast to char c to print it otherwise you get a number instead
  }
}

Pros: Extremely Simple, Non-blocking and Robust
Cons: Nothing really. You can do a lot with one char cmds (A..Z,a..z,0..9) + symbols

Arduino's Serial readStringUntil – Blocking read of input

The Arduino Serial class provide a number of read.., find.., parse.. methods.
SerialReadStringUntil.ino is an example of using Serial.readStringUntil to read a line of available input.

The Serial.setTimeout(50) reduces the default 1sec timeout of readString( ) / readStringUntil( ) and is suitable for input from the Arduino IDE Monitor for 9600baud and above.
The input.trim(); handles all types of line endings by removing any leading/trailing white space including any trailing \r\n
The if (Serial.available()) { skips the readStringUntil timeout if there is nothing to read. So when there is not input the loop() is not delayed at all.
When there is input the loop() blocks until the delimiter is read or until nothing arrives for 50mS. At 9600 it takes about 1mS per char for the chars to arrive at the Serial readStringUntil()
Increasing the baud rate to say 115200 reduces that to <0.1mS per char or about 15mS for 80chars of input.

The advantage of readStringUntil() is that it will return as soon as the delimiter, e.g. /n (NL) is read, but it will also handle un-delimited input but just timing out.

String input;
int number;
void setup() {
  Serial.begin(9600);
  Serial.setTimeout(50); // about 1mS/char at 9600baud
  for (int i = 10; i > 0; i--) {
    Serial.print(' '); Serial.print(i);
    delay(500);
  }
  Serial.println();
  Serial.println(F("SerialReadStringUntil.ino -- Enter number"));
  input.reserve(80); // expected line size, see Taming Arduino Strings
  // https://www.forward.com.au/pfod/ArduinoProgramming/ArduinoStrings/index.html
}

bool processInput() {
  if (Serial.available()) {
    input = Serial.readStringUntil('\n');
    if (input.length() > 0) {  // got some new input
      if (!input.endsWith("\n")) {
        // missing NL delimiter - is that a problem?
      }
      input.trim(); // remove delimiters
      // parse input
      int newNumber = input.toInt();
      if (input != String(newNumber)) { // check for valid number
        // invalid input
        Serial.print(input);  Serial.println(F(" is invalid number"));
      } else {
        number = newNumber;
        return true;
      }
    }
  }
  return false;
}

void loop() {
  if (processInput()) {
    // got a new number
    Serial.print(F("New number:")); Serial.println(number);
  }
  // other tasks here
}

Pros: Simple. Robust in that it will return all the available input even if there is no line ending specified. No delay if nothing to read. Small delay if reading input. Use 115200 or higher baud rate to reduce the reading delay to <10mS for 80 char line.
Cons: Blocking while reading, but increasing the baud rate minimizes that. This approach temporarily uses twice the RAM becuause Serial.readStringUntil( ) creates a String the size of the input and then copies it to input before releasing the String read in.
But the input String memory is completely recovered when processInput method exits. Useful for small sketches that only do one thing and have enough memory to hold the input String twice.

The other read.., find.. and parse.. methods can also block the loop for the setTimeout( ) setting and so are not recommended except for very simple sketches. See below for non-blocking alternatives. In addition the parseInt() and parseFloat() methods are not robust.

Read a line - non-blocking

A common request is to read a line of input from Serial. readStringUntil_nonBlocking.ino uses the readStringUntil method to read the input until the until_c char found.

// read Serial until until_c char found, returns true when found else false
// non-blocking, until_c is returned as last char in String, updates input String with chars read
bool readStringUntil(String& input, char until_c) {
  while (Serial.available()) {
    char c = Serial.read();
    input += c;
    if (c == until_c) {
      return true;
    }
  }
  return false;
}

void loop() {
  if (readStringUntil(input, '\n')) { // read until find newline
    Serial.print(F(" got a line of input '")); Serial.print(input); Serial.println("'");
    input = ""; // clear after processing for next line
  }
}

Pros: Simple. Non-Blocking
Cons: Does not return until the until_c char is found. i.e. no timeout or length limit

Read a line with line length limit – non-blocking

It is simple to add a line length limit which will detect if there is a missing newline resulting in an unexpectedly long input line. readStringUntilLimited.ino adds this check limit check.
Because the terminatingChar, if found, is added to the end of the input String, the loop() code can check if it was found or if the line limit was reached.

// read Serial until until_c char found or limit char read, returns true when found/limited else false
// non-blocking, until_c is returned as last char in String, updates input String with chars read
bool readStringUntil(String& input, char until_c, size_t char_limit) {
  while (Serial.available()) {
    char c = Serial.read();
    input += c;
    if (c == until_c) {
      return true;
    }
    if (input.length() >= char_limit) {
      return true;
    }
  }
  return false;
}

char terminatingChar = '\n';
void loop() {
  if (readStringUntil(input, terminatingChar, 20)) { // read until find newline or have read 20 chars
    if (input.lastIndexOf(terminatingChar) >= 0) {   // could also use check  if (input[input.length()-1] == terminatingChar) {
      Serial.print(F(" got a line of input '")); Serial.print(input); Serial.println("'");
    } else {
      Serial.print(F(" reached limit without newline '")); Serial.print(input); Serial.println("'");
    }
    input = ""; // clear after processing for next line
  }
}

Pros: Simple. Non-Blocking, more Robust against unexpectedly long input lines
Cons: Does not return short lines until the until_c char is found. i.e. no timeout

Read a line with line length limit and timeout – non-blocking (Recommended)

As well as a line length limit, adding a (non-blocking) timeout will detect if the newline is missing from a short input line. readStringUntilLimitedTimeout.ino adds this check timeout check.
Because the terminatingChar, if found, is added to the end of the input String, the loop() code can check if it was found or if the line limit or timeout was reached.
Using method static variables, timerRunning, timerStart and timeout_mS, hides these variables from the rest of the code and holds their values from on call to readStringUntil to the next one.

// read Serial until until_c char found or limit char read or timeout, returns true when found/limited else false
// non-blocking, until_c, if found, is returned as last char in String, updates input String with chars read
bool readStringUntil(String& input, char until_c, size_t char_limit) { // call with char_limit == 0 for no limit
  static bool timerRunning; static unsigned long timerStart;     // timeout static variables
  static const unsigned long timeout_mS = 1000; // 1sec  set to 0 for no timeout

  while (Serial.available()) {
    timerRunning = false; // set true below if don't return first
    char c = Serial.read();
    input += c;
    if (c == until_c) {
      return true;
    }
    if (char_limit && (input.length() >= char_limit)) {
      return true;
    }
    // restart timer running
    if (timeout_mS > 0) {  // only start if we have a non-zero timeout
      timerRunning = true;
      timerStart = millis();
    }
  }
  if (timerRunning && ((millis() - timerStart) > timeout_mS)) {
    timerRunning = false;
    return true;
  }
  return false;
}

char terminatingChar = '\n';
void loop() {
  if (readStringUntil(input, terminatingChar, 20)) { // read until find newline or have read 20 chars, use 0 for unlimited no chars
    if (input.lastIndexOf(terminatingChar) >= 0) {   // could also use check  if (input[input.length()-1] == terminatingChar) {
      Serial.print(F(" got a line of input '")); Serial.print(input); Serial.println("'");
    } else {
      Serial.print(F(" reached limit or timeout without newline '")); Serial.print(input); Serial.println("'");
    }
    input = ""; // clear after processing for next line
  }
}

Pros: Almost Simple. Non-Blocking, Robust against unexpectedly long input lines and missing termination, until_c, char
Cons: Nothing really. The readStringUntil is a little more complicated.

Flushing the Input

Sometimes you need to ignore input that has been received before start up or, in the loop(), after a command has been recognized. The methods below cover how to ignore not just the input that has already been received but the input that is still slowly arriving at the Serial UART.

How NOT to flush the input

Serial.flush() does not flush the input. Serial.flush() (when it is implemented) only forces the TX buffer to be emptied, it does nothing to clear about the incoming RX which is continuing to arrive.

To 'flush()' the Serial RX buffer you can use

while(Serial.available()) {
  Serial.read();
}

The problem with the above code is that at 9600 baud it takes about 1mS to receive a char. The while() loop above completes in uS and a 1mS later the next char arrives in the Serial RX buffer. The result is a corrupted input rather than an empty one.

How to flush the input – blocking

To clear all the input characters that are arriving, you can use just

Serial.readString();

This statement waits for up to 1sec for characters to start to arrive and then consumes all the characters until none arrive for 1sec.

Pros: Very simple. Completely safe on small memory AVR Arduino boards like Uno and Mega2560
Cons: Blocks the sketch for at least 1sec and relies on there being a natural time break in the input stream.

You can vary the 1sec timeout using Serial.setTimeout( ) i.e.

Serial.setTimeout(100); // set 100mS timeout 
Serial.readString();
Serial.setTimeout(1000); // reset default timeout

If you only want to flush input that has already started to arrive you can use

if (Serial.available()) {
  Serial.readString();
}

The Serial.readString() approach is particularly suitable when working with user input from the Arduino IDE. When used in the setup() method the 1sec delay is not a problem. However in the loop() code it will delay the loop() by at least 1sec each time it is used.

Memory considerations
Serial.readString() accumulates the input chars in an internal String and consumes more heap memory as more chars arrive. On small memory Arduino AVR boards like UNO and Mega2560, when there is only 128bytes of memory left, the String size stops increasing and the subsequent characters are just discarded. This prevents the sketch from crashing due to Out-Of-Memory.
When the Serial.readString() returns the internal String itself is discarded and all the memory it used is recovered without any memory fragmentation. So on those small boards you can safely handle an input of unlimited length. Other boards have more memory available and can usually handle at least a few thousand chars of input without problems. The following non-blocking flushing methods are completely safe for unlimited length inputs on all boards

How to flush the input - non-blocking

The flushUntil( ) method in flushingInput.ino flushes the input until either the until_c char is read OR the input stops coming for 1 sec.

// returns true if still flushing, else false if flushing finished either by finding until_c char or by timeout
bool flushUntil(bool flushing, char until_c) {
  static bool timerRunning; static unsigned long timerStart;     // timeout static variables
  static const unsigned long timeout_mS = 1000; // 1sec  set to 0 for no timeout
  if (!flushing) { // just return
    return false;
  } // else
  if ((timeout_mS > 0) && (!timerRunning)) {  // only start if we have a non-zero timeout
    timerRunning = true;
    timerStart = millis();
  }
  while (Serial.available()) {
    timerRunning = false; // set true below if don't return first
    char c = Serial.read();
    if (c == until_c) {
      return false;  // not flushing any more
    }
    // restart timer running
    if (timeout_mS > 0) {  // only start if we have a non-zero timeout
      timerRunning = true;
      timerStart = millis();
    }
  }
  if (timerRunning && ((millis() - timerStart) > timeout_mS)) {
    timerRunning = false;
    return false;  // not  flushing any more
  }
  return true;  // still flushing
}

void setup() {
  Serial.begin(9600);

  bool stillFlushing = true; // start flushing
  while (stillFlushing = flushUntil(stillFlushing, '\n')) {
    // loop here until input is emptied or find a newline
  }
  // stillFlushing == false here
  Serial.println(F(" Finished flushing Input"));
}

Pros: Simple method call. Non-Blocking, Robust against initial partial input
Cons: Nothing really, except you need to add the flushUntil method. Handles unlimited input and by stopping on the until_c char, does not need a time break in the input to work.

Input with Start and End Characters – non-blocking (Not Recommended, usually unnecesary)

It is vary rare that you need to send data with Start and End chars, e.g. { … } as used in pfodApp. Almost always you can just send lines of data terminated by '\n' or some other delimiter and use flushUntil to discard any initial partial startup data. Having to scan for start and end chars adds un-necessary complications to the code. The sketch startEndCharInput.ino shows how you can do this if you ever need to.

char startChar = '{';
char endChar = '}';
bool flushToStart = true;

void loop() {
  if (flushToStart = flushUntil(flushToStart, startChar)) {  // flush until find {
  } else {
    // flushing finished read until }
    if (readStringUntil(input, endChar, 0)) { // read until find }
      int idx = input.lastIndexOf(endChar);
      if (idx >= 0) {   
        input.remove(idx); // remove the }
        Serial.print(F(" got a line of input '")); Serial.print(input); Serial.println("'");
      } else {
        Serial.print(F(" reached limit or timeout without } '")); Serial.print(input); Serial.println("'");
      }
      input = ""; // clear after processing for next line
      flushToStart = true; // flush until next { found
    }
  }
}

Pros: Simple using the previous flushUntil and readStringUntil methods. Non-Blocking, Robust flushes the input until finds the start, {, then reads until } or timeout or input line limit
Cons: Nothing really, except it is unnecessary in almost all case.

SafeStringReader – Read a line with line length limit and timeout and flushing (non-blocking, Recommended)

The SafeString library provides a SafeStringReader that wraps up the read line length limit, the timeout and the input flushing in one simple method, read(). SafeStringReader_flushInput.ino
Calling flushInput() clears the any pending characters and Stream's RX buffer and then discards any new input up to the next terminator or timeout, if a timeout is set.

void setup() {
  Serial.begin(9600);
  SafeString::setOutput(Serial);
  sfReader.setTimeout(1000); // set 1 sec timeout
  sfReader.flushInput(); // empty Serial RX buffer and then skip until either find delimiter or timeout
  sfReader.connect(Serial); // read from Serial
}

void loop() {
  if (sfReader.read()) { // got a line or timed out  delimiter is NOT returned
    if (sfReader.hasError()) { // input length exceeded
      Serial.println(F(" sfReader hasError. Read '\\0' or input overflowed."));
    }
    if (sfReader.getDelimiter() == -1) { // no delimiter so timed out
      Serial.println(F(" Input timed out without space"));
    }
    Serial.print(F(" got a line of input '")); Serial.print(sfReader); Serial.println("'");
    if (sfReader == "flush") {
      sfReader.flushInput();
    }
    // no need to clear sfReader as read() does that
  }
}

Pros: Minimal Code. Non-Blocking, Robust flushes any initial partial input on startup, if flushInput() called in setup().
       Calling flushInput() clears the RX buffer and then discards input up to the next terminator or timeout, if a timeout is set.
       Skips un-expected long input lines (missing terminator).
       Returns un-terminated input, if a timeout is set. Option echoOn() setting to echo all input.
Cons: Nothing really, except needs SafeString library to be installed.

Number Conversions

Arduino String to Int/Long – does not handle leading zeros

Arduino String class provides a simple not very good toInt() method. The Arduino String toInt() does not look at the whole String but just stops at the end of the first number, so Strings like “abc” return 0 and “5a” return 5.
You can overcome most of these sort comings by comparing the result to the original String, ArduinoStringToInt.ino
This does not work if the number has leading zeros or if you are trying to convert floats/doubles. For better conversion routines use SafeString methods shown next.

  String numStr = " 43 ";
  numStr.trim();  // need to remove leading and trailing white space for the check to work
  int result = numStr.toInt(); // Arduino toInt() actually returns a long assigning to an int can be invalid for large numbers
  if (numStr == String(result)) {
    Serial.print(F(" result = ")); Serial.println(result);
  } else {
    Serial.print(F("Not a valid integer '")); Serial.print(numStr); Serial.println("'");
  }

Pros: Fairly robust, catches invalid numbers
Cons: Need to trim() String first. Does not handle leading zeros, i.e. "0044" flagged as an invalid integer

SafeString conversions – very robust, handles floats, hex etc

The Arduino String to int conversion above does not handle leading zeros and the processing of checking the result can not be used for floats and doubles.

The SafeString library provides a number of very robust conversion routines which return false if the text is not valid otherwise return true and set the argument to the result. They are:-
bool toInt(int&), bool toLong(long&), bool hexToLong(long&), bool octToLong(long&), bool binToLong(long&), bool toFloat(float&), bool toDouble(double&)

The sketch SafeStringToInt.ino shows how to convert SafeStrings and Arduino Strings to ints using the SafeString.toInt(..) method.

  sfString = " 0043 \n";  // sfString is a SafeString
  int result = 0;
  if (sfString.toInt(result)) { // ignores leading and trailing whitespace
    Serial.print(F(" result = ")); Serial.println(result);
  } else {
    Serial.print(F("Not a valid integer '")); Serial.print(sfString); Serial.println("'");
  }

  // To convert from an Arduino String
  // first wrap the Arduino String's underlying char[] in a SafeString
  String numStr = " 0004 \n"; // an Arduino String
  cSFP(sfStr, (char*)numStr.c_str()); // wrap String's char[] in a SafeString
  int numResult = 0;
  if (sfStr.toInt(numResult)) { // ignores leading and trailing whitespace
    Serial.print(F(" numResult = ")); Serial.println(numResult);
  } else {
    Serial.print(F("Not a valid integer '")); Serial.print(numStr); Serial.println("'");
  }

Pros: Very robust catches invalid numbers and handles leading zeros/white space and trailing white space, no need to trim()
Cons: Nothing really, except needs SafeString library to be installed.

The example sketch SafeStringToNum.ino provided with the SafeString library has examples of each of the other SafeString number conversion methods.

Parsing Input

Reading an Int – non-blocking replacement for Serial.parseInt()

Serial.parseInt() has number of problems. It stops at the first non-digit read and then next time around will return 0 for that non-digit. Serial.parseInt() blocks for 1sec waiting for a following char.

The complete sketch ReadToInt_String.ino combines Read a line – non-blocking and Arduino String to Int/Long above into a single sketch that reads a line of input with just one int and converts it, rejecting invalid inputs.

bool readIntNonBlocking(int& result) {
  if (readStringUntil(numStrInput, '\n')) { // got a line
    bool numberValid = false;
    // try and convert to int
    numStrInput.trim();
    int num = numStrInput.toInt(); // Arduino toInt() actually returns a long assigning to an int can be invalid for large numbers
    if (numStrInput == String(num)) {
      numberValid = true;
      // valid so update the result
      result = num;
      //Serial.print(F(" result = ")); Serial.println(result);
    } else {
      Serial.print(F("Not a valid integer '")); Serial.print(numStrInput); Serial.println("'");
    }
    numStrInput = ""; // clear after processing, for next line
    return numberValid;
  }
  return false; // still reading in line so no new number
}

void loop() {
  if (readIntNonBlocking(number)) { // got a new valid number input
    Serial.print(F(" got a new valid number:")); Serial.println(number);
  }
}

Pros: Simple. Non-Blocking, Fairly robust, catches invalid numbers
Cons: Does not return until the until_c char is found. i.e. no timeout or length limit, Does not handle leading zeros, i.e. "0044" flagged as an invalid integer

Conclusion

For use of the Arduino name see http://arduino.cc/en/Main/FAQ

Arduino Serial -- Reading/Parsing A Code Collection

A collection of Arduino Sketchesfor Reading and Parsing Serial input