EEPROM Handler

14 Mar 23  The EepromHandler replaces the functions of the LangloEEPROM.h file, which was previously the method by which the size and location of parameters stored in the EEPROM were specified, and the various node<id>config.h files, which were previously used to define the MAC addresses [used by the PacketHandler] of a Node and its intended LoRa Gateway, although either of these files can continue to be used as before if required. The use of EEPROM storage of parameters unique to an individual Node, however, ultimately allows the use of a single software image for any group of Nodes with a common function.

Regardless, while I am migrating sketches from using the help files to using the EepromHandler, please be aware that there may be a bit of a hardware mismatch if swapping between sketches that use different methods, since the two currently store data in different and conflicting EEPROM locations.

When migrating to the EepromHandler, the ResetEEPROM utility can be used to reset an EEPROM or establish the initial EEPROM contents.

The EepromHandler is used to store Node and application data, currently in an 'external' I2C EEPROM. It is noted that some MCUs include their own EEPROMs, or pseudo-EEPROMs, and while these alternatives are not currently supported, there is no reason why the EerpomHandler could not be extended to manage these alternatives in a seamless fashion. The primary purpose of this and the other 'Handlers' is to remove the need to consider the details of common underlying processes in the development of individual applications.

While EERPOMs of greater capacity are generally used (because there are little or no cost implications), the data structure overlay currently managed by the EepromHandler comprises only 256 bytes, as illustrated below. This is, nonetheless, the maximum capacity that can be accessed through a single I2C device address on an EEPROM of 16K or less (refer to the discussion on addressing in the following section).

h Langlo EEPROM Data Structure.docx [92 KB]

EEPROM Addressing

Accessing higher capacity (>16K) I2C EEPROMs is complicated somewhat by the addressing characteristics of the I2C bus. In the present case, there were several issues that required consideration in developing methods that could accommodate EEPROMs of different sizes.

First of all, while the size of an EEPROM is generally measured in bits (e.g. an 8K EEPROM comprises 8K bits, or 1K bytes of storage), its contents are accessed using byte offsets. Even so, a 'standard' internal address is limited to just 8 bits, so that only 256 bytes are directly addressable. To manage this issue, EEPROMs of more than 2K bits (256 bytes), up to 16K, are effectively addressed in 256 byte blocks identified by unique I2C device addresses—the first by address 0x50, the second 0x51, etc.

There is a side note to be made here in relation to configuring more than one EEPROM on a single I2C bus. This is permitted, using the appropriate hardware configuration, provided no more than 16Kbits of EEPROM in total are configured (when using 'standard', single byte internal addressing). These 16Kbits of EEPROM are then addressed as eight discrete 2K 'blocks', with I2C device addresses 0x50 – 0x57 respectively. Unfortunately, there is no way for library functions to devine what I2C device address a user might wish to access, so this choice must be managed within the relevant sketch, before invoking the EepromHandler.

An expanded addressing mechanism is required to support EEPROMs larger than 16K, with the result that a 'smart' two-byte internal address must be used on EEPROMs of 32K or more. The trick, from a software perspective, is to be able to determine the size of an EEPROM, or at least the required internal addressing mode, without its being explicitly defined, thus enabling software to be universal with regard to the size of EEPROM that might be configured.

A mechanism to achieve this end, and indeed to automatically determine the size of a configured EEPROM, is described in Microchip Application Note AN690. The process described therein for determining the required addressing mechanism is as follows:

We send a two-byte address, 00 00, followed by a single data byte, 01, to be written to that address, then attempt to read the value back using a single-byte address of 00. If the EEPROM under test requires ('standard') single-byte addressing, we will read a value of 00 at address 00. If ('smart') addressing is required, the original write request will have set the value of the byte at this location to 01.

The logic of this process is as follows:

  • Transmitted write request:

    –start– 00 00 01 –stop–

  • If a ('standard') single-byte address was required, the write request will have been interpreted as a single address byte, 00, followed by two data bytes, 00 and 01, which will be stored starting, as requested, at location 00.

    Result:

    Location0001
    Value0001
  • If a ('smart') two-byte address was required, the write request will produce its intended outcome, with two address bytes, 0000, identifying the location, followed by a single data byte, 01.

    Result:

    Location00000001
    Value01unknown

When we try to read the stored value using a single address byte, 00, the request will be interpreted as follows:

  • Transmitted read request:

    –start– 00 –stop–

  • If a ('standard') single-byte address is required, the [single byte] address provided will be valid and the content of EEPROM location 00 will be returned as 00 (see above).

  • If a ('smart') two-byte address is required, the address in our read request, which contains only a single address byte, will be incomplete, and the result dependent on how the system responds to this condition. The Application Note suggests that, for reasons discussed therein, the request would always return the value at location 0000, which would be 01.

    My own observation has been that, in this situation, the EEPROM address register appears to contain the address following the last valid access request. In fact, on closer inspection, the relevant Microchip [AT24C08, AT24C16, AT24C32] data sheets all state exactly this:

    "The internal data word address counter maintains the last address accessed during the last read or write operation, incremented by one. This address stays valid between operations as long as the chip power is maintained."

    In the present case, the value of the byte at that location, location 0001, is unknown.

In our write process then, we need to ensure that we always write some known value into location 01, or 0001, of the EERPOM, as, if we are careful with our sequence of instructions, this will be the value returned by a read request with an incomplete address specification (i.e. from an EEPROM that requires two-byte addressing). Accordingly, one small modification has been made to the Microchip algorithm. Instead of attempting to write just one byte to a known location, we write two, as follows:

  • Send two write requests:

    –start– 00 01 01 –stop–

    –start– 00 00 01 –stop–

  • If a ('standard') single-byte address was required, each write request will have been interpreted as a single address byte followed by two data bytes. After the first write, we will have the following:

    Location0001
    Value0101

    Then, after the second, which overwrites the same two bytes:

    Location0001
    Value0001
  • If a ('smart') two-byte address was required, these write requests will write the value 01 to each of the two distinct addresses specified. After the first write, to location 0001, we will have the following:

    Location00000001
    Valueunknown01

    Then, after the second, to location 0000:

    Location00000001
    Value0101

    The important thing about writing the two locations in this way is that the EERPOM address register is left containing the location following the last write, that being location 01 (or 0001), the content of which we know (see above). If we had simply written two bytes in sequence, the register would be left pointing to the third byte, location 02 (or 0002), the content of which would be unknown.

This process will have no impact on the result if 'standard' single-byte addressing is required—the subsequent read request will return the value stored at location 00, which will be 00.

If, however, 'smart' addressing is required, both EEPROM locations 0000 and 0001 will contain the value 01, which will thus be the value returned whether the incomplete address provided in the 'standard' single-byte address read attempt returns the value stored at location 0000, as suggested by the Microchip Application Note, or the location following that of the last successful operation, 0001, as the datasheets suggest.

It is also noted that this process overwrites the original contents of the first two bytes of the EERPOM. In the present implementation, the relevant EepromHandler class method (setSmartSerial()) saves the original EEPROM contents and restores them on completion of the test, so no data is lost in the process—we save using both addressing methods, noting that only one will produce a valid copy, but which one is valid will be known on completion of the test and it is this copy that is then used in the restoration.

Data Structures and Methods

The central data structure in the EepromHandler defines the arrangement of data elements within the EEPROM. The data elements themelves are itemised in the EH_dataType enumerated type, which can be expanded to accommodate additional elements as required. The size and location within the EEPROM of each of the identified data elements are then defined in appropriate array structures within the EerpomHandler class.

EepromHandler Data Structures & Methods

// Index to EEPROM Data Types

enum EH_dataType {
EH_GATEWAY_MAC = 0,// 0
EH_NODE_MAC,// 1
EH_DESCRIPTOR,// 2
EH_SEQUENCE,// 3
EH_RAINFALL,// 4
EH_TANKID,// 5
EH_PUMPID,// 6
EH_NUM_TYPES
};

// The following entity defines the storage location and size
// respectively of an element stored in EEPROM

struct EH_element {
const uint8_t location;
const uint8_t byteCount;
};

static constexpr EH_element _element[EH_NUM_TYPES] = {
{0,4}, // Gateway MAC Address[uint32_t]
{8,4}, // Node MAC Address[uint32_t]
{16, 48}, // Descriptor[48 bytes]
{ 128,2}, // Sequence Number[uint16_t]
{ 132,2}// Rainfall Counter[uint16_t]
{ 136,1}// Tank ID[uint8_t]
{ 138,1}// Pump ID[uint8_t]
};

// Methods

void begin(TwoWire* _wirePtr);
void begin(TwoWire* _wirePtr, uint8_t _i2cAddress);
void begin(TwoWire* _wirePtr, bool _modeFlag);
void begin(TwoWire* _wirePtr, uint8_t _i2cAddress, bool _modeFlag);

void setI2CBus(TwoWire* _wirePtr);
void setI2CAddress(uint8_t _i2cAddress);
uint8_t getI2CAddress();
bool initSmartSerial();
void setSmartSerial(bool _modeFlag);
bool getSmartSerial();
void setDebug(bool _debug);
bool getDebug();

uint8_t getParameterByteCount(EH_dataType _dataType);
uint8_t getParameterLocation(EH_dataType _dataType);

void write(uint8_t* _buf, uint8_t _byteCount, uint8_t _eepromAddress);
void writeUint32(EH_dataType _dataType, uint32_t _value);
void writeUint16(EH_dataType _dataType, uint16_t _value);
void writeUint8(EH_dataType _dataType, uint8_t _value);
void writeBytes(EH_dataType _dataType, uint8_t* _buf);
void writeBytes(EH_dataType _dataType, uint8_t* _buf, uint8_t _byteCount);

void read(uint8_t* _buf, uint8_t _byteCount, uint8_t _eepromAddress);
uint32_t readUint32(EH_dataType _dataType);
uint16_t readUint16(EH_dataType _dataType);
uint8_t readUint8(EH_dataType _dataType);
uint8_t* readBytes(EH_dataType _dataType);

void dump();
void dump(uint8_t _startByte, uint8_t _endByte);

void scrub();
void scrub(uint8_t _startByte, uint8_t _endByte);

An EEPROM instance is created and initialised using the begin method. Several versions of the method are available, providing varying degrees of initialisation:

void begin(TwoWire* _wirePtr);
void begin(TwoWire* _wirePtr, uint8_t _i2cAddress);
void begin(TwoWire* _wirePtr, bool _modeFlag);
void begin(TwoWire* _wirePtr, uint8_t _i2cAddress, bool _modeFlag);

An appropriate Wire instance must be created in the parent sketch and identified in the call (see below). By default, the I2C device address is set to 0x50, and the internal addressing mode is set to SmartSerial (two-byte) addressing (i.e. by default, an EEPROM of 32K or more is assumed), but either, or both of these parameters can also be explicitly specified if required—setting the Mode Flag to true enables two-byte addressing (set by default in any case), and false enables 'standard' (single-byte) addressing.

Methods are provided to enable parameters to be set or reset, if and as appropriate, at any point within a sketch:

void setI2CAddress(uint8_t _i2cAddress);
void setI2CBus(TwoWire* _wirePtr);
void setSmartSerial();
void setSmartSerial(bool _modeFlag);
void setDebug(bool _debug);

Calling the setSmartSerial() function without any parameters executes the algorithm discussed above to automatically set the type of addressing required, returning true if two-byte addressing has been set or false otherwise.

Methods are also provided to query internal parameter settings:

uint8_t getI2CAddress();
bool getSmartSerial();
bool getDebug();
uint8_t getParameterByteCount(EH_dataType _dataType);
uint8_t getParameterLocation(EH_dataType _dataType);

Then we have the workhorse methods to write data to, and read data from EEPROM:

void write(uint8_t* _buf, uint8_t _byteCount, uint8_t _eepromAddress);
void writeUint32(EH_dataType _dataType, uint32_t _value);
void writeUint16(EH_dataType _dataType, uint16_t _value);
void writeUint8(EH_dataType _dataType, uint8_t _value);
void writeBytes(EH_dataType _dataType, uint8_t* _buf);
void writeBytes(EH_dataType _dataType, uint8_t* _buf, uint16_t _byteCount);

void read(uint8_t* _buf, uint8_t _byteCount, uint8_t _eepromAddress);
uint32_t readUint32(EH_dataType _dataType);
uint16_t readUint16(EH_dataType _dataType);
uint8_t readUint8(EH_dataType _dataType);
uint8_t* readBytes(EH_dataType _dataType);

The write() and read() methods provide the ability to write and read arbitrary byte streams to arbitrary EEPROM locations. The more specific write and read methods are provided to simplify the process of writing and reading specific data types to predefined locations associated with the specified EH_dataType (see definition above).

Finally there are methods to generate a HEX dump of, or 'scrub' (reset to all-1s) the current contents of the EEPROM:

void dump();
void dump(uint8_t _startByte, uint8_t _endByte);

void scrub();
void scrub(uint8_t _startByte, uint8_t _endByte);

The dump() and scrub() methods currently only operate on the first 256 bytes of an EEPROM (that's all the EepromHandler uses), although this applies to any EEPROM segment that is addressed through a unique I2C device address. If only a subset of this block is required, the relevant start and end bytes can be specified.

Usage

Note that, as stated in the Microchip AT24C32/AT24C64 datasheet, when these EEPROMs are powered with less than 5V, the I2C bus should not be clocked above 100kHz. I've not had problems using a higher rate (400kHz) in conjunction with the Espressif MCUs, but my experience has been that this limitation must be observed when using the 3.3V Pro Mini.

Typical usage of the EepromHandler class and methods is illustrated in the following pseudo-sketch.

Typical Usage

#include <Wire.h>
#include <EepromHandler.h>

const uint8_t I2C_EEPROM_Address = 0x50;
EH_dataType parameter;
bool smartSerial;// Smart Serial Addressing is used for 32K and 64K EEPROMs
uint16_t sequenceNumber, rainfallCounter;
uint8_t* descriptor;

EepromHandler eeprom;

void setup() {
.
.

// Define the I2C bus
// Note that for EEPROMs 32K and larger, the I2C bus should not
// be clocked higher than [the default rate of] 100kHz when
// operating below 5V

Wire.begin(SDA,SCL);

// Initialise the EEPROM instance
// This form of the begin method identifies the I2C bus to be used,
// allows the use of the default EEPROM I2C device address (0x50)
// and sets the default, SmartSerial, addressing mode

eeprom.begin(&Wire);

// If the size of the EEPROM is unknown, the required addressing mode
// can be determined automatically as follows

smartSerial = eeprom.setSmartSerial();
if (smartSerial) {
Serial.println("[setup] Smart Serial Addressing (32K+ EEPROM)");
} else {
Serial.println("[setup] Standard Serial Addressing (16K- EEPROM)");
}
.
.
}

void loop() {
.
.
Serial.println("[loop] Retrieve data from EEPROM...");
Serial.print(" Gateway MAC (GM): 0x");
Serial.println(eeprom.readUint32(EH_GATEWAY_MAC),HEX);
Serial.print(" Node MAC (NM): 0x");
Serial.println(eeprom.readUint32(EH_NODE_MAC),HEX);
Serial.print(" Descriptor (DS): ");
descriptor = eeprom.readBytes(EH_DESCRIPTOR);
int byteCount = eeprom.getParameterByteCount(EH_DESCRIPTOR);
for (int i = 0; i < byteCount; i++) {
Serial.print((char)descriptor[i]);
}

// Note that descriptor is only a pointer to temporary storage used when
// retrieving data from EEPROM. Any subsequent EEPROM read will use the same
// location. If retrieved data needs to be preserved for later use, it should
// be copied to appropriate local storage.

Serial.println();
Serial.print(" Sequence # (SN): ");
Serial.println(eeprom.readUint16(EH_SEQUENCE));
Serial.print("Rain Counter (RC): ");
Serial.println(eeprom.readUint16(EH_RAINFALL));
Serial.print(" Tank ID (TD): ");
Serial.println(eeprom.readUint8(EH_TANKID));
Serial.print(" Pump ID (PD): ");
Serial.println(eeprom.readUint8(EH_PUMPID));
.
.
sequenceNumber++;
Serial.println("[loop] Write data to EEPROM...");
eeprom.writeUint16(EH_SEQUENCE, sequenceNumber);
}
07-09-2024