Register mapping of variable length data

S

Thread Starter

Sebastian Seidel

I am programming an ATmega 644p microcontroller which will be part (slave) of a master-slave sensor network over RS485. We decided to use modbus for protocol.

Two master applications are in developement and they asked me about the interface (registers and definition of corresponding values). Since I'm new to modbus, I'm unsure about how to define them.

First possibility might be a linear memory layout (first 32 bit data in HR 0000, second in HR 0002, ...). Since I do not yet know about the max data length of many variables this could cause severe limitations in the future. There would have to be zero-filled 'reserve registers' after each data.

Second possibility might be to take each register as a pointer or index to data-items of variable length (first 32 bit data referenced by HR 0000, second by HR 0001, ...). The data would still be read or written in units of words using functions 0x03 and 0x10, only it would be impossible to read several variables in one go.

On the other hand, accessing several variables at once would be impossible anyway because some variables do not even exist in memory but are computed after requested.

In general, I want the device to work with most existing applications and equipement and to keep it as close to the standard as possible but still simple and easy to adapt to future changes.

So, should I stick to the former or latter solution or do it completely different?

Many thanks and kind regards,

Sebastian Seidel
(eMail: seidel [at] seika.de)

PS:

Apart from the register mapping, I thought of the following layout:

- All four tables overlap, R and RW data mixes.
- Functions 0x03 and 0x10 used for data access.
- Floats transfered big-endian.
- Strings transfered like they would be read.
- Data saved in native (little endian) format and shuffled before sending.
- Multibyte data accessed by specifying its first register. Otherwise, exception code 0x02 returned (for example when trying to read the second part of a 32 bit value).
 
You said that you're not sure about "the max data length of many variables". They're 32 bit, aren't they? That's what you just said, anyway. If you change that to something else (16 bit, 64 bit) in a new version, your sensor isn't going to be compatible with the previous version anyway.

The "pointer" idea sounds like a bad idea. It will take the user twice as many read operations to do anything with it, and I suspect that the majority of users will never figure out how to use it.

Keep things as simple as possible or you (or your customers) will have no end of problems. Pick a register map, and stick with it. If it has to change in future in a new version, then it's a new model with a new register map.

As for reading multiple variables at once, your software should be able to figure out if a register is being read whether it is a single or multiple read.

As for your other points:
> All four tables overlap, R and RW data mixes.

This is sometimes done, but it can be complicated to figure out what to do if the user writes to something you would rather they didn't. It is usually easiest to separate RO and RW (input and holding registers). The user then only writes to holding registers, and only reads from input registers (the same with discrete inputs and coils).

> Functions 0x03 and 0x10 used for data access.

This is normal. You will also need function 4 for input registers if you use them.

> Floats transferred big-endian.

I don't think there is any standard for this. You will always find someone who wants it the other way. If you really want something to be flexible, you may want to make endian-ness an option.

> Strings transfered like they would be read.

I think this is normal. However, would you have 1 or 2 characters per register? Both ways have their uses. Also, are you specifying ASCII only? This is normal for industrial applications.

> Data saved in native (little endian) format and shuffled before sending.

Whatever is easiest for you. Endian-ness is always a problem though. No matter what you do, someone won't be happy.

> Multibyte data accessed by specifying its first register. Otherwise, exception code 0x02 returned (for example when trying to read the second part of a 32 bit value).

The user will be reading blocks of registers. It will be up to their software to figure out how to combine registers into larger values. Exception code 2 means that all the addresses that were attempted to be read are valid. It is not meant as an "application error". If the user attempts to read a register which exists in your device, they should get data, not an error.

If you return an exception 2 for a valid register, the user will be hunting down missing terminating resistors, bad connections, bad drivers, and everything except the real problem. If you need an "application error", that is exception 4.

However, you should be doing this one of the following ways:

1) Calculate the register values on demand. Your software will "know" if one or more affect registers is being read, and can calculate the correct value before providing the data.

2) Update all register values on a regular timed cycle. Any reads which occur between update cycles get the last calculated value. This is the most common way of doing things and is also usually the easiest.

You can come up with 100 complicated ways of doing things all based on "keeping your options open". However, you will regret them all once products are in the hands of customers or end users. Modbus is intended to be a very simple protocol. Keep it simple and you will have far fewer problems in the long run.
 
S

Sebastian Seidel

Thank you very much for the swift and in-depth response, without which I would most certainly have ended up with an unfortunate solution.

>You said that you're not sure about "the max data length of many variables". They're 32 bit, aren't they? That's what you just said, anyway. If you change that to something else (16 bit, 64 bit) in a new version, your sensor isn't going to be compatible with the previous version anyway. <

Good point.

>Keep things as simple as possible or you (or your customers) will have no end of problems. Pick a register map, and stick with it. If it has to change in future in a new version, then it's a new model with a new register map. <

Right, thus I will do it.

>As for reading multiple variables at once, your software should be able to figure out if a register is being read whether it is a single or multiple read. <

However, the difficult part might be to arrange the bytes of all requested registers properly for output, especially since multi-register numerical values and single-byte strings are used.

>> All four tables overlap, R and RW data mixes.
---- snip ---
>easiest to separate RO and RW (input and holding registers). The user then only writes to holding registers, and only reads from input registers (the same with discrete inputs and coils). <

Each sensor input to the ATmega will have at about the same variables / constants (measurement value, resolution, range, ...). So, I decided to use an register offset of 0x100 for each such channel and put the RO data into the beginning of each 256 byte block, then zeros, then RW data.

>> Functions 0x03 and 0x10 used for data access.
>
>This is normal. You will also need function 4 for input registers if you use them. <

Does 0x03 read holding registers, only, and 0x04 read input registers, only? Even if the tables map to the same memory? Should exception 0x02 be returned if there is a read request of 0x03 to input or of 0x04 to holding? In that case, reading larger blocks of registers with a single operation could also be difficult if RO and RW data mixes too much.

>I think this is normal. However, would you have 1 or 2 characters per register? Both ways have their uses. Also, are you specifying ASCII only? This is normal for industrial applications. <

I tend toward using ASCII-only English-language single-byte strings (readily readable everywhere with low memory footprint).
Wide character strings might map more naturally to the 16 bit registers, though.

>> Data saved in native (little endian) format and shuffled before sending.

>Whatever is easiest for you. Endian-ness is always a problem though. No matter what you do, someone won't be happy. <
----- snip -----
>You can come up with 100 complicated ways of doing things all based on "keeping your options open". However, you will regret them all once products are in the hands of customers or end users. Modbus is intended to be a very simple protocol. Keep it simple and you will have far fewer problems in the long run. <

Agreed. In fact, with our current setup the register update will be fast enough to update all the registers on demand. If that changes in the future, I might think about partial or timed updates, too.

Still another question that came up is identification. The master applications will have to make sure that it's really our device they are talking with.

What's preferable: using function 17 (Report Slave ID) or function 43/14 (Read Device Identification)? Maybe the former is just simpler, the latter more clearly defined? What is more often used? And what is the meaning of 'Run Indicator Status' of function 17? A Boolean that tells if the device is in normal operation or sleep mode?

Again, many thanks for your patience and detailed answers.

If you're interested - I'll append how I now think that I will implement this to the end of the message.

Kind regards,
Sebastian Seidel
(eMail: seidel [at] seika.de)

----------------

Preliminary idea about simple implementation of modbus protocol (and one more question):

As to pre-send-processing of registers in the little-endian environment, three cases are distinguished:
- register: send 2nd byte first, 1st byte second
- string: send 1st byte first, 2nd byte second
- multi-register-value: revert all bytes before sending

(Btw., management of 8 bit data seems to be a problem. I made it all 16 bit in the register map, because otherwise one 16 bit unit in memory would correspond to two register numbers. How could it properly be dealt with?)

There must be a table that tells:
- Where values start and how long they are
- Which pre-send operation shall be performed.

Since that table exists anyway, it will also be extended to contain the flags for read-enable (input) and write-enable (holding).

Two arrays are mapped into memory:
(1) Register array with version-specific register map.
(2) Corresponding flagbyte-array with the same number of elements as there are registers.

Each flagbyte describes exactly one register (so that I can refer to them both by the same register number) and contains this:
- Value-start flag.
- Value-length (2 ^ n registers, 3 bits).
- Pre-send processing (2 bits).
- Read flag.
- Write flag.

When a read request arrives, the program will:
(1) Update all registers, like you suggested.
(2) Read each value (one or several registers), successively.
(3) For each value, shuffle the bytes using the information of the flagbyte and append the result to the output puffer.
(4) Package the output and send it over serial.
 
I will start off by saying that what would probably help a lot is to sit down and imagine yourself as the user or customer (whatever you want to call them). Then when you are thinking like a customer, design one of your devices into a few typical applications. Base the use of your device in your sample customer application only the information you have written down. You will probably get a completely different perspective on the whole situation.

When I am working on something, I find it very handy to write the user manual as I go along. If I find something to be hard to explain, then it will probably also be hard to understand. Often that means it needs a new approach to make it easier to use.

> So, I decided to use an register offset of 0x100 for each such channel and put the RO data into the beginning of each 256 byte block, then zeros, then RW data. <

I'm not sure just how your users are supposed to be using this, but most people would usually like to be able to read all the essential data in one operation rather than needing to sequence a series of read operations for each poll. You can read up to 125 16 bit registers with one Modbus command. If you can arrange the data so that they can do that, it makes the system much easier to use.

What some designers do is they provide two copies of the essential data at different addresses. They provide one copy with all the possible data, and they provide a second copy at another address as a summary with just the essential data grouped together.

I'm not familiar with your system, but in most applications the users have to configure or program their PLC ( DCS, HMI, or whatever) to make use of each data item they read. However, time is money, and most people won't spend the time (money) trying to make use of information they don't really need. So, you have to ask yourself just what it is that people will actually do with the information you are giving them. If it is a case that 90% of the people won't use all the extra data, but 10% will, then that is the reason for providing both basic and detailed data at different addresses.

Another thing to consider is bandwidth. If you have a serial (RS-485, RS-232) network, then network speed can be a problem in many applications. In this case, users won't want to read any more information than they absolutely have to because it slows down polling of the entire system (including any other unrelated nodes on the same network).

>Does 0x03 read holding registers, only, and 0x04 read input registers, only? <

Yes.

>Even if the tables map to the same memory? <

You can have the two tables occupy the same physical memory, or have different blocks of registers mixed together. There are examples of that in the Modbus specification. The idea though is that any memory locations that you want to have be "read-only" are input registers, and any memory locations that you want to have be read-write are holding registers.

Where designers get into problems is where they don't use input registers for data that needs to be read-only, and then tie themselves into knots trying to figure out what they should do if the user goes ahead and writes to that memory. Modbus has a simple solution to that. There is no command for writing to input registers, so the problem never arises.

>Should exception 0x02 be returned if there is a read request of 0x03 to input or of 0x04 to holding? <

Exception 2 means you either tried to read an address that doesn't exist. That can happen one of two ways, Either the starting address doesn't exist, or else one or more of the block addresses that you tried to read doesn't exist (e.g. you started at a valid address, but used a quantity that caused it to read beyond the end of memory). If a user does not do either of those, then the user should not see exception 2.

>In that case, reading larger blocks of registers with a single operation could also be difficult if RO and RW data mixes too much.<

You can have a copy of the same data in an input register and in a holding register. They don't even have to be at the same numerical address. For example, the user could write to holding register 100, and you could copy that same number to input register 10.

>Still another question that came up is identification. The master applications will have to make sure that it's really our device they are talking with.<

I'll take your word for it in this application, but in most application people don't worry about that. They just assume that if they are talking to the right network address, then they are talking to the right device.

>What's preferable: using function 17 (Report Slave ID) or function 43/14 (Read Device Identification)? Maybe the former is just simpler, the latter more clearly defined? What is more often used? And what is the meaning of 'Run Indicator Status' of function 17? A Boolean that tells if the device is in normal operation or sleep mode?<

You need to do a bit of market research to find out what it is your customers will be hooking your device up to. Once you know that, you will be able to find out functions are supported by the master. You can usually assume that a Modbus device will support the basic functions of 1,2,3,4,5,6, 15, and 16, (although not always all of those) but beyond that function support varies a lot.

The function 43 stuff is related to tunnelling other protocols over Modbus. E.g. Sending CANopen data over Modbus to a CANopen bridge. If you don't know what that is, then you probably don't need it.

The 'Run Indicator Status' for function 17 is for if your device has an "off-line" mode where it is able to respond to communications requests, but can't perform its normal job because of a mode selection. For example, a PLC can have "run" and "stop" modes. If you put the PLC in "stop" mode, it can talk to the network, but it can't run it's PLC program. This provides a standard means to lets your device report its status. Most field devices only have one mode, so it is really only useful for special cases.

>Btw., management of 8 bit data seems to be a problem. I made it all 16 bit in the register map,<

That sounds like the best solution. If you packed two values into one register, the user would have to do a bunch of mask and shift operations to work with the data, which would make them rather unhappy over all the extra work they had to do.
 
S

Sebastian Seidel

Now, that took me some time! But I have defined the registers (and am very glad that's finally done) and just wanted to thank you for providing that excellent information!

Like you suggested, I thought about what the user might want and grouped the configuration data into RO-only and RW-only blocks (mostly 4 sensor channels x 16 registers each) each of which serves a special purpose and can easily be read / written in one operation.

All measurement values reside in successive registers now, too, so they can be polled in one go. So, I now do have some hope I'll - with some further experimentation (have yet to implement the modbus, timing and all) - come up with a useable interface.

Identification might still be something to worry about later on (we have a master application to configure our devices - until now, that was done with broadcast using a home-grown protocol, but only one device can be on the bus that way).

CAN is no topic for now. Maybe we'll look into that later. Also, we might later check which converters to other networks exist (Profibus, for example).

I really like the register layout now and am very glad we decided to use modbus protocol.

Again, many thanks and kind regards,
Sebastian Seidel
 
It's always good to hear feedback. The feedback helps other people who are following the discussion as well, as they see how things turned out in practise. Good luck with the implementation.
 
Top