Getting wrong values when I stitch 2 shorts back into an unsigned long

▲
0
▼

I am doing BLE communications with an Arduino Board and an FPGA. I have this requirement which restraints me from changing the packet structure (the packet structure is basically short data types). Thus, to send a timestamp (form millis()) over, I have to split an unsigned long into 2 shorts on the Arduino side and stitch it back up on the FPGA side (python).

This is the implementation which I have:

  // Arduino code in c++
  unsigned long t = millis();
  // bitmask to get bits 1-16
  short LSB = (short) (t & 0x0000FFFF); 
  // bitshift to get bits 17-32
  short MSB = (short) (t >> 16);

  // I then send the packet with MSB and LSB values


  # FPGA python code to stitch it back up (I receive the packet and extract the MSB and LSB)
  MSB = data[3]
  LSB = data[4]
  data = MSB << 16 | LSB

Now the issue is that my output for data on the FPGA side is sometimes negative, which tells me that I must have missed something somewhere as timestamps are not negative. Does any one know why ?

When I transfer other data in the packet (i.e. other short values and not the timestamp), I am able to receive them as expected, so the problem most probably lies in the conversion that I did and not the sending/receiving of data.

python
c++
arduino
asked on Stack Overflow Feb 20, 2020 by Rajdeep • edited Feb 20, 2020 by Rajdeep

3 Answers

▲
2
▼

short defaults to signed, and in case of a negative number >> will keep the sign by shifting in one bits in from the left. See e.g. Microsoft.

From my earlier comment: In Python avoid attempting that by yourself (by the way short from C perspective has no idea concerning its size, you always have to look into the compiler manual or limits.h) and use the struct module instead.

answered on Stack Overflow Feb 20, 2020 by guidot • edited Feb 20, 2020 by guidot
▲
0
▼

you probably need/want to first convert the long to network byte order using hotnl

answered on Stack Overflow Feb 20, 2020 by nivpeled
▲
0
▼

As guidot reminded “short” is signed and as data are transferred to Python the code has an issue:

For t=0x00018000 most significant short MSB = 1, least significant short LSB = -32768 (0x8000 in C++ and -0x8000 in Python) and Python code expression
time = MSB << 16 | LSB
returns time = -32768 (see the start of Python code below).
So, we have incorrect sign and we are loosing MSB (any value, not only 1 in our example).
MSB is lost because in the expression above LSB is extended with sign bit 1 to the left 16 bits, then new 16 “1” bits override with “|” operator whatever MSB we have and then all new 16 “1” bits are discarded and the expression returns LSB.

Straightforward fix (1.1 Fix) would be fixing MSB, LSB to unsigned short. This could be enough without any changes in Python code.

To exclude bit operations we could use “union” as per 1.2 Fix.

Without access to C++ code we could fix in Python by converting signed LSB, MSB (2.1 Fix) or use “Union” (similar to C++ “union”, 2.2 Fix).

C++

#include <iostream>
using namespace std;

int main () {
  unsigned long t = 0x00018000;
  short LSB = (short)(t & 0x0000FFFF);
  short MSB = (short)(t >> 16);
  cout << hex << "t = " << t << endl;
  cout << dec << "LSB = " << LSB << " MSB = " << MSB << endl;

  // 1.1 Fix Use unsigned short instead of short
  unsigned short fixedLSB = (unsigned short)(t & 0x0000FFFF);
  unsigned short fixedMSB = (unsigned short)(t >> 16);
  cout << "fixedLSB = " << fixedLSB << " fixedMSB = " << fixedMSB << endl;

  // 1.2 Fix Use union
  union {
    unsigned long t2;
    unsigned short unsignedShortArray[2];
  };

  t2 = 0x00018000;

  fixedLSB = unsignedShortArray [0];
  fixedMSB = unsignedShortArray [1];

  cout << "fixedLSB = " << fixedLSB << " fixedMSB = " << fixedMSB << endl;
}

Output

t = 18000
LSB = -32768 MSB = 1
fixedLSB = 32768 fixedMSB = 1
fixedLSB = 32768 fixedMSB = 1

Python

DATA=[0, 0, 0, 1, -32768]

MSB=DATA[3]
LSB=DATA[4]
data = MSB << 16 | LSB
print (f"MSB = {MSB} ({hex(MSB)})")
print (f"LSB = {LSB} ({hex(LSB)})")
print (f"data = {data} ({hex(data)})")

time = MSB << 16 | LSB
print (f"time = {time} ({hex(time)})")


# 2.1 Fix
def twosComplement (short):
  if short >= 0: return short
  return 0x10000 + short
fixedTime = twosComplement(MSB) << 16 | twosComplement(LSB)

# 2.2 Fix
import ctypes
class UnsignedIntUnion(ctypes.Union):
    _fields_ = [('unsignedInt', ctypes.c_uint),
                ('ushortArray', ctypes.c_ushort * 2),
                ('shortArray', ctypes.c_short * 2)]

unsignedIntUnion = UnsignedIntUnion(shortArray = (LSB, MSB))

print ("unsignedIntUnion")
print ("unsignedInt = ", hex(unsignedIntUnion.unsignedInt))
print ("ushortArray[1] = ", hex(unsignedIntUnion.ushortArray[1]))
print ("ushortArray[0] = ", hex(unsignedIntUnion.ushortArray[0]))

print ("shortArray[1] = ", hex(unsignedIntUnion.shortArray[1]))
print ("shortArray[0] = ", hex(unsignedIntUnion.shortArray[0]))

unsignedIntUnion.unsignedInt=twosComplement(unsignedIntUnion.shortArray[1]) << 16 | twosComplement(unsignedIntUnion.shortArray[0])

def toUInt(msShort: int, lsShort: int):
  return UnsignedIntUnion(ushortArray = (lsShort, msShort)).unsignedInt

fixedTime = toUInt(MSB, LSB)
print ("fixedTime = ", hex(fixedTime))

print()

Output

MSB = 1 (0x1)
LSB = -32768 (-0x8000)
data = -32768 (-0x8000)
time = -32768 (-0x8000)
unsignedIntUnion
unsignedInt =  0x18000
ushortArray[1] =  0x1
ushortArray[0] =  0x8000
shortArray[1] =  0x1
shortArray[0] =  -0x8000
fixedTime =  0x18000
answered on Stack Overflow Mar 9, 2020 by Val • edited Mar 10, 2020 by Val

User contributions licensed under CC BY-SA 3.0