Using gm30 on a UV13-Pro

[mabnz]

Hi,

I've got a Baofeng UV13-Pro, and I've managed to get the Python script running using test_read.sh, but when I attempt:

./gm30 read -c config.bin

I get:

(env) Mikes-Air:gm30 mab$ gm30 mr -f full_memory.bin
Using serial device: /dev/cu.usbserial-1110
Traceback (most recent call last):
  File "/Users/mab/Downloads/gm30/env/bin/gm30", line 33, in <module>
    sys.exit(load_entry_point('radioddity-gm30', 'console_scripts', 'gm30')())
  File "/Users/mab/Downloads/gm30/radioddity_gm30/cli.py", line 154, in main
    read_memory(device_path=device_path, data_file=args.data_file)
  File "/Users/mab/Downloads/gm30/radioddity_gm30/cli.py", line 42, in read_memory
    protocol.unknown_init()
  File "/Users/mab/Downloads/gm30/radioddity_gm30/protocol.py", line 230, in unknown_init
    assert fw_variant == 'P13GMRS'

I assume this means it's seeing a different variant to that which is expected, is this correct?

Is there anything I can provide to help with diagnosing where the problem lies so that I can use the tool?

[CtrlC-Root]

I don't have a UV13-Pro but yes I'd say it's a fair bet the firmware reports a different variant than the Radioddity GM30. I added the assert statements into the code so I would know right away if the radio I was testing with ever responded differently than it had in the past. If you just want to see if the rest of the code will work you can remove or comment out that assert (radioddity_gm30/protocol.py line 230). The variant name is used later on in the protocol to enter programming mode so I would probably do a sanity check first by replacing the assert with this line instead:

print("FW Variant:", fw_variant); import sys; sys.exit(1)

Then run the script and see what the variant value is in the console output.

[mabnz]

Looks like it's a UV15999. Should I replace all instances of P13GMRS with that string?

Tried this, and got a little further:

Entering programming mode
00 00 00 6d c9 00 08 01
00 00 00 6d c9 00 08 01
00 00 00 6d c9 00 08 01
00 00 ff 1f 03 00
Traceback (most recent call last):
  File "/Users/mab/Downloads/gm30/env/bin/gm30", line 33, in <module>
    sys.exit(load_entry_point('radioddity-gm30', 'console_scripts', 'gm30')())
  File "/Users/mab/Downloads/gm30/radioddity_gm30/cli.py", line 160, in main
    read_config(device_path=device_path, config_file=args.config_file)
  File "/Users/mab/Downloads/gm30/radioddity_gm30/cli.py", line 80, in read_config
    radio_config.read_radio(device_path)
  File "/Users/mab/Downloads/gm30/radioddity_gm30/radio_config.py", line 145, in read_radio
    protocol.unknown_init()
  File "/Users/mab/Downloads/gm30/radioddity_gm30/protocol.py", line 287, in unknown_init
    assert response == bytes([0x00, 0x00, 0xFF, 0xFF, 0x00, 0x00])
AssertionError

[CtrlC-Root]

There should only be the one instance of assert checking for P13GMRS. Looks like the radio responded with 00 00 ff 1f 03 00 instead of the value that next assert is checking for. So you can update the next one to match (this is what I would do, so if the radio responds differently in the future, you'll know) or drop it.

[CtrlC-Root]

You can pretty much remove all the assert statements from the code if you want. I've never been able to break or otherwise harm a radio by using the read commands implemented here (although it's possible given the firmware it's talking to is different!) so if you're not interested in confirming the radio responds the same way each time they don't really matter. You could disable them with an environment variable:

export PYTHONOPTIMIZE="True"

See this for more information on disabling asserts:

• https://docs.python.org/3/using/cmdline.html#cmdoption-O
• https://docs.python.org/3/using/cmdline.html#envvar-PYTHONOPTIMIZE

[CtrlC-Root]

If you get it to run and you share the output files captured by the test_read.sh script and I could probably modify the code to handle both radios.

[mabnz]

I've disabled asserts using PYTHONOPTIMIZE as you suggest, and it's got further:

Detecting memory segments
Reading unknown memory from segment 0x0 @ 0x1000
Reading frequency memory from segment 0xd @ 0xe000
Reading channel memory from segment 0x2 @ 0x3000
Reading general memory from segment 0xb @ 0xc000
Traceback (most recent call last):
  File "/Users/mab/Downloads/gm30/env/bin/gm30", line 33, in <module>
    sys.exit(load_entry_point('radioddity-gm30', 'console_scripts', 'gm30')())
  File "/Users/mab/Downloads/gm30/radioddity_gm30/cli.py", line 160, in main
    read_config(device_path=device_path, config_file=args.config_file)
  File "/Users/mab/Downloads/gm30/radioddity_gm30/cli.py", line 80, in read_config
    radio_config.read_radio(device_path)
  File "/Users/mab/Downloads/gm30/radioddity_gm30/radio_config.py", line 164, in read_radio
    memory.import_data(data)
  File "/Users/mab/Downloads/gm30/env/lib/python3.10/site-packages/mrcrowbar/blocks.py", line 280, in import_data
    self._field_data[name] = klass._fields[name].get_from_buffer(
  File "/Users/mab/Downloads/gm30/env/lib/python3.10/site-packages/mrcrowbar/fields.py", line 965, in get_from_buffer
    result = super().get_from_buffer( buffer, parent=parent )
  File "/Users/mab/Downloads/gm30/env/lib/python3.10/site-packages/mrcrowbar/fields.py", line 293, in get_from_buffer
    element, end_offset = self.get_element_from_buffer( pointer, buffer, parent, index=len( result ) if is_array else None )
  File "/Users/mab/Downloads/gm30/env/lib/python3.10/site-packages/mrcrowbar/fields.py", line 1016, in get_element_from_buffer
    data = data.decode( encoding )
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

[CtrlC-Root]

Ok so the read command tries to locate the memory segments currently in use by the radio (out of the 15 it has available internally to store data), parse them, and then write them to a local file in the same format as the OEM CPS software. It looks like the format for the general memory segment is different for your radio in an incompatible way. I'd guess it's one of these two lines that are causing the issue:

• https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/memory/general.py#L102
• https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/memory/general.py#L108

Well this is the fun annoying part. You can:

1. Start with the radio in a known state and run the mr command (like the test script does) to read all the radio memory.
2. Change a single setting (on the radio or using the CPS software).
3. Run the mr command again and compare the output against the previous run.
4. Figure out what part changed and how the data is encoded.
5. Update the memory configuration to reflect this.
6. Repeat until you have all the settings figured out.

You could speed up this process if you capture all the data between the radio and the OEM CPS software during a read or write cycle. Either by capturing the USB traffic to the programming cable or assuming it's just a USB-to-serial adapter by capturing the serial data directly.

In the meantime if you want to just skip the general memory for now you can comment out this line and try again: https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/radio_config.py#L56. The only one left is the DTMF settings (i.e. PHONE data).

[CtrlC-Root]

Also, I'm sure you're aware of this if you discovered this repository from the CHIRP issue tracker, but this project is very much a work in progress. I can reliably read GM30 radios and I've figured out how most of the settings are encoded in it's memory. Writing to them currently breaks them in a way that I can't recover from (you'll notice the write commands are disabled :D).

From what I know of how the OEM market works for these radios Radioddity probably got a copy of the firmware from BaoFeng and made some random changes to "brand" it and/or improve various things. So it's probably pretty similar to the UV13 Pro but it could also be way different. No way to know until someone checks.

[drzraf]

I got this devices too: UV15999:
and this is my "intro":

00 00 00 6d c9 00 08 01
00 00 00 6d c9 00 08 01
00 00 00 6d c9 00 08 01
00 00 ff 1f 03 00

But test_read (only) complains about Reading unknown memory from segment 0x0 @ 0x1000

Attached is the result of mr on a genuine device:
mem.bin.txt

Hopefully it could help you figure out the difference between this memory map and other (P)15UV devices assuming that GM30 is really about BF UV13 / P15UV... but is that assumption correct?

NB: I couldn't get "P15UV-UV13 1.01" to work under wine (the mapping COM1 to /dev/ttyUSB0 doesn't seem to work).
NB: Not sure what a ch34x-specific Wireshark dissector would bring over native usbmon
NB: https://github.com/juliagoda/CH341SER + echo module ch34x +p |sudo tee /sys/kernel/debug/dynamic_debug/control get the logs in syslog.

[isomer]

I have a Baofeng UV-17, and started reverse engineering the protocol but found this repository before I sunk too much time into it (found this by searching for "PSEARCH"). I've captured the messages from the P15UV-UV13 1.01 to the radio (but didn't get to capture the other direction) by using kvm to run windows.

The radio responds to PSEARCH with "UV15999" which I assume means "newer than UV15" and you're supposed to use something else to determine the exact model number.

After commenting out assert fw_variant == 'P13GMRS' (but leaving all the other asserts in place), I too ran into the problem with the boot message having a \xFF and failing to decode to ascii. So I changed the boot message and dumped the memory again.

Then the bootmessage is at address 0xC010 and 0xC020, so I changed the "CONFIG_FILE_ADDRESS" for GENERAL_DATA to 0xC000 and it got a lot further, crashing trying to decode the frequency table.

Hrm, now when I dump the memory it's at a different offset (0x4010), and now when I look it's at offset 0x4010 and 0xE010. I powered it off and on again, and now it's at 0x2010. Data at 0x0000-0x1FFF seems static, but the rest seems to change based on the devices whim. Is there something about how this maps memory that the state varies based on what's going on with the device?

Do you have any advice before I spend some more time digging into this?

[CtrlC-Root]

I don't know for sure so this is all a guess based on what I've seen in the GM-30s (which may not be what other variants do).

The configuration data is stored in a flash IC which is divided into equally sized segments (https://github.com/CtrlC-Root/gm30/blob/master/notes/addresses.txt#L1). There are several types of data (https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/radio_config.py#L14) and the last byte of each segment is used to identify what kind of data is stored there (https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/radio_config.py#L101).

When you read data from the radio you need to read the last byte of each segment to see what kind of data is stored there (https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/radio_config.py#L95) and then parse it appropriately. The OEM CPS software does this as well (which you can see in USB/serial traces). When the CPS software saves the configuration data to a file it stores it at consistent offsets though (https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/radio_config.py#L43) for convenience. The CONFIG_FILE_ADDRESS dictionary stores the offset for each type of data in CPS files (and not in the radio's memory).

When you write data to the radio (at least from what the CPS software seems to do) you overwrite it in the segments that are currently storing it, the firmware then does some kind of integrity check, and then the radio reboots. If that integrity check fails the firmware will pick new segments to use (for all data types), initialize them, and update the first bytes of the previous and new segments to mark them appropriately. I suspect this is why I am breaking radios when I try to write data to them; the integrity check is somehow failing, no doubt because I am missing some part of the protocol or data, so the firmware decides to switch to new segments and the data I wrote to the existing ones is ignored. Unfortunately it seems like the way the firmware is initializing the new segments is broken and at that point when the radio reboots the firmware breaks in all kinds of ways.

If you go through the firmware upgrade process with the CPS and reinstall the same firmware version it will get further and the data will be valid enough that the radio boots but it's still broken in ways the CPS can't seem to fix (i.e. random icons are enabled on the screen in modes that don't make sense, can't select channels from memory, VFO mode uses wild frequency ranges). I suspect one of the segments (the UNKNOWN_DATA one) stores some factory calibration / configuration data which neither the firmware upgrade nor CPS write procedures fixes correctly.

I was basically working through the radio memory to figure out what every byte represents with the hope that I could start writing the segments I had fully figured out. Unfortunately this doesn't seem to work and I have now broken three of these in ways I can't fix. So I think the next step is to start reverse engineering the firmware, figure out what the post-write integrity check is doing, and go from there. I've kind of put this on hold until I have at least a few contiguous days to look into that because I don't want to stop part-way.

So the tl;dr is: I am sure with enough patience you can (1) change something in the CPS (2) write to the radio with the CPS (3) dump the memory segments (either all of them or only the active ones) and (4) reverse engineer how most of it is stored. Unfortunately I think in order to change any of it we need to do it in a way that's valid and I'm not sure how the firmware checks for that.

EDIT: It's the last byte in each segment that identifies it not the first.

[isomer]

Ah that's really interesting. My radio seems to have \xFF in annoying, uninitialised areas which cause the program be upset. Setting the welcome message means it no longer complains about being unable to decode it.

Reading the frequency table complains because channel b is set to 0xFF. I extended the validator to allow range(0,256) and now I can read the contents of my radio, and everything seems to decode correctly.

So it seems that what you have does generally decode all these different radios, it's just a little strict on validation (which is reasonable).

I can fully appreciate having time to work on writing (and it's a bit expensive if you keep bricking radios). When I have a little time I might take a bit of a look and see if I can help out figuring out what's going on, but you seem to have gotten amazingly far already. If/when you get some spare time, let us know and we'll see if we can help out in one way or another.

[CtrlC-Root]

Ah that's really interesting. My radio seems to have \xFF in annoying, uninitialised areas which cause the program be upset. Setting the welcome message means it no longer complains about being unable to decode it.

I would not be surprised at all to see that each firmware (for each variant) is subtly different. On the GM-30s the welcome message is set by default at least. I would also not be surprised if the hardware is subtly different as well with no easy way to tell the revision from the outside. Looking at the Baofeng UV-17 I can see at least the LCD is different (color versus B/W on the GM-30) and the battery pins are in a different orientation (top versus bottom on the GM-30).

Reading the frequency table complains because channel b is set to 0xFF. I extended the validator to allow range(0,256) and now I can read the contents of my radio, and everything seems to decode correctly.

So it seems that what you have does generally decode all these different radios, it's just a little strict on validation (which is reasonable).

Well that's good to know. Yes the strictness was intended to support reverse engineering so if the firmware changed anything on subsequent reads/writes I would notice it right away and could investigate it. Again I'd expect the data layout and values to differ based on hardware/firmware revision.

I can fully appreciate having time to work on writing (and it's a bit expensive if you keep bricking radios). When I have a little time I might take a bit of a look and see if I can help out figuring out what's going on, but you seem to have gotten amazingly far already. If/when you get some spare time, let us know and we'll see if we can help out in one way or another.

Ok, well on a whim I pulled out the GM-30 I disassembled and identified all the ICs. It's using an ARM Cortex-M23 CPU which has a bootloader burned into ROM. The SWD debugging interface is also exposed via pads on the side of the PCB. Assuming there's nothing crazy happening with the PCB layout or the OTP settings in the CPU I expect it should be trivial to dump the entire firmware and get custom code running on it. I strongly suspect the firmware upgrade procedure is really just a thin layer (via USB to serial adapter in the PC cable) over the bootloader upgrade procedure the MCU datasheet mentions. I ordered some adapters so I can hook up a JLink debugger to the SWD interface which should arrive sometime this week. In the meantime I'm going to look into the firmware available from the OEM and see if I can figure out what's in it now that I know at least what instruction set and memory layout should be in use.

What would be useful is to have a chart of all the variants somewhere so we can start documenting the differences (i.e. the variant the firmware responds with, any changes to the data layout or the protocol, etc). Not sure if the CHIRP project might have this somewhere or not. It's hard to keep track when the information is spread out across several issues.

[drzraf]

The above linked issue to the radio_tool repository is about an effort to handle firmware r/w for this class of devices.

[drzraf]

When dumping using the official program, the result is slightly different.
There is a 8k block header which start by
0a0d 0000 2021 0000 0020 0000 0000 0000 .... !... ......
repeated 3 times (and I believe is program-specific).

But there is also a difference after 4k: where this tool dumps the 0100 0000 0000 ... WELCOME (welcome message), the windows provides a different organization:

First comes this particular header:

00001000: e703 0a00 0000 0000 0000 0000 0000 0000  ................
00001010: 0025 5043 0000 0000 00ff ffff ff06 1100  .%PC............
00001020: 0025 5015 0000 0000 00ff ffff ff06 1100  .%P.............
00001030: 0025 2145 0025 2145 0093 0693 0606 1100  .%!E.%!E........
00001040: 0025 3245 0025 3245 0015 0915 0906 2100  .%2E.%2E......!.
00001050: 0025 4345 0025 4345 0065 1365 1306 3100  .%CE.%CE.e.e..1.
00001060: 0025 5445 0025 5445 0014 1514 1506 4100  .%TE.%TE......A.
00001070: 0025 6545 0025 6545 0028 1928 1906 5100  .%eE.%eE.(.(..Q.
00001080: 0025 7645 0025 7645 0018 2418 2406 6100  .%vE.%vE..$.$.a.
00001090: 0025 8745 0025 8745 0025 8025 8006 7100  .%.E.%.E.%.%..q.
000010a0: 0025 9845 0025 9845 0034 8134 8106 8100  .%.E.%.E.4.4....
000010b0: 0025 1946 0025 1946 0074 8274 8206 9100  .%.F.%.F.t.t....
000010c0: 0025 2246 0025 2246 0046 8346 8306 a100  .%"F.%"F.F.F....
000010d0: 0025 3346 0025 3346 0003 8503 8506 b100  .%3F.%3F........
000010e0: 0025 4446 0025 4446 0073 c073 c006 c100  .%DF.%DF.s.s....
000010f0: 0025 5546 0025 5546 0003 c703 c706 d100  .%UF.%UF........
00001100: 0025 2240 0025 2240 00ff ffff ff06 e100  .%"@.%"@........
00001110: 0025 7443 0025 7443 00ff ffff ff04 f100  .%tC.%tC........
00001120: 0075 9947 0075 9947 00ff ffff ff06 1100  .u.G.u.G........
00001130: 0050 8513 0050 8513 00ff ffff ff06 1100  .P...P..........
00001140: 0050 7615 0050 7615 00ff ffff ff06 1100  .Pv..Pv.........
00001150: 0050 2717 0050 2717 00ff ffff ff06 1100  .P'..P'.........
00001160: 0000 8543 0000 8543 00ff ffff ff04 1100  ...C...C........
00001170: 0000 5715 0000 5715 00ff ffff ff06 1100  ..W...W.........
00001180: ffff ffff ffff ffff 00ff ffff ff06 1100  ................

Followed by 3x 4k-sized blocks (the ones normally at 0x4000) like ffff ffff ffff ffff 00ff ffff ff06 1100

Then followed by the ffff ffff ffff ffff ffff 00ff ffff ffff-kind of blocks.

Then only the WELCOME stuff at 0x8000.

I don't know if the exe interprets the dumps and reorganize them before storing them to file, but if it's not, it could be interesting to consider this alternative dump format.

[CtrlC-Root]

When dumping using the official program, the result is slightly different. There is a 8k block header which start by 0a0d 0000 2021 0000 0020 0000 0000 0000 .... !... ...... repeated 3 times (and I believe is program-specific).

But there is also a difference after 4k: where this tool dumps the 0100 0000 0000 ... WELCOME (welcome message), the windows provides a different organization:

First comes this particular header:

00001000: e703 0a00 0000 0000 0000 0000 0000 0000  ................
00001010: 0025 5043 0000 0000 00ff ffff ff06 1100  .%PC............
00001020: 0025 5015 0000 0000 00ff ffff ff06 1100  .%P.............
00001030: 0025 2145 0025 2145 0093 0693 0606 1100  .%!E.%!E........
00001040: 0025 3245 0025 3245 0015 0915 0906 2100  .%2E.%2E......!.
00001050: 0025 4345 0025 4345 0065 1365 1306 3100  .%CE.%CE.e.e..1.
00001060: 0025 5445 0025 5445 0014 1514 1506 4100  .%TE.%TE......A.
00001070: 0025 6545 0025 6545 0028 1928 1906 5100  .%eE.%eE.(.(..Q.
00001080: 0025 7645 0025 7645 0018 2418 2406 6100  .%vE.%vE..$.$.a.
00001090: 0025 8745 0025 8745 0025 8025 8006 7100  .%.E.%.E.%.%..q.
000010a0: 0025 9845 0025 9845 0034 8134 8106 8100  .%.E.%.E.4.4....
000010b0: 0025 1946 0025 1946 0074 8274 8206 9100  .%.F.%.F.t.t....
000010c0: 0025 2246 0025 2246 0046 8346 8306 a100  .%"F.%"F.F.F....
000010d0: 0025 3346 0025 3346 0003 8503 8506 b100  .%3F.%3F........
000010e0: 0025 4446 0025 4446 0073 c073 c006 c100  .%DF.%DF.s.s....
000010f0: 0025 5546 0025 5546 0003 c703 c706 d100  .%UF.%UF........
00001100: 0025 2240 0025 2240 00ff ffff ff06 e100  .%"@.%"@........
00001110: 0025 7443 0025 7443 00ff ffff ff04 f100  .%tC.%tC........
00001120: 0075 9947 0075 9947 00ff ffff ff06 1100  .u.G.u.G........
00001130: 0050 8513 0050 8513 00ff ffff ff06 1100  .P...P..........
00001140: 0050 7615 0050 7615 00ff ffff ff06 1100  .Pv..Pv.........
00001150: 0050 2717 0050 2717 00ff ffff ff06 1100  .P'..P'.........
00001160: 0000 8543 0000 8543 00ff ffff ff04 1100  ...C...C........
00001170: 0000 5715 0000 5715 00ff ffff ff06 1100  ..W...W.........
00001180: ffff ffff ffff ffff 00ff ffff ff06 1100  ................

Followed by 3x 4k-sized blocks (the ones normally at 0x4000) like ffff ffff ffff ffff 00ff ffff ff06 1100

Then followed by the ffff ffff ffff ffff ffff 00ff ffff ffff-kind of blocks.

Then only the WELCOME stuff at 0x8000.

I don't know if the exe interprets the dumps and reorganize them before storing them to file, but if it's not, it could be interesting to consider this alternative dump format.

The CPS uses a fixed layout for the data while the radios move the sections around in memory every time they are programmed. The layout for the CPS is defined at https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/radio_config.py#L43 and the layout in the radio's memory is detected at https://github.com/CtrlC-Root/gm30/blob/master/radioddity_gm30/radio_config.py#L95.

[CtrlC-Root]

The above linked issue to the radio_tool repository is about an effort to handle firmware r/w for this class of devices.

I tried that tool and could not get it to work. I'll look at it again if I get stuck. For the moment I'd like to try and dump the firmware using the debug port inside the radio in order to avoid the obfuscation between the CPS and bootloader.

[CtrlC-Root]

I'm going to make some significant changes to this repository in the next few days as I start working on this again. I'm going to close this issue in favor of separate issues for each of the variants.