How to create hex-dump .bin file from powershell or cmd?

The main purpose is to write hex-coded string or just hex data into a main bin file, whether or not, hex-dumped.

Using windows powershell or cmd, we're trying to create a bin file in order to concatenate it with the main one using opensource srecord execution scripts, like srec_cat.exe.

We've managed to hex-dump some stuff like a string or so in powershell:

"hello_world" would be:

PS C:\Users\user> echo hello_world | format-hex > output.bin

The concent of that "bin" file is:

           Ruta:

           00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000   68 65 6C 6C 6F 5F 77 6F 72 6C 64                 hello_world

However, this output above, is string formatted, and what is needed is to dump that as a bin type file. This is because srecord's srec_cat tool needs it to be that way in order to write (concat) that data at some memory region of the main bin file that will be flashed into devices.

The output needed would be something like:

00000000: 68 65 6C 6C 6F 5F 77 6F 72 6C 64 00 00 00 00 00  #hello_world

Again, as binary type file.

Thanks a lot for your time and attention!

---

First edit:

So, we're using Atmel Studio to make a program that will work with SAMD21 devices. Along with SAM ICE via JTAG for flashing and debugging. The result of the compilation, and hence the final executable that is mounted for debugging is a *.bin file. The encoding maybe is UTF-8.

What I'm trying to do, is to add data like IDs, keys, and so, into specific memory directions of the final firmware *.bin file in order to be flashed.

Making use of srec tools, something like this is done:

First conversion:

srec_cat data_input_file.bin -o data_input_file.srec

Second conversion:

srec_cat main_file.bin -binary -o main_file.srec

The concat at specific memory region, this command is generic (no address targeted):

srec_cat data_input_file.srec main_file.srec -o final_firmware.bin

I don't know how to create such binary file with hex-coded strings, starting at zero address, up to space needed to hold the data.

With srec tools, we're able to put that data into the directions needed.

---

Second Edit

The sources of raw bytes that will be used are two.

The first one is the *.bin result of the Atmel Studio's compilation that is intended to be flashed into the hardware devices we're using.

The content of the file, for example would be something like, Main_File_1.bin:

00000000: 30 30 30 30 09 66 66 20 66 65 20 36 38 20 30 30  #0000.ff fe 68 00
00000010: 20 36 35 20 30 30 20 36 63 20 30 30 20 20 36 63  # 65 00 6c 00  6c
00000020: 20 30 30 20 36 66 20 30 30 20 35 66 20 30 30 20  # 00 6f 00 5f 00
00000030: 37 37 20 30 30 20 20 20 2E 2E 68 2E 65 2E 6C 2E  #77 00   ..h.e.l.
00000040: 6C 2E 6F 2E 5F 2E 77 2E 0D 0A 30 30 31 30 09 36  #l.o._.w...0010.6
00000050: 66 20 30 30 20 37 32 20 30 30 20 36 63 20 30 30  #f 00 72 00 6c 00
00000060: 20 36 34 20 30 30 20 20 30 64 20 30 30 20 30 61  # 64 00  0d 00 0a
00000070: 20 30 30 20 20 20 20 20 20 20 20 20 20 20 20 20  # 00
00000080: 20 20 6F 2E 72 2E 6C 2E 64 2E 2E 2E 2E 2E 0D 0A  #  o.r.l.d......

Again, is not mandatory to be in hex-dump fashion, the hex-dump is for visual reference, I know that it can be something like, Main_File_1.bin:

X"  ¡@  @  @                              @          @  @  @  @  @  @  e)  @  @  @  @  a/  q/  /  ‘/  ¡/  ±/  @  @  @  @  @  @  @  @  @  @  @  @  @  µL#x +ÑK +ÐH à ¿##p½Ø       ,h  µK +ÐHI à ¿Hh +ÐK + ИG½ÀF    ,h  Ü   ,h      µ  K˜GLú!‰  GK`ú!‰  GK`"K`½ÀFU>  E         à à0µKhJ %€!Ià ,ÐT`•`hBüÐ8öÒ0½   à àµKh + ИG½ÀFô   pµ‚° #J`¬%ep£p%p& !N°G% !°G' !°GK !Ya@"ZaYaZaKhBÐ
KhšÔ    K˜G"Kp¿ó_b¶  °p½ô   å*  €D Aà à     8µ (ÐM ¨G< ,úÑ8½ÀFM  µL #£a K˜G@#£a½€D A  µ„°K`¬ 
K˜G$#“ #c`##r#ãr !K˜GH! "K˜G  !K˜G°½ÀFô   Y*  m*  y  õ(  %)  µ (Ð  !K˜Gà  !K˜G½ÀF%)  E)  0µ‘°.L##p #cp£p©"Jp‹p
p )K˜G#xÙ    " )ÑZ  Ò&H!@!˜@aªp #S`“`tSt‘tÑtu$!ST   “
“€#[Ó`K“b#[BÓbKcKScK“aHIK˜G (ÑM,h K˜G#@#ƒ@K`*hÓi +üÑ"h#C#`K˜G K˜G   àú @²°0½¨  å*   D A      €–˜ ¬

The content descriptions above are just example information, not the actual bytes no be written, just theoretical context.

Just as any device out there, the main target is to write the IDs, frequencies, data, etc, in each one of the hardware devices.

I thought about a set of JSON files in local windows machine, that will read the content, in order to write it to the current device connected, and ready to be flashed (no fuse usage at this point).

So, that would be, something like, Data_File_1.json:

{
    'device_1_data' : {

        'device_uuid' : {
            'low_uuid' : '0x00000001',
            'high_uuid' : '0xABCD000A'
        },

        'key_1' : ['0x2E', '0x34', '0x72', '0x0A', '0x45', '0x57', '0x41', '0x17', '0x07', '0x52', '0xCA', '0x4C', '0x0D', '0x7B', '0x41', '0x57'],

        'module_data' : "0x387A3830",
        'frequency_0' : '390200000',
        'frequency_1' : '392100000',

        'generator_syncword' : '0xCF'

    }
}

The project reads specific memory addresses in order to get up and running, setting up the IDs and the information needed to work. So, making use of the information that the JSON files are holding, that data needs to be written as a binary file, in order to convert it to SREC, so it can be written into the main binary file at the memory address specified with srecord tools.

So it's configured to read the information as it is. You can notice that the JSON holds hex data, but the JSON holds data as text, so the hex strings, need to be treated as hex, and the integers, need to be converted to hex in order to be appropiately written as expected by the application made.

So, that would be something like, Data_To_Be_Written_1.bin:

00000000:   01 00 CD AB 0A 00 00 00 2E 34 72 0A 45 57 41 17  #..Í«.....4r.EWA.
00000010:   07 52 CA 4C 0D 7B 41 57 30 38 7A 38 C0 FA 41 17  #.RÊL.{AW08z8ÀúA.
00000020:   A0 F8 5E 17                                      #ø^.

Data above is taken from the JSON file, so, it's expected as that, so we can make it work with the SRecord tools in order to write that data at memory address of the main binary file holding the project that will make the hardware module work.

---

Third edit (solution and summary):

Thanks a lot to everyone!

The answer of @AnsgarWiechers indeed shows what's needed to get the *.bin expected.

If we take the example of Data_File_1.json:

{
    'device_1_data' : {

        'device_uuid' : {
            'low_uuid' : '0x00000001',
            'high_uuid' : '0xABCD000A'
        },

        'key_1' : ['0x2E', '0x34', '0x72', '0x0A', '0x45', '0x57', '0x41', '0x17', '0x07', '0x52', '0xCA', '0x4C', '0x0D', '0x7B', '0x41', '0x57'],

        'module_data' : "0x387A3830",
        'frequency_0' : '390200000',
        'frequency_1' : '392100000',

        'generator_syncword' : '0xCF'

    }
}

We can proceed then to perform some operations in powershell as follows:

# JSON processing.
$json = Get-Content 'C:\<path to json file>\device_data.json' -Raw | ConvertFrom-Json

Now that we got the json object to work, we proceed to start taking the data.

First, fetch the UUID. and declare the byte array to be used concatenating the first byte values.

BEWARE HERE, if you know a workaround here, or the best programming practice, that will be awesome, I'm starting with this, so, I came down to what is below by reading and coding on-the-go.

# Process the whole UUID from the two JSON fields.

[int64] $device_uuid = [int64] ($json.device_1_data.device_uuid.low_uuid + (($json.device_1_data.device_uuid.high_uuid).split('x')[1]))

$device_uuid_byte_array = [BitConverter]::GetBytes($device_uuid)

# Final Byte array for first target address.

[byte[]] $final_byte_array = $device_uuid_byte_array

Above I took advantage of the BitConverter function usage with int values. First a String concat is made with the high and low address, you can notice the order of the concatenation, and is because the endianness processed by powershell, so, in order to get the bytes in the following order 01 00 CD AB 0A 00 00 00, powershell needs the string like "0x00000001ABCD0001", not like 0xABCD00100000001.

Then it converts the value to an integer, the result of that is 7177306113. And making use of the function:

[BitConverter]::getBytes(7177306113)

We can get the Byte value array of that. So it can get concatenated, or in this case, be the starting value of the byte array that will be at the content of the binary file.

Next up is concatenation of the key string byte array of the JSON.

# Now process the key byte array from JSON.
[int[]] $key_1_value = [int[]] $json.device_1_data.key_1
[byte[]] $key_1_byte_array = [byte[]][int[]]$key_1_value

$final_byte_array += $key_1_byte_array

Now, can proceed with the module_data JSON field.

[int64] $module_data = [int64] $json.device_1_data.module_data

$module_data_byte_array = [BitConverter]::GetBytes($module_data)

$final_byte_array += $module_data_byte_array

Then the frequencies:

[int] $frequency_0 = [int] $json.device_1_data.frequency_0
[byte[]]$frequency_0_byte_array = [BitConverter]::getBytes($frequency_0)
$final_byte_array += $frequency_0_byte_array

[int] $frequency_1 = [int] $json.device_1_data.frequency_1
[byte[]] $frequency_1_byte_array = [BitConverter]::getBytes($frequency_1)
$final_byte_array += $frequency_1_byte_array

Finally the syncword:

[int] $generator_syncword = [int] $json.device_1_data.generator_syncword

$syncword_byte_array = [BitConverter]::GetBytes($generator_syncword)

$final_byte_array += $syncword_byte_array

Once we have everything fetched and concatenated at the byte array we can perform a pipe operation to get a glimpse of what we're getting.

That would be this preliminar output at powershell:

PS C:\Users\user> $final_byte_array | format-hex

               Ruta:

               00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

    00000000   01 00 CD AB 01 00 00 00 2E 34 72 0A 45 57 41 17  ..Í«.....4r.EWA.
    00000010   07 52 CA 4C 0D 7B 41 57 30 38 7A 38 C0 FA 41 17  .RÊL.{AW08z8ÀúA.
    00000020   A0 F8 5E 17 CF 00 00 00                           ø^.Ï...

Now we can send that byte array to a file with extension *.bin, but as the data is appropiatelly written in bytes, there will be no issues with further steps with SRecord tools.

# File storage
[IO.File]::WriteAllBytes('C:\<path desired to save>\input_file.bin', $final_byte_array)

Once we have the file, we can work with srec_cat.exe, for illustrative purposes, let's suppose we're working with CMD at certain directory location, in which srec_cat.exe is located, as well as the input_file.bin we've just created.

>srec_cat input_file.bin -binary -o srec_input_file.srec

If we open the file srec_input_file.srec it will show something like:

S0220000687474703A2F2F737265636F72642E736F75726365666F7267652E6E65742F1D
S12300000100CDAB010000002E34720A455741170752CA4C0D7B415730387A38C0FA4117D5
S10B0020A0F85E17CF000000F8
S5030002FA

Then, we can convert it again, with hex-dump in order to see if we get the same byte values we saw back then at powershell output:

>srec_cat srec_input_file.srec -o hex_input_file.bin -hex-dump

If we open hex_input_file.bin, we can see that:

00000000: 01 00 CD AB 01 00 00 00 2E 34 72 0A 45 57 41 17  #..M+.....4r.EWA.
00000010: 07 52 CA 4C 0D 7B 41 57 30 38 7A 38 C0 FA 41 17  #.RJL.{AW08z8@zA.
00000020: A0 F8 5E 17 CF 00 00 00                          # x^.O...

And we can use this file to concat it at the offset desire, let's say that we intend to write it from location 0x1A010, we do something like:

>srec_cat srec_input_file.srec -offset 0x1A010 -o srec_input_file_offset.srec

And we get something like:

S0220000687474703A2F2F737265636F72642E736F75726365666F7267652E6E65742F1D
S22401A0100100CDAB010000002E34720A455741170752CA4C0D7B415730387A38C0FA411723
S20C01A030A0F85E17CF00000046
S5030002FA

Then we review it with:

>srec_cat srec_input_file_offset.srec -o hex_input_file_offset.bin -hex-dump

And open it to see:

0001A010: 01 00 CD AB 01 00 00 00 2E 34 72 0A 45 57 41 17  #..M+.....4r.EWA.
0001A020: 07 52 CA 4C 0D 7B 41 57 30 38 7A 38 C0 FA 41 17  #.RJL.{AW08z8@zA.
0001A030: A0 F8 5E 17 CF 00 00 00                          # x^.O...

After this, we fill with 0xFF the empty bytes of the srec_input_file_offset.srec:

>srec_cat.exe srec_input_file_offset.srec -fill 0xFF 0x0001A030 0x0001A040 -output final_srec_input_file.srec

Finally, only append the final_srec_input_file.srec with the srec of the main bin file.

Let's suppose we didn't convert the main_file.bin:

>srec_cat main_file.bin -binary -o main_file.srec

Then we get everything together:

>srec_cat main_file.srec srec_input_file_offset.srec -o final_file.bin -binary

And that's about that. Thanks a lot again for everything. Feedback and suggestions are very welcome.

Kind regards.

The man page tells me:

It is possible to read and write binary files using srec_cat(1).

(Having no personal experience with SRecord,) I take this to mean that you can use your input file as-is, using its raw bytes - no need for hex dumps and reformatting; you merely need to follow its filename on the command line with option -binary.

This assumes that your input file uses the desired character encoding; if not, convert it first, such as with iconv, or even a Get-Content -Encoding ... / Set-Content -Encoding ... combination.

The acceptable input formats are described here; format Ascii-Hex is similar to what Format-Hex produces, but would require non-trivial tweaking.

---

Generally, note that representations such as the ones created by Format-Hex are themselves invariably text, not "binary".
They describe a sequence of byte values textually, which allows them to represent arbitrary data, including binary (non-textual) data.