Monitoring Communication  

I used a program called "USB Monitor" from HHD Software to monitor the communications between the PIC and the PC during bootloading using the Microchip-supplied demo program, PDFSUSB.  This allowed me to see everything the PC-side program needed to do in order to perform the bootloading functions with the PIC, using the standard Microchip bootloader firmware.  This particular USB monitor program did a good job of formatting the packets so they were easy to follow.  Unfortunately, it's commercial software and costs $65 which is a bit much for a hobbyist who will only be using it very occasionally, but they offer a 14-day (max 100 sessions) free trial, which should be enough to finish this project.

The hardware is the same barebones setup I used in my initial experimentation, seen in the first article , schematic shown below: 

I just used the buttons to put the PIC in bootload mode, started up PDFSUSB.exe, and started playing around with it with the USB monitor running.  Anyway, there are a lot of transactions, but here is what I picked out of it so far:

Packet Format:

Based on the communcation I've monitored, I believe the packet format is:
<command><length><address (LSB)><address><address(MSB)>
The address is transmitted with the least significant byte first, followed by the next most significant byte, followed by the most significant byte.  Note that for all program memory and EEPROM read/writes, the MSB is 0 (at least for the 18F4550) because these memories do not extend past address 0x7FFF.  Only when reading/writing configuration bytes is the MSB non-zero.

See the below communication transcripts for examples.

Read Operation:

First, the program reads all the PIC program memory, using the command READ_FLASH (0x01) 

Each response from the PIC is an echo of the 5 bytes from the PC, followed by 59 data bytes of program memory starting at the specified address.  (59 + 5 = 64 bytes, which I believe is the maximum for one packet)

After the program finishes reading all the program memory, it reads the EEPROM, using READ_EEPROM (0x04), in exactly the same way it read the program memory:

 After reading the EEPROM, the program then reads the special ID locations, 0x200000 to 0x200007.  For whatever reason, it does this by reading them one byte at a time:

 Then it reads the actual configuration bytes, which reside in special locations 0x300000 to 0x30000D, 0x3FFFFE, and 0x3FFFFF.

Erase Operation: (command = 0x03)

Since the bootloader takes up the space before 0x800, it looks like each command erases 0x40 (64) memory locations, starting with 0x800.  When it finishes erasing all the program memory, it verifies by simply reading it all; it's the same as the read operation listed above, except it only reads the program memory, not the EEPROM or the configuration.

Write Operation:
Well, this one is a pain, so bear with me.  To give you an idea of how much USB traffic this involves, when I exported capture log from the USB monitor into simple HTML format, the file size was 3.52 megabytes, and contained 7661 lines of text, meaning somewhere around 1700 transactions to sort through.

The first thing that happens is that it performs an erase of the program memory, just as in the erase operation above, except it does not perform a full read to verify; instead it verifies as it programs in the new code. 

Next, it repeats one process, in which it writes and then verifies one chunk of data.  Each chunk is 16 bytes; I'm not 100% sure of the reasoning for this, but that also happens to be the number of data bytes seen on each line of a standard PIC HEX file, which may be the reason.  It does not attempt to program any memory locations before 0x0800, even if they appear in the hex file, because that is reserved for the bootloader code.

After that, it does the same kind of write-verify process to write the EEPROM:

Then, it does one erase-write-verify to write the ID locations (starting at 0x200000):

One confusing thing here is that when doing the read operation seen above, it used the command READ_CONFIG (0x06) to read the ID locations; here, it is using ERASE_FLASH, WRITE_FLASH, and READ_FLASH.  I can understand the erase, since there is no command to erase config locations, but why use read/write flash instead of read/write config?  Perhaps the functions are interchangeable for these locations? I guess this is one of those things where I'll just have to keep my mouth shut and do it the way they did it, and then go back and test my theory later once I have a working program.

Finally, it does one write-verify on the configuration bytes, 0x300000-0x30000D (note that 0x3FFFFE-0x3FFFFF are not written as these are read-only values representing the PIC hardware device ID and revision number)

Note that it only writes 14 data bytes this time, as opposed to the fixed 16-byte packets it used for everything else, for whatever reason.

Execute:

Just like a regular reset, the PIC goes into user mode and begins executing the user program, as long as the bootload button isn't held down at the time of reset.

Next up: Starting to put together a windows bootloader application. 

{mos_sb_discuss:8}