Cycle 12, day off #1:
The heap corruption solution hope went away. Finally, the test program dump was a problem in the test program. It was still a good exercise as it did allow me to improve the code in many ways.
So, for now, I'm going to conclude that the heap corruption was the result of build problem and I'm going let go this issue.
Yesterday, I was ready to close this chapter and move on to something else and the JSON library did byte back at me again.
I reserve 2 KB on the stack for the JSON parsing. If memory needs exceed this limit, I added code to track the parsing memory usage and log some traces if the limit is exceeded. I want to know which messages is causing trouble and if I need to adjust the allocated space.
By adding this simple code snippet, I made my programs crash on launch. First, I detected that a 1000 bytes long msg was needing 64KB of storage.
In itself, this is a non-sense. I suspect that the library is preallocating space for arrays and for this pathological msg, it preallocates a lot because it is a msg composed of dozens of nested arrays. I don't have an answer yet for this issue. I want to look into it today.
I did address the crash. I forgot about that but many months ago when I wrote the decoding code, I was taking the timestamp fields having the format "seconds since epoch"."microseconds". I was replacing the dot for a NULL character to ease the ts extraction. I didn't bother to put back the dot once done. IOW, it was destructive. Because now, I'm reencoding JSON to log them since I did switch libray, I got bitten back by this small detail. It took me a good hour to figure out what in hell was happening...
This investigation did make me realize something else. The library is a template library and provides some specializations so it can use assembly code for faster processing when possible. I assume that my code was using those specialization. Guess, it wasn't... I did fix that as well...
Now, the only thing that I need to figure out why this particular msg type is making memory needs explode...
I like few good challenges once in a while... but I feel like a little break would be welcome now...
The heap corruption solution hope went away. Finally, the test program dump was a problem in the test program. It was still a good exercise as it did allow me to improve the code in many ways.
So, for now, I'm going to conclude that the heap corruption was the result of build problem and I'm going let go this issue.
Yesterday, I was ready to close this chapter and move on to something else and the JSON library did byte back at me again.
I reserve 2 KB on the stack for the JSON parsing. If memory needs exceed this limit, I added code to track the parsing memory usage and log some traces if the limit is exceeded. I want to know which messages is causing trouble and if I need to adjust the allocated space.
By adding this simple code snippet, I made my programs crash on launch. First, I detected that a 1000 bytes long msg was needing 64KB of storage.
In itself, this is a non-sense. I suspect that the library is preallocating space for arrays and for this pathological msg, it preallocates a lot because it is a msg composed of dozens of nested arrays. I don't have an answer yet for this issue. I want to look into it today.
I did address the crash. I forgot about that but many months ago when I wrote the decoding code, I was taking the timestamp fields having the format "seconds since epoch"."microseconds". I was replacing the dot for a NULL character to ease the ts extraction. I didn't bother to put back the dot once done. IOW, it was destructive. Because now, I'm reencoding JSON to log them since I did switch libray, I got bitten back by this small detail. It took me a good hour to figure out what in hell was happening...
This investigation did make me realize something else. The library is a template library and provides some specializations so it can use assembly code for faster processing when possible. I assume that my code was using those specialization. Guess, it wasn't... I did fix that as well...
Now, the only thing that I need to figure out why this particular msg type is making memory needs explode...
I like few good challenges once in a while... but I feel like a little break would be welcome now...
