02-06-2020, 06:46 PM
Stage 2, day 6:
Arbitrage code is done. Not tested yet. I had to revisit a communication library that is a building block of my system first.
The code is very well done but documentation is poor. So to figure out anything about how things are done, the only way is to roll your sleeves in dive in the code.
The purpose of revisiting this part is that enabling the arbitrage module will multiply the input data amount by 131. Current bitrate is a bit less than 10 kbps. So we are talking about 1.3 Mbps. Not too big but I'm concerned about its processing time. By just writing those numbers, I think that something is going to blow when I start up the arbitrage module. I currently use 15% CPU. My current CPU won't be able to keep up with a 131x bigger load... I guess hardware upgrade is now just around the corner. TBH, my current system is roughly 10 years old so this is very venerable age for a computer system... I shouldn't be ashamed thinking replacing it...
but anyway, before I realize that detail, the goal was:
1. Figure out how to integrate libev with the other communication lib.
2. Make sure that TCP parameters were well tweaked to support the increased volume
3. While, I'm at it, knowing that the very next step will be to have a client connecting to 2 different services using 1 thread per service, I better figure out how this is going to work while the not so well documented library code is fresh in my mind...
It took about 1.5 day to complete those 3 tasks. I have a clear view in my mind how #3 is going to be done in my own codebase refactoring (either tonight or tomorrow).
This has been a bit painful and stressful... Reverse engineering someone else code isn't easy. Plus, on the surface, not a lot of code is written during that time. That leaves the impression of stalling but it isn't the case. Collecting information is an important step in creating nice solution to complex problems.
Beside, I thought that I knew it all when it comes to linux system programming and I have just discovered 2 curious and useful subsystems that I have never heard about during my investigation. Not sub related but for those who like that type of details, it is eventfd and signalfd! Discovering those 2 little gems has been fun and enjoyable.
Arbitrage code is done. Not tested yet. I had to revisit a communication library that is a building block of my system first.
The code is very well done but documentation is poor. So to figure out anything about how things are done, the only way is to roll your sleeves in dive in the code.
The purpose of revisiting this part is that enabling the arbitrage module will multiply the input data amount by 131. Current bitrate is a bit less than 10 kbps. So we are talking about 1.3 Mbps. Not too big but I'm concerned about its processing time. By just writing those numbers, I think that something is going to blow when I start up the arbitrage module. I currently use 15% CPU. My current CPU won't be able to keep up with a 131x bigger load... I guess hardware upgrade is now just around the corner. TBH, my current system is roughly 10 years old so this is very venerable age for a computer system... I shouldn't be ashamed thinking replacing it...
but anyway, before I realize that detail, the goal was:
1. Figure out how to integrate libev with the other communication lib.
2. Make sure that TCP parameters were well tweaked to support the increased volume
3. While, I'm at it, knowing that the very next step will be to have a client connecting to 2 different services using 1 thread per service, I better figure out how this is going to work while the not so well documented library code is fresh in my mind...
It took about 1.5 day to complete those 3 tasks. I have a clear view in my mind how #3 is going to be done in my own codebase refactoring (either tonight or tomorrow).
This has been a bit painful and stressful... Reverse engineering someone else code isn't easy. Plus, on the surface, not a lot of code is written during that time. That leaves the impression of stalling but it isn't the case. Collecting information is an important step in creating nice solution to complex problems.
Beside, I thought that I knew it all when it comes to linux system programming and I have just discovered 2 curious and useful subsystems that I have never heard about during my investigation. Not sub related but for those who like that type of details, it is eventfd and signalfd! Discovering those 2 little gems has been fun and enjoyable.
