cycle 2. Stage 3, day 27:
Despite what I wrote in my last entry instead of doing right away what I said that I would do, I did work on something else. I had several reasons for doing so.
First, 2 months ago when I got my last breakthrough with the fee free trading, I was in the middle of implementing an idea. This task has been put on hold since then. I hate placing an idea on hold for so long because all the small details that were clear in my mind have the time to disappear and it is extremely inefficient to return back to old tasks and rediscover all the details that were available in the past. The longer you wait, the costlier it is.
Second, I am competitive in nature. Despite not representing a big chunk of my trading volume, it piss me off to miss a trade because my system has been too slow. Despite all the optimizations that I did bring to it in the last month, I was witnessing juicy trade misses in my log window as I was in front of my computer.
The idea that has been put on hold is that currently when my system receive a price update. It puts the new price in place and then reevaluate all the trade combinations that may be affected by the change and if it finds a profitable combination, it launch a trade. All this computation is kinda complex and expensive. I got the idea of batch updating. That is instead of reevaluating everything at every update, I should empty the TCP reception buffers and place all the updates it contains and only when all the updates have been put into place, perform the reevaluation. This small change had the potential to spare thousands of duplicate reevaluations to my system hence make it more reactive and increase its processing capacity.
The concept is simple but the implementation did require some design modification. I rarely do that but this time, I had to sit down and write some design document to figure out how I would do it. Usually, I just have a mental picture of what I want to do and I go straight to coding but I felt that this was not an option this time. It must have taken a day or 2 to just create the design.
and all in all, I spent the whole week on this effort including coding. It appears that everything went fast but it is me that was so absorbed by the task that I did not see the time pass. (we are already Friday!)...
Now the expected results. I was expecting that I would make my system faster and it would finally succeed grabbing the few trades that it currently miss. Yes, I got that but I got way much more than that. Before this change, fee free trading was representing 95% of all my trading. It steadily profit every day but due to its nature, it was literally generating only pennies of profit. Since yesterday, regular arbitrage trading has become the predominant type of trade. It is beautiful to see the system operating because the last time where it was doing a decent amount of arbitrage, my system did immensely gain in maturity and robustness with all the bug fixes that I did put into it in the last several months... and regular arbitrage trades are much juicier than fee free trading. We are talking 2,3,4% and sometimes even more on amounts of few hundred dollars.
So yesterday my monthly trading volume fell below 600K and was getting dangerously close to the value of 500K that I absolutely did not want to see my volume go below. I did wake up this morning and the volume downtrend has finally broke up! My volume must be about 650K and in a single evening, I must have generated as much profit than I did in the last week!
I am not too sure why I got this result. My best explanation is that the exchange was throttling down the pace it was sending me updates to adapt to my system processing capacity and several updates were just dropped making several trading opportunities simply invisible to me.
In conclusion, it seems like stage 3 theme was system performance improvement. 2 major breakthrough in 2 consecutive months! It is also a nice example of not doing what your conscious mind tells you to do and you go against reason and stick to what your guts is telling you... It would have been so easy to pass beside this hidden opportunity and not even suspect that it was there... I am so grateful for everything that is happening. My multi year effort finally starts to pay off... For the records, I started this effort in December 2019...
Once again, it seems like I am going to confirm the 3 years rule... This is the amount of time that it usually takes to bring a new venture from scratch to profit...
Update:
I got few glitches during the last run. I never stop to amazed by how many random ways that stuff can go wrong... The luck to stumble into all those glitches is awesome
Despite what I wrote in my last entry instead of doing right away what I said that I would do, I did work on something else. I had several reasons for doing so.
First, 2 months ago when I got my last breakthrough with the fee free trading, I was in the middle of implementing an idea. This task has been put on hold since then. I hate placing an idea on hold for so long because all the small details that were clear in my mind have the time to disappear and it is extremely inefficient to return back to old tasks and rediscover all the details that were available in the past. The longer you wait, the costlier it is.
Second, I am competitive in nature. Despite not representing a big chunk of my trading volume, it piss me off to miss a trade because my system has been too slow. Despite all the optimizations that I did bring to it in the last month, I was witnessing juicy trade misses in my log window as I was in front of my computer.
The idea that has been put on hold is that currently when my system receive a price update. It puts the new price in place and then reevaluate all the trade combinations that may be affected by the change and if it finds a profitable combination, it launch a trade. All this computation is kinda complex and expensive. I got the idea of batch updating. That is instead of reevaluating everything at every update, I should empty the TCP reception buffers and place all the updates it contains and only when all the updates have been put into place, perform the reevaluation. This small change had the potential to spare thousands of duplicate reevaluations to my system hence make it more reactive and increase its processing capacity.
The concept is simple but the implementation did require some design modification. I rarely do that but this time, I had to sit down and write some design document to figure out how I would do it. Usually, I just have a mental picture of what I want to do and I go straight to coding but I felt that this was not an option this time. It must have taken a day or 2 to just create the design.
and all in all, I spent the whole week on this effort including coding. It appears that everything went fast but it is me that was so absorbed by the task that I did not see the time pass. (we are already Friday!)...
Now the expected results. I was expecting that I would make my system faster and it would finally succeed grabbing the few trades that it currently miss. Yes, I got that but I got way much more than that. Before this change, fee free trading was representing 95% of all my trading. It steadily profit every day but due to its nature, it was literally generating only pennies of profit. Since yesterday, regular arbitrage trading has become the predominant type of trade. It is beautiful to see the system operating because the last time where it was doing a decent amount of arbitrage, my system did immensely gain in maturity and robustness with all the bug fixes that I did put into it in the last several months... and regular arbitrage trades are much juicier than fee free trading. We are talking 2,3,4% and sometimes even more on amounts of few hundred dollars.
So yesterday my monthly trading volume fell below 600K and was getting dangerously close to the value of 500K that I absolutely did not want to see my volume go below. I did wake up this morning and the volume downtrend has finally broke up! My volume must be about 650K and in a single evening, I must have generated as much profit than I did in the last week!
I am not too sure why I got this result. My best explanation is that the exchange was throttling down the pace it was sending me updates to adapt to my system processing capacity and several updates were just dropped making several trading opportunities simply invisible to me.
In conclusion, it seems like stage 3 theme was system performance improvement. 2 major breakthrough in 2 consecutive months! It is also a nice example of not doing what your conscious mind tells you to do and you go against reason and stick to what your guts is telling you... It would have been so easy to pass beside this hidden opportunity and not even suspect that it was there... I am so grateful for everything that is happening. My multi year effort finally starts to pay off... For the records, I started this effort in December 2019...
Once again, it seems like I am going to confirm the 3 years rule... This is the amount of time that it usually takes to bring a new venture from scratch to profit...
Update:
I got few glitches during the last run. I never stop to amazed by how many random ways that stuff can go wrong... The luck to stumble into all those glitches is awesome
