cycle 2. Stage 2, day 8:
You have no idea the amount of luck that I am getting.
My system is generating maybe 10 thousands of trades daily so it is very unlikely that upon my first look on the tailing log when I wake up in the morning, I stumble on the only trade that went wrong because of a bug...
This is exactly what did happen this morning! This is impossible to imagine being more lucky than that... This is absolutely amazing luck!
Yes, I am annoyed since I had to dig into the problem and it took me the whole day... but if it was not for this amazing moment of luck, the nasty bug would still be lurking in the codebase...
and there was not just a single bug... it was like a dog running in a bowling alley and striking all the pins... I got several serious bugs in this single investigation session!
I'm a bit frustrated to not have move forward my main tasks but that was really a very productive day... A very lucky day...
Luck is so enjoyable... this is fun... UMS, if it is responsible of all this, is an amazing luck generator... I keep receiving all of it very often... Amazing!
cycle 2. Stage 2, day 10:
I have my 2 hands into refactoring my code base to be able to support multiple exchanges...
It is a mess and a really daunting task but I am confident that I will succeed... A common pattern is emerging into how to transform the code to make it exchange neutral... I am applying it everywhere and this seems to be working... It is just a daunting titan size task...
I don't think that I am going to be done today... It is a multi days long effort I am affraid...
but I'll be happy about the result when it is done...
cycle 2. Stage 2, day 12:
The last 2 days have been painful. I have been experimenting with the builder pattern. I ended up refining the design 2-3 times before reaching a satisfying pattern. The end result deviates a bit from the pure official pattern. I did try search for someone having the same set of requirements and I could not find anything on the net... It is very hard to believe but I might have invented a new design pattern...
Bottomline, I start to see the light at the end of tunnel... I am almost done... When it is over, it is going to be an incredible relief...
I knew this exercise would be painful and this is probably why I did procrastinate for so long before tackling this challenge. This is some takeaway... I guess that the idea of pain, painful tasks is a factor stopping me from succeeding. I avoid unconsciously painful tasks to not feel the pain... but sometimes... taking a deep breath... and going through the pain is required to achieve some goals...
As I said, I have drifted a bit into doing into this task. So probably attacking this painful task was an unconscious decision... Therefore... UMS might be behind this result...
cycle 2. Stage 2, day 14:
Refactoring allowing to support new exchanges is almost over...
Objectively, I cannot say exactly what is so terrible in this task but there is a part of me that hates doing it. I am going override this feeling and I will go through no matter what but I surprise myself procrastinating instead of speed up this task that I hate doing...
Almost done... almost... This will have been a 1 week effort but I 100% sure that this will handsomely pay off very fast...
cycle 2. Stage 2, day 17:
I have made some progress since last entry but I am not done yet. Yesterday and today have seemed like slow days to me. Yesterday, I did stumble on a difficulty that got me blocked. I was not finding a solution for the issue so I decided to sleep on it. This morning, a solution did pop up in my mind as if I did work on it all night.
I got distracted today with the new idea to perhaps run a Bitcoin Lightning network node. I have a server that I am far from using to its full capacity.
The prerequisite for operating a Lightning network node is to also run a bitcoin full node... I got surprised by the current blockchain size that has now surpassed 400GB... I don't have enough disk space to run that today.
I am still learning about linux machine management. I am kinda stuck with my current setup which is configured as a single ext4 partition on top of a software RAID1 setup made of 2 480GB SSD. If it was to be redone, I would add LVM to the mix and I am also considering replacing EXT4 partitions to give BTRFS a go. BTRFS appears to be really nice with the option to compress files on the go.
I kinda have found a solution. The hosting company is offering to install free of charge a 3rd 500GB HDD to my server. This little guy, partionned as a compressed BTRFS is going to become my dedicated mount just for holding the BTC blockchain... With compression, this should last for some time...
Beside that, I am approaching another important milestone. I was expecting it to happen today but maybe sometime during next night. I must be about $13K away from reaching a monthly trading volume of a million...
So while I am busy refactoring my code base, the latest server build has been running nonstop for possibly at least a week without any problems. Things are finally working fine...
Reaching the million dollar trade volume might bring something nice. Fiat pairs trading becomes free of fee at this volume. I expect this event to maybe unlock new arbitrage opportunities...
things are going nicely. I just feel like the time is passing way too fast. I am not seeing the days go by...
cycle 2. Stage 2, day 19:
I have been feeling exhausted for the last 1-2 days. IDK if it is some UMS programming side-effect (ie: my subconscious resisting) or if I am fighting some sort of spring virus...
I have a small pet animal that has its health deteriorating quickly. Since last week-end, it started to have difficulty to just move. It is a generalized cancer. There is nothing that can be done to save it. Palliative treatment is pretty much the only thing I can do for it. The thing, I have been thinking since Monday last week that it was only few hours away from passing away. It is still of this world as I write this... I admire its will to live, its combative bravery... but it has been very taxing emotionally... shortening its suffering has crossed my mind but despite its condition deteriorating, it seems still happy to be here but this is hard and tough to experience... A real emotional roller coaster... and this is exhausting..
Every day that I have with it, I accept it as a gift life is giving me... Those last moments are precious...
My refactoring task is progressing. It is still slower than I wish it would go but it is progressing so as long as I work on it and it is moving forward, I guess that this is good. I have pretty much completed the changes that I wanted to perform but the changed code is in the middle of the code base so it has ripple effect all over the place and I basically have to go over almost every files to adapt the code to the new change. I wonder if there is not some sort of software that could automate much of what I am doing...
a week or 2 ago, I did stumble into a dev IDE named CLion. Its business model is different than what I am used to when I purchase software. It is based on a subscription model... It is roughly $20/month with a free 1 month trial...
I should give it a try... If it boost my productivity, that would be a real winner move...
I had a realization maybe last week or this week that I am the bottleneck of my venture. So far, this is a one man show which limits a lot what I can accomplish... I mean, on one hand, I do not care... My craft is my passion... I love what I do and I don't care to work on it from morning to evening every day but if I could reach my goals faster... that could be cool as well...
My main dilemma is how to outsource dev without giving away my trading "secret sauce"? My wealth relies mostly on the IP that have developed over the time. I start to realize that the current refactoring is a step toward outsourcing. I am designing some sort of plugin framework where my trading engine can interface with an exchange. This framework, once completed, will allows me to outsource the uninteresting and boring to develop exchange plugins for the various exchanges that I want to support. Outsourcing those plugins would allow me to onboard other developers without giving away my secret sauce...
I am surprising myself having those growing organization thougths... this is usually outside my regular thinking... I love where UMS is bringing me as I suspect that it is not stranger to this new mindset...
Yesterday, my system has finally reached the very important symbolically milestone of 1 million dollar trading volume... The celebration lasted few minutes but AFAIK, beside the symbolic success, nothing else special did happen. I was expecting FIAT pairs trading with 0 fee to unlock something but it did not...
Congratulations on reaching 1 million in volume lano1106
Concerning CLion I have used it superficially in the past and am a regular user of PyCharm and WebStorm by the same company. So I can vouch that they make good IDEs and encourage you to give the trial a shot - see if it is a fit for you.
In WebStorm there is the function to refactor variables and then it searches through the whole project and also refactors it in other files, hope there's a similar thing in CLion.
Wishing you the best
MM
cycle 2. Stage 2, day 22:
I think that I am getting very close to having worked 2 weeks on the refactoring effort to support multiple exchanges.
This is an enormously huge task. It is a bit scary to have change code for 2 weeks without having an opportunity to validate the intermediate iterations. The initial change is so core to the system that it practically requires to revisit almost every corners of the system to generalize and extract new abstractions from the code. I think that the biggest chunk is done. The most important classes have all been modified.
I am eager to complete this one and move on to funnier tasks!
cycle 2. Stage 2, day 25:
Almost done with stage 2 for a second time. This month theme has definitely been the month of adding support multiple exchanges feature to my system.
This is not over yet but I suspect that I might finish the task sometime today. I haven't done more refactoring since Sunday. I am at the step of compiling the whole thing and fixing the various compilation errors that the changes did induce.
It is very boring and tedious but at the same time there is nothing hard or difficult left to figure out. I am pretty sure that I am going to feel some great satisfaction once this is finally completed...
I guess that I am going to reap the benefits sometime during stage 3 but what I can say is that this initiative feels like it is coming straight from UMS programming has it as come out out of the blue suddenly, in a compulsive way, organically without really thinking about it. Clearly this idea has emerge from my subconscious...
cycle 2. Stage 2, day 28:
I have completed the refactoring allowing to add support for multiple exchanges. Almost a month later, the black box output is identical to what it was before the effort started.
It might include minor cleanups and improvements but nothing ground breaking.
I still need at least few days up to a full week to adapt my client GUI program and test through fully the refactoring before deploying it on the server.
The value is what it will allow me to add next starting from here. That should be the stage 3 story...
Update:
Because I have been busy with the refactoring, I did not really improve the trading code for the last month. That, the fact that markets have been pretty much side ways and my last breakthrough is now over 30 days old have made my monthly trading volume go below a million.
It went back above a million few minutes ago but I feel that keeping this volume above the magic threshold is going to be a constant struggle until I improve the trading code...
cycle 2. Stage 3, day 1:
I have put in production the refactored version of my server. Beside big obvious easy to fix bugs at startup, it seems like the upgrade has been a success.
I feel relieved and some satisfaction to have reached completion to the hard task. Sadly, this has no effect of performance. The benefits will be realized when the system will be adapted to other exchanges which I should be doing later this month...
cycle 2. Stage 3, day 6:
The first stage 3 week is over.
I felt extreme fatigue during the whole week. I was waking up still tired in the morning. I felt some sort of brain fog throughout the day.
I did not make the association before today but this state coincide with the start of stage 3. It seems like run 2 of stage 3 is stirring much more stuff than during the first run...
cycle 2. Stage 3, day 7:
Energy level that I feel appears to be improving a bit.
A small pet animal did pass away this morning which saddens me. I am pretty sure that I did mention it in a previous journal entry... Roughly 1 month ago, the animal was in such a bad shape that I was believing that it was a matter of hours before it pass away. With a lot of love and good care, I have been able to extend its life a whole month. That is quite an achievement. The animal was not eating at all. Something was stopping it from eating solid food. The miracle came from a spark of genius to try to feed it with yogurt. This has turned out to be an awesome idea and it did give it a boost of vitality... It went from being immobile for 2 straight days to be running all over the house with an excellent appetite!
About my trading system project... I got a somehow funny surprise this morning. Now that the refactoring is over. I was returning to the original task that did trigger the whole refactoring thing a month ago only to find out that the idea is finally a useless dead-end.
I am not ready to call the major refactoring decision a mistake. I deeply believe that having finally done it (I was procrastinating doing it because I knew that it would be a painful experience), will turn out to be a profitable move...
It is just that the trigger motivation justifying doing it was fake... Time will tell if I have been lucky with this judgement error... Track record of UMS in regards to luck tells my gut feeling that it will be the case...
cycle 2. Stage 3, day 16:
almost a week has passed since last entry.
last week has been very busy for crypto trading. My system trading pattern did change a lot compared to what it used to be in the last month.
Due to the volatility spike, my system made more normal size arbitrage trades than my new micro trading strategy.
My system did stumble on a bunch of small bugs that I have kept me busy pretty much the whole week putting on hold my regular planning tasks.
The crypto crash is a bit worrying to me... The amount of income that my system is generating has remained constant when nominated in crypto but once converted in fiat, it got practically slashed in half.
I guess that if I leverage this situation to maintain my income in fiat stable, it will have the effect of doubling it when crypto value returns to its previous value down the road.
I was fixing what I though to be the very last remaining glitch from the current server version when I spotted code in a very low-level module that is called millions of time in the hot path. I have found a way to simplify it by removing a conditional branch and by removing an unnecessary variable assignment.
It seems like nothing. 2 lines max got removed but this code is called so much and branching is so expensive on x86-64 platform that this very simple change made a very significant difference in the system execution time. This is cool. This code is over 10 years old. I did not expect to find improvement opportunities there. That has been a very nice surprise today.
Bottomline, I am drifting a bit since last week but good stuff is happening nonetheless...