+- Subliminal Talk (https://subliminal-talk.com)
+-- Forum: Men's Journals (18+ NSFW) (https://subliminal-talk.com/Forum-Men-s-Journals-18-NSFW)
+--- Forum: Men's Journals (https://subliminal-talk.com/Forum-Men-s-Journals)
+--- Thread: lano1106 UMSv2 (/Thread-lano1106-UMSv2)
RE: lano1106 UMSv2 - lano1106 - 01-02-2022
Stage 4 (Month 2) day 1:
I haven't given up on speeding up my reaction time despite the last setback.
The last 2-3 days have been very quiet in terms of trading activity. I guess that this is expected considering that people are more likely than not in holiday and not working. It should ramp up starting tomorrow morning.
In the last few days, I have kept looking very hard to optimize my code. I did find a way to merge 2 virtual function calls into a single one. That didn't make any measurable difference but there is one since not calling a function has to be faster than calling it.
I got another cool idea. This time, it should be the right idea. I am in the process of modifying how my code is interacting with the websocket library. This is a bit tricky to implement but the concept is brilliant and should this time make a noticeable difference in the reaction time.
I like how all those ideas are coming in my mind... It almost seems like I am able to see in the future... I have those cool ideas and once implemented, it is done exactly like I did imagine them... Well, as long as inspiration keep coming, I'll keep trying to make my system successful...
RE: lano1106 UMSv2 - lano1106 - 01-03-2022
Stage 4 (Month 2) day 2:
I have completed another round of optimization. I am perplex about the result.
25% speed-up in the baseline case... No difference at all in the more complex more important use case.
I have a hard time making any sense from those observations.
For one, the measured time is between the insertion time and the write confirmation time. Confirmation notification is different than the actual write. The confirmation could somehow be delayed so there is a speedup but is not perceivable. I find this explanation very fishy and not very good...
I will let the system run for some time and see if I can notice performance improvement from the speed up...
Anyway, I cannot dig into the question any longer. I need to put this problem on hold until I take case of all the other things that I have neglected while focusing on the trading project. I guess that I will revisit the question eventually...
RE: lano1106 UMSv2 - lano1106 - 01-06-2022
Stage 4 (Month 2) day 6:
There was an email laying in my mailbox for a week or 2. It was from someone at Cloudflare that I did talk with few times last year.
Despite all the attempts their engineers tried to find a solution to speed up my connection time with the exchange, nothing good never came out of that...
I like the guy, he keep buzzing me once in a while with new ideas... Like the email he sent me between christmas and NYE.
I was thankful for the ideas the email was containing but I was skeptic that nothing it contains would actually be of any help...
Despite not believing in the idea, I gave it a shot... After 2 days of fumbling around installing proprietary Cloudflare software on my server and modifying my software to be able to interface with Cloudflare software, the whole setup finally started to work by the end of the afternoon...
Initial test numbers did show some substantial improvement... I have tried to stay prudent... I have been fooled so many times lately with false hopes. I said to myself that this could be luck... Latency with server fluctuates a lot depending of the exchange trading volume...
I stopped working to relax and cook up dinner... Finalizing the Cloudflare software setup was the milestone I was waiting to reach before calling it a day
So far so good... When I came back to my computer after dinner in late evening, the trading system did trade very well during my absence... It starts to feel real this time but there is a part of me that remains skeptic... crypto markets happens to be very volatile tonight... This type of volatility is usually very fertile for my system... The skeptic part of me thinks that this might be some coincidence....
I need more time to be 100% convinced that the latest experiment is actually fruitful....
but it starts to look like I did finally stumble into something that makes a real difference...
RE: lano1106 UMSv2 - lano1106 - 01-11-2022
Stage 4 (Month 2) day 10:
Finally, the cloudflare product testing initial result has been a little bit misleading. Believe it or not, the good trading session on day 1 ended up being a cruel coincidence.
maybe it speeds up things a bit but speed is not my #1 issue anymore. Maybe 6 months ago, my system was missing several hundreds of dollars worth of trades daily because of speed but I guess that markets have matured since then. They are more efficient or bigger traders with 1+ million trading volume reap everything with their smaller fees before anything reach what can be profitable to me with my higher fees. The end result being that those several hundreds of dollars of potential profits have dried out... They are not there anymore... Is it a cyclical thing or they are gone for good? idk but knowing the answer won't help me. I need keep going and explore other directions
Bottomline, I have declared optimization hunting season over. There is no point for me to actively focus on speeding up further the system. In a strange way, this realization feel like a relief.
This allows me to move to the next todo element in my roadmap which is porting my system to other exchanges. I have the feeling that this is going to be a painful exercise and some part of me must have postponed for as long as possible this task to avoid the pain... but the decision has been taken that this would be the next task. There is a part of my mind that has already started the design to make the code support multiple exchanges in the mental space... Based on prior experience, the work is halfway done when I start visualizing it as done in my mind... This is where my mental state is...
Success is not easy... There is a lot of fumbling around before finally getting there...
RE: lano1106 UMSv2 - lano1106 - 01-22-2022
Stage 4 (Month 2) day 22:
I spent the whole week tackling with tasks that I have been procrastinating doing for months.
It is pretty much a full break from my trading project which has mostly not traded at all during that time except for the last 48h with the major crypto crash currently going on.
Concerning the unrelated tasks that I have cleared, it provides a feeling of relief and satisfaction. When I postpone tasks for that long, it is always to avoid pain that I imagine doing it will bring. It is never as bad as I imagine but the thought of that pain is making me procrastinating over those tasks.
One of the task was to replace a hard disk for a SSD in my previous main desktop computer to convert it into a Monero miner. With Linux, doing something is never simple. I had to relearn intricacies about rsync, bash curly braces expansion, grub mkinitcpio and so on... Now, that it is over, nothing was very complicated, but I have burned few days on doing it when it could have taken hours. idk, maybe it is me getting older and slower...
The desired outcome was to increase my overall mining pool hash rate. That is the type of thing that once setup, it requires very little attention but relentlessly bring back some money 24h a day 7 days a week. You set it up and you can forget about it. If it runs for months or years, sooner you launch your miner, the better. Well, that is the theory. Since it is a somehow old machine, it does not seem to make a noticeable difference on my overall hash rate.
Idk if something is wrong with me. It seems like I did focus mostly on working on low profit tasks resulting in making me waste my precious time...
At least, it is now done. I can move on.
Concerning the trading system activity, it did pretty well. For one, no crashes at all. I suspect that it did stumble into few snags... Its state did not seem pristine when I restarted it earlier this afternoon. I have roughly 100 trades to analyze in a 5MB big log file. I am sure that this trading activity will make me discover bugs that did escape my attention until now. My system is most likely than not going to be better than it was by the end of the week-end...
I am not sure I should be happy or sad. If I look the amount of cryptos that I own, I did a good gain. However, my assets value nominated in dollar got a major hit in the balls...
On a different topic, because I have kinda completed a major milestone with my trading project and I am getting close from completing a super UMSv2 cycle and considering that I haven't gotten any intimacy with a woman since I have started UMS (at least), I am considering making a small DMSI test run... Those pesky intimacy urges are in the process of resurfacing and because I have worked really hard in the last few months, I certainly deserve a small pleasure reward...
RE: lano1106 UMSv2 - lano1106 - 01-25-2022
Stage 4 (Month 2) day 25:
I am not done yet analyzing the latest 5MB big log file. So far, doing so has allowed me to discover and squash at least 8 small bugs...
I am looking forward deploying the next server version and seeing those fixes in action...
RE: lano1106 UMSv2 - lano1106 - 01-30-2022
Stage 4 (Month 2) day 30:
Log analysis is completed. Final fix count is 9 and I did add a small optimization. The updated server did run for few days without much trading. Last night, I got a new idea on how I can improve the system.
This is a radical departure to how the system is currently doing things but given how competitive the market is becoming, this change appears necessary. Again, I feel the excitement of having maybe discover a game changing idea.
Implementing it might be a bit complex... At least a week of work but the potential appears so good that I think that I am going to find the dedication needed to make that happen...
RE: lano1106 UMSv2 - lano1106 - 02-06-2022
After 32 days of stage 4, I have put aside UMS to make a 1 month DMSI v3.5 experiment. I will open a new journal to document the experience.
After the month done, I'll return to the next UMS 4 months stages cycle. I just wanted to document the end of the first cycle.
Trading activity is dead. The environment is quite different than what it was 1 year ago. January 2021 is the first month where I did finally break through the monthly trading volume of $50K.
If I recall correctly, last February was the month where DOGE did experience a major surge and did make my system profit like a thief.
In contrast this year, I have several days long trading dry spell. At most, I get a small trade per day that generate few cents of profit.
So what is the next step?
I have 2 ideas.
1. Add support to other exchanges
Idk, maybe the current exchange that I used is now sucking with my current trading strategy and could blossom elsewhere... but there is no garanty about the result
2. Implement a strategy enhancement idea that has a market making component in it instead of passively waiting on the side that some profitable trade shows up on the radar...
If this one works, it should work everywhere.
I didn't know which one to tackle first. Last week, out of the blue, An Equinox representative sent me an email to know if I wanted to discuss about some hosting opportunities with them. For those who don't know, Equinox is simply a HUGE multinational data centers operator. They operate multiple data centers everwhere across the world. I told them what I was looking for. If they can propose me some hosting very close to some cypto exchange servers, I would be interested about that.
For one, I know that Gemini is hosted in an Equinox data center. So if they ended up offering me a cool hosting spot, this would have solve my dilemma but they didn't come back to me after my last email...
So I think that if I have no obvious new exchange to target, I will explore fist the current strategy enhancement idea.
Another thing that I did push aside a lot is searching for a buyer for my digital business...
So this is pretty much the state of my trading project... I will continue working it in the next month but I'll probably won't update the journal about new development until I return back to UMS...
RE: lano1106 UMSv2 - lano1106 - 02-11-2022
here is quick update... Even if I did take a break, I believe I still have residual effects of the program.
I did a small change. In fact, I had no idea if it would have any positive effect on the system performance. My expectations were very low. It was more like an experiment to perform for the sake of completeness so that I check a mark beside this idea in the list of things that I have tried to improve the system performance.
Concretely, the idea is that inside my event loop, the thread is blocking and the kernel is setting its state to sleeping. When stuff happens, the kernel change back the thread state to running and set it back in the list of processes to schedule. Even if my thread is configured to be Real-Time and have the highest scheduling priority, simply putting the task to sleep and make a context switch to make it run again when events are available appears to have a very high overhead.
The experiment was to configure the thread to create a busy loop. It is loop without never blocking. The trade-off of doing so is that you use more the CPU to essentially doing nothing and this increase the electricity consumption of the machine... but you skip the context switch overhead when something to do happens...
I got a big surprise. That must be the best single improvement that I have done so far to improve the system reaction time. I must have reduced the reaction time by about 20% with this single change...
Next time crypto markets experience a bumpy period, I should get better execution rate with this...
this is cool!
RE: lano1106 UMSv2 - lano1106 - 03-02-2022
cycle 2. Stage 1, day1:
Program change has been felt very fast. Since 2 days, I make a lot of dreams.
Yesterday, I went to bed very late at night. I had to finish something before going to sleep because money never sleep. While I rest, my trading system can work...
I have not felt this energy, desire and motivation to work since a while. The DMSI me would have stopped much sooner and indulge myself some time for my pleasures. Not the UMS me. This one relentless and wants success.
RE: lano1106 UMSv2 - lano1106 - 03-02-2022
cycle 2. Stage 1, day2:
I think that I am experiencing some turbulence today due to the rapid switch from 1 program to the other...
Granted, I am tired because I went to bed very late yesterday... but beside that, I feel a mix of sadness and anger today... Minor setback in my project. The experiment that I have tried yesterday did not go as planned. This made my system lose few dollars... Nothing serious but due to my current state of mind... This affects me a little bit emotionally...
RE: lano1106 UMSv2 - lano1106 - 03-05-2022
cycle 2. Stage 1, day5:
Not too long after having restarted to listen UMS, a problem in my trading system did show up out of a sequence of very events occuring in a fast succession. On the surface, this could be interpreted as bad but I am not. I see that as UMS LM module manifestation. I am given the opportunity through luck to detect unsuspected bugs, correct them and make my system better.
RE: lano1106 UMSv2 - lano1106 - 03-06-2022
cycle 2. Stage 1, day6:
So far, the mentioned problem did result in 7 changes. Only 1 of them would have been sufficient to avoid the problem. All the other changes have been for adding robustness to the code for situations that should not happen if the problem would have been avoided in the first place.
I like to work with robust code. If the situation arise from some other causes, at least this part of the code will well behave and the likelihood that the system survive this unexpected situation will increase.
The problem did also invalidate some general assumption that I had about margin trading. This did not cause any problem this time but I am preemptively addressing it to avoid waiting something nasty to happen before looking into it. This change was a somehow very fundamental change in how my system core is working and despite this fact, adapting its behavior has been pretty smooth. I guess this is me benefiting from my well thought designing efforts put into it over time.
All that to say that I have done all that and this has kept me busy pretty much all the time this week-end... All that work was unplanned and did distract me away from what I was currently working on before the problem did show up in the system operation. Without the problem distraction, the idea could have been completed 2-3 days ago.
I was working on a new feature. A new idea that I got again by accident from an observation that I have made by looking at my trading results. Again, this is another unlikely event generated out of random that did help me discover some profit potential technique. Without that luck (thank UMS LM module), I would have never been aware that this possibility was even existing! I am very excited to try it out... For as long as I can remember, I did put the blame for the system poor performance on competitors using more or less the same strategy than me and reaping all the available profit on the table simply by being bigger than me and thus have access to lower trading fees allowing them to grab everything available before it even falls in my system radar.
I have found a way to trade without any fees at all. So if my explanation about poor performance is valid, I expect putting this new idea on the field could result into thousands of trades daily...
I did continue moving forward the project while I was on DMSI... but now it seems like everything is finally falling in place very fast since I have restarted using UMS...
RE: lano1106 UMSv2 - lano1106 - 03-08-2022
cycle 2. Stage 1, day7:
Last night, I have finally upgraded the server with all the new fixes/optimizations and the famous new feature. There is a good and bad news.
The good news is that my idea turn out to work perfect. I have found a way to make profit without paying any fee! My system made about 20 trades over the night using this new idea.
The bad news. The profit is 1 cent or less per trade so in order to be something more than a curiosity, the system needs to make thousands of these trades per day...
There is hope. There seems to be a glitch in the implementation stopping the system exploiting the technique at its full potential. It is applied to very specific pairs only...
I'll try to find what is wrong and maybe something beautiful will unlock...