lano1106 OF 5.75G journal

[lano1106]

Cycle #6, day 1:

I made a small trip with family. I made a very particular observation during the trip. I exhibit a behavior that is against the current social norm and not only I am unafraid of what others may think, I do receive absolutely zero negative energy while others that do that same behavior, they are saying that they receive aggressive reaction in return from others.

The explanation must be a mix of my expectations and what DRS is doing for protecting me.

Last night, OF has been very naughty with me by mixing probably 2 of my biggest fears together. I have a homework to return to some professor that was late AND, I had to receive about a dozen of attractive women into my home office which is kinda messy because I never receive no one in it. I was feeling ashamed to let them see my working environment that isn't meant to be shared with no one and TBH, it is mess for that reason.

I did enjoy what OF and/or my subconscious mind were trying to communicate with me.

During my trip, my trading system did continue to run unsupervised and with its new slightly increased budget. It did lose some money. I have just discovered that a recent fix did in fact introduced a new problem as well. Regression is part of the game. Fortunetely, I did catch it this morning and this will stop the fund bleeding...

[lano1106]

cycle #6, day 2:

only 2 trades this morning. It has been a very quiet day trading wise...

I did a very small but clever fix on the trading client side for a performance issue. The issue show up after having run for 24h so I will see tomorrow if it is effective.

This fix is good news as I have been dragging this issue on my TODO list for some time.

[lano1106]

cycle #6, day 3:

I woke in the middle of the night to go to the restroom. I remember having paused OF playback before leaving but I forgot to resume it when I came back to bed.

I realize what did happen this morning. I'm currently finishing day #3 playback. I am very tired today.

The trading system went crazy this night. It executed 8 trade plans with 33 orders. The majority of them were profitable but 2 of them resulted in major losses. -6 and -4%. Ouch.

Those execution did allow me to spot a bug in the code and also refine my trading rules to not go after rotten opportunities.

So because, I'm learning something from the experience, it has been positive.

And my 30 days trading volume is now above $27K!

[lano1106]

cycle 6, day 4:

I feel like my goals are getting closer to completion.

My system did 1 trade this morning. Trading volume has shrinked a little bit compared to yesterday. I'm currently at $26.9K.

I'm currently putting the finishing touch on my engine test application that is using the newly developped testing framework consisting of a bunch of abstract interfaces allowing me to replace the real thing with some test mock-up objects. This is another huge task that was heavy on my shoulders. I feel like I may have procrastinated a bit its realization by working on some other issue instead of attacking this one.

The engine code did change so much in the last 2 months but I'm not even sure that I'll be able to reproduce the recorded mysterious error that I have seen in late June. It is very possible that I got rid of that problem without even been aware of it. Nonetheless, the testing framework is a feature that did force me to improve the code quality. So in the end, it was a good move. It will be useful. If not now, later in the future.

[lano1106]

Cycle #6, day 5:

I am done with my engine testing program. I haven't been able to reproduce the weird problem that I have seen back in June that I couldn't solve back then because of insufficient info.

It seems like I have fixed the issue after having observed different manifestations of the same problem.

Having completed this task is removing a big burden from my shoulders. This completion is marking the return to fun and important stuff that I did put on hold to do the testing program.

In terms of trading, since yesterday morning, there has been no trading at all. There has been few false starts. If the system cannot buy the first book entry of a trade plan with a MARKET order because it doesn't stay there long enough for me to grab it (ie: less than 150ms), this abort the execution of the whole trade plan. I start to notice a recurring pattern in the trading that my system is doing. Very often trading happens very late in the evening and in the middle of the night which must correspond to the opening of the Asian and European markets.

Another pattern that I have discovered it is that very often, the day after having gone to the gym, I'm waking up very tired the morning after. This is what did happen today. I'm starting to feel better. I guess that I could have gone to the gym this afternoon but I felt something sore in my right arm and my guts are telling me to rest my arm at least today to not transform this mild pain into a more serious injury.

This energy pattern associated with my workout is probably not a permanent state but this is how it felt this week and maybe last week too.

Update:
I did pursue the execution engine refactoring by encapsulating an extra 314 lines of code into a small utility classes. More to come!

[lano1106]

Cycle #6, day 6:

Before going to bed, I did restart my trading server with the latest code changes in it. It did 1 profitable trade about 2 hours after the restart. Prior to the restart, it has been running for possibly at least since last week-end and it didn't trade for the last 2 days. This anecdote makes me wonder if there is something in the internal data that decays with time that makes harder for the system to launch a trade or if it is only a strange coincidence...

I'll need to find a way to validate this thought or find out the problem and fix it...

Despite having generated roughly $450 trading volume today. I'm still at around 27K for the last 30 days trading volume due to my low volume in the last few days... I wonder if the 50K milestone will get closer or farther in the coming days...

[lano1106]

Cycle #6, day 1 Off:

I did pursue my execution engine refactoring. I created a new class to encapsulate a core functionality of the engine in it. This did allow me to rip off an extra 500 lines from the execution engine main file. I think that before the refactoring it was 2000 lines long and I am now down to about 1,200.

This is introducing a very minor performance cost as I am creating communication interfaces between the various sub-components but the code is becoming so much cleaner. Easier to navigate and read. This is going to pay off in terms of maintainability and flexibility when the time will come to add features to it.

Update: Another round of refactoring is completed. I created a 4th utility class to encapsulate another execution engine sub-component. This has remove another 400 lines from the execution engine main file.

It is getting pretty slick. Just for the kick, I did return to my BASE journal. At the start of March, I had 40K lines of code written for my trading system. At the end of that month, I was up to 50K. I have written that in the first 2 months of the project, I wrote 40K lines.

I just checked how many lines there was currently in the current code baseline. It is now up to close to 64K lines. It seems like the pace has slowed down but this is probably ok. It is not the only metric to measure productivity. A lot of time has been spent to improve and fix existing code. 15K new lines in the last 5 months but a system that runs pretty smoothly for few weeks now... If only I could have some crazy volatility to trigger my system...

[lano1106]

Cycle #6, day 2 Off:

I made a strange dream this morning. I thwarted a blackmail scheme that was setup on me...

Update:
I hae broke down one of the newly created utility class into a base class and derived class.

This is the first step into having a simple and dumb execution engine in my client and an advanced execution engine in the server sharing the same codebase.

This part of refactoring existing code into a nifty, flexible and powerful Object Oriented design is something that I extract a lot of enjoyment from.

[lano1106]

Cycle #6, day off 3:

During last night, trading system has been very active. It made over 20 trades. Volatility around GNO that did gain over 40% during that time has been the main driver for all this activity. It hasn't been a good session. It lost about 5% of its budget.

30 days trading volume is now at $28.5K

I'm trying to understand what went wrong. First impression is that I could fingerpoint the latest improvements added in the algo but this isn't it.

My system got fooled by price spikes that made the price move by 2+% instantaneously but because that market (mostly GNO) is low-volume, the spike that was nothing else than thin air did disappear as fast as it came.

I must definitely include more variables into the trading decisions such as spread size, trading volume and the 24h price change to protect the system against those traps.

I'm about to complete what might be the last execution engine refactoring for some time... I could have finished doing it yesterday evening but I got distracted by all the market activities done by my system...

[RTBoss]

That's cool you've programmed your own trading system. Have you ever used or considered something like Pionex, where you run a bot with either their built-in AI algorithm or your own manual settings?

[lano1106]

RTBoss,

no I didn't. I guess Pionex might be a good platform for creating some prototypes for the sake of proof of concept.

but my thing is creating high performance low latency C/C++ linux software. One of my former position was being a developer at Morgan Stanley. building a trading system has been a pet project for a very long time and I decided to finally realize it last November just before going through BASE.

I feel like the field that I want to play in wouldn't be serve that well with an easy to use platform. I'm not just creating a bot that only generates buy and sell signals. A big chunk of my system is the execution engine (which I spent the last few days to improve).

TBH, I didn't look what Pionex offer but I would be surprised to learn that they manage execution as flexibly as I'm able to do.

My execution engine allows to almost exclusively use maker fees (there are few situations where I still want the taker fee). I hand it a trade that I want to make and it will perform optimal execution with active management giving most of the time maker fees.

Update:
Tonight cycle 7 is starting and I'm very eager to start the first cycle playback...

Update2:
Want to see what a pump and dump looks like?

Here is one:


The early signs are the several comb like spikes before the pump. It is the pumpers that accumulate big positions before pumping the currency. Because of the low volume of this pair, this is what create the comb like spikes.

By looking at the time the pump started, I would think that they based in Europe.

Those early pump spikes are what did trigger my system. I'll be more careful about those fake signals... but actually, it should be pretty easy to detect those pump and dump and profit from them as well...

I'll look into this idea...

[lano1106]

Cycle #7, day 1:

A new cycle is starting. While OF playback was on during my sleep, I slept like a rock. I was very tired when I went to bed last night.

I have not much to say today except that I'm starting to get concrete results as an indication that I'm approaching some of my goals...

It is pretty cool to think that maybe fears were holding me from success and that I am in the process of liberating myself from those blocks.

Next milestone that I'm looking forward is the completion of cycle 9 which will mark the third OF run being completed...

[lano1106]

Cycle #7, day 2:

For a second night in a row, I'm having a very deep and recovering sleep.

Something have changed either in my algo or in market conditions but it did stop from being profitable for about a week.

Since yesterday, 2 trades have occured and both have resulted into a loss.

Update:
I was in the process of revisiting old code that I wrote last December that creates pair markets stats and I have found out 2 errors that may have in impact in the trading decisions. I'm going to include the fixes into the server and see if the trading outcome/volume is any different with those fixes in ...

[lano1106]

Cycle #7, day 3:

Funny thing, during my sleep during day #2 playback, I must have gone to the restroom during the night. What I do in those time, is to pause the program playback to resume it when I come back.

I must have been sleepwalking or close to it but when I came back to bed, I forgot to restart the playback. I did realize that when I wanted to start playback for day 3 and see that I didn't complete the playlist. I missed about 1.5 loop on day 2.

I did complete day #2 playback followed by the day 3 6 loops. I guess that it is not too big a deal to have listened 7.5 loops on day 3....

Yesterday has been very productive. Having put in place a solid framework is finally starting to pay off.
I wanted to add a new variable to the collected market stats for each pair.

I did that on both the server and client side.

Yesterday morning, I was looking at what I could add and there was 3 candidates. At first, I was a bit uncertain about which candidate should be handled first. I must have done some mental judo with myself for an hour. I'm not exactly sure which criteria made me jump the fence for 1 candidate more than the others but this is what I have done.

The result:

The bad news is:
The last 2 trades have been negative. The current sequence of losing trades are now up to 5.

The good news is:
- It is always the same pattern that makes me go after a bad trade and the newly added variable seems to be what I need to detect and avoid falling for the trap.
- Execution is now flawless. I remember few months ago... I was complaining about execution halting midway because of some random glitches.... It seems to not happen anymore...

It is weird that this is happening now. I have been using my system for months and I have never experienced that issue so much. It must be some new market participant that has some weird algo messing with my own system...