lano1106 BASE v2.1 journal

[lano1106]

Stage 1, day 13:

I have been busy yesterday evening and today to add up code on top of what I have. My program has been running smoothly for close to 24h without any glitch. I'm very happy about that. Stability is very important for software meant to run 24/24 7 days a week....

Then out of nowhere. Boom. SEGV signal. Core dump.. no more program running. This has been very brutal. I did inspect the core dump.... My first suspicion was some heap corruption. That wasn't good news. If it takes 24h or more before manifesting itself... I could take a while before I find out the root cause.... I became obsess by this new problem. It has totally disrupted my thought process.

I went to dinner... Made some reading... I came back to the computer in the evening. I have learned about a tool to track down heap corruption named AddressSanitizer:
https://en.wikipedia.org/wiki/AddressSanitizer

It did turn out that the problem was much more simpler than that. I took a multi year break from programming that must be why but I caught myself doing junior errors. Maybe it is utopic to expect to write code, compile it the first time with no compile error, and run it and it works flawlessly on the first try. Very often, I'm very close from that utopia. but I did type of very silly errors.

1. Create a for loop with iterators but forget to increment the iterator causing an infinite loop upon entering the loop.
2. Call it mathematical dyslexia if you want but I wrote B - A when I meant A - B. The first time, I caught myself doing that, what it caused was cute. I was printing a negative time period instead of printing a positive one. but in what I did that caused the last crash. I have a time window that I start sliding when it reach 24h in length. To access the point array, I compute the index by subtracting the point time from the number of msecs from Epoch (big number). I was computing the number of second that I needed to move the point time zero. So I was doing 24h in seconds minus current time window size that is now bigger than 24h. this resulted into a negative number that ended up causing an out of bound array access.

Something that I did notice about my behavior.. It is that I naturally give myself some time to relax and enjoy myself. This must be from the script:

- Paced Working & Entrepreneurial Efforst To Steady The Approach/Drive/Motivation & Prevent Burnout

Something did happen yesterday... I was sit in front of the computer... and I was searching for the best name for a new variable. Then a diagram from a book that I must have read 15 years ago did pop into my head. I went to my bookshelf. try to find the damn book that I haven't seen for many years... I did never found it. I must have lend it to someone and the book never came back my way...

But my urge to look into the book was very persistent... I did remember what the book cover did look like... A vague memory of the title. So I went on amazon and I found the damn book... It did clicked when I saw the author picture... It looks like a funky groovy cool Indian guy... A very memorable picture. I bought the Kindle version for immediate access...

It turns out that the diagram inside the book didn't help me much for finding a good variable name.... but somehow reading again this book at this very moment was the right thing to do...

I started reading the first 2 chapters. and I made 2 realizations:

1. With the background and experience that I now have today... Reading the book now makes much more sense than when I was total finance newbie 15 years ago.
2. What is currently in my program current implementation/plan is documented as being the standard industry practices... that is:

trend following
mean reversal
and technical sentiment...

So that was very comforting that the simple design that I had in mind and wrote on the back of a napkin is actually what the bests are also doing....

I keep my program running in one of my monitors (I have 2). there is the hope that having it live in my face very often, it will allow me to confirm intuitions that I have and observe new market phenomenons that I previously didn't realize were present. It did happen this afternoon or yesterday evening. When something very specific happens, in all occurrences that I have witnessed, the market go in the predicted direction. When I have these ah ah moments, I write those ideas down. That is the best way to not forget good ideas.

As I'm adding more windows to the program, I'm entertaining the idea to have a 4-6 monitors setup with information popping in my face giving a good feeling of the market as I work... This sounds cool to me... Some are dreaming of possessing a powerful gaming rig... My thing is some weird nerdporn fantasy. 6 30 inches 4K monitors mounted on a wall... This insane dream is going add extra motivation...

Another weird desire... I want to purchase some powerful bitcoin miner hardware. My office is in the basement. It is winter and it is too cool for me in here... I could turn up the heater but why not purchasing a heater making money while it is heating the room.... This will be another purchase once bitcoins start to come in....

[lano1106]

Stage 1, day 14:

plugging the trading black box with live data is a little bit longer than expected.

Saturday evening, I draw on the back of a sheet a small Object-Oriented design showing all the various components. Lets call this methodology divide and conquer. I did wrap the concept that I did discover while writing the prototype into nice objects. It makes the code easier to understand and extensible by having created abstractions and is hiding a lot of small details.

The only remaining task will be to write a derived class that simply add markers on the graph to show where and when the black box is opening and closing trading positions...

Around dinner time, I got another seg fault. This one was satisfying to solve. It did happen in one of the oldest module that I created for that project. The header comment says that I started it on December 7 last month.

I thought that it was finally 100% correct but I was wrong. I made some assumptions about the input coming from a WebSocket that proved to be incorrect. It could be some corner case situation that may happen maybe once per week or maybe less often. I'm quite happy to have seen it. It did allow me to modify the function in which the problem did occur and I believe that the end result is a better function. This function is very important in the system. It must be called any number of times between 15 to 30 times per second.

The fix is in and running for the last hour without any new problems.

Bitcoin trading is busy tonight. At the end of the afternoon and for the most part of today, the market was pretty quiet with the price at around $8100.

Now in a matter of 1 hour or less, the price did bump up by at least $300.... This makes me even more eager to complete my first operational system ASAP....

Last time bitcoin price has been that volatile was when Iran sent its missiles.... I wonder what is triggering this activity tonight... I haven't seen any news possibly explaining the current move...

Update: I did some quick math about my goal. Preliminary results is showing that a realist yield might be around 0.75%. In my simulation, I used a $200 trading budget. That is probably what I'll use when the system goes live until my confidence level into the viability in the system increase.

With that budget, as I wrote somewhere on page 2, I think, this gives peanuts. Few bucks at the end of the day.... That is a lot of work for just few bucks...

That being said, I did some math. What would be the required daily volume at 0.75% yield to earn $10K per day....

It is north of $100K... That sounds huge, but when you put it in BTC, it is just 10 trades with one coin.... Said that way, this sounds totally doable!

[lano1106]

Stage 1, day 17:

The observation about the cool new signal that I have observed was basically big orders on the book coming out of nowhere. I'll need to automate and collect stats about the phenomenon but I haven't seen it happen much yesterday... In retrospect having witnesses those big moves just hours before a major breakout, it seems like some big players knew what was about to come... I'm not judging... just taking note... Next time, I see it, I'll know better how to interpret it.

I did finish the task that I refer to in my last post. I did finish it on Day 15 instead of in the evening on Day 14. I added some bell and whistle such as the program playing a wav file when opening and closing position with a different sound associated to profit or loss when closing the position. When the program was starting to make some noise, it was quite exciting.

But at first, the timing did look random. You know the simple principle: Buy low, sell high... It did seem that the program was doing exactly the opposite.

I spent the whole day 16 for a major rewrite of the rules of the black box. Yesterday evening, it was making moves that were making much more sense. A great source of inspiration for this last refactoring come in great part from the book that I did pick Saturday evening after having a flash of having read it more than 15 years ago. If THAT isn't subconscious helping doing the right thing, I don't know what is...

I woke up this morning. Open my computer only to find my software having coredumped at 6:35AM this morning. No big deal, by simply opening the core dump, the problem was trivial to identify. It is going to take 5 minutes to fix and it will never happen again.

The funny thing is the crash did occur while closing a position. The details of the transaction have been added to the log. That is the very last thing the program did before crashing. AFAIK, it is the best performing transaction performed by my creation so far. A 0.53% yield! Woohoo! A profit of one virtual dollar.... Definitely if this thing is to be sustainable, it will require a LOT of flesh on the table.... Right now, the thought is a bit scary...

3 more random thoughts for today.

1. I did notice that 2-3 seconds before the signal that indicates me to close a position, the price was sharply dropping. This tells me I am not alone monitoring the same signals and this is essentially a game of frontrunning the competition... TBH, I don't fully understand the whole situation yet. I'll need to add some few wise traces to gain a better insight of what is going on but that is one possible lead for improvement.

2. Timing now seems potent enough to be usable. What needs to be added is some risk/reward signal. Due to the transaction fees, the required swing to just break even needs to be accounted. Opportunities that simply don't appears to have a great potential should simply be passed.

3. I love how my mind is attracted to important tasks to achieve the goal rather than letting itself being distracted by interesting but non-essential ideas... That is something that did hurt me a lot in past projects... Possibly something BASE is doing for me...

[lano1106]

Stage 1, day 18:

I thought that last time that I did wrote in my journal was few days ago but it seems like it was only yesterday!

I have the feeling that so much thing did happen.

I have added 3 more improvements/features to my software. On top of still tweaking the core of it... It is a very sensible piece of code. Not yet fully understood... And there is, of course a lot of variability in the input...

I tweak something to improve the reaction to some input and this creates some regression and other issues somewhere else... Sometimes, the system simply becomes unreactive or is too much trigger happy.. Neither are good... Slowly but surely things are improving....

I'm stunned to see how useful BASE is... I feel motivated... productive... focused... creativity is peaking... ideas about what to do next flow and appears naturally.

There is a price however for those nice benefits. Yesterday after dinner, I was feeling mentally exhausted.... I also feel some stress that manifest as muscular tension in my upper back... I guess that as long those are under control... A little bit of them is a good thing...

[ianmarconi]

Hey man, I am also using BASE ( day 14 stage 1) and I am loving it. I can see how my whole being is being directed to the entrepreneurial route, I am constantly thinking and developing my ideas around my life purpose and how to materialize it and turn it into my career. It is like slowly throwing away of every bullshit and petty thoughts and focusing on what really matters.

[lano1106]

Stage 1, day 21:

Saturday evening, I was invited for dinner at a friend's place. Alcohol did flow a lot. This kinda ruined my Sunday productivity. I guess that as long as it isn't a regular occurence, once in a while, it is ok.

Sunday morning while I was asleep (around 6AM), bitcoin price did drop by about $400 in less than 5 minutes. My simulation was running and I did discover that a piece of code called 'loss stop' was instead a 'profit stop'. Program did profit from the drop but not as much as without the 'loss stop'. It kept exiting the trade at the smallest bump where it would have stayed in without the 'loss stop' module. This is very expensive problem because every time you exit/reenter, you get shaved by transaction fees.

I had a small moment of despair yesterday. This project scope is huge. To have a working profitable system does require a lot of time. This makes me neglect my marketing biz a little bit. Giving sufficient time to both at the same time is simply impossible. And given the current system state... I'm not seeing it working very soon. And my profit/loss sound system start to hit on my nerve as the loss sound is heard more often than the profit one...

It works decently by almost always buying low and selling higher but at the end of the day, I'm at loss due to the fees...

I have few ideas to address this problem:

1. Add a filtering method based on risk/reward (I mentioned this idea few days ago). This would work by considering the current trading channel to filter out signals that have low potential. This idea is still a bit fuzzy. This will require some research... There is a lot of unknown on this about how long it will take to develop. Currently my system does way too much trades per day. It is in the range of 100s of trades daily instead of roughly 10-20 ideally.
2. Analyze if the fee discount given with higher volume would make my trades positive. Maybe I can tolerate some pain in order to reach a volume that will allow to profit.
3. Zoom out my timeframe. There is chance instead of looking at seconds, I was looking at minutes or hours. The system could be profitable as-is. When I look at a 24h graph, my intuition tells me that it is the case...

The path to use to reach the goal isn't 100% clear right now... but I know that I have many building blocks to build... So I'll unroll my sleeves. Keep going and I hope that the roadmap will get clearer as I go...

[Omni3]

I'm watching your bot updates

[lano1106]

I did something pretty cool today. The crypto exchange offers to use 2FA (aka Google Authenticator) when using their API.

Of course, you don't want to sit in front of the computer all day to enter the 6 digits code whenever the bot wants to make an API call.

I did some research and found out that Google Authenticator is essentially just an implementation of a 10 years old standard Internet protocol:

https://tools.ietf.org/html/rfc6238

A decent (but not working) implementation can be found at:
https://github.com/jamesliu96/totp

With that done, it is pretty cool, I can feed automatically 2FA codes from software. I have no need for Google Authenticator anymore to use 2FA. As an extra benefit, you can tweak the encryption used.

By default Google app use the SHA1 20 bytes digest to generate a 6 digits code. I did beef up the crypto to use SHA512 64 bytes long digest to generate a 8 digits code.

It works flawlessly... I can go to bed with a smile... I had fun day today...

[lano1106]

Stage 1, day 26:

I spent almost the last week tweak the system parameters. The system behavior did oscillate between oversensitive to totally nonreactive...

It is very frustrating state of things but I feel like I'm getting close to break the dam wall...

I was thinking putting aside the trend following strategy aside to explore a simpler arbitrage strategy... but I cannot let go the trend following system as I keep having new ideas to have it work...

What it took me some time to realize (and really integrate) is that market conditions widely changes and parameters tweaking that would work well yesterday may miserably fail today or tomorrow... You have clearly trending markets, you have quiet, low volume flat market and you have widely oscillating market that goes nowhere in a tight channel. It is the latter that cause me the most trouble and frustration. It is one thing to observe some quality by looking at a graph. It is quite another to detect the pattern computationally... but I'm slowly getting there...

Right now I'm only simulating the BTC/USD market... The dream is once, I have a working system, even if it generates pennies. It is not harder to trade on the 130+ available trading assets. Therefore, it is the multiplying capability that the machine is offering that is simply not possible for a human.

[lano1106]

I went to the shopping mall this afternoon... I found myself looking at expensive gadgets and seeing myself owning them in a near future.

One of those items was a LG gaming monitor. The price tag is 1.7K not that tall about a feet high. but it is curved and 48 inch wide.... I'm dreaming building a wall of those. At 4 of them mounted on wall....

I guess this looks like a little bit to the crazy things shown in Kol video...

Concerning work related stories... A new idea to solve my current problems with market trend following is to use some computational geometry old code of mine.

I wrote some Voronoi diagram solver from a set of points back in 1998 when I was at university. Voronoi diagran was considered bach then as one of hardest and complex algorithm to implement when I was reading about it 20 years ago...

idk, my subconscious mind did give me this inspiration out of the blue to dig out this 20 years old code...

oh and you know the old book that I was looking like crazy the other week-end and I ended up buying the kindle version of it because I couldn't find it... I had flash... There was a moving box laying down in the basement that they gave me to pack my stuff 8 years ago when I have been fired from Morgan Stanley... idk why, the knowledge that the book might be in that box that I haven't reopened for 8 years did come out in my mind this morning out of knowhere this morning..

Since I have started BASE, this out of knowhere thoughts events are occuring very often... I'm assuming that this is my subconscious collaborating the best it can to assist me achieving my success goals...

[lano1106]

Stage 1, Day 29:

I'm pulling the plug on the channel tracking code... It is a complex solution to a problem that could be solved in a much simpler way.

I came to the conclusion that doing too much to front run signals and not prematurely exit positions simply result into unexpected result at least 50% of the time which result into losses at the end of the day.

Conclusion: just tweak the Moving Averages period values to something that give a sensible result and be done with all the shenanigans that adds complexity with no benefits.

One new addition to the system is tracking my trading volume. I used to hardcode the base fee. The first discount happens when you reach $50K in volume. It seems like a very far away target to reach with a $200 trading budget but I have been surprised that if anything and not profitable yet... I was moving a lot of volume. Making roughly 100 trades per day and losing about 5% of my capital (so 10 bucks), I was creating about 20K in volume... So this is a not so bad baseline...

So, I had some trade_signal_gen class from which I ripped its core into a new class called trade_strategy. I'm going to make a new round of refactoring. Extract all the stuff that I'm going to keep from trade_strategy and put it into a new base class called trade_strategy_base and derived a new class called trade_strategy_simple where I'm just going to rewrite the algo without all the extra experimental junk that did turn out to not work well at all...

My sleep cycle is a bit screwed and I didn't expect that it could be possible with the stage 1 'Maximum Sleep Quality' script... I work till the middle of the night... tbh, toward the end of the night, I'm not that productive... and this is messing my energy level in the morning... The urgency feeling that I impose on myself makes me try to finish stuff before going to bed... but overall... this seems to reduce my productivity instead...

[lano1106]

Stage 1, day 30:

Trading markets are maddening. Any small bumps in the price was making the moving averages 2 short crossovers. Just enough to make my system exit its position before continuing the same trend once I exited the positions.

The solution to that was to increase the moving average period to require more than a small bump to trigger a signal. The trade off to that solution is that the system has become slower to respond.

The price just dropped by more than $100 in about 4 minutes. When my system gave the signal to sell, the price was already at the bottom...

It is not that my moving averages new values aren't good. Simply that market conditions have changed and currently trend following strategy isn't appropriate... I'm going to crack the code eventually...

Every assumptions that I have made so far are getting discredited by current observations. I was calculating some technical Sentiment attempting to quantify the market mood. Whether it is more bearish or bullish... So right now, it doesn't work at all... When my technical sentiment measurement says that the market is bearish, the price goes up.... and vice-versa...

On the bright side, when I observe some patterns in my graph, I start to grasp the psychology behind what I am observing... I guess this insight is critical to success as I'm not observing a natural phenomenon such as planets moving in the solar system following rigorous physical equations... I'm looking at market participants having feelings and taking moves that are influenced by the herd mentality.

I quickly learned about a small tool called graphviz. It draws graphs from a small spec file. I wanted to visualize the interconnection of all the markets available on my exchange. It has 39 supported currencies with close to 140 supported pairs (ie: market)... Without any visualization help, it is close to impossible to have mental picture of what the interconnection looks like. I'm far from having become a graphviz expert, but my first result is interesting enough to be worthwhile for sharing...

[lano1106]

Here is a trader having a YT channel



Pay attention to the monitor behind him... The LG 49 monitor is even more monstruous....

Somewhere on the salespage there is a 2 monitor setup showcase... Just imagine stacking 4 of those $1.7K babies one on top of the other....

This is real geek porn....

https://www.lg.com/ca_en/desktop-monitors/lg-49WL95C-W

[lano1106]

I am currently going through the list of scripts included in BASE to see what the difference was between stage 1 and stage 2. Here are some random thoughts:

1. Ultra Success is still in Stage 2 but not Luck Magnifier... I wonder why...
2. Improved Communication added
3. Charismatic Presence added
4. Successful Negotiator added
5. Tempered Gratitude added
6. Networking Skills added
7. Always Let Your Market Be Your Guide (hehe, in my situation, market has a different meaning than the one Shannon intended...) added
8. "Have The Courage To Ask For/Expect/Demand Fair Compensation For The Value You Bring To Market" added - This one used to be a challenge for me when I was in the job market that I have eventually overcome...
9. "Learn How To Reasonably & Objectively Determine Fair Value For Your Product/Service Value" added
10. Effective & Successful Competition Where Necessary: Super interesting as success in my new niche is pure and raw competition with others in a zero sum game...
11. Only Compete Where Necessary & Use The Competition Or Lack Thereof To Your Advantage added: This one too is super interesting
12. Protect Your assets added: This one is going to be crucial when I'm going to transition from simulation to the real thing...
13. Become The Best Of the Best In Your Field added: Super interesting. That is my ambition for doing BASE. And I knew that my current professional transition is in order to better leverage my best skills.
14. Express Success In All Ways, Even Before You Have Achieved It added: I LOVE this one. Purchasing a new crazy ass computer system that is as expensive as a car is something that I would do if I was successful to the level that I dream to reach... I'll say no more put just plant that seed in my mind and see what happens...
17. Get Out Of Debt is still in stage 2 and as a review, I feel like my finance has improved effortlessly during stage 1!
18. I didn't notice it the last time I went through this list the first time one month ago but the script 'Dream Your Best Solution' is interesting. I wonder if it is the one behind all those intuitions/flash/memories coming out in my conscious mind from nowhere...
19. Disconnect From Distraction: Still another script that was in stage 1 that is still present in stage 2: Now that I think about it, I had a Facebook tab alway open that was distracting me all the time with all the freaking distracting notifications. I decided to remove myself from a FB group that was generating a lot of notifications and FB tab is now permanently closed at all time...

So in summary, Stage 2 contains 75 different scripts. 13 added over stage 1 and only 1 removed.

This is increasing even more my amazement that it is possible to pack so much subliminal instructions in only 1 hour of audio...
Going through the exercise to reread the program scripts one by one did make me realize that the program has influenced me more than I have been aware of!

Weak points that I think are important to focus on to improve them during stage 2 are pretty much still the same ones than last month. That is:

1. Ultra Motivation
2. Ultra Success
3. Overcoming Guilt, Shame & Fear - I don't feel that I need it but there might be something hidden since despite all my efforts for the last few years, I haven't still reached my goal yet...
4. Maximum Sleep Quality
5. End Clutter & Get Organized
6. Get Out Of Debt
7. Prioritize Effectively/Mindfulness Of How You Are Spending Your Time
8. Paced Working & Entrepreneurial Efforts To Steady The Approach/Drive/Motivation & Prevent Burnout

Scripts that I am curious/eager to experiment:
1. Ego Balancing
2. Extreme Self Esteem
3. Become The Best Of the Best In Your Field
4. Express Success In All Ways, Even Before You Have Achieved It

I'm very happy of results so far now that I'm getting close to complete stage 1... Very curious to see what Stage 2 will bring...

It seems like stage 2 is very similar to stage 1 except with MORE stuff in it...