Watcher/dev_log.txt at main · No-LAN-B/Watcher · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
Starting a lil devlog here to keep track of my progress in the project and to share some insights into the development process.
I will be updating this file regularly with new features, bugs, and other things I encounter along the way.

April 29th, 2025
- git issue regarding a dereferencing to my local head branch. I had to create a new master branch publish to the remote repo
- them merge that master branch into my remote main branch as that was the default branch for the repo and ideally id like to keep a linear history.
- I had to delete my local main branch and create a new one from the remote main branch. ie. just a simple checkout to main and sync with the remote.

# 4:52pm
- Current Work & Plans:
    - I have begun training and have been testing a variety of methods for the model.
    - I have been using a variety of datasets to train the model and have been testing the model on a varient of metrics
    - I have found that the model is not performing well with linear regression & random forest regression.
    - This makes sense as the model is not able to capture the non-linear relationships in the data like in the stock market.
    - While there is a lot of overlap in the data, What im looking for is a general UP / DOWN and the degree of that change.
    - Afterwards I will be looking to integrate a sentiment analysis model to help with the prediction of the stock market.
    - This will be done through an LLM likely a fine-tuned version of on OpenAI / Gemini / Claude model likely OpenAI as the developer API is what I am most familiar with.
    - I will then create a model that fuses these prediction lines with a heavier weight on sentiment with either a supervised learning algorithm as the data will be fairly simplistic once I quantify the sentiment analysis
    - Or I will attempt to use a neural network and take multiple inputs such as sentiment analysis, live stock data (stuff that I'll probably have to pay for), and use the neural network as a stream for other ml models
    - My current solution is to try out gradient boosting as it ties up a few different models in a nice package that I hope will be able to capture that UP / DOWN relationship.
    - I'll be using XGBoost as it is a very popular and powerful library for gradient boosting.
    - Then to fine turn the model I'll start feuture engineering a little more (adding RSI, MACD, etc) and then use a grid search to find the best hyperparameters for the model.
    - I'm curious if the RSI MACD line crossovers will be able to help the model learn the UP / DOWN relationship better. and develop an understanding of "trends" in the data.

#6:16pm
- Current Work & Plans:
    - I have been working on the model for a few hours now and XGBoost actually performed worse than other regression models.
    - I have been trying to figure out why this is the case and I just don't think that the model is able to understand the data well enough.
    - As much as data like macd or rsi are great indicators applying it to a regression model is not the best way to go about it as the model is not able to learn the relationships between the data points and develop strategies obviously.
    - Because of this were going to take a different approach. Instead of switching models ill try using XGBoost as a classifier and see if that works better. ie determing if the stock will go up or down.
    - I will then use the classifier to determine the direction of the stock and then use a regression model to determine the degree of that change if possible.
    - Once I can reliably predict the direction of the stock I will then be able to use that information and determine how I will proceed around that.

#9:01pm
- Current Work & Plans:
    - I now have a working classifier trainer for models. I'll be using this to try and determine my best choice of model for the classification task and fine tuning the model.
    - I have gotten to about 55-60% accuracy (determining Up or Down) on the S&P 500 data set which is not great but its a start.
    - I want to try fooling around with the training split something like 95-98% training and 2-5% testing. As I think more current data along with a smaller period of fluctuation will help me tune in for day to day trading which is what I am looking for.
    - I have also added a trading strategy seperate from the model more similar to a trading bot that is more of a proof of concept than anything else.
    - By creating multiple strategies the idea is to allow an "overseer" model to determine which strategy is the best for the current market conditions and then use that strategy to trade.
    - And / Or use that accuracy data along with overlaying the strategies to find a "best" strategy for the current market conditions. Like a "best of" strategy or a nested ML model.
    - Currently the MACD and RSI strategy is performing the best with a 60% accuracy on the S&P 500 data set. and a profit of ~$1000 trading 1 share at a time over a year, and over other stocks like AAPL and TSLA. the model is not performing AS well but
    - is still performing well enough to be profitable.
    - Calling it a night for now. I will be back tomorrow to continue working on the models and strategies and continuing research!

April 30th, 2025
#4:02pm
- Current Work & Plans:
    - I have been working on the model for a few hours now and I have been trying to figure out how to improve the accuracy of the model.
    - Ive decided to stick to a classification model regression just isnt in tune with multivariable stock data like this. At least for an intraday model.
    - I'm currently looking in to LSTM models as they are able to capture the relationships between the data points and develop strategies.
    - Today will be more of a research day as I want to gather as much info regarding technical approximations of data that other people have accomplished and see if I can find a way to implement that into my model and trading in general.
    - I have also been looking into the possibility of using a reinforcement learning model to determine the best strategy for the current market conditions and then use that strategy to trade.
    - Admittedly I got a little sidetracked with n8n as I was trying to see if agentic ai could be used to converse with eachother and interlink with eachother to create a more complex model.
    - This took a little longer than expected and I will be sticking to my initial approach not because it wont or didnt work but because this project is specifically focused on machine learning and I want to keep it as linear as I can.
    - Until I have a working model with my current approach or a strong understanding of the ML models ill be using and why or why not they are working.
    - This devlog is infact helping alot as it helps to get my thoughts organized and helps me to see the progress that I am making even if it is back forth back forth and not linear at all.
    - Main conclusion for today's testing and work is I'll have to implement a multiclass model that utilizes a best fit of technical models.
    - An interesting issue i ran into on my test branch that i will not be committing as I have since wiped it is that different models had the exact same up down predictions despite entirely different approaches to the data.
    - Current theory is that there is a "dominant" feature in the data that is causing the model to learn the same thing over and over again.
    - I will be looking into this more and try to instead of increasing features to decrease features and see if patterns emerge from that.
    - After of which I'll try to reoptimize.

April 30th, 2025
#11:50am
- Current Work & Plans:
    - We are up and awake bright-eyed and bushy-tailed I have a mocha on my desk and a breakfast wrap in my hand and today we are going to kill this thing... I hope :D.
    - Okay I may be a little over my head I've decided in order to get the results I want I will need to implement an LSTM model. I have very very bareboned knowledge of RNN's so this will be a learning experience for me.
    - I'll be taking a bit of a crash course and hit the books this is going to be a long day.
    - Not much work may be done today I'll probably play around with code and try little isolated projects like replicating tensorflow examples from hand written numpy code.
    - This project has jumped from basic machine learning regression to a full blown deep learning project and I am all for it. However I'll have to step back and read, read, read.


May 1st, 2025
#9:00pm
- Current Work & Plans:
    - Was busy all day today but I squeezed in a couple hours of research and I'm starting to get a solid understanding of LSTM models and how they work.
    - I have been looking into the Keras library and how to use it with Tensorflow and I think I have a solid understanding of how to implement an LSTM model.
    - I found multiple resources / examples of how to implement an LSTM model and I've considered using those examples as a skeleton for my model.
    - This way I can have my own version to play around with different parameters and see how they affect the model which I find I learn best by doing.
    - I've dug up some great online resources and I'm chugging through data science and machine learning books to reinforce my knowledge of the subject.
    - All in all I think I'm on the right track and its becoming less of a daunting task and more of a fun project.
    - Tomorrow I'll be prototyping the LSTM model and seeing how it performs on the data.
    - I will be using the S&P 500 data set as a test case and then I will be using the NVDA and TSLA data sets to see how well the model generalizes to other stocks that have had more recent "explosive" performance.

May 2nd, 2025
#5:00pm
- Current Work & Plans:
    - Taking a break day today. I am working on a professional certification and I will be focusing on that for today.

May 4th, 2025
#8:26pm
- Current Work & Plans:
    - The professional certification I'm working on is an ML AI certification and I have been working on that for the past few days.
    - I have been learning alot about deep learning supervised and unsupervised learning.
    - I'll be back doing light work here and there on the project but I will be focusing on the certification for a while.