My First Semester at Georgia Tech

This month, I finished my first semester of the OMCS program at Georgia Tech. I have focused my degree to have a machine learning specialization and my first two classes were Machine Learning for Trading (ML4T) and Software Architecture and Design CS 6310.

My Machine Learning for Trading Class focused on building a simulated trading system incrementally through the use of machine learning. I was assigned 8 projects throughout the class that helped me building this trading system while covering topics such as: market simulation, portfolio optimization, technical indicators, supervised learning, and reinforcement learning. Taking this class showed me the realistic capabilities of machine learning in the field of trading and understanding its limits in financial markets.

Project 1: Martingale:

This project involved simulating an American Roulette Wheel to show us how the odds are set up for the House to always win. We are given the following betting strategy to implement:

• Start with 1 dollar and betting on Black only
• If you win, keep betting 1 dollar, if you lose double your bet
• Don’t stop playing until you win 80 dollars.

With this strategy I ran two experiments. The first experiment ran this strategy with an unlimited bankroll which meant you can keep betting forever until you win 80 dollars. This experiment’s expected value and probability had made it seem like a great strategy. In the second experiment, a bankroll was added to make it more realistic so that the person cannot keep betting if they keep losing forever. This experiment’s expected value and probability had showed that in the real world this scenario was very risky and not profitable. Thus through the expected value I learned that this strategy can still lose money long term even if I “win most of the time” which is just what the casinos want you to focus on.

Project 2: Optimize Something:

In this project I was exposed to metrics to evaluate portfolios such as:

• Cumulative Return: total gain or loss of portfolio.
• Average Daily Return: mean of portfolio’s daily returns.
• Volatility(std of daily returns): measure of how much a portfolio’s daily returns fluctuate.
• Sharpe Ratio (return relative to risk): risk adjusted metric to measure how much return a portfolio gets per unit of volatility or risk.

I got to use Scipy’s minimize function to minimize the negative Sharpe Ratio and return the portfolio statistics above to find out the optimal stock allocations or weights from a set of givens stocks to maximize returns.

Project 3: Assess Learners

This project focused on supervised machine learning. I implemented the following supervised learners from scratch:

• Decision Tree learner: makes decisions by splitting data based on feature values. Easy to understand but can overfit and become too specific to training data.
• Random Tree learner: works the same as a decision tree but adds randomness when choosing which feature values to split the data on to help prevent overfitting.
• Bag Learner: trains the same model on slightly different variations of data and averages the results to get a more general result.
• Insane Learner: Combines a bunch of bag learners together and takes their average to further reduce variance.

This project helped me understand the concept of overfitting and how a model can perform amazing on training data but then do poorly on out-of-sample data because it failed to properly generalize and pick up on the patterns in the data. I tested to see if leaf size is related to overfitting using the RMSE metric. I also got to see how using ensemble methods like bagging can reduce overfitting.

Project 4: Defeat Learners

This project built on project 3 and I had to try to change up the datasets to be able to defeat one of my learners. Here I got to see the strengths and weaknesses of different supervised learning models. I created:

• A dataset where linear regression outperformed the decision trees
• A dataset where decision trees consistently outperformed linear regression

Through this project I learned that there is no single best performing learned and that model performance relies on the data being used.

Project 5: MarketSim

Here I got to build a market simulator that keeps track of how much money the portfolio given makes over time. This simulation took into account:

• Holdings and cash
• Portfolio value computation
• Commission and market impact

I was given a list of stocks and their orders and had to calculate how much money is accumulated at the end as shown below: Input:

# Input
Date,Symbol,Order,Shares 
2008-12-3,AAPL,BUY,130 
2008-12-8,AAPL,SELL,130 
2008-12-5,IBM,BUY,50

Output:

# Output
2008-12-3 1000000 
2008-12-4 999418.90 
2008-12-5 999754.30

Project 6: Indicator Evaluation

In project 6 I was introduced to and implemented the following 5 technical indicators:

• Bollinger Bands: measures how for stock price deviates from its moving average using standard deviation.
• Stochastic Oscillator: Compares current stock price to its recent price range to indicate momentum and potential price reversals.
• MACD: Looks at difference between two moving averages to identify changes in trend direction and momentum.
• Simple Moving Average (SMA): Calculates average price over fixed window to identify changes in trend direction and momentum.
• Momentum: Measures rate of change in price over time to indicate strength and direction of trend.

These technical indicators were then visualized with graphs and used to determine the overbought and oversold conditions (short or long signals) for each indicator. To have an upper bound I implemented a Theoretically Optimal Strategy that assumes knowledge of future prices and always takes the correct short or long position. This strategy was evaluated against a benchmark which was buying and holding shares of a stock for the entire trading period.

Project 7: QLearning Robot

In this project I got to implement a Q-Learner from scratch and got a deeper understanding of reinforcement learning. This Q Learner learns through finding out the optimal action to take through trial and error with its environment and updating its Q table with the observed rewards and state transitions. I also learned and implemented a technique called Dyna-Q to improve learning speed by allowing the learner to simulate experiences to better accelerate convergence. This project showed me the importance between exploration and exploitation and how there needs to be a balance between the learner taking actions which it already knows the outcome/reward to versus taking new unexplored actions.

Project 8: Strategy Evaluation

For this final project I got to implement what I learned in the previous projects to create a stock trading bot. I implemented the following 2 strategies:

• Manual Strategy — a manual trading strategy based on 3 indicators of my choosing with manual thresholds on when to buy and sell for each indicator
• Learner Strategy — Here I used my Q learner to use my 3 chosen indicators to learn the best trading strategy.

On in sample data my strategy learner was able to outperform the manual and benchmark but when tested with out of sample data, my strategy learner did poorly which is shows that it overfitted to the training data and was not able to generalize properly.

Taking this Machine Learning for Trading class introduced be to the different supervised learning and reinforcement learning algorithms, how they work, and more importantly, when and how to use each one for real world applications such as trading.

My second class: Software Architecture and Design, helped me think beyond writing code, about designing systems that can scale and withstand change. Here I analyzed why systems fail, where complexity comes from and how design decisions early on can prevent costly problems later.

I was taught to slow down and ask these questions when thinking about implementing a system:

• What are the core responsibilities of this system?
• Which components interact with each other and which are isolated?
• How accepting to change will this system be as requirements evolve?

This helped me think beyond files and functions and instead think in terms of modular components each with their own responsibilities and interactions.

Through this course I was introduced to UML (Unified Modeling Language). This language provides a visual representation of how a software system is structured and how its parts interact. I got hands on experience with class and sequence diagrams when I had to design a pokemon battle simulation. The class diagrams helped me take verbal descriptions of a system and be able to identify and categorize these requirements into classes while the sequence diagrams helped me visualize all the possible flows of the system to get a thorough understanding.

I learned about key system design concepts such as:

• Single Responsibility Principle: making sure each class only does one thing.
• Coupling: how much components depend on each other.
• Cohesion: degree to which components work together to fulfill a purpose

and got to put these concepts to use through peer reviewing my fellow classmate’s submissions. This taught me how to spot inefficiencies in a system’s design such as spotting tight coupling, lack of modularity, and being able to explain why a system’s design is strong or weak.

I got to learn about the following architectural styles as well:

• Layered Architectures: a system with layers where each layer has a clear responsibility and only depends on the layer below it.
• Object-Oriented Systems: software with interacting objects that combine data and behavior and promote encapsulation, modular design and code reuse.
• Pipe-and-Filter: process in which data is passed through a sequence of independent steps otherwise known as filters, where each stage transforms the data and passes it onto the next filter.
• Event-Driven Systems: Components communicating by emitting and responding to events at their own time allowing the system to be loosely coupled and react asynchronously.

At the end of this class, I was left with these key takeaways:

• Design decisions matter most before code implementation
• A good system design makes change easier not harder
• Extracting requirements/diagraming out the system and having discussions regarding architecture are paramount to creating an efficient scalable system.

Both these classes have taught me a lot and I can’t wait to get started on my next semester where I will be taking Machine Learning and Human Computer Interaction!