Cross-Sectional Momentum in Algo Trading — underdogs of Machine Learning Approaches — Part 1

This article is one of a multi-part series on a failure-aware study of LambdaMART and Neural LambdaRANK applied to the time-tested cross-sectional momentum algo trading strategy.

This first part outlines the lay of the land, some challenges, and the unexploited potential when Machine Learning (ML), including Neural Network (NN) models are carefully used. It also shows how new GenAI concepts can boost its potential. The first test is a disappointment, but an honest revelation of the fallacies of applying standard financial machine learning approaches to a different paradigm.

Later sections will aim to build upon the lessons learnt in a controlled, ablation process to see if some of the potential of these models can be unlocked.

1.Cross-Sectional Momentum (CSM) is a time and battle tested strategy, but Machine Learning (ML) applications are rare

Every day, retail traders globally wake up to hit Scanners on ‘Top Movers’ in the pre-market and open sessions. They filter price and volume metrics, go long the top gainers and short the bottom few. The strategy has been alive for at least 25 years (to my observation!). One might argue that alpha decays, and they’re partially right. The exact timing, selection of assets, and some metrics change, but the strategy remains the same. There are 11k+ tradeable stocks just in the US, add to that ETFs, Futures, Commodities and such, there is enough room to move around!

2. Time Series Momentum (TSM) is statistically fragile, and CSM is considered superior in academic circles for reasons we’ll see later

Prado(2019) published an entire ML framework around avoiding the fragility of autocorrelation, the core basis for time series momentum. The algo trading bandwagon a few years ago was calling it superior to “Time Series Momentum”.

Meanwhile, in 1993, on the academic side, Jegadeesh and Tittman published a seminal paper triggering a debate amongst many in academic and professional circles. While the arguments loosely characterized below may seem trivial, this was 1993 — no internet, AI or machine learning!

Loosely characterized illustration for entertainment

It’s no secret that ML and NNs can be notoriously difficult in the low signal-to-noise financial markets. However, if you already know markets and some strategies, you can enhance them with the right architecture designs, some adjustments and a bit of art. So, how does such a battle-tested and time-tested strategy like CSM not have plugin-ready ML and Neural Network (NN) frameworks? That question brings us to Learning-to-Rank (LTR) models, of which two of the more relevant ones are LambdaMART (tree-based) and LambdaRank(NN)

3. Cross-sectional ML models have unique designs, nuanced parameters, and evaluation metrics!

These models are less popular than ranking regression and classification models because their advantages come at a higher upfront cost. The tools used for these models are borrowed from search engines, and they operate differently from typical models like Random Forest, LSTMs, or even GenAI models like Transformers. Even the gradient loss function and other evaluation metrics are different.

LambdaRank and LambdaMART optimize ranking through surrogate gradients rather than differentiating through the sorting operation itself. LambdaRank scales pairwise logistic gradients by the change in NDCG that would result from swapping two items. LambdaMART uses those lambda-weighted pseudo-gradients to fit boosted decision trees. NDCG is the ranking objective and evaluation target; the neural model still learns through an ordinary differentiable pairwise loss, while the tree model fits gradients stage by stage.

In similar situations, I have often resorted to my favorite fallback, the less mathematically elegant LeakyRELU, another surprisingly rare and workable fallback that I found while sniffing around for more forgiving loss curves. It has gotten me to the finish line a few times, and apparently is also used by production engineers quietly, with caution of course!

4. A key upside is that ranking models are an inherent edge on their own, unlike other algo methods that work you hard to fine one!

LambdaMART and LambdaRank are inherently tail-alpha generating approaches. They are built to find the top and bottom asset scores and relevance. What remains is how to decide to execute them, your position sizing, holding period, dynamic trading vs. portfolio approaches, etc. They pick the best performing asset relative to others in the universe, given their relative features at a point in time. By relying on this relative hierarchy rather than absolute price trajectories, you are fundamentally immune to broad market beta drawdowns.

If the whole market drops 10%, but your long tail drops only 5% and your short tail drops 15%, your cross-sectional alpha is pristine, and your BPR (Buying Power Reduction) remains stable. Here is an example output from one of many backtest trials.

Random selection from one of hundreds of trials . All of them escaped the Covid drop, while trading actively

5. Unlike other ML algo finance approaches. the trading strategy with these models is decoupled from the model. You get the top k candidates and then construct dynamic strategies or portfolios.

The focus of the study has been the ML part, rather than feature engineering or the actual strategy, which the framework allows you to separate. Poh et al. (2021) used a portfolio approach, while this study considers a dynamic daily strategy, just like the age-old manual scanner technique. The strategy is fundamentally “de-linked” from this model architecture. You get your top k (and bottom k if you wish!) and their relative scores. You can apply a hundred different position sizing methods, holding periods, volatility targeting, combine with time-series strategies, etc.

I’m not a traditional researcher and a big fan of action-oriented, outcome-driven methods and of course alpha, although I maintain my reservations about the Sharpe Ratio, perhaps a topic for another day. The model forced me to consider its inherent dynamics and I’m glad I did. They are outlined in a later series on lessons learnt and suggestions for improvement.

6. The results of Phase 1 was an honest disappointing start.

It is not uncommon to get awkward results in MART or NN models in low signal-to-noise ratio markets. The bigger problem is about to do next, given their unique design and sparse research!

Phase 1 was a deliberately simple, frozen baseline — not a hyperparameter search. The values below are cross-checked against the reproducible Phase 1 contract, saved model summaries, and training histories. Where an older screenshot or narrative differs, I label that as historical evidence rather than silently combining it with the reconstruction.

LambdaMART parameters used in Phase 1: Tree ranker

Neural LambdaRank parameters — PyTorch ranker

Reproduced Phase 1 results

Both test pairwise accuracies are effectively chance-like (~50%). LambdaMART retained somewhat better tail-ranking quality and a positive reconstructed static-hold result, but this was one frozen split with zero costs — not evidence of a production-ready edge. These numbers describe the reproducible Phase 1 package; any conflicting values from older screenshots should remain explicitly labelled as historical rather than merged with this table.

7. Loss Trajectories in high Signal-to-Noise financial markets are different from Textbook Baseline often built in high-signal, deterministic domains like image classification on MNIST datasets

MLFlow: NeuralNine Youtube, check them out for applied AI tutorials, MLOps and similar quick crash courses

The Loss Curves: Both training and validation loss exhibit steep, synchronized exponential decay. The model is efficiently mapping the feature space without encountering structural friction.

The Target Metric: Validation accuracy climbs smoothly and monotonically. There is no overfitting, and the continuous function approximation is working perfectly.

8. In Comparison…….the reality of applying ML LTR on low-signal financial cross-sections!

Note that the curves of the type above are rarely seen in regular ML financial models without major feature smoothing etc. in the first pass. Furthermore, as explained above the loss curves are not meant to be smooth, but also not as brutal as the below.

MLFlow: LambdaMART AND LAMBDARANK Losses in the first 1st run

9. The most critical observation: the 0.693 Pairwise Trap

Overall, LambdaMART extracted modest tail-ranking power in the reproducible package: test macro-tail NDCG@2 was 0.4169, while test pairwise accuracy remained chance-like at 49.75%. Its preserved Phase 1 training history records LightGBM NDCG—not binary pairwise loss—and contains 26 boosting rounds with iteration 16 selected. Any older chart showing LambdaMART loss around 0.693 therefore belongs to separate historical evidence and should not be presented as if it came from this saved reconstruction.

LambdaRank did not have much luck. Its best validation macro-tail NDCG@2 was 0.4034 at epoch 1, and the selected checkpoint was therefore the first epoch. By epoch 13, the validation pairwise loss had moved toward 0.6943, while the untouched-test macro-tail NDCG@2 was 0.3915 and pairwise accuracy was 50.59%. This is consistent with a weak, unstable signal, although “actively destroying” the signal would be stronger than the evidence supports.

For the neural pairwise logistic loss, ln(2) ≈ 0.693 is a useful diagnostic reference: it is the loss obtained when a pair receives no directional margin. The Phase 1 validation loss approaching that level is consistent with weak separation between preferred and non-preferred assets. It is not, by itself, proof that every predicted pair has an explicit 50/50 probability, and it should not be transferred directly to LightGBM's NDCG training history or to listwise objectives with different loss definitions.

Loss flattening: The neural training and validation losses moved toward a narrow band without producing durable validation-ranking improvement. LambdaMART must be assessed from its recorded NDCG trajectory rather than from the neural 0.693 loss reference.

8. The journey and lessons learned documented in this series will be hopefuly be useful to a disciplined trader, willing to do ablation studies with sound causal knowledge of markets

The ablation approach will very likely lead to stable, reproducible and robust results, rather than a than a polished success claim after-the-fact. You will have the lego bricks with some lessons learnt. Read between the lines! E.g. While the 11 sectors were selected to manually track causal cross-sectional relationships, these model performs best with a massive datasets as explained in . Poh et al(2021) studied over 1,000 equities.

However, the journey documented in this multi-part series will be useful to a disciplined trader, and very likely lead to stable, reproducible and robust results, rather than a than a polished success claim manufactured after the fact. More importantly, the code and the full lifecyle is transparent and available to you to audit, replicate, or improve in hundreds of ways!

9. Watch out for Part 2….In the meantime, play around with the code (published with guidance and report generation) in the Github repo!! https://github.com/0xpoorman/

Follow, connect and share feedback? 0xpoorman@gmail.com, 0xpoorman@proton.me; Consider donating if you want to support my work.

Disclaimer: Note that content in this article is not financial advice. It’s for infotainment purposes only. You can lose large amounts of money, especially with obscure ML methods like these. Do your own research if at all you wish to deploy your own capital. The data used is from Yahoo Finance (implicitly web-scraped!) and old (cutoff 2025). It does not violate any distribution agreement to my knowledge. Also, nothing in this document should be considered credible claims for profits or financial returns.

Sources:

Jegadeesh & Tittman (1993): Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency https://www.google.com/search?q=https%3A%2F%2Fdoi.org%2F10.1111%2Fj.1540-6261.1993.tb04702.x

Poh, D., Lim, B., Zohren, S., & Roberts, S. (2021). Building Cross-Sectional Systematic Strategies by Learning to Rank. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3751012

Marcos Lopez de Prado (2019): Advances in Financial Machine Learning https://www.amazon.com/Advances-Financial-Machine-Learning-Marcos-ebook/dp/B079KLDW21

Hamlet Medina & Ernest Chan (2025): [Generative AI for Trading and Asset Management [https://www.amazon.com/Generative-AI-Trading-Asset-Management/dp/1394266979)]

MLFlow: NeuralNine Youtube, check them out for AI tool tutorials, MLOps and similar quick crash courses, no fluff I’m I just didn’t get a time at all