Implementing DeepSipred: Building a Deep Learning Model for siRNA Inhibition Prediction

Simplified diagram of siRNA and shRNA processing to facilitate gene knockdown

Navigating the Complexities of siRNA Design & Testing

In the realm of genetic medicine, small interfering RNA (siRNA) has emerged as a powerful tool with immense therapeutic potential. These tiny molecules hold the promise of silencing disease-causing genes with unprecedented precision. However, the journey from concept to clinic is filled with challenges that demand significant investments of time, money, and scientific expertise.

The Quest for Efficacy

Designing efficient siRNAs is like finding a needle in a haystack. Despite advances in bioinformatics and molecular biology, predicting which siRNA sequences will effectively silence a target gene remains a complex task. Factors such as:

• Off-target effects
• Stability in cellular environments
• Delivery to specific tissues
• Activation of the immune system

All play crucial roles in determining an siRNA’s efficacy. Each of these factors introduces variables that can make or break a potential therapeutic. But what if there was a way to significantly reduce the candidate pool to, hopefully, reduce the resource-intensive screening process?

DeepSipred

Thus begins my journey to reconstruct the deep learning siRNA inhibition architecture proposed by Bin Liu et al., which can be found here. My interest in this paper stemmed from a previous project where I witnessed the power of convolutional neural networks in melanoma prediction. From a biological perspective, I was intrigued by how CNN architectures can detect position-specific motifs and complex patterns that might not be apparent. Most importantly, I wanted to explore how deep models can integrate diverse information sources , from sequence context to thermodynamic properties to structural characteristics, into a unified prediction framework for siRNA efficacy

Building the Model: 3 Main Modules

In recreating DeepSipred, I structured the architecture around three primary modules as described in the original paper. First, a comprehensive feature engineering module extracts meaningful representations from siRNA sequences and their targets. Second, a convolution and pooling module identifies critical motifs and patterns across these features. Finally, a full connection layer aggregates all this information to produce accurate inhibition predictions.

While the original paper utilized multiple datasets (Dataset-H, Dataset-RVHU, and Dataset-T), I focused exclusively on the Huesken et al. dataset, the largest among them with 2,431 experimentally validated siRNAs targeting 34 mRNAs. This allowed me to save time while still working with sufficient data to test the model’s core capabilities.

Feature Engineering: La crème de la crème

The first and probably most important step was to convert the siRNA and mRNA sequences into meaningful features that the model can understand.

• Sequence Composition: Both sirna and target sequences required a structured one-hot encoded method to ensure a proper array was being fed to the neural network.
• Thermodynamic Properties: The popular ViennaRNA was used to extrapolate the rna secondary structures, minimum free energies (MFE), and the G/C content knowing stability plays an important role in siRNA potency.
• Seed Region Characteristics: The seed regions were used to find the binding region on the mRNA, but the concept of a “local_mRNA” was employed to try and capture local relationships and how they attributed to gene knockdown.
• RNA-FM: This model was used to construct a rich vector embedding representation of the sirna sequence in hopes of capturing important functional, sequential, structural, and evolutionary information.

Convolution and Pooling

The convolution and pooling module forms the critical pattern recognition component of my DeepSipred implementation. This module processes three distinct input features — siRNA sequences, UTR regions, and RNA-FM embeddings through parallel convolutional pathways. Each pathway employs a Conv1d layer to detect position-specific motifs, followed by ReLU activation to introduce non-linearity. The subsequent pooling operations (max pooling and adaptive average pooling) extract the most significant features while reducing dimensionality. This design allows the model to identify crucial sequence patterns like seed regions, thermodynamic stability indicators, and secondary structure elements that influence siRNA efficacy.

# Convolutional layers definition
self.sirna_conv = nn.Conv1d(seq_input_dim, 64, kernel_size=3)
self.utr_conv = nn.Conv1d(utr_input_dim, 64, kernel_size=3)
self.rnafm_conv = nn.Conv1d(640, 64, kernel_size=1)

# Pooling layers definition
self.max_pool = nn.MaxPool1d(kernel_size=2)
self.adaptive_pool = nn.AdaptiveAvgPool1d(1)

# Implementation in forward method
sirna_feat = F.relu(self.sirna_conv(sirna))
if sirna_feat.size(-1) > 1:
    sirna_feat = self.max_pool(sirna_feat)
    sirna_feat = self.adaptive_pool(sirna_feat).squeeze(-1)

utr_feat = F.relu(self.utr_conv(utr))
if utr_feat.size(-1) > 1:
    utr_feat = self.max_pool(utr_feat)
    utr_feat = self.adaptive_pool(utr_feat).squeeze(-1)

rnafm_feat = F.relu(self.rnafm_conv(rnafm))
if rnafm_feat.size(-1) > 1:
    rnafm_feat = self.max_pool(rnafm_feat)
    rnafm_feat = self.adaptive_pool(rnafm_feat).squeeze(-1)

The Full Connection Module: Architecture Overview

The full connection module serves as the decision-making component of the DeepSipred implementation. After feature extraction, this module integrates diverse information through a series of linear transformations with progressive dimensionality reduction (128→64→32→1). Each layer incorporates batch normalization to stabilize learning, ReLU activation to maintain non-linearity, and dropout (p=0.5) to prevent overfitting. This architecture gradually refines the combined signal from sequence patterns, thermodynamic properties, and structural features into a single inhibition prediction. The final sigmoid activation ensures outputs fall within the normalized 0–1 range, directly corresponding to expected siRNA efficacy levels.

Architecture Overview from the literature

Training the Model

My training results demonstrated solid convergence and model performance. The loss curve shows healthy learning dynamics, with both training and validation loss steadily decreasing over the first 40 epochs before stabilizing. The final training loss settled around 0.11, while validation loss plateaued slightly higher at approximately 0.13, indicating good generalization without severe overfitting. Most encouragingly, the PCC metric (Pearson Correlation Coefficient) steadily improved throughout training, starting near 0.25 and ultimately reaching approximately 0.58 after 100 epochs. This represents reasonable predictive performance for siRNA inhibition, though it falls short of the paper’s reported 0.745 PCC. This difference likely stems from my simplified implementation and use of only the Huesken dataset but still demonstrates successful capture of the core relationships between siRNA features and inhibition efficacy.

Takeaway

My hands-on approach to reimplementing the DeepSipred model yielded valuable insights into both the technical challenges and biological implications of siRNA inhibition prediction. While my implementation achieved a respectable PCC of 0.58 it fell short of the paper’s reported 0.745 PCC. This gap highlights the complexity involved in fully reproducing models within literature without the original code or dataset.

In future efforts, I plan to explore how a BERT model (specifically, DNABERT6) might perform using the same datasets and engineered features. Could the transformer architecture better capture underlying relationships or potentially long-range dependencies between siRNA sequences and UTRs that my CNN implementation missed? The self-attention mechanism might offer advantages for understanding context-dependent interactions that influence inhibition efficiency.

Thanks for tuning in!