Few-shot learning with LLM for prediction of binary molecular Models

Large language models (LLMs) have garnered significant interest across various domains everywhere and scientist and people day by day finding its unique uses in day to day life. Even in cheminformatics scientist have used them to generate molecules , automate tasks, synthesis planning like chemcrow. Doing chemistry and LLM is challenging but as we progress we see will new possibilities . Despite the broad popularity of descriptor-based QSAR modeling — where features such as molecular weight, hydrogen-bond donors, morgan fps and topological indices from RDKit are computed — this method deliberately avoids all classical descriptors. Instead, it relies on a raw, large language model (LLM) to interpret SMILES strings through a series of prompts, treating them purely as textual input rather than structured chemical information. This post presents an in-depth exploration of a code snippet that demonstrates how large language models (here, GPT based) perform classification predictions on the blood brain barrier dataset (TDC) , herg (TDC) and bace (β-site amyloid precursor protein cleaving enzyme) dataset( provided at github repo) . We will discuss both two methods i used one random sampling and scaffold-based sampling strategies, how they compare, and why LLM based predictions may still lag behind RDKit based approaches, while hinting at the exciting possibility of future improvements. I must say it is very much possible that one day these models can do predictions with high accuracy and while performing 5–7 experiments i am amazed to see how well the predictions are getting. I have provided the code at my github repo with bace dataset.

We use a dataset from TDC’s **ADME** module, specifically the “BBB_Martins” data. The task is to classify whether given molecules, represented by SMILES, can cross the blood-brain barrier (BBB). This classification is simplified into two classes (0 or 1):

1. Class = 1 might correspond to crossing the BBB (or in some use cases, inhibiting a target).
2. Class = 0 indicates not crossing the BBB (or no inhibition).

Instead of using RDKit descriptors or other traditional cheminformatics features, the code below demonstrates how we employ a GPT based model (e.g., gpt-4o) to generate predictions. Though this LLM based approach is intriguing and easy to adapt, it is notoriously less reliable at capturing nuanced molecular attributes when compared to descriptor-based QSAR or machine-learning models. As a result, its performance may not match that of RDKit based or deep-learning descriptor-based approaches just yet, but continued development could bring these methods much closer in the future.

The function below sends a prompt to a GPT based model. It adjusts temperature and returns multiple responses (in this case, 5). Each response predicts whether the molecule is “1” or “0.” By collecting multiple predictions at once (n=5), we gain richer insight into the model’s level of certainty and variability. If three out of five responses consistently predict a molecule to be “1,” for example, it might signal higher confidence in that classification. This multi-response strategy can reveal areas of consensus and help shed light on the subtlety or ambiguity the model perceives in each SMILES pattern.

def generate_response_by_gpt4(prompt, model_engine="gpt-4o"):
    completion = client.chat.completions.create(
        model=model_engine, temperature=1, n=5,
        messages=[{"role": "user", "content": prompt}],
    )

    message = completion.choices
    message = [i.message.content.strip() for i in message]
    return message

Sampling Methods for Prompt Examples

To guide GPT predictions, we provide a small set of known SMILES/class pairs. This is critical because LLMs benefit from “few-shot” examples that clarify the format and nature of the prediction. Two main strategies are employed for selecting these examples from the training set:

Random Sampling

Random sampling is straightforward. It selects examples randomly from the training set, ensuring a balanced number of positive and negative samples. By adjusting sample_size, you can choose how many total molecules to feed as prompt examples. For instance, using a sample_size of 10, 20, or 30 allows you to experiment with the influence of example quantity on model predictions.

def radom_sample_examples(data, sample_size):
    positive_examples = data[data["Class"] == 1].sample(int(sample_size/2))
    negative_examples = data[data["Class"] == 0].sample(int(sample_size/2))
    smiles = positive_examples["mol"].tolist() + negative_examples["mol"].tolist()
    class_label = positive_examples["Class"].tolist() + negative_examples["Class"].tolist()
    data_examples = list(zip(smiles, class_label))
    return data_examples

def top_k_scaffold_similar_molecules(target_smiles, _data, k):
    _data = _data[_data["mol"] != target_smiles]
    molecule_smiles_list = _data['mol'].tolist()
    label_list = _data['Class'].tolist()

    target_mol = Chem.MolFromSmiles(target_smiles)
    if target_mol is not None:
        target_scaffold = MurckoScaffold.GetScaffoldForMol(target_mol)
    else:
        print("Error: Unable to create a molecule from the provided SMILES string.")
        return None

    target_fp = rdMolDescriptors.GetMorganFingerprint(target_scaffold, 2)
    warnings.filterwarnings("ignore", category=UserWarning)

    similarities = []
    for i, smiles in enumerate(molecule_smiles_list):
        mol = Chem.MolFromSmiles(smiles)
        try:
            scaffold = MurckoScaffold.GetScaffoldForMol(mol)
            scaffold_fp = rdMolDescriptors.GetMorganFingerprint(scaffold, 2)
            tanimoto_similarity = DataStructs.TanimotoSimilarity(target_fp, scaffold_fp)
            similarities.append((smiles, tanimoto_similarity, label_list[i]))
        except:
            continue
    similarities.sort(key=lambda x: x[1], reverse=True)
    top_similar_molecules = similarities[:k]
    return top_similar_molecules

Scaffold-based sampling is constructed on the principle that structurally related compounds share similar properties. By using Bemis–Murcko scaffolds, the code finds molecules in the training set that most resemble the target (the test molecule) in scaffold structure. Here, you can experiment with different values of k, such as 10 or 20, to retrieve the top k scaffold-similar molecules from the training set. These top-k molecules serve as the few-shot examples.

Prompt

LLMs rely on well-structured prompts for effective responses. The create_pred_prompt function creates a text prompt containing “few-shot” examples, instructing the GPT based model to output only a “1” or “0”. Each test molecule from test dataframe is fed into this function, along with the examples either the random subset or the top-k scaffolds.

def create_pred_prompt(input_smiles, pp_examples):
    prompt = (
        "You are an expert chemist, your task is to predict the property of molecule "
        "using your experienced chemical property prediction knowledge.\n"
        "Please strictly follow the format, no other information can be provided. "
        "Given the SMILES string of a molecule, predict the molecular properties of "
        "a given drug SMILES string, predict whether it blocks (1) or not blocks (0). "
        "Consider factors such as molecular weight, atom count, Veber's Rule, "
        "Physicochemical Properties, bond types, and functional groups, "
        "Graph-Based Descriptors, morgan descriptors in order to assess the compound's "
        "drug-likeness and its potential to serve as an effective therapeutic agent, "
        "please answer with only 1 or 0. A few examples are provided in the beginning.\n"
    )
    for example in pp_examples:
        prompt += f"SMILES: {example[0]}\n Inhibit: {example[-1]}\n"
    prompt += f"SMILES: {input_smiles}\nInhibit:\n"
    return prompt

Finally, the main part of the script loops over test molecules, retrieves examples (random or scaffold-based) from bbb train set, constructs prompts, queries the GPT based model for predictions, and saves the results to CSV files for further analysis. The sample_numsis the samples to select from train set and i see 100 samples are good enough to get a decent prediction. I will share the results for some datasets .

I got convinced with several experiments to use 100 samples and results of the datasets are given below ,

Performance with different datasets

When the model is trained or prompted with random samples (e.g., “bace random 50” or “bbb random 50”), the recall values are quite low (often near 0.10–0.26). This indicates the model struggles to correctly identify positive cases in the dataset.Random sampling examples may not provide enough structural or chemical insight to guide the LLM to robust classification decisions.

On the other hand when using a scaffold-based approach (e.g., “bace scaffold 50,” “bbb scaffold 50,” and “bbb scaffold 100”), the model typically has a better balance between precision and recall. For instance, in “bace scaffold 50,” recall jumps to ~0.70 and F1 score ~0.71–0.73. This improvement occurs because scaffold sampling selects examples structurally similar to the test compound, helping the model identify relevant molecular features. Doubling the sample size from 50 to 100 examples in scaffold sampling (see “bbb scaffold 100” and “herg scaffold 100”) generally boosts performance further. Precision, recall, and ROC AUC all climb, and the F1 Score often surpasses 0.80. This suggests a richer set of structurally relevant examples helps the LLM discriminate subtle patterns more effectively.

Ultimately, while descriptor-free LLM predictions remain in an early stage, they open the door to flexible, text-based experimentation. As LLM architectures continue to evolve — and as more tailored prompt engineering techniques emerge — this purely language-based approach may steadily narrow the gap with classical descriptor-based modeling. As the underlying language models continue to evolve, we can expect their predictive capabilities to improve and possibly inch closer to traditional QSAR approaches in certain niches. For now, though, practitioners should remain aware of their inherent limitations and consider coupling them with or comparing them to well-established cheminformatics pipelines. Although LLMs can show promising precision, their recall can be finicky when prompts are not well-aligned or when examples are too few or poorly matched. One can craft various based on given type of dataset and verify the results.

If you experiment with both random and scaffold-based sampling, observe how the final predictions differ. For scaffold-based sampling, you might notice more chemically relevant examples guide the LLM to more consistent answers. Meanwhile, random sampling offers a more general approach that does not rely on scaffold structure.

If you enjoyed reading this, and/or want to be kept in the loop about the next blog, follow me on Medium.

Feel free to connect with me on LinkedIn and if you feel this can be revolutionary in your industry research/job implementation.

If you feel to leave some tips send it here .