Microsoft Fabric DataScience — Pipeline or Data Pipeline (Part 1)

Before creating make sure you have an account of Microsoft Fabric, Open the data science project. There are two ways you can create the pipeline 1- Create the notebook, and add the new data pipeline.

2- Or directly add the data pipeline.

Then add a new pipeline, and create a new pipeline with the name “pipeline” (or any name as want), Add the notebook and write something else. Clicking the run button from the top menu, you will be able to run the notebook, before running make sure you have validated it.

Now I'm going to explain how many ways we can pass the data from one notebook to another book.

Case 1: Pass the variable from the pipeline notebook parameter to the notebook

In this case, select the notebook, open the settings, and add the base parameters with any name i.e. “node_param (or any name)” and assign any value.

Now run the pipeline, by clicking the run button, in the output section you will see the run’s details.

Open the activity name, this opens the popup, we can check the notebook cell execution detail by clicking the “notebook snapshot“, After opening the notebook, here one thing notice a new cell created with “node_param =1010” created automatically. All the cells were executed successfully.

Case 2: Pass Parameter from Parent Notebook to child Notebook (using the Pipeline)

In this case, I will pass the pipeline parameter from the parent notebook to the child notebook. First, create notebook B, define the variable, and assign any value. click on the three dots (…) and select the “Toggle parameter cell”. Or we can define the node parameter.

using this code of line: mssparkutils.notebook.run(“NotebookB”, 90 , {“notebook2_param” : pipeline_param }). This code is called the NotebookB and passes the new value and executes it.

Case 3: Pass Parameter from Parent Notebook to child Notebook (without the pipeline)

in this case, don’t need any kind of pipeline, simply we have two notebooks A and B passing the value from A to B notebook.