Mapping Flood Susceptibility: Using Random Forest and MLP Classifier to Predict Vulnerable Areas

Author 1: Abhigyan Chakraborty, Department of Civil Engineering, IIT Hyderabad. correspondence email: ce23resch12001@iith.ac.in

Author 2: Barun Kumar, Department of Climate Change, IIT Hyderabad. correspondence email: cc23mtech11001@iith.ac.in

Introduction:

Floods are devastating natural disasters that affect millions of people each year. To mitigate these impacts, it is crucial for policymakers and planners to identify areas that are prone to flooding. This enables targeted investments and the development of effective risk management strategies. Flood susceptibility mapping plays a key role in pinpointing these vulnerable regions. In this article, we will develop a flood susceptibility map using two machine learning models: the Random Forest Classifier and the MLP Classifier.

The process is organized into five key steps:

1. Exploratory Data Analysis: We begin by examining the dataset through summary statistics, correlation analysis, and checking for missing values.
2. Feature Engineering: We employ the Sequential Feature Selection method to refine the feature set.
3. Model Development: With optimal features selected, we proceed to tune the hyperparameters of our models, followed by training and testing. We assess model performance using ROC_AUC plots and statistics derived from the confusion matrix.
4. Interpretability of the Models: To understand the influence of various features, we generate partial dependence plots and permutation feature importance plots.
5. Flood Susceptibility Mapping: After training and testing the models, we use the probabilistic outputs from model.predict_proba(features) to create a preliminary flood susceptibility map. This map is further refined using the Inverse Distance Weighting (IDW) technique in GIS and classified into five categories: very low, low, moderate, high, and very high.

This approach aims to provide a comprehensive tool for flood susceptibility assessment, aiding decision-makers in their efforts to safeguard vulnerable communities.

Study Area:

Two Flood Prone districts of Assam, India were selected for this project: Dhemaji & Kamrup Metro.

Data used:

1. Flood conditioning Factors: Twelve Flood Conditioning Factors were selected namely — Distance to River [DR] (m), Drainage Density [DD], DEM (m), slope, flow Accumulation [FA], MNDWI, NDBI, NDMI, NDVI, TWI, Average Daily Precipitation [Precip] (mm/hr.), and Curve Number [CN].
2. Flood Inventory: 1000 data points were collected from each map. 500 from flooded locations, 500 from non-flooded location.

Fig.: Flooded & Non-Flooded Locations obtained from NRSC WebService (Bhuvan | Thematic Data dissemination | Free GIS Data | OGC Services | Clip and Ship (nrsc.gov.in)

Exploratory Data Analysis:

In this phase of our study, we conducted a thorough examination of our dataset to lay the groundwork for more detailed analysis. We generated summary statistics to get an overview of central tendencies and variability. To understand the distribution patterns of the data, histograms for each variable were created. We also constructed correlation plots to identify relationships and dependencies between variables. Additionally, we checked for missing values and ensured that our dataset includes an equal number of observations from flooded and non-flooded locations, which is crucial for maintaining balance and reducing bias in model training. This comprehensive exploratory analysis helps us confirm the robustness and readiness of the data for subsequent modeling stages.

import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import numpy as np
import math

# Load dataset
data = pd.read_csv("/path/data.csv")  # Update the path to your actual data file

# Descriptive Statistics and Null Values Check
print("Descriptive Statistics:")
print(data.describe())
print("\nNull Values Check:")
print(data.isnull().sum())

# Histograms for All Variables in a Single Image
num_variables = data.select_dtypes(include=[np.number]).columns.tolist()
num_plots = len(num_variables)
cols = 4  # Adjust the number of columns
rows = math.ceil(num_plots / cols)

plt.figure(figsize=(cols * 4, rows * 3))  # Adjust figure size based on number of subplots
colors = plt.cm.viridis(np.linspace(0,1, num_plots))  # Generates a color map
for i, var in enumerate(num_variables):
    plt.subplot(rows, cols, i + 1)
    data[var].hist(color=colors[i], bins=15)
    plt.title(var)
plt.tight_layout()
plt.show()

# Correlation Analysis Plot (Heatmap)
corr_matrix = data.corr()
plt.figure(figsize=(10, 8))
sns.heatmap(corr_matrix, annot=True, fmt=".2f", cmap='coolwarm', cbar=True)
plt.title('Correlation Matrix Heatmap')
plt.show()

descriptive_stats = data.describe()
descriptive_stats.to_csv("/path/descriptive_statistics.csv")
print("Descriptive Statistics saved to 'descriptive_statistics.csv'")
print(descriptive_stats)

Summary Statistics:

Table.: Summary Statistics of data

Histograms:

Fig.: Histograms of all data

Inferences gained from summary statistics table & Histograms:

1. Label: Binary classification (0 or 1), evenly split with a mean of 0.5.
2. DR: Ranges from 0 to 10,821.60 with a mean of 853.36, indicating a right-skewed distribution.
3. DD, DEM, FA: Similar observations with skewness, especially FA (Flow Accumulation) which ranges up to 763,891 but mostly concentrated near zero.
4. MNDWI, NDBI, NDMI, NDVI: These are vegetation and water indices, showing a range of values that are typical for such indices. The standard deviations suggest variability in the data which is expected for environmental measurements.
5. Precip (Precipitation): Ranges from 0 to 5.78 with a mean of 2.46, indicating variability in precipitation across observations.
6. Slope: Highly skewed with most data near 0 but extending up to 29.75.
7. TWI (Topographic Wetness Index): Also skewed, with a significant number of zeros but stretching up to 24.67.
8. CN (Curve Number): Mostly ranges between 70 and 79, indicating standardized site conditions for most observations.

Correlation plots:

Fig.: Correlation Matrix Heatmap

This correlation matrix heatmap represents the pairwise correlation coefficients between different variables:

1. Variables like DD and TWI, DEM and TWI, and NDVI and Precip have strong positive correlations, suggesting they tend to increase together.
2. NDWI and NDVI show a significant negative correlation, meaning as one increases, the other tends to decrease.
3. FA displays very low correlation with other variables, indicating it doesn’t have a strong linear relationship with them.
4. Perfect correlations along the diagonal are self-correlations of each variable.
5. The heatmap highlights clusters of related variables and suggests which ones might be important for further analysis or modeling.

Feature Engineering:

Sequential Feature Selection (SFS) is a process of selecting features for a model by adding (forward selection) or removing (backward selection) attributes until a desired number of features is reached. For models like the RandomForest Classifier and the MLPClassifier, SFS can help in improving model performance by:

1. Reducing overfitting: Selecting a subset of relevant features can reduce the complexity of the model, which can decrease the chance of overfitting, especially important for models like MLPClassifier that are prone to fit noise in the training data.
2. Improving accuracy: By keeping only the most informative features, both models may show improved predictive accuracy on unseen data.
3. Increasing interpretability: Fewer features can make the model more interpretable, as each feature’s impact on the prediction is clearer, which is beneficial when using a model with many decision trees like the RandomForest.
4. Speeding up training: SFS can lead to faster model training by reducing the dimensionality of the data, which is particularly useful for computationally expensive models like MLPClassifier.

We employed Sequential feature selection on our Random Forest and MLP Classifier Model:

Random Forest Classifier Feature Selection:

# Random Forest Feature Selection
# Forward Selection
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from mlxtend.feature_selection import SequentialFeatureSelector as SFS

# Load dataset
data = pd.read_csv("/path/data.csv")

# Split the data into features and target
X = data.drop('label', axis=1)
y = data['label']

# Split the dataset into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# Initialize the Random Forest classifier
rf_classifier = RandomForestClassifier(n_estimators=100, random_state=42)

# Configure the Sequential Feature Selector for forward selection
sfs = SFS(rf_classifier,
          k_features='best',  # Adjust this to select up to the maximum number of features
          forward=True,       # Forward Selection
          floating=True,
          scoring='accuracy',
         # verbose=2,
          n_jobs =-1,
          cv=4)

# Fit SFS on the training data
sfs.fit(X_train, y_train)

# Extract performance metrics
results = sfs.get_metric_dict()

# Prepare the plot data
features = []
scores = []

for i in range(1, len(results)+1):
    features.append(', '.join(list(results[i]['feature_names'])))
    scores.append(results[i]['avg_score'])

# Plotting
plt.figure(figsize=(10, 6))
plt.plot(features, scores, marker='o')
plt.title('Performance of Features During Sequential Forward Selection')
plt.xlabel('Features')
plt.ylabel('Accuracy')
plt.xticks(rotation=90)
plt.grid(True)
plt.show()

# Backward Elimination
sfs = SFS(rf_classifier,
          k_features=1,  # Minimum number of features to select
          forward=False,  # Backward elimination
          floating=True,
          scoring='accuracy',
          n_jobs=-1,
          #verbose=2,
          cv=4)

# Fit SFS on the training data
sfs.fit(X_train, y_train)

# Get the metric dict from the SFS object
metric_dict = sfs.get_metric_dict()

# Prepare the data for plotting
selected_features = []
accuracy_scores = []

# Start with the full feature set and end with the last one
for i in sorted(metric_dict.keys(), reverse=True):
    selected_features.append(', '.join(metric_dict[i]['feature_names']))
    accuracy_scores.append(metric_dict[i]['avg_score'])

# Plot the accuracies with the corresponding selected features
plt.figure(figsize=(12, 10))
plt.plot(selected_features, accuracy_scores, marker='o')
plt.title('Performance of Features During Sequential Backward Elimination')
plt.xlabel('Features')
plt.xticks(rotation=90)
plt.ylabel('Accuracy')
plt.tight_layout() 
plt.grid(True)
plt.show()

Fig.: Sequential Feature Selection for Random Forest Classifier

For Random Forest we considered the features: [‘DR’, ‘DD’, ‘DEM’, ‘Precip’, ‘Slope’, ‘CN’]

MLP Classifier Feature Selection:

## Forward Selection
from sklearn.preprocessing import StandardScaler
from sklearn.neural_network import MLPClassifier
from sklearn.pipeline import make_pipeline
# Initialize a StandardScaler
scaler = StandardScaler()

# Fit the scaler on the training data and transform it
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Initialize the ANN classifier
# For example, a simple neural network with 1 hidden layer with 100 neurons
ann_classifier = MLPClassifier(hidden_layer_sizes=(100,), random_state=42)

# Create a pipeline that first standardizes the data then fits the ANN
pipeline = make_pipeline(StandardScaler(), ann_classifier)

# Configure the Sequential Feature Selector for forward selection with ANN
sfs = SFS(pipeline,
          k_features='best',  # Adjust as necessary
          forward=True,       # For forward selection
          floating=True,
          scoring='accuracy',
          #verbose=2,
          n_jobs=-1,
          cv=4)

# Fit SFS on the scaled training data
sfs.fit(X_train_scaled, y_train)

# Extract performance metrics
results = sfs.get_metric_dict()

# Prepare the plot data
feature_names = list(X.columns)  # Get the feature names from the DataFrame
feature_combinations = []
scores = []

# Iterate over the results and get the feature names using their indices
for i in range(1, len(results) + 1):
    feature_combination = [feature_names[int(idx)] for idx in results[i]['feature_idx']]
    feature_combinations.append(', '.join(feature_combination))
    scores.append(results[i]['avg_score'])

# Plotting
plt.figure(figsize=(10,10))
plt.plot(feature_combinations, scores, marker='o')
plt.title('Performance of Features During Sequential Forward Selection')
plt.xlabel('Features')
plt.ylabel('Accuracy')
plt.xticks(rotation=90)
plt.grid(True)
plt.tight_layout()  # Adjust layout to fit everything
plt.show()

##Backward Elimination
# Initialize the ANN classifier
ann_classifier = MLPClassifier(hidden_layer_sizes=(100,), random_state=42)

# Create a pipeline that first standardizes the data then fits the ANN
pipeline = make_pipeline(StandardScaler(), ann_classifier)

# Configure the Sequential Feature Selector for backward elimination with ANN
sfs = SFS(pipeline,
          k_features=1,  # We want to end with 1 feature
          forward=False,  # Now we use backward elimination
          floating=False,
          scoring='accuracy',
          #verbose=2,
          n_jobs=-1,
          cv=4)

# Fit SFS on the scaled training data
sfs.fit(X_train, y_train)

# Extract performance metrics
results = sfs.get_metric_dict()

# Prepare the plot data
feature_names = list(X.columns)  # Get the feature names from the DataFrame
feature_combinations = []
scores = []

# Iterate over the results in reverse for backward elimination and get the feature names using their indices
for i in sorted(results.keys(), reverse=True):
    feature_combination = [feature_names[int(idx)] for idx in results[i]['feature_idx']]
    feature_combinations.append(', '.join(feature_combination))
    scores.append(results[i]['avg_score'])

# Plotting
plt.figure(figsize=(10, 10))
plt.plot(feature_combinations, scores, marker='o')
plt.title('Performance of Features During Sequential Backward Elimination')
plt.xlabel('Features')
plt.ylabel('Accuracy')
plt.xticks(rotation=90)
plt.grid(True)
plt.tight_layout()
plt.show()

Fig.: Sequential Feature Selection for MLP Classifier

For MLP we considered the features: [‘DR’, ‘DD’, ‘DEM’, ‘MNDWI’, ‘NDMI’, ‘Precip’, ‘CN’]

Model Development:

The Random Forest and MLPClassifier are both powerful classification algorithms used in our model development, each fine-tuned using GridSearchCV. The Random Forest, an ensemble of decision trees, excels in capturing complex interactions and providing robust predictions. The MLPClassifier, a neural network algorithm, is adept at modeling complex, non-linear relationships in data.

Both models underwent hyperparameter optimization to enhance performance. For the Random Forest, we adjusted ‘n_estimators’, ‘max_depth’, ‘min_samples_split’, and ‘min_samples_leaf’. The MLPClassifier’s hyperparameters included ‘hidden_layer_sizes’, ‘activation’, ‘solver’, ‘alpha’, and ‘learning_rate_init’.

Model validation was carried out with ROC-AUC plots for both classifiers to assess their discriminative capabilities. High AUC values indicate strong predictive power. The performance of both classifiers was also dissected using confusion matrices, from which precision, recall, F1-score, and accuracy were calculated, offering a multi-faceted view of model quality.

Both models aim to balance the trade-off between sensitivity and specificity, striving for high true positive rates while keeping false alarms to a minimum. These models, one leveraging a collection of decision boundaries and the other the power of a simulated neural network, showcase the diversity of approaches in machine learning to tackle classification challenges.

Random Forest Accuracies:

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.inspection import PartialDependenceDisplay, permutation_importance
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score

# Load the dataset
data = pd.read_csv("/path/data.csv")
X = data[['DR', 'DD', 'DEM', 'Precip', 'Slope', 'CN']]  # Specify your features
y = data['label']

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# Define the model and hyperparameters for GridSearchCV
param_grid = {
    'n_estimators': [100, 200, 300],
    'max_depth': [None, 10, 20, 30],
    'min_samples_split': [2, 5, 10],
    'min_samples_leaf': [1, 2, 4]
}

random_forest = RandomForestClassifier()
grid_search = GridSearchCV(random_forest, param_grid, cv=5, scoring='accuracy', n_jobs=-1, verbose=1)

# Perform grid search to find the best hyperparameters
grid_search.fit(X_train, y_train)
best_rf = grid_search.best_estimator_
grid_search.best_params_

# Accuracies from ROC_AUC plots & Confusion Matrix
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import (roc_curve, auc, confusion_matrix, precision_score,
                             recall_score, f1_score, accuracy_score)

y_probs = best_rf.predict_proba(X_test)[:, 1]

# Generate ROC curve values
fpr, tpr, thresholds = roc_curve(y_test, y_probs)
roc_auc = auc(fpr, tpr)

# Plot ROC curve
plt.figure()
plt.plot(fpr, tpr, color='darkorange', lw=2, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic')
plt.legend(loc="lower right")
plt.show()

# Predict the test data
y_pred = best_rf.predict(X_test)

# Compute the confusion matrix
conf_matrix = confusion_matrix(y_test, y_pred)

# Print the confusion matrix
print("Confusion Matrix:\n", conf_matrix)

# Calculate precision, recall, F1-score, and accuracy
precision = precision_score(y_test, y_pred)
recall = recall_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)
accuracy = accuracy_score(y_test, y_pred)

# Display the confusion matrix with seaborn heatmap
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix, annot=True, fmt="d", cmap="Blues", 
            xticklabels=['Predicted No', 'Predicted Yes'], 
            yticklabels=['Actual No', 'Actual Yes'])
plt.title('Confusion Matrix')
plt.ylabel('True Class')
plt.xlabel('Predicted Class')
plt.show()

# Extract TP, TN, FP, FN
TP = conf_matrix[1, 1]
TN = conf_matrix[0, 0]
FP = conf_matrix[0, 1]
FN = conf_matrix[1, 0]

# Calculate FAR and FPR
far = FP / (FP + TP)  # False Alarm Ratio
fpr = FP / (FP + TN)  # False Positive Rate

# Print statistical results
print(f"Precision: {precision:.2f}")
print(f"Recall (Sensitivity or TPR): {recall:.2f}")
print(f"F1 Score: {f1:.2f}")
print(f"Accuracy: {accuracy:.2f}")
print(f"POD (Probability of Detection): {recall:.2f}")
print(f"FAR (False Alarm Ratio): {far:.2f}")
print(f"False Positive Rate (FPR): {fpr:.2f}")

Fig.: ROC_AUC plot & Confusion Matrix of Random Forest Classifier

Statistics from Confusion Matric:

Precision: 0.82 Recall (Sensitivity or TPR): 0.65 F1 Score: 0.72 Accuracy: 0.75 POD (Probability of Detection): 0.65 FAR (False Alarm Ratio): 0.18 False Positive Rate (FPR): 0.14

Best Model Architecture of the Random Forest Classifier:

{‘max_depth’: 20, ‘min_samples_leaf’: 1, ‘min_samples_split’: 2, ‘n_estimators’: 300}

MLP Classifier Accuracies:

import pandas as pd
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.inspection import PartialDependenceDisplay, permutation_importance
import numpy as np
import matplotlib.pyplot as plt

# Load the dataset
data = pd.read_csv("/path/data.csv")
X = data[['DR', 'DD', 'DEM', 'MNDWI', 'NDMI', 'Precip', 'CN']]  # Specify your features
y = data['label']

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

# Define the MLPClassifier and hyperparameters for GridSearchCV
pipeline = Pipeline([
    ('scaler', StandardScaler()),
    ('mlp', MLPClassifier(random_state=42, max_iter=1000))
])

param_grid = {
    'mlp__hidden_layer_sizes': [(50,), (100,)],
    'mlp__activation': ['tanh', 'relu'],
    'mlp__solver': ['sgd', 'adam'],
    'mlp__alpha': [0.0001, 0.05],
    'mlp__learning_rate_init': [0.001, 0.01, 0.1]
}

grid_search = GridSearchCV(pipeline, param_grid, cv=3, scoring='accuracy', n_jobs=-1, verbose=1)
grid_search.fit(X_train, y_train)
best_pipeline = grid_search.best_estimator_

grid_search.best_params_

# Accuracies from ROC_AUC plots & Confusion Matrix
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import (roc_curve, auc, confusion_matrix, precision_score,
                             recall_score, f1_score, accuracy_score)

y_probs = best_pipeline.predict_proba(X_test)[:, 1]

# Generate ROC curve values
fpr, tpr, thresholds = roc_curve(y_test, y_probs)
roc_auc = auc(fpr, tpr)

# Plot ROC curve
plt.figure()
plt.plot(fpr, tpr, color='darkorange', lw=2, label='ROC curve (area = %0.2f)' % roc_auc)
plt.plot([0, 1], [0, 1], color='navy', lw=2, linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic')
plt.legend(loc="lower right")
plt.show()

# Predict the test data
y_pred = best_pipeline.predict(X_test)

# Compute the confusion matrix
conf_matrix = confusion_matrix(y_test, y_pred)

# Print the confusion matrix
print("Confusion Matrix:\n", conf_matrix)

# Calculate precision, recall, F1-score, and accuracy
precision = precision_score(y_test, y_pred)
recall = recall_score(y_test, y_pred)
f1 = f1_score(y_test, y_pred)
accuracy = accuracy_score(y_test, y_pred)

# Display the confusion matrix with seaborn heatmap
plt.figure(figsize=(8, 6))
sns.heatmap(conf_matrix, annot=True, fmt="d", cmap="Blues", 
            xticklabels=['Predicted No', 'Predicted Yes'], 
            yticklabels=['Actual No', 'Actual Yes'])
plt.title('Confusion Matrix')
plt.ylabel('True Class')
plt.xlabel('Predicted Class')
plt.show()

# Extract TP, TN, FP, FN
TP = conf_matrix[1, 1]
TN = conf_matrix[0, 0]
FP = conf_matrix[0, 1]
FN = conf_matrix[1, 0]

# Calculate FAR and FPR
far = FP / (FP + TP)  # False Alarm Ratio
fpr = FP / (FP + TN)  # False Positive Rate

# Print statistical results
print(f"Precision: {precision:.2f}")
print(f"Recall (Sensitivity or TPR): {recall:.2f}")
print(f"F1 Score: {f1:.2f}")
print(f"Accuracy: {accuracy:.2f}")
print(f"POD (Probability of Detection): {recall:.2f}")
print(f"FAR (False Alarm Ratio): {far:.2f}")
print(f"False Positive Rate (FPR): {fpr:.2f}")

Fig.: ROC_AUC plot & Confusion Matrix of MLP Classifier

Statistics from Confusion Matric:

Precision: 0.78 Recall (Sensitivity or TPR): 0.63 F1 Score: 0.70 Accuracy: 0.73 POD (Probability of Detection): 0.63 FAR (False Alarm Ratio): 0.22 False Positive Rate (FPR): 0.18

Best Model Architecture of the MLP Classifier:

{‘mlpactivation’: ‘tanh’, ‘mlpalpha’: 0.05, ‘mlphidden_layer_sizes’: (50,), ‘mlplearning_rate_init’: 0.1, ‘mlp__solver’: ‘sgd’}

Interpretability of the Models:

Partial Dependence Plots (PDP) and Permutation Feature Importance are two techniques used to interpret machine learning models, helping to understand the effect and importance of the input features.

Partial Dependence Plots (PDP) illustrate the effect of one or two features on the predicted outcome of a machine learning model, averaged over a dataset. A PDP can show whether the relationship between the target and a feature is linear, monotonic, or more complex. For instance, it can show if the probability of a certain prediction rises as a feature value increases.

Permutation Feature Importance measures the increase in the prediction error of the model after we permute the feature’s values, which breaks the relationship between the feature and the true outcome. This method is model agnostic and can be used with any model. A higher value indicates that scrambling the feature’s values leads to a bigger drop in model accuracy, suggesting that the feature is important for the model’s predictions.

In our study, we utilized these techniques to gain insights into two machine learning models: the MLPClassifier and the Random Forest.

Partial Dependence Plot & Permutation Feature Importance Plot of Random Forest Classifier:

# Generate partial dependence plots with histograms for all features
fig, axs = plt.subplots(3, 2, figsize=(15, 20))  # Adjust the subplot grid based on the number of features
fig.subplots_adjust(hspace=0.2, wspace=0.25)  # Adjust space between plots
for i, feature in enumerate(X.columns):
    ax = axs[int(i / 2), i % 2]  # Determine the position of the subplot
    disp = PartialDependenceDisplay.from_estimator(best_rf, X_train, [feature], ax=ax, grid_resolution=50)
    ax.set_title(feature)  # Set title to the feature name
    # Add histogram to each PDP
    ax_hist = disp.axes_[0, 0].twinx()
    ax_hist.hist(X_train[feature], bins=30, alpha=0.3)
    ax_hist.set_ylabel("Frequency")

plt.suptitle('Partial Dependence Plots for All Features with Tuned RandomForest')
plt.show()

# Permutation feature importance
result = permutation_importance(best_rf, X_test, y_test, n_repeats=10, random_state=42, n_jobs=6)
sorted_idx = result.importances_mean.argsort()
plt.figure(figsize=(10, 6))
plt.boxplot(result.importances[sorted_idx].T, vert=False, labels=X_test.columns[sorted_idx])
plt.title('Permutation Feature Importance')
plt.xlabel('Decrease in Accuracy')
plt.show()

Fig.: PDP of all features for Random Forest Classifier

Fig.: Permutation Feature importance plot of all features for Random Forest Classifier

Partial Dependence Plot & Permutation Feature Importance Plot of MLP Classifier:

# Generate partial dependence plots with histograms for all features
num_features = X.shape[1]
cols = 2
rows = (num_features + cols - 1) // cols

fig, axs = plt.subplots(rows, cols, figsize=(15, rows * 5))
fig.subplots_adjust(hspace=0.2, wspace=0.25)

for i, feature in enumerate(X.columns):
    ax = axs[i // cols, i % cols]
    PartialDependenceDisplay.from_estimator(
        best_pipeline,
        X_train,
        features=[feature],
        ax=ax,
        grid_resolution=50
    )
    ax.set_title(feature)
    # Add histogram to each PDP
    ax_hist = ax.twinx()
    ax_hist.hist(X_train[feature], bins=30, alpha=0.3, color='grey')
    ax_hist.set_ylabel("Frequency", color='grey')

plt.suptitle('Partial Dependence Plots for All Features with Tuned MLPClassifier')
plt.show()

# Permutation feature importance
result = permutation_importance(best_pipeline, X_test, y_test, n_repeats=10, random_state=42, n_jobs=-1)
sorted_idx = result.importances_mean.argsort()
plt.figure(figsize=(10, 6))
plt.boxplot(result.importances[sorted_idx].T, vert=False, labels=X_test.columns[sorted_idx])
plt.title('Permutation Feature Importance')
plt.xlabel('Decrease in Accuracy')
plt.show()

Fig.: PDP of all features for MLP Classifier

Fig.: Permutation Feature importance plot of all features for MLP Classifier

From the PDPs for the MLP Classifier:

1. DR and DEM show a decreasing trend, indicating that the predicted outcome is likely to decrease as these feature values increase.
2. MNDWI and Precip exhibit a positive relationship with the predicted outcome.
3. The shape and slope of these plots can help in understanding the non-linear behavior captured by the MLP model.

From the PDPs for the Random Forest:

1. DR shows a sharp decline initially and then levels off, suggesting that there might be a threshold effect.
2. DD shows a non-linear relationship with several peaks and valleys, indicating complex interactions.
3. Precip displays an increasing trend, highlighting its importance in the Random Forest predictions.

Permutation Feature Importance plots for both models:

1. DEM appears to have a significant impact on model accuracy, as indicated by a wider box, especially for the MLP Classifier.
2. DD has a high importance for the Random Forest, with some outliers suggesting variations in how it affects model performance.
3. The features CN and MNDWI have varying levels of importance across both models, which could indicate model-specific reliance on these features.

In both models, certain features consistently appear to be of high importance (DEM, DD), which means they are key drivers in the prediction outcomes. The impact of other features varies between the models, indicating that the Random Forest and MLP Classifier may be leveraging the information in the data differently due to their distinct learning algorithms.

Results:

Flood Susceptibility Maps using GIS:

In the results section, we present the Flood Susceptibility Maps for the Dhemaji and Kamrup Metro Districts, showcasing the areas potentially at risk for flooding. We obtained the flood susceptibility values in the form of probability estimates from the .predict_proba() method applied to both the MLP Classifier and Random Forest models. These values reflect the likelihood of flooding occurring at various points within the districts.

Leveraging Geographic Information System (GIS) software, we utilized Inverse Distance Weighting (IDW) interpolation to convert discrete probability values into a continuous flood susceptibility surface. This process allows for a nuanced spatial representation that highlights varying degrees of flood risk across the landscape.

To make the information more accessible and actionable, we further processed the continuous susceptibility map through a reclassification tool within the GIS software. The map was stratified into five distinct categories representing varying levels of flood risk: Very Low, Low, Moderate, High, and Very High. This categorical map provides clear visual guidance for planners and decision-makers to identify critical areas that require the most attention regarding flood preparedness and mitigation efforts.

1. Dhemaji District:

Fig.: Flood Susceptibility Map of Dhemaji District using MLP & Random Forest Classifier

2. Kamrup Metro District:

Fig.: Flood Susceptibility Map of Kamrup MetroDistrict using MLP & Random Forest Classifier

Conclusion:

In conclusion, our comprehensive study provides a nuanced analysis of flood susceptibility in the Dhemaji and Kamrup Metro Districts, bridging advanced data exploration techniques with sophisticated machine learning algorithms. The initial phase of Exploratory Data Analysis (EDA) ensured a robust foundation for our models, where we assessed the distribution, central tendencies, and correlations within our data, alongside a rigorous check for balance and completeness.

Leveraging Sequential Feature Selection (SFS) allowed us to refine our feature set, enhancing model performance by mitigating overfitting, bolstering predictive accuracy, clarifying interpretability, and expediting training times — factors that are critically important for complex models such as the Random Forest Classifier and MLP Classifier.

Post-training, the validation of our models through ROC_AUC plots and Confusion Matrix statistics affirmed the reliability and precision of our predictions. These steps, from meticulous EDA to careful model validation, culminated in the generation of Flood Susceptibility Maps. These maps provide an invaluable tool, transforming model outputs into a visually interpretable format, aiding in risk assessment and strategic planning for flood mitigation.

Overall, this work not only underscores the synergy between data science and geospatial analysis but also offers actionable insights for disaster management authorities, helping to fortify communities against the perennial threat of floods. Through this fusion of analytics and applied science, we set a new benchmark for environmental risk assessment and the proactive management of natural catastrophes.