10 More Useful Python Packages for Data Science Projects

Photo by RealToughCandy.com

Python is the most beginner-friendly programming language. With the help of fundamental packages like numpy and scipy, Python stands out as the best language for data processing and machine learning. Thanks to the contributions of experts and enthusiastic developers, numerous Python packages have been developed to assist data professionals in their work.

In this post, we will introduce 10 more unique and useful Python packages that can assist you in building your Python projects in various ways.

1. tqdm

When you need to iterate or loop, and you want to display a progress bar, tqdm is what you need. This package provides a simple progress meter in your notebook or command prompt.

First, let’s start with installing the package.

pip install tqdm

Then, you can use the following code to display a progress bar during a loop.

from tqdm import tqdm
q = 0
for i in tqdm(range(10000000)):
  q = i +1

Just like the gif image above, it can show a nice progress bar in a notebook. It becomes very handy when dealing with complex iterations and you want to track the progress.

2. Emoji

As the name suggests, Emoji is a Python package that supports emoji text parsing. Usually, dealing with emojis in Python can be challenging, but the Emoji package can assist in the conversion.

Install the Emoji package with the following command.

pip install emoji

Then input the code below:

import emoji
print(emoji.emojize('Python is :thumbs_up:'))

Output is : Python is 👍

With this package, you can easily output emojis.

3. Numerizer

Numerizer can convert written numerical text into corresponding integers or floating-point numbers.

pip install numerizer

Now, let’s try a few inputs for conversion.

from numerizer import numerize
numerize('forty four')

Output : 44

It can also work with alternative writing styles.

numerize('forty-four')

Output : 44

numerize('two and three quarters')

Output : 2.75

If the input is not a numerical expression, it will be preserved:

numerize('maybe around two and three quarters')

Output : may around 2.75

4. WeightedCacls

Weightedcalcs is used for statistical calculations. Its usage ranges from simple statistics, such as weighted averages, medians, and standard deviations, to weighted counts and distributions.

pip install weightedcalcs

Calculate a weighted distribution using the available data.

import seaborn as sns
df = sns.load_dataset('mpg')

import weightedcalcs as wc
calc = wc.Calculator("mpg")

Then, we perform weighted calculations by passing a dataset and computing the desired variables.

calc.distribution(df, "origin")

output is:
origin
europe 0.208616
japan 0.257042
usa 0.534342
Name: mpg, dtype: float64

5. Cerberus

Cerberus is a lightweight Python package for data validation.

pip install cerberus

The basic usage of Cerberus is to validate the structure of a class.

from cerberus import Validator
schema = {'name': {'type': 'string'}, 
          'gender':{'type': 'string'}, 
          'age':{'type':'integer'}}
v = Validator(schema)

Once the structure to be validated is defined, instances can be validated.

document = {'name': 'john kerry', 'gender':'male', 'age': 55}
v.validate(document)

output: True

If a match is found, the Validator class will output True. This way, we can ensure that the data structure is correct.

6. Maya

Maya is used to parse DateTime data as effortlessly as possible.

pip install maya

Then, we can easily obtain the current date using the following code.

import maya
now = maya.now()
print(now)

It can also be used for tomorrow’s date.”

tomorrow = maya.when('tomorrow')
tomorrow.datetime()

output :
datatime.datatime.(2023, 12, 6, 8, 44, 10, 141499,tzinfo=<UTC>)

7. category_encoders

category_encoders is a Python package for encoding categorical data (converting it into numerical data). It is a collection of various encoding methods that can be applied to various categorical data as needed.

pip install category_encoders

The transformation can be applied using the following example.

from category_encoders import BinaryEncoder
import pandas as pd

df = pd.read_csv("engine.csv")
enc = BinaryEncoder(cols=['origin']).fit(df)
numeric_dataset = enc.transform(df)
numeric_dataset.head()

8. Multiset

The Multiset class is similar to the built-in set function, but it allows the same character to appear multiple times.

pip install multiset

You can use the following code to utilize the Multiset function.

from multiset import Multiset
my_set = Multiset('aab')
print(my_set)

output : 
Multiset({'a': 2, 'b':1})

9. handcalcs

handcalcs is used to simplify mathematical formula processes in notebooks. It converts any mathematical function into its equation form.

pip install handcacls

Use the following code to test the handcalcs package. Use the %%render magic command to render Latex.

import handcalcs.render
from math import sqrt
%%render
a = 5
b = 8
c = sqrt(3*a + b/7)

10. Combo

Combo is a Python package designed for combining machine learning models and ensembling arrays. The package provides a toolbox that allows training various machine learning models into one integrated model, enabling the integration of models.

pip install combo

We’ll use the breast cancer dataset from scikit-learn and various classification models to create a machine learning ensemble.

from sklearn.tree import DecisionTreeClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.neighbors import KNeighborsClassifier

from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer

from combo.models.classifier_stacking import Stacking
from combo.utils.data import evaluate_print

Next, let’s examine a single classifier for predicting the target.

# Define data file and read X and y
random_state = 42

X, y = load_breast_cancer(return_X_y=True)X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4,random_state=random_state)

# initialize a group of clfs
classifiers = [DecisionTreeClassifier(random_state=random_state),
                   LogisticRegression(random_state=random_state),
                   KNeighborsClassifier(),
                   RandomForestClassifier(random_state=random_state),
                   GradientBoostingClassifier(random_state=random_state)]

clf_names = ['DT', 'LR', 'KNN', 'RF', 'GBDT']

for i, clf in enumerate(classifiers):
    clf.fit(X_train, y_train)
    y_test_predict = clf.predict(X_test)
    evaluate_print(clf_names[i] + '   |   ', y_test, y_test_predict)
    print("\n")

Output :
DT   | Accuracy: 0.9386, ROC:0.9383, F1:0.9521
LR   | Accuracy: 0.9693, ROC:0.962,  F1:0.9766
KNN  | Accuracv: 0.9561, ROC:0.9519, F1:0.9662
RF   | Accuracy: 0.9781, ROC:0.9716, F1:0.9833
GBDT | Accuracy: 0.9605, ROC:0.9524, F1:0.9699

Utilize the Stacking model from the Combo package.

clf = Stacking(classifiers, n_folds=4, shuffle_data=False,
                   keep_original=True, use_proba=False,
                   random_state=random_state)

clf.fit(X_train, y_train)
y_test_predict = clf.predict(X_test)

evaluate_print('Stacking | ', y_test, y_test_predict)

output:
Stacking | Accuracy: 0.9781, ROC:0.9745, F1:0.9832

Summary

In this post, we summarized 10 unique Python packages useful in your projects. Most of the packages are easy to use and straightforward, but some may have more advanced features that require further reading of their documentation. Have a fun to use them in your projects.

Thanks for your reading.