How Do I Quantize Data In Pandas?

November 19, 2023 Post a Comment

I have a DataFrame like this a = pd.DataFrame(a.random.random(5, 10), columns=['col1','col2','col3','col4','col5']) I'd like to quantize a specific column, say col4, according to

Solution 1:

Perhaps qcut() is what you're seeking. Short answer:

df['quantized'] = pd.qcut(df['col4'], 5, labels=False )

Longer explanation:

>>>import pandas as pd>>>import numpy as np>>>df = pd.DataFrame(np.random.randn(10, 5), columns=['col1','col2','col3','col4','col5'])>>>df
       col1      col2      col3      col4      col5
0  0.502017  0.290167  0.483311  1.755979 -0.866204
1  0.374881 -1.372040 -0.533093  1.559528 -1.835466
2 -0.110025 -1.071334 -0.474367 -0.250456  0.428927
3 -2.070885  0.095878 -3.133244 -1.295787  0.436325
4 -0.974993  0.591984 -0.839131 -0.949721 -1.130265
5 -0.383469  0.453937 -0.266297 -1.077004  0.123262
6 -2.548547  0.424707 -0.955433  1.147909 -0.249138
7  1.056661  0.949915 -0.234331 -0.146116  0.552332
8  0.029098 -1.016712 -1.252748 -0.216355  0.458309
9  0.262807  0.029040 -0.843372  0.492120  0.128395

You can use pd.qcut() to get the corresponding range.

>>> q = pd.qcut(df['col4'], 5)
>>> q
0       (1.23, 1.756]
1       (1.23, 1.756]
2     (-0.975, -0.23]
3    [-1.296, -0.975]
4     (-0.975, -0.23]
5    [-1.296, -0.975]
6       (0.109, 1.23]
7      (-0.23, 0.109]
8      (-0.23, 0.109]
9       (0.109, 1.23]
Name: col4, dtype: category
Categories (5, object): [[-1.296, -0.975] < (-0.975, -0.23] < (-0.23, 0.109] < (0.109, 1.23] < (1.23, 1.756]]

You can set parameter labels=False to get the integer representation

>>>q = pd.qcut(df['col4'], 5, labels=False)>>>q
0    4
1    4
2    1
3    0
4    1
5    0
6    3
7    2
8    2
9    3
dtype: int64

First argument is an array or Series.
Second argument is number of quantiles you'd like.
Documentation here for more options. http://pandas.pydata.org/pandas-docs/stable/generated/pandas.qcut.html

Solution 2:

Most pandas objects are compatible with numpy functions. I would use numpy.digitize:

import pandas as pd

a = pd.DataFrame(pd.np.random.random((5, 5)), columns=['col1','col2','col3','col4','col5'])
#       col1      col2      col3      col4      col5#0  0.523311  0.266401  0.939214  0.487241  0.582323#1  0.274436  0.761046  0.155482  0.630622  0.044595#2  0.505696  0.953183  0.643918  0.894726  0.466916#3  0.281888  0.621781  0.900743  0.339057  0.427644#4  0.927478  0.442643  0.541234  0.450761  0.191215

pd.np.digitize( a.col4, bins = [0.3,0.6,0.9 ]  )
#array([1, 2, 2, 1, 1])

Solution 3:

Pandas has a built in function pd.cut which allows you to specify bins and labels. Following Dermen's example:

Baca Juga

df = pd.DataFrame(pd.np.random.random((5, 5)), columns=['col1', 'col2', 'col3', 'col4', 'col5'])
#        col1      col2      col3      col4      col5# 0  0.693759  0.175076  0.260484  0.883670  0.318821# 1  0.062635  0.413724  0.341535  0.952104  0.854916# 2  0.837990  0.440695  0.341482  0.833220  0.688664# 3  0.652480  0.271256  0.338068  0.757838  0.311720# 4  0.782419  0.567019  0.839786  0.208740  0.245261

pd.cut(df.col4, bins = [0, 0.3, 0.6, 0.9, 1], labels=['A', 'B', 'C', 'D'])
# 0    C# 1    D# 2    C# 3    C# 4    A# Name: col4, dtype: category# Categories (4, object): [A < B < C < D]