Pandas: Remove Nan Only At Beginning And End Of Dataframe

I've got a pandas DataFrame that looks like this: sum 1948 NaN 1949 NaN 1950 5 1951 3 1952 NaN 1953 4 1954 8 1955 NaN and I would like to cut off th

Solution 1:

Use the built in first_valid_index and last_valid_index they are designed specifically for this and slice your df:

In [5]:first_idx=df.first_valid_index()last_idx=df.last_valid_index()print(first_idx,last_idx)df.loc[first_idx:last_idx]1950 1954Out[5]:sum1950    51951    31952  NaN1953    41954    8

Solution 2:

Here is one way to do it.

importpandasaspd# your data# ==============================dfsum1948  NaN1949  NaN1950    51951    31952  NaN1953    41954    81955  NaN# processing# ===============================idx=df.fillna(method='ffill').dropna().indexres_idx=df.loc[idx].fillna(method='bfill').dropna().indexdf.loc[res_idx]sum1950    51951    31952  NaN1953    41954    8

Solution 3:

Here is a an approach with Numpy:

importnumpyasnpx=np.logical_not(pd.isnull(df))mask=np.logical_and(np.cumsum(x)!=0,np.cumsum(x[::-1])[::-1]!=0)In [313]:df.loc[mask['sum'].tolist()]Out[313]:sum1950    51951    31952  NaN1953    41954    8