Saving A Pandas Dataframe To Separate Jsons Without Nans

I have a dataframe with some NaN values. Here is a sample dataframe: sample_df = pd.DataFrame([[1,np.nan,1],[2,2,np.nan], [np.nan, 3, 3], [4,4,4],[np.nan,np.nan,5], [6,np.nan,np.n

Solution 1:

Use apply to drop NaNs, groupby to group and dfGroupBy.apply to JSONify.

s = sample_df.apply(lambda x: x.dropna().to_dict(), 1)\
        .groupby(sample_df.index // 2)\
        .apply(lambda x: x.to_json(orient='records'))
s    

0            [{"0":1.0,"2":1.0},{"0":2.0,"1":2.0}]
1    [{"1":3.0,"2":3.0},{"0":4.0,"1":4.0,"2":4.0}]
2                            [{"2":5.0},{"0":6.0}]
dtype: object

---

Finally, iterate over .values and save to separate JSON files.

import json
for i, j_data in enumerate(s.values):
    json.dump(j_data, open('File{}.json'.format(i + 1), 'w'))

Solution 2:

I suggest:

with open("data.json","w") as fpout:
    fpout.write("{\n")
    for row_id in range(sample_df.shape[0]):
        fpout.write("\t"+ str(sample_df.index[row_id]) +":"+ sample_df.iloc[row_id].dropna().to_json(orient="index") +"\n")
    fpout.write("}\n")