The way that pandas handles empty values is diabolical

See if you can predict the exact output of this code block:

import pandas as pd
    
values = [0, 1, None, 4]
df = pd.DataFrame({'value': values}) 

for index, row in df.iterrows():
    value = row['value']
    if value:
        print(value)

Answer and explanation:

!Output:

1.0
nan
4.0

Explanation:

• The list of values contains int and None.
• Pandas upcasts the column to float64 because int64 cannot hold None.
• None values are converted to np.nan when stored in the dataframe column.
• During the iteration with iterrows(), pandas converts the float64 scalars. The np.nan becomes float('nan')
• Python truthiness rules:
  • 0.0 is falsy, so is not printed
  • 1.0 is truthy so is printed.
  • `float('nan') is truthy so it is printed.
  • 4.0 is truthy and is printed.!<

I've faced a lot of bugs due to this behavior, particularly after upgrading my version of pandas. I hope this helps someone to avoid the same woes.