Using fan daggers to deal with ritual talismans

Why do people get mad when i use fan daggers to deal with their ritual talismans? I don’t want to swap talismans every fight so i don’t use them myself. Is throwing one fan dagger to level the playing field really that annoying?