So I can't summon more than one phantom now.

When I started the game I could summon 2 phantoms to help me out with no problem. Once I got to Road of Sacrafices, I can only summon one phantom at a time. I see tons of signs, but when I summon someone, they all disappear and no more show up. I use the dried fingers and get constantly invaded, but no more signs show up. Anyone else experiencing this or am I doing something wrong?