Skills question

I must’ve not read it at all or misread, but am I not able to utilize passives from separate skill trees?

For instance on Vex, I can’t utilize the “Grave Sustenance” passive from the Here Comes Trouble tree while using the Fourth Seal skill tree?