Understanding multiple network interfaces, network topology (and a bit of LACP)

I've been running TrueNAS on my dedicated server for a couple of years now and haven't had any major issues with it. The hardware has two 1GbE network interfaces, so I've typically mapped all file shares to the primary port and tried to direct apps (immich, bittorrent) to use the secondary port. It *kinda* works.

Recently I upgraded one of the switches in my network to one that can handle faster throughput (2.5G/10G ports) but I have limited ports on it so I can't dedicate two ports to the NAS.

I thought it would be efficient to connect TrueNAS port 2 to the switch built into the router (to be closer to the WAN) and TrueNAS port 1 to the faster switch (to be closer to all the file share clients). I got error messages from the router complaining that duplicate IP addresses had been found - it was reading both ports as having the port 2 IP address (despite this being hard coded).

I'm not exactly sure what is going on - but I suspect that traffic from apps comes in via port 2 but goes back out via port 1 with the IP address of port 2 (I have no direct evidence to prove this). Somehow this doesn't happen when both ports are plugged into the same switch.

To resolve this I moved both ports to a nested 1Gb switch and this seems to have solved the problem (though now all traffic is nested and capped at 1Gb combined).

Anyone have any experience with this - am I doing something stupid with my network configuration? Is something set up wrong with my apps? Would link aggregation do *anything* useful for me?