Weird cause of oscillation on the output of linear regulator (dodgy ground connection fixes it)

I've been working on a discrete linear regulator for a power supply project and overall it has been going well besides one problem with unusual symptoms.

​

The regulator is a constant current and constant voltage control loop which was stable on the breadboard after quite some work. I've put the circuit onto something more permanent now and the current limiting function causes the output to oscillate. I don't think this is caused by too much phase shift in the control loop however. The oscillation is ~62kHz, triangular and looks like one of the op amps is at max slew rate,

​

The circuit needs voltage and current references which are being generated through a separate (5V) supply. This is what it looks like. Where the ground of the circuit has been labelled on the picture is a ground bus in a breadboard and at this point all the grounds are combined and then all go back to the rectifier. When the connection at A is kinda a little loose the oscillation disappears, and the circuit otherwise works fine. Supplies power up to 2A without a problem and the current limiting is completely stable. If the connection at A is completely snug the system will oscillate. Jiggling around the wire at A which goes from the regulator board to the breadboard ground point will make the output randomly stabilize and oscillate.

​

I can even connect the linear regulator ground directly to the ground of the rectifier and it oscillates. It seems the circuit almost wants the connection to be a bit crappy and then it's fine. Does anyone have any ideas what could be causing this? The only thing I haven't tested yet is NOT using a different supply to generate the references. I've run into grounding issues before in this project when a large filter capacitor was sharing the same ground as the control circuity where it was producing 50Hz ground currents which showed up on the output. I'm not very good at knowing the best way to ground things. Any help would be appreciated, because I'm a novice and I could be mistaken that it has anything to do with the grounding in the first place. Thanks in advance I know this question is oddly specific.

​

Also I just realized the load is grounded through the control circuits ground which might also be bad.