<< UPDATE >>96 XJS Rear Break Issues - Take a look and listen

Kevin,

Only way to find out is to start dismantling.
Remove the brake disc, start the car and verify that your stub axle is absolutely straight.
If it’s indeed straight the rest is relatively easy. If not you need a new one.
Next remove the hub and inspect the bearings and U-joint.
New discs for sure.
And since you will be there, check also you lower pivot bearings and seals.