Hi Richard-
A few new thoughts and summarizing some of what has been said above:
-
All of my statements below assume that you are SURE your battery is good. It’s been load tested (with a load tester like at autozone, not a volt meter) and has a full charge.
-
From the symptoms you are describing, it sounds like you have a parasitic draw. Simply put, when the key and everything else is “off,” something is still drawing current and eventually draining the battery. It could be a short (it seemed like that might be it when you said the problem was the starter, it is directly connected to the battery) or a faulty electronic component (e.g. trip computer, relay, etc.) The draw is “concealed” when the car is running because the alternator overcomes it by generating more power, just like it does when you turn on the headlights or AC.
-
Attempting to troubleshoot a parasitic draw on an XJS based on what happened on Other People’s cars is likely an exercise in frustration. There have been many different causes recorded over the years, from trip computers (me) to relays to the trunk light switch. Thus, a methodical approach is needed. It’s not difficult, just tedious. By using a methodical approach, your mechanic should be able to locate the draw with a very high degree of certainty. This will cost you money for time, but he should not be swapping out parts until he is sure he has it located. That’s just trial and error and expensive. Mark outlines the proper methodical approach on this other thread about as well as I’ve ever seen it done: Need input about battery voltage?
-
It seems like your car has a significant draw. Current is measured in amps, and most of us consider anything over 50mA (50/1000ths of an amp) to be too much draw. Even that will flatten the battery over the course of a few weeks.
-
Your measurement of the battery at 4.85V after a couple of days indicates a DEAD battery. Like really dead. Which means that IF the battery is GOOD, your draw is significant. Well over the 50mA threshold. Which means you’d get a pretty good spark when you hook up the last cable (positive or negative) to the battery.
-
Your mechanic will likely test for the draw using a multitester and the procedure Mark described. If he is patient and methodical, he’ll nail it down. Sometimes mechanics who are not patient or don’t understand electrics will just swap out the “likely suspect” parts, which is hard on your wallet and confidence.
Not sure that there is much new in the above, but maybe it helps you think through this as you work with your mechanic. I tend to question the experts, even when they are my friends. 
Good luck!
Bob