What could this be..misfire on first trip.. then good on return trip

I have seen many posts about the X-Type running rough or misfiring when it’s cold. But this is weird. My 2001 X-Type misfires on cylinder 5 on my first trip off the day. ok, simple enough… but the weird part is it will drive rough, and misfire throughout my first journey whether that is a 5 minute trip to the supermarket or 15 mins across town. Park the car and do my shopping, when I return to drive home the car runs perfectly. I may have only been 10minutes. It does this cycle every time. I recently replaced the large O ring seals on the IMT valves, no change. Number 5 cylinder is under the manifold, will change out the coil and plug when I get there soon but this business of the car instantly coming right is baffling me. Other codes I have are for the oxygen sensor 2/2 and a knock sensor. But these readings I had prior to this current behaviour.

My first XJ40 2.9 used to run on 5 1/2 cylinders until it was warm, turned out one of the plugs had a small amount of build up on the part that screws onto the top of the plug, cleaned that off and it was back to running on 6.

Thanks Robin. This afternoon I swapped out the coil on cyl 5, seems better. But I need the wet and cold morning to see if it handles it. All my coils were replaced about 5 years ago, seems they’re not lasting.