How do you replace bulbs on Xenon headlights?
 

I can't find any guide to replace the bulbs for a xenon headlight on a V40/S40.

I took some pictures for reference https://imgur.com/a/ajiP3Tg
Do I have to remove the battery, and then undo the black cover to access the bulbs?

Which bulbs do you need to change?
For the H7 highbeam you need to remove the ballast (which you have done in photos) and then the rubber cap.
For the xenon you remove the black rubber cap, loosen (but not remove ) the screw and twist the retainer.
TBH due to access its easier to unscrew the 4 bolts and remove the complete headlight, there is a guide in the articles section or just search the forum.