The only time I noticed it swap sides is when he was looking in a mirror. We were seeing his reflection.
I could be wrong , but I noticed in some scenes it was on the left and right. Maybe we need to watch more closely, I believe they're trying to play a joke on us.