Whilst i agree the guy was dangerous. I also do not think that you and the person in front should be sitting in the over taking lane... neither of you was overtaking at that point.
If you had both been in the left lane until ready to overtake he would have zoomed by on the right, illigally. But it would have reduced the risk.
Teeny tiny little willy