You need to set the ellipsoidalMode to True:
distance.setEllipsoidalMode(True)
And then you might need to set the ellipsoid over which QGIS will perform calculations, e.g., WGS84:
distance.setEllipsoid('WGS84')
Now you can measure the distance once again:
m = distance.measureLine(point1, point2)
Which will give you ~322.475m.