I am looking for a Python based tool/API (PyProj?) that will help me transforming a set of XY points from within a custom flat map to latitude and longitude.

I have information about the latitude and longitude of (0, 0) point and also latitude and longitude of (Xmax, Ymax) point. The total area is not wider or longer than 400 meters.

It looks to me that PROJ/PyProj can do the job, however I am still lacking understanding how to initialize the thing.

Also, these XY points can be given as a matrix/table, so it would be nice if the tool can transform the whole data in vector implementation without the need to loop over all values.