0

I am currently working on analyzing data structure and size of a shapefile. To analyze it, I was wondering how much geometry column consumes the size of a shapefile. Unlike other features, geometry(in this case, polygons) consists of numerous x,y coordinates values, pretty clear it occupies a lot of memory.

So I tried to check it using Geopandas; a GIS python library. After I read my shapefile, I checked the memory usage of each feature in a data frame using sys.getsizeof() function. But what I found was quite different from what I expected.

This is df.info()

<class 'geopandas.geodataframe.GeoDataFrame'>
RangeIndex: 384 entries, 0 to 383
Data columns (total 11 columns):
BAS_MGT_SN    384 non-null object
CTP_KOR_NM    384 non-null object
SIG_CD        384 non-null object
SIG_KOR_NM    384 non-null object
BAS_ID        384 non-null object
BAS_AR        384 non-null float64
NTFC_DE       384 non-null object
MVMN_DE       0 non-null object
MVMN_RESN     0 non-null object
OPERT_DE      0 non-null object
geometry      384 non-null object
dtypes: float64(1), object(10)
memory usage: 33.1+ KB

My dataframe consists of 10 feature data columns and one geometry column.

And I checked how much memory a row(a record) uses by using sys.getsizeof() and it shows like this:

col name ->  BAS_MGT_SN         value :  4146000265     dtype :  class 'str'  size :  59
col name ->  CTP_KOR_NM         value :  Kyungki    dtype :  class 'str'  size :  80
col name ->  SIG_CD     value :  41460  dtype :  class 'str'  size :  54
col name ->  SIG_KOR_NM         value :  Yongin    dtype :  class 'str'  size :  80
col name ->  BAS_ID     value :  16868  dtype :  class 'str'  size :  54
col name ->  BAS_AR     value :  0.141141       dtype :  class 'numpy.float64'        size :  32
col name ->  NTFC_DE    value :  20121221       dtype :  class 'str'  size :  57
col name ->  MVMN_DE    value :  None   dtype :  class 'NoneType'     size :  16
col name ->  MVMN_RESN  value :  None   dtype :  class 'NoneType'     size :  16
col name ->  OPERT_DE   value :  None   dtype :  class 'NoneType'     size :  16
col name ->  geometry   value :  POLYGON ((965732.3805047646 1925904.324787525, 965732.3724840151 1925904.298925773, 965728.1308961953 1925890.622536143, 965709.8040791262 1925855.932488598, 965665.4649813089 1925773.647927837, 965628.0741399537 1925697.184387997, 965589.7351665759 1925621.005746239, 965575.007362393 1925588.275926026, 965563.072969694 1925556.380975007, 965562.0070495281 1925551.673805262, 965556.4711060586 1925527.226804454, 965551.9772138182 1925499.73081674, 965546.2518685962 1925451.089254094, 965550.1711595334 1925413.452517548, 965554.6643679787 1925392.215312426, 965550.1400685157 1925390.53029377, 965543.0127732882 1925387.875918985, 965531.3777784419 1925383.62694081, 965508.8709748472 1925375.408849849, 965490.4752945246 1925364.857950098, 965475.1869396837 1925356.86667663, 965460.2656082716 1925347.579959064, 965455.2881759857 1925339.749208257, 965443.0769325568 1925337.306582622, 965414.741067638 1925327.140993681, 965358.0324741391 1925308.728278397, 965357.5379455724 1925345.163022841, 965357.1247426887 1925375.601619988, 965357.0046657993 1925484.529800648, 965357.2732194763 1925743.095984805, 965357.4419149611 1925905.801176198, 965376.0565883868 1925905.376087978, 965419.6142546471 1925904.474529698, 965437.527387828 1925903.707274571, 965494.7467207256 1925903.349401935, 965503.9531586345 1925905.233092366, 965592.9315690342 1925905.097507654, 965603.3192982257 1925905.037703028, 965610.4763579519 1925904.996942925, 965692.697546525 1925904.523152333, 965732.3805047646 1925904.324787525))
dtype :  class 'shapely.geometry.polygon.Polygon'     size :  56

To me, it is very wierd that a single string like CTP_KOR_NM data consumes 80 bytes while geometry(82 floats) consumes only 56 bytes.

Geometry should take more memory allocation in my perspective, I guess it is because sys.getsizeof function only counts visible data, doesn't count internal data that are linked by pointers.

I googled to find a way to calculate actual memory use in python, but no way to find it. (I tried to make my own size counting function, doesn't work... There was no way to disassemble(decompose) polygon obj and count it)

Is there any way I can find how much memory(or in total file size)the features of a shapefile consume?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.