I'm in arcpy scripting for couple of years and after this time i've got a whole lot of scripts, packages and so on. The problems is that many of this scripts should be used over and over, many of them use the same logic (couple of point-to-point spatial joins with different limitations). Additional problem is that I reuse some part of this scripts: extentded arcpy.da.Cursor class for example (not only this one, it's about 5 classes and 5 modules with functions. So, 10 "scripts" in total)
Now the time to orginize this mess has came. At this moment I use project structure. One project - one toolset with it's own pack of scripts. I just copy from base location to project's one if I need my class in another project
Additional problem is that this "package" should be portable as easy as possible.
Almost all of this scripts/tools are used inside ArcMap environment.
Now I think of a structure like this:
_docs # sphinx-generated docs gptools # main "toolbox" --- __init__.py # access everything from outside and generate docs --- gptooset1 # same logic script. Just as ArcMap toolboxes. --- __init__.py # i could import tools from here --- gptooset1.tool1.py # single tool --- gptooset1.tool2.py --- gptooset2 --- ... scripts # main wideused classes, functions, etc. --- __init__.py # use it from outside --- class1.py --- ... --- functions1.py --- ... site-packages # third-party modules, should be imported from here --- module1 toolboxes # python-toolboxes to access tools from ArcMap
There is few issues. First of all I can't import class1 to gptoolset1.tool1.py with relative import. There is seems to be only one way to do it through
sys.path.append() which is
The second, the main one is that this structure is against Python basic structure.
scripts --- class1.py --- functions1.py tool1.py # tool1.py from scripts import class1 from scripts import functions1
Does anyone has an experience of organizing such large packages? How to organize it well from Python point of view and keep it as simple is possible? (bulk of
sys.path.appends would solve it but its awkward.)
UPD I don't think it's a general software question because it's ArcGIS specific. Python "software" uses up->down (complicated->easy) logic, when main modules are uses "small pieces of logic" by importing it. It gives a strict scheme (packages, subpackages and so on).
ArcGIS uses reversed logic: tools (python modules actually) are on the bottom of project structure and oftenly uses one common parts of code. If you put tools on top of your project you would get a mess of scripts or should copy-paste common parts from one location to anoter.
The question is how to combine this two completely different approaches. Why not SO? Because it's not that frequent case in python scripting but it is a part of ArcGIS environment.