1

How do you keep your development scripts in sync with your source code management (SCM) and your projects?

For instance right now I have things like D:\code\reusable-stuff that goes to an SCM (e.g. GitHub, Bitbucket, ...). Then in my projects I copy the particular scripts I want to use. However this is a pain and inefficient because it's very rare the script is just used as is. So I tend to avoid keeping things in sync, and rely on occasional big all-in-one lump commits that wave away many benefits of using revision control and inevitably leading to more headaches later.

To illustrate with a single concrete example:

There is a script D:\code\reusable-stuff\conversion\clip_all_layers.py (along with a dozen other scripts in the same folder, none of which I care about right now).

Clip_all is replicated in Project_2015_01\scripts\clip_all_layers.py, which is used in Project_2015_01\Toolbox.tbx\Awesome_Hicky_Model.

I discover an off-by-one bug which meant in some circumstances every 2nd layer is skipped.

I change and test in situ with the project data where the bug was encountered. It takes 3 hours to come to a clean working copy, with a dozen intermediate changes and tests. The first 3 changes are dutifully replicated to D:\code and committed, but the ones between r3 and r12 go bye-bye; too much work. Oh yeah, when committing r12-final from d:\code, another fix was discovered, so we had to replicate back to the project.

2 weeks later, I'm looking at the undocumented jump between r3 and r13 and scratching my head. Important logic is missing and I no longer remember enough to know why solution X was used, and today I need to know that.

There has to a smoother way.

Improve this question
3
  • 1
    Yes this seems like a pure programming Q and off topic for GIS SE. I don't think so though. Software development practices are increasingly front and central for GIS Professionals and how this is solved for us, but more importantly how best practices are documented, will be subtly but usefully different. With that in mind, please answer with educating GIS users as your target. – matt wilkie Feb 9 '15 at 19:13
  • We use version-ing clip_all_layers_v1.2.py, clip_all_layers_v1.2.tbx with clip_all_layers_v1.2.mxd etc. – Mapperz♦ Feb 9 '15 at 19:39
  • I am using the same clean (no layers, removed results) _wbench.mxd to do all processing. Toolboxes (with toolsets) all pointing to the same folder with subfolders. Scripts with relative paths option checked. Separate BACKUPS folders contains yes backups. – FelixIP Feb 9 '15 at 20:44
2

This is a Subversion centric answer as I don't use git et al..

For library code that gets reused in multiple python add-ins/python toolboxes/script tools, we include the library package/module in the add-in/toolbox/tools repo as an external file via an svn:externals property.

Updating the external files is automatic when an svn update is run on the add-in/toolbox/tools working directory

Although it can be a little dangerous, changes to external file get committed back to the main library code when changes to the add-in/toolbox/tools working directory are committed which helps keeping everything in sync (you can probably disable this, but I haven't looked into it).

Improve this answer
1
  • oh, externals is new to me, thanks. The git corollary seems to be submodules, with some notable differences, while mercurial has subrepository though it's tagged a "feature of last resort". – matt wilkie Feb 10 '15 at 17:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.