Working as a Python Development Project Team with ArcGIS?

Question

We have a development project in Python (ArcGIS 10). This project involves a mixture of toolboxes, map templates, layer files, file geodatabase templates (acting as templates being imported into a map by scripts) and various other things.

We use Eclipse as our source editor and SVN as our Source Code Repository.

Though we have a problem with keeping all the files (that aren't py files) in a synchronised project by everyone. The toolbox routinely gets messed up by multiple people editing the toolbox and then template files get adjusted and then not updated for other people as they are not checked back in.

How are people in organisations with more than one python developer on a company toolbox project ensuring that the project and all the different files are get versioned and managed correctly? Or is it a case through everything goes into Eclipse (including template layers and GDB's used by the scripts) into the project and hope that people check out the files correctly?

Answer 1

If I understood well, one of your problems is that developers are not using properly the SVN and this, lets the content in the SVN repository unstable.

So maybe you can try a couple of things:

Set a clear repository use policy

Make it clear to all the developers the way to use the repository and when and what to commit. So the repository always have working copy of the project.

Use a distributed control system

If you use a distributed control system like Git or Mercurial, each user can commit to their repository and only send their versions to a centralized one when they are sure it will work, you can maybe even set windows of commit for each user so they don't step on each others code.

Saying this, in your case I'd go for Mercurial, because it's developed in Python and you can create hooks to tailor it to your needs. And because the learning curve from SVN it's quite easy... a good point to start is hginit tutorial, that actually have a section called SVN re-education.

Answer 2

If I know that I am going to working with other developers, one of the first things I do nowadays is to setup a Continuos Integration server like Jenkins.

The idea is to always trigger your test suite after every check-in and you will get an automated e-mail right away if it fails. In your case, it could be a simple Selenium script. That clicks around a browser, or some ArcObjects script that automates ArcMap. There are several presentations out there about Selenium.

The cool thing about Jenkins, is that there are several plugins that allow you to integrate/leverage other technologies (build systems, lint, etc). You can get awesome reports about how much of your code is being covered by the test. They are really easy to setup.

Personally, instead of SVN, I like to integrate with Git and GitHub... there are several advantages of doing this like relying on GitHub for authentication.

But of course, the first step is to get Jenkins running. If you have never done it, reserve one day and breathe a lot since it can be very quirky... but once you have it running, it is really awesome.

Answer 3

I think you need somebody in charge and more accountability. My suggestion would be to appoint or recruit an administrator for the toolbox(es). Make the public toolbox and everything in its space read-only except to the administrator. The administrator can be responsible for making sure things get tested, checked in (or managed--in the case of items outside of SVN space). Since the administrator will get a chance to see what people are doing, he/she will know when someone needs training, i.e., he can catch people doing things improperly.

Answer 4

This is a people problem rather than a technology problem. The toolbox and the template files are being edited outside of Source Control, so there is no control over it. These files should be in version control even though they are binary files and you can't diff or compare them. As a general rule of Thumb, anything that is not generated from your code, and is required to run or compile the code, should be under Source Control.

That way the entire project will be under the source control, and there will always be a working copy. The developers should edit the toolbox and template in their local version after locking and commit back when their local copy is working.

As to

...not updated for other people as they are not checked back in

This is a people problem, and unless all your developers understand why this is important, no amount of technology is going to help.

Answer 5

The toolbox routinely gets messed up by multiple people editing the toolbox

For this particular problem, we put our toolbox in an ArcSDE Database. I haven't try with other type of database! Unless two persons edit the same tool at the same time, it work great. Really less problems than with file toolbox (.tbx).