Comparison of open data portal solutions

Question

For many organizations, such as local/county/state governments, providing public access to GIS data is a challenge.

Simply knowing what data is available, what it is, who owns and maintains it, and where to get it is probably the most immediate challenge. Therefore a GIS data inventory is a necessity. Of course, creating one is one thing, getting people to maintain it (and assume responsibility for their data) it is quite another. Therefore getting buy-in from internal stakeholders is also very important.

Another important consideration is a solution's support for metadata. If the data cannot be described, it is almost useless. With some solutions there may be limited support for metadata in the form of a limited number of proprietary fields. Support for metadata in its full and original form seems to be rare.

There may be limitations on the size, number of records/attributes, field lengths, or the geometric complexity of common GIS data, such as address points or parcel data that might make some solutions unusable. Often, the sheer size of some types of data (e.g. LiDAR, aerial imagery) makes it difficult or impossible to host them over the internet in their raw form, and instead they must be delivered via physical media (e.g. hard drives). At best, these large files may be hosted on something like an FTP server or Dropbox/Box, but it may be quite costly to do so.

Additionally, in many implementations, the spatial nature of the data may not considered to be important - or considered at all - making finding, viewing and using spatial data spatially difficult or impossible. Things like being able to symbolize the data in any useful way may be nonexistent.

Lastly, keeping the data up-to-date is vital. If there is no automated way to update data then the whole thing becomes unworkable for non-trivial amounts of data.

So with that out of the way, the main question is:

• What open data portal solutions are available and how do they compare?

Side questions:

• What are the most important considerations when deciding on an open data portal solution?

• Are there scenarios where using more than one portal is beneficial (and cost-effective)?

• Can spatial and non-spatial data be successfully integrated into the same portal?

• Are there any examples of open data portal implementations you would consider to be successful or exemplary?

  • A list of some implementations is in List of US government free GIS data online as well as the Wikipedia article on geoportals.

Addendum:

• A list of portal solutions that we are currently evaluating:

  • ArcGIS Online
  • ArcGIS Open Data
  • Portal for ArcGIS Server (on-premises)
  • Socrata and Mondara
  • Google Maps Engine and Gallery

• Other portals we haven't yet considered:

  • DataHub
  • CKAN
  • DKAN
  • OpenGeoportal

• Not really portals but work well for serving flat files:

  • Dropbox
  • Box
  • FTP (on-premises)

• Rolling your own, either using hosted storage/servers or on-premises

In our case, because we have an ESRI ELA and have an ArcGIS-based spatial data infrastructure, it seems quite likely that at least one of the ArcGIS solutions will be selected.

Personally, I'd rather see us use more open source software, but it seems unlikely we'd be able to put something together that could compete with Esri's offerings, at least not without a paradigm shift in the way people use GIS in our organization.

Answer 1

This question has been converted to Community Wiki and wiki locked because it is an example of a question that seeks a list of answers and appears to be popular enough to protect it from closure. It should be treated as a special case and should not be viewed as the type of question that is encouraged on this, or any Stack Exchange site, but if you wish to contribute more content to it then feel free to do so by editing this answer.

---

One very popular and open source solutions is missing from this list: GeoNode.

GeoNode is 100% built on a open source stack, and it allows the development of geo-portals and SDI based on technologies such as GeoServer, PostGIS and pyCSW.

GeoServer provides mapping services based on OGC standards (WMS, WCS, WFS, WPS etc), PostgreSQL and PostGIS provides the spatial storage, pycsw (or alternatively GeoNetwork) provides catalogs services (OGC CSW).

Spatial datasets exposed by GeoNode can be easily harvested by CKAN through pycsw. GeoNode itself can harvest and expose standard remote services published by MapServer, GeoServer, ArcGIS Server and other mapping engine implementing OGC standards.

GeoNode is widely used from many organizations including the World Bank, the United Nations World Food Programme, the European Commission, the US State Department and many others and has a vibrant, friendly and very active open source community.

For more information visit the website: http://geonode.org/ and the documentation site: http://docs.geonode.org/en/master/

---

Another option, which may especially be suitable for ArcGIS users, is Esri Geoportal Server (not to be confused with Portal for ArcGIS.)

Esri Geoportal Server is not a part of the ArcGIS platform, and is a stand-alone application:

Esri Geoportal Server is a free, open source product that enables discovery and use of geospatial resources including datasets, rasters, and Web services. It helps organizations manage and publish metadata for their geospatial resources to let users discover and connect to those resources

One advantage for existing ArcGIS users is the ArcCatalog publish client, which allows existing metadata, which was created in ArcCatalog, to be pushed directly to the Geoportal Server.

---

The Open Data Catalog at http://www.opengeocode.org/opendata/ (I am a co-founder of) is a crowd-sourced listing of dataset related portals around the globe. In the GIS category, there are over 120 sites listed - mostly governmental (e.g., federal, state, county, municipal, etc).

---

This question is one my organization (a large Midwestern research university) is actively engaged in. We've been focusing our attention on OpenGeoportal (OGP), but as we're in many ways a very Esri-centric place, a lot of our development effort has been implementing support for working across Esri products to the open source side (previewing ArcGIS Server services in OGP, for example). I'll focus my answer on OGP and our experience with it.

I think a big question is how much developer time you have access to. We've been able to implement OGP with basically one part time graduate research assistant doing the coding and sys admin bits. Having Esri support would be attractive in some ways, but I so far I think having the code base open for hacking and adaptation outweighs it.

OGP utilizes a Solr index on the Java (Spring framework) backend, which is very speedy with tens of thousands of records, and via sharing potentially scales really well.

With Solr on the backend, the real power of OGP, and the aspect seeing a lot of active development (this, for example), is the ability to harvest metadata from other instances of OGP. With a robust governance structure currently in the works, there's an eye towards a future of a federated system of OGP instances sharing metadata across institutions and really improving the discoverability of spatial data.

That said, there are always concerns about FOSS projects running out of developer juice and dying on the vine. Yet, its embrace of open standards would make any future move out of OGP far easier than could be the case with more "secretive" solutions on the list.

A small note is that the state of Minnesota is launching a CKAN powered geospatial commons this summer, hopefully tying together what had long been a laundry list of different agencies housing and distributing GIS data in myriad ways. You can view a BETA version here.