Skip to main content

Questions tagged [spatial-etl]

ETL (extract, transform, load)

Filter by
Sorted by
Tagged with
0 votes
0 answers
23 views

OData Copernicus API single bands

I'm writing a python pipeline to download and process some S2 data and I am using the Copernicus API for the download part. As I only need two bands (B04 and B08), I was wondering whether it's ...
EAlbergo's user avatar
0 votes
0 answers
49 views

Upload a CAD file into ArcGIS Portal as a hosted layer

ArcGIS Portal has the ability upload a shapefile and have this become a hosted layer. Is there any pathway to upload a CAD file and have this become a hosted layer? CAD is a supported data type for ...
Stephen Lead's user avatar
  • 21.2k
0 votes
1 answer
1k views

Using dynamic feature types in FeatureReader (Excel) constraints

I am new to FME. I have 100+ Excel workbooks, each of which has 2-4 worksheets in them. To prep to do some schema mapping, I'm trying to read out all of the attribute names to a new spreadsheet (Excel ...
traitortots's user avatar
1 vote
1 answer
336 views

How to resolve this error in my Python code: Warning: not all FMESessions that were created were destroyed before shutdown. This may cause instability

I am using the Data Interoperability extension. I created 2 spatial ETL tools and added each to a model in my ArcGIS Pro project. When run manually, both tools and models run just fine but when I put ...
Tiffany WB's user avatar
0 votes
2 answers
290 views

FME 2019: Generate a new PK based on a concatenation of a string and a three digit increment

I'm transforming a number of geographic tables from one MCD to another, so I'm using FME (I'm new to FME). I can map 80% of my fields with out any problem (MCD_A.fld_1 = MCD_B.fld_1) but in the in the ...
lhowarth's user avatar
  • 613
-1 votes
1 answer
103 views

How to create polygons from box type coordinates using ET GeoWizards?

I have a table that has an ID and coordinates for Left, Right, Top, Bottom and I would like to create rectangles out of those coordinates. This rectangles should have around 4 km per side. Using ET ...
alecsx's user avatar
  • 618
-1 votes
1 answer
765 views

Best (Spatial ETL) Tool open Source [closed]

What is the best Spatial ETL Tool opensource can I use it?
GIS_Q's user avatar
  • 57
0 votes
0 answers
174 views

Use a map service exposed via ArcGIS REST API as a data source in Data Integration Extension ETL tool

I am trying to use the SpatialETL tool in ESRIs Data Integration Extension to cut out rasters from a map service exposed via a REST URL on the following format http://utility.arcgis.com/usrsvcs/...
lijat's user avatar
  • 153
1 vote
2 answers
90 views

Merging feature class with field mappings using datum transformation?

I suspect that there is no easier way to do this, other than developing this as a two-step process with a temporary output (which seems somewhat wasteful). If I have to, I'll just create a temporary ...
jbchurchill's user avatar
  • 4,489
2 votes
2 answers
484 views

FME and WebDAV resources

FME is capable of reading and writing from and to a broad variety of data sources and formats, but is it somehow possible to read/write from/to a WebDAV resource with FME? A cloud storage solution ...
Eva Großmann's user avatar
6 votes
3 answers
4k views

How to create autoincrementing primary key with dynamic writer in FME

I'm writing 3 shapefiles to an MSSQL Spatial database with FME. I'm using a writer with dynamic settings to simplify future updates of the input files. In order to create a spatial index with the ...
hexamon's user avatar
  • 2,656
4 votes
1 answer
1k views

FME Error when writing to relationship class in ArcSDE Geodatabase writer type

I'm working in the FME workbench through ESRI Data Interoperability extension (ArcGIS 10.4.1). I'm writing to an SDE geodatabase with relationship classes established. When attempting to setup the ...
Tiffany's user avatar
  • 41
2 votes
1 answer
295 views

Transforming from BOX2DF to BOX

I am using some legacy code which requires my data in PostgreSQL box type e.g. this format (37357,50073),(34242,357). However, I am using PostGIS with box2df i.e. I get: BOX2DF(588893 4515997, 600342 ...
audlift-bit's user avatar
2 votes
1 answer
254 views

How can I get GeoKettle (or any other Java based ETL) to run on MacOS Sierra? [closed]

Has anyone managed to get GeoKettle or Spatial extension for Talend running on MacOS Sierra? If so, how did you do it? These packages have not been updated since 2015 and Apple has more or less ...
Aron's user avatar
  • 163
1 vote
1 answer
698 views

Execute a batch process of a spatial tool within an ArcGIS Toolbox

I have created a Data Interoperability Spatial ETL tool. I do have a Data Interoperability license. I want to create a nightly batch process to run this tool. As suggested, I have created a python ...
Snehil Kandpal's user avatar
8 votes
1 answer
2k views

Is GeoKettle still under development? [closed]

GeoKettle (http://www.spatialytics.org/projects/geokettle/) is (or was?) a very promising approach to spatial ETL with an intuitive graphical process design, at least introduced an open source ...
Jochen Schwarze's user avatar
3 votes
0 answers
1k views

Can QGIS do automated spatial ETL? [closed]

Can QGIS do automated spatial ETL? ... especially the automated point I'm aware of tools like FME, Talend, etc. but price is a concern, so I'm looking for FOSS alternatives that provide spatial ETL ...
SteveC's user avatar
  • 547
3 votes
2 answers
94 views

Simplifying a feature class to feature class ETL using multiple input feature classes with varied schemas

I'm wondering if anyone can help me brainstorm how to simplify this ETL process? The way I have it written, it just feels very repetitive, but I can't think of a way to avoid the repetition. I'll ...
Kristen G.'s user avatar
  • 1,100
1 vote
1 answer
326 views

Spatial ETL for MutliLineString [closed]

I'm pretty new to Spatial ETL. In accordance with this question I have experience the geokettle v3 but unfortunately i'm having a several problems working with it. There is some issues that aren't ...
ePascoal's user avatar
  • 951
1 vote
0 answers
322 views

Migrating ArcSDE data to Oracle Spatial?

We've got all the geospatial information of our company within an ArcSDE Database (over oracle Server): points lines polygons We consume these data with ArcGIS Desktop and with ArcGIS Server. We ...
Tharu's user avatar
  • 11
2 votes
1 answer
986 views

How to print log from FME when run from python script?

I created a Spatial ETL ToolBox in ArcCatalog. When I run the tool from ArcCatalog I see log massages. But when I run this tool from a python script, the logs do not print to the python console. ...
user7172's user avatar
  • 1,529
1 vote
0 answers
149 views

ArcGIS - Spatial ETL Tool - Checking fields in input shapefile

How to check if the input file fields have the expected name and type? Which transformer will be useful to validate names and types of attributes of reader. edit: I have a shapefile that transforms ...
Stan's user avatar
  • 11
0 votes
2 answers
1k views

Spatial ETL Tools won't run "AttributeError: 'module' object has no attribute, etc."

I have a python script that runs 5 Spatial ETL tools that I wish to run as a scheduled task. Each of the tools will run successfully from ArcCatalog and also from the ArcCatalog Command line. If I ...
Chris Rado's user avatar
2 votes
1 answer
1k views

How to transform JSON with field x,y to GDB in FME

I have JSON file with x,y field. I wonder how to transform geometry to GDB? Example of JSON: [ { "OBJECTID": "1", "x": "15", "y": "21" }, { "OBJECTID": "2", "x": "16", "y"...
user7172's user avatar
  • 1,529
1 vote
2 answers
1k views

"ERROR 000819: Tool is not assigned" While Running Spatial ETL Tool?

I have Created a spatial ETL tool to do small operations like adding an ID, updating different fields, etc. While running it from the Edit Session of Spatial ETL it runs well and good. http://help....
Mayur Sharma's user avatar
1 vote
0 answers
840 views

Seeking Spatial ETL or some other tools? [closed]

I need to read geographic data from an ORACLE spatial database in Gauss Boaga SRS, convert them in WSG84 (EPSG 32632), e put them in another ORACLE spatial data base. I've tried to use GeoKettle 2.5 ...
Cesare's user avatar
  • 2,011
2 votes
1 answer
2k views

FME: how to extract the coordinate of the North West corner

In an FME translation from a polygon geodatabase feature class to a DGN I am trying to place a label in the top left corner of a box. I have tried using CoordinateExtractor to find the coordinates of ...
whatahitson's user avatar
  • 1,896
1 vote
0 answers
545 views

Geokettle transformation for FileGDB to SHP

I want to transform FileGDB layer to Shapefile on Geokettle. Firstly i added OGR Input and than i added a Shapefile Output. And i entered parameters shown on the screenshot but i got errors for ...
mete7's user avatar
  • 607
1 vote
1 answer
1k views

tools and ETLs for migrating data benchmarking - from shapefile to SQL SERVER

i need to migrate my shapefiles to my database SQL SERVER, and i don't know what's the best tool or ETLs to choose, i already tried FME Desktop and i succeded to load the shapefiles on my db, but ...
Geographic's user avatar
5 votes
3 answers
4k views

Migrating PostGIS spatial data to SQL Server Spatial [duplicate]

I am trying to migrate PostGIS geometry to SQL Server Spatial data. It's the first time I'll work with SQL Server Spatial. Is there any Data ETL model that does this migration?
Geographic's user avatar
13 votes
2 answers
2k views

Migration from ArcSDE (SQL Server) to PostGIS (PostgreSQL)

We've got all the geospatial information of our company within an ArcSDE Database (over SQL Server): points (locations of the antennas), lines (elevation, rivers), polygons (protected areas, houses). ...
SonOfabox's user avatar
  • 809
3 votes
1 answer
1k views

Is Data Interoperability Extension for ArcGIS Server required to extract data in .dgn format?

I am trying to create a Geoprocessing Task in my .mxd to in order to extract data to .SHP and .DGN formats which will be utilized in a data extraction tool for a web application. Right now, there is ...
carriewiley's user avatar
10 votes
4 answers
10k views

Transforming from DWG/DXF to GIS?

I've got tons of DWG/DXF files with information (in several layers) dealing with: Contour lines and elevation (lines) positions of antennas (points) protection areas (polygons) administrative ...
SonOfabox's user avatar
  • 809
9 votes
0 answers
3k views

Open source alternative to FME from Safe Software for Spatial ETL? [duplicate]

Just wondering if there is any open source solution which comes close to safe FME? I would like to integrate it into my workflow, but it´s just another few thousand which my employer has to give out ...
Robert Buckley's user avatar
1 vote
2 answers
5k views

Using FME to Parse An Attribute Field

I have a table that contains a field with street names. Each street name ends with the street type. Fore example, Washington St, Happy Valley Rd, Redstone Dr. I need to extract the last part of the ...
dchaboya's user avatar
  • 984
3 votes
0 answers
118 views

Determine the Degree of Data Lossiness when Converting Data

I'm working through some of the ESRI tutorials on Data Interoperability and it briefly mentions the issue of data conversion being a lossy process. Some questions that come to mind are: What is ...
dchaboya's user avatar
  • 984
41 votes
6 answers
23k views

Migrating geodatabase data into PostGIS without Esri apps?

Can you tell me how I can load geodatabase data into PostGIS without having to install Esri apps? The solution I found online talked about using ArcGIS to convert GDB to SHP and then importing SHP ...
Gerson Galang's user avatar
5 votes
2 answers
704 views

Data ETL workflow optimization

I have a legacy workflow that I would like to optimize and automate (at least partially). The current workflow goes like this. I receive a MS Access .mdb file containing a single table. This table ...
Chris M's user avatar
  • 1,013
4 votes
2 answers
1k views

Setting output of spatial ETL tool in ModelBuilder?

I am using the data interoperability extension and building spatial ETL tools, but I can't seem to specify the output location of the ETL tool so it's impossible to use this dynamically. Has anyone ...
GuidoS's user avatar
  • 1,174
13 votes
3 answers
1k views

Centralizing shapefile data into database

I've got hundreds of shapefiles from various different GIS projects that I want to start consolidating into a single database platform, currently attempting this with Postgres / PostGIS. Hardly any ...
colemanm's user avatar
  • 1,347
33 votes
1 answer
20k views

Seeking options for Spatial ETL (Extract, Transform, Load)

This question has been converted to Community Wiki and wiki locked because it is an example of a question that seeks a list of answers and appears to be popular enough to protect it from closure. It ...
10 votes
1 answer
3k views

Seeking tools for uploading GIS data to database

I would like a nice graphical tool that allows me to take standard GIS data such as shapefiles and KML files and upload them to a database such as PostGIS, MySQL or Oracle. Are there any such tools?