3

I am trying to open an ASC file in QGIS for the first time. When I open the file below I get an error message:

Couldn't determine X spacing Raster layer Provider is not valid (provider: gdal, URI:

I noticed there is no header when I open the ASC file in notepad. The information looks like XYZ data.

enter image description here

The data is in this format.

enter image description here

How can I arrange the header for the ASC file or otherwise open this data in QGIS so it will load as a raster.

QGIS version 3.18.3-Zürich

0

4 Answers 4

7

The ASC it isn't

The cryptic error message you get from QGIS is because it sees the .asc file extension and thinks this is in Esri ASCII raster format. But then the file doesn't start with NCOLS, so it's clearly some other file format that happened to choose the .asc to show that it's ASCII Text. Despite the error message it is vector data (points, specifically), and nothing to do with rasters.

The ASC it is

What this does look like is a fixed length text file. It's theoretically possible that it could be tab delimited, but the file specification looks like it's defining field widths.

It looks like it conforms to the IBM DB2 Non-delimited ASCII (ASC) file format. However, the supporting file starting "MAGNETIC DATA FORMAT" does not seem to be part of the DB2 standard. However, we can take a guess at what it means:

  1. The first column lists the field names.
  2. The second column is the field type and length. The a8 means ASCII text, 8 characters wide. The f10.0 means floating point number, with 10 characters before the decimal point and zero characters after. The f9.5 means floating point with 9 characters before the decimal point and 5 after.
  3. The third column is the value used to indicate NULL within that column. If it appears, it will be in the format of that column. So an easting of -9999999.0 should be treated as NULL, while in the local time column it will be -99.00000 instead.

Fixed Length Text

QGIS must have libraries for dealing with fixed length text. The widely-used Shapefile format has attributes are stored in fixed length .dbf files in the dBase IV specification. But it doesn't seem to offer any way to open fixed length table. So we're going to need to convert it into a delimited text format before going near QGIS. There are at least three ways to do this.

(We used to use fixed length formats a lot more. In the days of slow storage like floppy disks and tape drives, you could read straight to the relevant row of a fixed-length file. But they were also forced a lot of limits on our storage, so we started using delimited text for flexibility. The lowest and most optimised layers of most databases still used fixed lengths, but the database driver means we don't have to handle this implementation detail.)

Option 1 - Import in a spreadsheet, export to CSV


Advantage - Allows you to see the table representation of your file.

Disadvantage - May round off floating point numbers losing precision.

  1. Open your .asc file in Microsoft Excel or LibreOffice Calc. (Other spreadsheets may work too, but I can confirm it for both of these. The screenshots below are from Excel, but Calc matches it very closely.)
  2. Under "Choose the file type that best describes your data" choose "Fixed width". Leave "Start import at row" at "1". Uncheck "My data has headers". You can probably leave "File origin" as "MS-DOS (PC-8)", but if this corrupts text, change to "20127 : US-ASCII".

Dialog box labelled 'Text Import Wizard - Step 1 of 3'. 'Choose the file type' is set to 'Fixed width', 'Start import at row' is'1', 'File origin' is 'MS-DOS (PC-8)' and 'My data has headers' is not checked.

  1. Select "Next".
  2. You will see options to drag lines separating the columns around. I had to drag the first column 2 spaces to the right, but it should be quite accurate.

Dialog box labelled 'Text Import Wizard - Step 2 of 3'. Under 'Data preview', lines divide the columns.

  1. Select "Next".
  2. The file specification appears to say that the first two columns are text, even though they appear to contain only digits. So change these columns from "General" to "Text".

Dialog box labelled 'Text import wizard - Step 3 of 3'. Under 'Data preview', the first two columns have had 'General' changed to 'Text'.

  1. Select "Finish".
  2. You should now see your table as a spreadsheet. If any columns seem broken, go back to step 1 and place the lines differently at step 6.
  3. Run a Find and Replace for "-9999999" and replace it with the empty string "". Note that the spreadsheet has probably discarded the .0 that was on the the end of the text version.
  4. Insert a row at the top of your sheet containing the 11 names from the format: line number, date, fiducial and so on, up to local time.
  5. Save the file with type "CSV (Commma delimited) (*.csv)". This is now delimited text with a comma as the delimiter.

Option 2 - Replace the spaces in a text editor


Disadvantage - If any fields contain text with spaces, this will corrupt your file!

Disadvantage - Horribly clunky method.

Advantage - Preserves floating point numbers and other details exactly as they were in the original file.

Advantage - Can be done anywhere with a text editor.

  1. Open your file in a text editor, like Windows Notepad as used in your screenshot.
  2. Search and replace for two spaces " " and replace with one space " ".
  3. Repeat step 2 until the file stops changing. The longest field length is 14 characters, so if you just repeat this 14 times you're definitely done.
  4. This particular format also uses -9999999.0 to define a null easting or northing, so search for that and replace it with an empty string "".
  5. Search for the remaining single spaces " " and replace them with commas ",".
  6. Add a line at the top with "line number","date","fiducial","easting","northing","raw magnetics","diurnal","levelled magnetics","radar altimeter","gps height","local time".
  7. Save the file with a .csv extension.

Option 3 - Write a script to convert to CSV


If you have a lot of files in this format, it's worth writing a script to automate the process. Look up the reading text files and the "substring" function in your language of choice.

Importing Your New CSV into QGIS

If you followed one of the three steps above, you are finall ready to open QGIS and go to "Layer" > "Add Layer" > "Add Delimited Text Layer". Under "Geometry Definition", your data are clearly "Point coordinates". However, you will need to specify a Coordinate Reference System. Your coordinates are in Easting and Northing, so are probably in Universal Transverse Mercator, but you need to know the zone and datum.

For these particular data you may want to specify a Z coordinate, using either the "radar altimeter" (height above ground) or "gps height" (height above sea level). If you do this, you will need to also blank the "-999" values from the relevant column before importing.

Good luck! Do you have many files like this?

6
  • Hello Thanks you the help
    – qgis Perth
    Commented Feb 20, 2022 at 6:42
  • I was able to import the file using you instructions Thanks!!
    – qgis Perth
    Commented Feb 20, 2022 at 7:03
  • Good to know. Which of the three methods did you use? And are you likely to have any more of these files?
    – Cowirrie
    Commented Feb 20, 2022 at 10:57
  • Hi Crowirrie , i used import version 1. thanks , now i trying to vector to raster with mixed results
    – qgis Perth
    Commented Feb 20, 2022 at 11:35
  • @qgis Perth, if this answer solved your problem you should accept it using the check mark. See: gis.stackexchange.com/help/someone-answers
    – Ben W
    Commented Feb 20, 2022 at 11:46
4

The posted magnetic survey data looks like comprised of Survey line number, Recorded date (YYMMDD), Fiducial (sequence) number, Easting (X), Northing (Y), and several measurements.

I would suggest working on these files in three steps.

(1) Pre-processing

If you have -9999999.0 in Easting (X) or Northing (Y), such records are useless and troublesome. Remove these lines beforehand, by a text editor. (You may need to bulk remove lines - find *-9999999.0* and replace it with nothing).

[Note] -99999.00 in the other fields (which contain measurements) are okay and you can set it as NoData in QGIS later.

(2) Read in the file by Add Delimited Text Layer. Use Space as the custom delimiter. Assign the 4th column (easting) to X field, and the 5th column (northing) to Y field. You will obtain a point layer.

(3) Final step is Rasterize (Vector to Raster). Choose your preferred field as the burn-in value.

1

that is a CSV or Delimited text file not an ASC (or ASCII Grid) file so you need to import it using the delimited text tool. See this tutorial for more details on how to do that.

0

Option 1 - Import in a spreadsheet, export to CSV

Open your .asc file in Microsoft Excel or LibreOffice Calc. (Other spreadsheets may work too, but I can confirm it for both of these. The screenshots below are from Excel, but Calc matches it very closely.)

Under "Choose the file type that best describes your data" choose "Fixed width". Leave "Start import at row" at "1". Uncheck "My data has headers". You can probably leave "File origin" as "MS-DOS (PC-8)", but if this corrupts text, change to "20127 : US-ASCII". Dialog box labelled 'Text Import Wizard - Step 1 of 3'. 'Choose the file type' is set to 'Fixed width', 'Start import at row' is'1', 'File origin' is 'MS-DOS (PC-8)' and 'My data has headers' is not checked.

Select "Next".

You will see options to drag lines separating the columns around. I had to drag the first column 2 spaces to the right, but it should be quite accurate. Dialog box labelled 'Text Import Wizard - Step 2 of 3'. Under 'Data preview', lines divide the columns. Select "Next".

The file specification appears to say that the first two columns are text, even though they appear to contain only digits. So change these columns from "General" to "Text". Dialog box labelled 'Text import wizard - Step 3 of 3'. Under 'Data preview', the first two columns have had 'General' changed to 'Text'.

Select "Finish".

You should now see your table as a spreadsheet. If any columns seem broken, go back to step 1 and place the lines differently at step 6. Run a Find and Replace for "-9999999" and replace it with the empty string "". Note that the spreadsheet has probably discarded the .0 that was on the end of the text version.

Insert a row at the top of your sheet containing the 11 names from the format: line number, date, fiducial and so on, up to local time. Save the file with type "CSV (Comma delimited) (*.csv)". This is now delimited text with a comma as the delimiter.

1
  • Hello, it looks like you want to isolate the part of my answer that you actually used. Would you like me to edit mine to move options 2 and 3 to separate answers so you can mark the existing option 1 answer as correct? Or, if you want to separate out option 1 because you followed a different sequence, you need to copy and paste the Markdown source for it. You can see the Markdown source of you choose the option to Edit the existing answer.
    – Cowirrie
    Commented Feb 20, 2022 at 23:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.