0

The Census OnTheMap download links seems regular in formatting 1. This suggest someone may have already written an R package to download these files. Does such a package exist?

Other Comments:

If not, how might I do so? (I have a list of about 30 counties I wish to do it for). The links seem regular. Regular links, for SLC, Utah and Denver, Colorado:

https://onthemap.ces.census.gov/cgi-bin/report.py?report_id=otm_d2c5f4708fff45dd8a153aeea29b66ae&settings=%7B%22analysis_type%22%3A%22area_profile_report%22%2C%22view%22%3A%22summary%22%2C%22characteristic%22%3A%22c000%22%2C%22year%22%3A2015%2C%22job_type%22%3A%22jt00%22%2C%22ap_segment%22%3A%22s000%22%2C%22origin%22%3A%22work%22%2C%22color%22%3A%5B%22%230000AA%22%5D%7D&mode=export_geography&format=shp

https://onthemap.ces.census.gov/cgi-bin/report.py?report_id=otm_b874bc58baa84cbea0d57c2f654450ef&settings=%7B%22analysis_type%22%3A%22area_profile_report%22%2C%22view%22%3A%22summary%22%2C%22characteristic%22%3A%22c000%22%2C%22year%22%3A2015%2C%22job_type%22%3A%22jt00%22%2C%22ap_segment%22%3A%22s000%22%2C%22origin%22%3A%22work%22%2C%22color%22%3A%5B%22%230000AA%22%5D%7D&mode=export_geography&format=shp

The string for the download link consists of 3 parts, the first and third of which are identical, viz:

  • 1) https://onthemap.ces.census.gov/cgi-bin/report.py?report_id=otm_
  • 2a) d2c5f4708fff45dd8a153aeea29b66ae
  • 2b) b874bc58baa84cbea0d57c2f654450ef
  • 3) &settings=%7B%22analysis_type%22%3A%22area_profile_report%22%2C%22view%22%3A%22summary%22%2C%22characteristic%22%3A%22c000%22%2C%22year%22%3A2015%2C%22job_type%22%3A%22jt00%22%2C%22ap_segment%22%3A%22s000%22%2C%22origin%22%3A%22work%22%2C%22color%22%3A%5B%22%230000AA%22%5D%7D&mode=export_geography&format=shp

In other words, is there a way to use R to automate a query on a webmap server?

Improve this question
11
  • 1
    where have you looked already?
    – Spacedman
    Mar 6, 2018 at 19:21
  • 1
    Why do you want a package for a single static download? If you want to download a file in R you can just use download.file(url(...)) but, in this case any type of URL scrubbing is unnecessary. If there are multiple files that you want to download then you need to quit giving us the single file download link!
    – Jeffrey Evans
    Mar 6, 2018 at 19:23
  • 1
    Please do not just rephrase as a new question because you are not getting the response that you would like. I am voting to close because you could have easily modified your original question at: gis.stackexchange.com/questions/273670/…
    – Jeffrey Evans
    Mar 6, 2018 at 19:26
  • The original question was different to this. As it was written in the there, it was not clear that there were meant to be multiple files, and hence the solution I provided was only for a single download link. This question is looking for a way to download multiple files. I advised him to include the URL as it gives an idea of how data could be downloaded.
    – Michael Harper
    Mar 6, 2018 at 19:50
  • 1
    @MikeyHarper these are not static links, they are produced by a query on a webmap server, which is why the URL contains a "report_id" string. Unfortunately, there is nothing to scrape as the data is buried in the service and they did not provide an API for automated queries.
    – Jeffrey Evans
    Mar 6, 2018 at 20:29

1 Answer 1

Reset to default
3

You do not need to query a webmap serve if you are looking for LEHD employment data. If you're searching for WAC, RAC, or OD data you can query the LED FTP directly or you can use a package I've designed. It is not yet on CRAN but here is the package link on github (https://github.com/jamgreen/lehdr). The package queries the LEHD FTP directly, so you can download preferred LED tables for any available year and state. Using this, you could download the data for your states of interest and then using a county file or matching block IDs you can subset.

If you don't want to use the package you can still download tables from the LODES FTP directly. This is the link to the latest version of the LEHD tables. From there you can write your own scripts if desired to download files directly from there.

Improve this answer
6
  • Thank you, but I was looking for the points shapefile (geometry) not the tabular data.
    – Mox
    Mar 16, 2018 at 21:16
  • What do you mean by points shapefile here? The LEHD provides the employment data and on the map simply displays it in different ways. There is no "point" LEHD data. That's a cartographic representation of the undelying employment data at the block level. If you want to reproduce a map like you see on OnTheMap, then that is a cartographic question and there are multiple options for you including dot density, proportional symbols or kernel density plots.
    – elmuertefurioso
    Mar 17, 2018 at 15:45
  • So, this gets a bit awkward. Depending on which year of LEHD data your download, the points are in different locations. (Or were, last time I tested in 2011). They are also not the same as block centroids. I don't know why.
    – Mox
    Mar 19, 2018 at 19:41
  • What were the points of, exactly? Block centroids? Employment?
    – elmuertefurioso
    Mar 19, 2018 at 20:49
  • I don't know. I've not been able to find out. My working theory was that they were specially generated for reasons only the Census knows.
    – Mox
    Mar 20, 2018 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.