0

I am geocoding a batch of addresses with the US Census Bureau's free geocoding tool. I am following the guide at https://geocoding.geo.census.gov/geocoder/Geocoding_Services_API.pdf.

I have a correctly formatted .csv file (Confirmed it works with the census bureau's geocoding tool at https://geocoding.geo.census.gov/geocoder/geographies/addressbatch?form) . I would like to send a request from our Java code.

As a first step, I attempted to submit a cURL statement from git bash on Windows 10. With the US census bureau geocoder API guide as reference, here is what I typed...

curl --form addressFile=@C:\Users\User\Documents\Work\xxxx\new.csv --form benchmark=9 https://geocoding.geo.census.gov/geocoder/locations/addressbatch --output geocoderesult.csv

This successfully fetched geocoded results from the census bureau site and stored them in geocoderesults.csv on my local machine.

What I ultimately want to do is send this post request from our Java server code. I understand the geocoding service requires the user to post a form that contains two fields (the benchmark number and a file). Here is what I have tried...

I built and executed the request with the apache.http client library...

    String benchmark = "9";
    CloseableHttpClient httpClient = HttpClients.createDefault();
    HttpPost uploadFile = new HttpPost("https://geocoding.geo.census.gov/geocoder/locations/addressbatch");
    MultipartEntityBuilder builder2 = MultipartEntityBuilder.create();
    builder2.addTextBody("benchmark", benchmark, ContentType.TEXT_PLAIN);
    // This attaches the file to the POST:
    File f = new File("UscbgeocodingInput.csv");
    FileInputStream is = new FileInputStream(f);
    AnalyzeInputStream(is);

    builder2.addBinaryBody(
            "addressFile",
            new FileInputStream(f),
            ContentType.APPLICATION_OCTET_STREAM,
            f.getName());

    HttpEntity multipart = builder2.build();
    uploadFile.setEntity(multipart);
    CloseableHttpResponse response = httpClient.execute(uploadFile);
    HttpEntity responseEntity = response.getEntity();

The response was a successful 200. I am trying to figure out how to pull the resulting csv file out of this response.

HttpResponseProxy{HTTP/1.1 200 OK [Cache-Control: private, Date: Tue, 05 Nov 2019 22:45:24 GMT, Content-Length: 515, Content-Type: text/plain, Content-Disposition: attachment; filename=GeocodeResults.csv, X-ORACLE-DMS-ECID: cb584342-ba3d-4c6a-bd04-2b681f7f063f-0026951d, X-Frame-Options: Deny, X-ORACLE-DMS-RID: 0, Set-Cookie: JSESSIONID=--U9vnuwE9vrmFvu-D6lJXVqOlhnG0dQIJFpXeBlDWKUR7_ZF-3v!-328210670; path=/; secure; HttpOnly, X-Content-Type-Options: nosniff, Strict-Transport-Security: max-age=31536000, Vary: Origin] ResponseEntityProxy{[Content-Type: text/plain,Content-Length: 515,Chunked: false]}}

I have also tried with the Java.ws.rs.client library...

public void batchGeocodeWithUSCensusBureau(File addressesCSVFile, Integer list_id) throws IOException {
    String benchmark = "9";
    String params = "";
    RestClient client = getCensusBureauClient("geocoder/locations/addressbatch", params);
    InputStream targetStream = new FileInputStream(new File("./new.csv"));

    FormDataMultiPart part = new FormDataMultiPart();
    part.field("benchmark", benchmark);
    part.field("addressFile", targetStream, new MediaType("text", "csv"));
    File response = client.postForm(part, File.class);
}

private RestClient getCensusBureauClient(String path, String params) {
    String url = "https://geocoding.geo.census.gov/";
    RestClient restClient = RestClient.builder().url(url + path + params).build();
    return restClient;
}

public class RestClient {

private Logger logger = LoggerFactory.getLogger(getClass());
private Client client;
private String url;

private RestClient(Client client, String url) {
    this.client = client;
    this.url = url;
}

public static Builder builder() {
    return new Builder();
}

public static class Builder {
    private String username;
    private String password;
    private String url;

    public RestClient build() {

        ClientConfig conf = new ClientConfig();
        conf.property(ClientProperties.CONNECT_TIMEOUT, timeout);
        conf.property(ClientProperties.READ_TIMEOUT, timeout);

        JacksonJsonProvider jjp = new JacksonJsonProvider();
        jjp.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
        conf.register(jjp);

        if (username != null) {
            HttpAuthenticationFeature httpAuth = HttpAuthenticationFeature.basic(username, password);
            conf.register(httpAuth);
        }

        return new RestClient(ClientBuilder.newClient(conf), url);
    }

public <T> T postForm(Object payload, Class<T> responseType) {
    try {
        Entity<Object> entity = Entity.entity(payload, MediaType.MULTIPART_FORM_DATA_TYPE);
        T t = client.target(url).request(MediaType.MULTIPART_FORM_DATA_TYPE).post(entity, responseType);
        return t;
    } catch (Throwable t) {
        logError(t);
        throw t;
    } finally {
        client.close();
    }
}
}

My csv file does indeed exist in that location. The targetStream variable is not null.

When the program arrives at the line that actually posts the request...

        T t = client.target(url).request(MediaType.MULTIPART_FORM_DATA_TYPE).post(entity, responseType);

I see this error.

org.glassfish.jersey.message.internal.MessageBodyProviderNotFoundException: MessageBodyWriter not found for media type=multipart/form-data, type=class org.glassfish.jersey.media.multipart.FormDataMultiPart, genericType=class org.glassfish.jersey.media.multipart.FormDataMultiPart.

I was unable to find a working solution. Any ideas? Also, I'm not married to this approach. If it won't work, then any approach to posting a form that includes a file and a string to the US Census Bureau's geocoding API will do just fine.

  • Well I can read the csv file. It is actually the response entity object! And it is possible to write this to an input stream...HttpEntity responseEntity = response.getEntity(); if (responseEntity != null) { ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream(); try { responseEntity.writeTo(byteArrayOutputStream); } – GNG Nov 6 '19 at 8:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.