2

So I have a gdb, and the feature class I'm working on has a field that is formatted in html for ArcGISOnline. This is great if I'm only viewing the data online. What I'm trying to do is copy that attribute data to a new field and then convert the data from html. There is a great tool that does this well, html2text.

My question is: How do I go about getting the tool to play nice with arcpy?

Here's where I am right now and it's not converting the value for me

import html2text, arcpy
from arcpy import env
arcpy.env.workspace = "C:/Workspace/Data_National/RGOV/temp/20170330_RGOV.gdb/"
Rgdb = "C:/Workspace/Data_National/RGOV/temp/20170330_RGOV.gdb"
FFA = Rgdb + "/RSites/Facapp"

arcpy.AddField_management(FFA, "FFA", "TEXT", "", "", "8000")
with arcpy.da.SearchCursor(FFA, "FACILITYDESCRIPTION") as cursor:
    for htmlformat, in cursor:
        Expression = html2text.html2text(htmlformat)
        arcpy.CalculateField_management(FFA, "FFA", html2text.html2text('!FACILITYDESCRIPTION!'), "PYTHON_9.3", Expression)

Any pointers on where I'm going wrong? I also tried throwing a

arcpy.CalculateField_management(FFA, "FFA", html2text.html2text('!FACILITYDESCRIPTION!'), "PYTHON_9.3")

but it didn't work...

When I used the CalculateField tool, the script runs through with no errors, but keeps the html formatting.

When I try it with the SearchCursor section of the script I get an error of "data = data.replace("/' + 'script)", "(/ignore)")

error message

  • Is the FACILITYDESCRIPTION field for every record formatted as HTML? – Fezter May 2 '17 at 23:58
  • 1
    Please edit your question to give the error message text in full – Midavalo May 3 '17 at 0:04
  • Yes @Fezter every value is formatted in HTML – morgan May 3 '17 at 15:20
1

This is one of the tricky parts of using CalculateField in a script. You are importing html2text into the script but the field calculator does not know it is imported as it starts a new internal scripting session. As far as I can see you are already calculating value for this additional field with Expression variable and there is nothing inhibiting you to use it directly, e.g., arcpy.CalculateField_management(FFA, "FFA", Expression, "PYTHON_9.3"). However it will recalculate this column with every iteration, which is a terrible practice. I recommend you to use the ArcMap's field calculator directly as shown below:

  1. In ArcMap, create a new text field, as you defined for FFA

  2. Calculate this field with the code block and expression with PYTHON below:

Pre-logic script code: import html2text

(Field to calculate) FFA=: html2text.html2text(!FACILITYDESCRIPTION!) if !FFA! else ""

More clearly, see below the snapshot:

enter image description here

  • his method runs through fine with no errors, but the new field values are still in html format. Maybe I didn't script it correctly? This is what I tried Expression = html2text.html2text("!FACILITYDESCRIPTION!") arcpy.CalculateField_management(FFA, "FFA", Expression, "PYTHON_9.3") I tried single and double quotes around !FACILITYDESCRIPTION! both methods yielded the same results – morgan May 3 '17 at 15:24
  • Most probably the quotation mark in the expression is the problem which cause html2text to get everything as string literal. Please be aware, this is not a scripting solution but simple field calculator operation. I have also updated the expression to avoid NULL values to throw any exception. – fatih_dur May 4 '17 at 0:29
  • ok, using this method in script works, but for some reason in ArcMap, only the first 170 characters are displayed, but on ArcGISOnline all is well. Here is the script I used import arcpy, html2text Expression = "html2text.html2text(!FACILITYDESCRIPTION!)" arcpy.CalculateField_management(FFA, "FFA", Expression, "PYTHON_9.3") – morgan May 24 '17 at 17:37
  • I think this is exactly the same problem that you directed to @Midavalo, special character issue at a specific position in the string, most probably a line break/feed. There are options in html2text module as explained in the documentation. I cannot test it right now but by first creating a parser after import html2text (e.g., text_maker = html2text.HTML2Text()) and then setting escape parameter, text_maker.escape_snob = True, you may overcome this issue. – fatih_dur May 24 '17 at 22:46
  • But in this case your (Field to calculate) FFA= should be text_maker(!FACILITYDESCRIPTION!) if !FFA! else "" – fatih_dur May 24 '17 at 22:47
5

Instead of using Calculate Field within a Search Cursor, why not update your fields with an Update Cursor?

import html2text, arcpy
arcpy.env.workspace = "C:/Workspace/Data_National/RGOV/temp/20170330_RGOV.gdb/"
Rgdb = "C:/Workspace/Data_National/RGOV/temp/20170330_RGOV.gdb"
FFA = Rgdb + "/RSites/Facapp"

arcpy.AddField_management(FFA, "FFA", "TEXT", "", "", "8000")

with arcpy.da.UpdateCursor(FFA, ["FACILITYDESCRIPTION", "FFA"]) as cursor:
    for row in cursor:
        if row[0]:
            row[1] = html2text.html2text(row[0])
            cursor.updateRow(row)
  • I'm still getting an error, shown above, with this method. – morgan May 3 '17 at 15:16
  • I checked the table again, and noticed that there were several <null> values. I added text in there and this method work. Thanks! – morgan May 3 '17 at 21:24
  • 1
    @morgan I have updated my answer to check that the value in FACILITYDESCRIPTION isn't NULL. If it is then it'll skip it, if it has a value then it will proceed. – Midavalo May 3 '17 at 21:35
  • I let this take a backseat for a while and tried the script again. I noticed that for some reason the script fixes the first 340 characters, and drops all the rest of the value when this method is used. – morgan May 23 '17 at 22:08
  • @morgan What happens if you (separate to this script) set a variable containing one of your values that's failing, then use the html2text into another variable. Does this output the complete expected string, or is this also only the first 340 characters? – Midavalo May 23 '17 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.