14

I have a geodatabase table with a field that I'm trying to remove return characters (new line). I found this post (How can I remove (chomp) a newline in Python?) of how to do it, however it is not working within field calculator. Here are the code snippets that I tried: Note return character is not at the end of the string.

!myField!.rstrip()

OR

!myField!.rstrip('\n')

OR

!myField!.rstrip('\r\n')

OR

!myField!.replace('\n', '')

000539 error given for this option:

Description The calculation used by the Calculate Field or Calculate Value tool is invalid. This error message provided will list the specific Python error.

Solution This error code covers a number of Python errors:

Example error 1: exceptions.TypeError: cannot concatenate 'str' and 'int' objects. The above is a Python-specific error. The calculation is attempting to add or concatenate a string and a number.

Example error 2: Invalid field shape@distance The above is an error using the geometry object. The distance method is not a valid method of the geometry object.

For specific Python issues, consult the external Python help for more information, or consult the Calculate Field or Calculate Value help for more information on these tools.

OR

import os
def removeReturn(myField):
  s = myField.rstrip(os.linesep)
  return s

Any ideas of how to remove return characters using field calculator?

5
  • Are you sure it's a newline character ("\n")? Could it possibly be a carriage return ("\r") instead? Also, are all values in all rows alpha, or maybe its failing on a numeric value in one of the fields? Possibly str(!myField!).rstrip('\n') Dec 20, 2012 at 23:43
  • Please post the full Python traceback with error message and line number, not the ArcGIS error you are getting (which is generic and unhelpful).
    – blah238
    Dec 21, 2012 at 5:21
  • It's starting to look like something in ArcGIS itself, because the various Python answers given are all correct. Dec 21, 2012 at 5:43
  • Are you getting a SyntaxError: EOL while scanning string literal error?
    – blah238
    Dec 21, 2012 at 6:03
  • I am not sure how is calculator handling importing modules, but you may also try: import re and use re.sub.
    – Tomek
    Dec 21, 2012 at 6:03

8 Answers 8

9

I think this is just a bug/limitation of the Python parser with the field calculator/Calculate Field tool. If a newline is encountered within the text field, a SyntaxError: EOL while scanning string literal occurs no matter what you try.

I can reproduce the issue at 10.1 SP1 by importing the following CSV file into a file geodatabase, adding a field and simply attempting to copy the Text field to the new field using the Field Calculator with the Python parser and and the expression !Text!.

ID,Text
1,"this is a multiline 
example"

Try switching to the VB parser, or using an UpdateCursor and avoiding the Field Calculator entirely.

This issue is also discussed on the ESRI forums, with the same conclusions:

The only relevant NIM I could find was this:

  • NIM085499 - CalculateField calculations using non-ASCII characters fail on a Linux engine with: "ERROR 000539: SyntaxError: EOL while scanning string literal (, line 1)".
5

There's two possible solutions I've found to be reliable. For some reason CartoPac allows people to put a Return in the REMARKS field where I work. To get rid of those, the solution that I've found to work best is using the Field Calculator.

Change your Parser to Python. Put the following in Pre-Logic Script Code:

def carriageReturnRemoval(remark):
    remark = remark.splitlines()
    separator = " -- "
    return separator.join(remark)

Put the following in the next text box (this one is working on a field called REMARKS):

carriageReturnRemoval( !REMARKS! )

enter image description here

When your carriage returns get removed, it will add a space, --, and a space between each line. You can modify " -- " to a different character or set of characters if you'd like. But I've found this to work best for me based on the way my construction crews enter data. It is easier to do Find/Replace operations if there are recognizable character patterns breaking up each carriage return, which they use in the field to denote a new attribute (even though it is often an attribute they could've just put into the proper field and saved me the headache in the first place).

If you'd prefer to use the Python Console in ArcGIS, you can modify the above to work. However I've also tried this with some success in the Python Console:

rows = arcpy.UpdateCursor("Assets\Welds")  
for row in rows:  
    hexString = str(row.REMARKS).encode("hex")  
    if "0a" in hexString:  
        hexString = hexString.replace("0a","")  
        row.REMARKS = hexString.decode("hex")  
        rows.updateRow(row)  

Replace "Assets\Welds" with the appropriate fc name and replace row.REMARKS with row.(insert field name here)

You may or may not need to run the import lines below before attempting the code example above:

import arcpy  
import string
2
  • I give it a shot, your statement after your if statement needs indented.
    – artwork21
    Aug 6, 2014 at 21:24
  • Good catch...it didn't copy over properly when I copied from the working Python command window script. Aug 7, 2014 at 23:59
4

!FieldName!.replace(chr(10), "").replace(chr(13), "")

Field calculator is a bit confused when it comes to escape sequences like \n and \t. Just use the raw character codes.

3
  • Still getting a SyntaxError: EOL while scanning string literal with this. VB parser works fine.
    – blah238
    Dec 21, 2012 at 6:19
  • In the expression and not the code block? And only on 10.0? Dec 21, 2012 at 8:13
  • In the expression, on 10.1 SP1.
    – blah238
    Dec 21, 2012 at 8:15
4

The work around for this since my feature class table originated in excel was to use the following excel command:

=CLEAN

method to remove all return or new line characters. You can then join or import the table into your GIS database.

3

I've used a SQL statement to first select the new line characters. In the Select By Attributes dialog window:

"MY_FIELD" LIKE '%
 %'

You have to press Enter after the first %.

2
  • 2
    I just realised that while this showed me where the new line characters were, I also could not get MY_FIELD.strip() to work, but because I only had 5 records, I manually edited them. Dec 21, 2012 at 5:24
  • You could probably select them like this and then use field calculator to re-calculate the values with the Python expression !MY_FIELD![:-1] -- the string slice should remove the newline character (assuming nothing comes after the newline, which seems to be the case for the OP).
    – nmpeterson
    Aug 6, 2014 at 19:50
2
myString = "My text\n"

print myString.strip()
13
  • that did not work in field calculator.
    – artwork21
    Dec 20, 2012 at 15:34
  • Do you need a Python solution or a VBA (ArcGIS 9) solution? strip(), rstrip() and lstrip() only deal with characters on the ends. If it is in the middle of a string, try .replace("\n", "")
    – Dan
    Dec 20, 2012 at 15:42
  • Yes, I need a python solution within field calculator. The return is within the string not at the end. I tried .replace("\n", ""), however I got a general 000539 error.
    – artwork21
    Dec 20, 2012 at 15:48
  • What is the specific Python error given with this number?
    – Dan
    Dec 20, 2012 at 15:53
  • Posted error message in my question.
    – artwork21
    Dec 20, 2012 at 15:57
2

I think this should work as Pre-logic Script Code in the Field Calculator - but unfortunately it seems not to. I've posted it anyway in case it gives you an alternative approach you can tweak to get working.

def fix(teststring):
    parts = teststring.splitlines()
    newtest = ""
    for part in parts:
        newtest+=part
    return newtest
2

Try wrapping your !field! declaration in the expression with quotes and a raw string initializer, e.g. r"!field!".

Try str(!field!) as well.

2
  • It doesn't work. I also tried MYFIELD.replace(r"\n",""), but it doesn't see that I've escaped the newline character, it just fails with a EOL` error. Dec 21, 2012 at 5:33
  • You are correct, see my other answer. Leaving this one for posterity.
    – blah238
    Dec 21, 2012 at 7:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.