Logging ArcPy error messages?

Question

I am trying to write any error messages that arise in my script and record it in a text file. When I check my errorLog, the error messages are not recorded. Below is my code: Note the parameter 'filePath' is the file path to the file that produced the error.

except:
    addError = arcpy.GetMessages(2)
    with open(errorLog, 'a') as errorMsg:
        errorMsg.write('%s, %s\n' % (filePath,addError))

Can anyone offer insight?

Answer 1

I've written a python log handler that I'm more than happy to share. The idea being that you can use the standard Python logging framework and also have the messages reflected back to ArcGIS through the arcpy messages.

import logging
import logging.handlers

import arcpy

class ArcPyLogHandler(logging.handlers.RotatingFileHandler):
    """
    Custom logging class that bounces messages to the arcpy tool window as well
    as reflecting back to the file.
    """

    def emit(self, record):
        """
        Write the log message
        """
        try:
            msg = record.msg.format(record.args)
        except:
            msg = record.msg

        if record.levelno >= logging.ERROR:
            arcpy.AddError(msg)
        elif record.levelno >= logging.WARNING:
            arcpy.AddWarning(msg)
        elif record.levelno >= logging.INFO:
            arcpy.AddMessage(msg)

        super(ArcPyLogHandler, self).emit(record)

You can then use the handler with a standard python logging object:

logger = logging.getLogger("LoggerName")
handler = ArcPyLogHandler(
    "output_log.log",
    maxBytes=1024 * 1024 * 2, #2MB log files
    backupCount=10
)
formatter = logging.Formatter("%(levelname)-8s %(message)s")
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.setLevel(logging.DEBUG)

And then you can write log messages like so:

logger.debug("A debug message")
logger.info("An info message")
logger.warning("A warning message")
logger.error("An error message")
logger.critical("A critical error message")

Which will write the messages of levels info and above through arcpy to the processing window, and all messages from debug and above to the log file specified with the format specified above (see the documentation for other formatting options).

Lastly you can use logger.exception in an except block which will give you a logging message at ERROR level, and also dump the stacktrace to the log.

try:
    #Something that might fail
except Exception as e:
    logger.exception(e)

Answer 2

A simple test script:

errorLog = r'd:\log.txt'
filePath = errorLog
try:
    with open(errorLog,'a') as errorMsg:
        errorMsg.write("%s,%s\n" % (errorLog,filePath))
except RuntimeError:
    print("Unable to log")
    print RuntimeError.message

Note: no imports, it's all internal..

This works from a pythonic sense which makes me wonder what's up with yours not working.

A more complicated script using arcpy:

import arcpy

errorLog = r'd:\log.txt'
filePath = errorLog
try:
    arcpy.Delete_management("Data doesn't exist, lets' just force the exception")
except:
    arcpy.AddMessage(arcpy.GetMessages(2)) # see if there is some error there
    try:
        with open(errorLog,'a') as errorMsg:
            errorMsg.write("%s,%s\n" % (errorLog,arcpy.GetMessages(2)))
    except RuntimeError:
        arcpy.AddMessage("Unable to log")
        arcpy.AddMessage(RuntimeError.message)

puts this in the text file:

d:\log.txt,Failed to execute. Parameters are not valid. ERROR 000732: Input Data Element: Dataset Data doesn't exist, lets' just force the exception does not exist or is not supported Failed to execute (Delete).

So, there's nothing wrong with your code, you will need to look elsewhere. Note the 2nd try in the except, it may not log to a file but at least you should get an error message on screen; from here it's likely to be 'file not found' or 'unable to access' that's causing the logging to fail.

Answer 3

thanks for the responses guys. I am still not 100% why the original code didn't work but I ended up going with a modified version of the trace back code shown here. In the end my coded ended up looking like this (see below), which produced the results that I was after:

import sys
errorLog = r'd:\log.txt'
filePath = errorLog
try:
'''Code not included in example'''
except:
    with open(LOG_FILE, 'a') as errorMsg:
        errorMsg.write('%s, %s\n' % (filePath, str(sys.exc_info()[1])))

Answer 4

Add logger.handlers = [] before adding the arcpy handler to avoid script crashes when executed more once

logger.handlers = [] 

logger.addHandler(handler)