1

I have been using ArcMap for a few years, but I have zero experience with Python or ArcPy. I have been looking for a way to check the links in my attribute table to see if they are broken, and by that I mean, which ones open to a 404-file not found page in a web browser . All my hyperlinks in the attribute table look like https://GIS.com/AsBuilts/PDF-NAME-HERE.pdf. I have had some exposure to other languages so I've been familiarizing myself with the code from this post: Using ArcPy to verify paths of Hyperlinks?. I got it working, but it always outputs all of rows in my attribute table to the broken links text file.

Which would mean that it thinks that all of my hyperlinks are broken?

I know for sure that there are some broken links and some that are good. I have linked them to the hyperlink tool in Arc and they open just fine.

I am not sure where to go from here, any suggestions of things to check?

Here's my code:

import arcpy
import os

fc = r"Z:\workplace\AsbuiltPoly_copy.shp"
fields = ["ASBUILT_RE"]

with arcpy.da.SearchCursor (fc,fields) as cursor:
    for row in cursor:
        Roll = str(row[0])
        if os.path.lexists(Roll):
            pass
        else:
            logfile = open(r"Z:\workplace\BrokenLinks.txt","a")
            logtext = Roll + os.linesep
            logfile.write(logtext)
            logfile.close()

I cannot download a link checker onto this computer cause I am not the admin :/ so I cannot go that route. But if you know anyone that are good, send them my way.

2
  • 2
    The question you link to is looking for links to files on a local filesystem. Are you sure that os.path.lexists works on URLs? I have a feeling it only works on file systems. Try using urllib.urlopen("myurl").getcode() instead.
    – Dowlers
    Jun 26 '18 at 20:00
  • I changed all of my links to the local file names on my computer and then ran my code. lol that works for now I guess... might be back
    – T Mai
    Jun 26 '18 at 22:22
4

To check the status of a webpage you'll want to use the built in urllib library. Then open the url and check the response code.

import arcpy
import urllib

fc = r"Z:\workplace\AsbuiltPoly_copy.shp"
fields = ["ASBUILT_RE"]

with arcpy.da.SearchCursor (fc,fields) as cursor:
    for row in cursor:
        Roll = str(row[0])
        a = urllib.urlopen(Roll)
        if a.getcode() is 404:
            logfile = open(r"Z:\workplace\BrokenLinks.txt","a")
            logtext = Roll + os.linesep
            logfile.write(logtext)
            logfile.close()

Update: To prevent the script from failing if the url is not present or formatted incorrectly I added a try block to catch those exceptions.

import arcpy
import urllib

fc = r"Z:\workplace\AsbuiltPoly_copy.shp"
fields = ["ASBUILT_RE"]

with arcpy.da.SearchCursor (fc,fields) as cursor, \
    open(r"Z:\workplace\BrokenLinks.txt","a") as logfile:
    for row in cursor:
        Roll = str(row[0])
        try:
            a = urllib.urlopen(Roll)
            if a.getcode() is 404:
                logfile.write(Roll)
        except:
            logfile.write(Roll)
4
  • I ran this, got errors with the urllib.py file on my comp... Traceback (most recent call last): File "Z:\workplace\urllib_CODE.py", line 10, in <module> a = urllib.urlopen(Roll) File "C:\Python27\ArcGIS10.6\lib\urllib.py", line 87, in urlopen return opener.open(url) File "C:\Python27\ArcGIS10.6\lib\urllib.py", line 210, in open return self.open_unknown(fullurl, data) File "C:\Python27\ArcGIS10.6\lib\urllib.py", line 222, in open_unknown raise IOError, ('url error', 'unknown url type', type) IOError: [Errno url error] unknown url type: 'k'
    – T Mai
    Jun 26 '18 at 22:23
  • The first error has something to do with the code...? while the others are with the urllib file. I'm not sure what do do with this type of error. The last one has to do with the files that do not have a url in them right now... I think it does not know what to do with the name.
    – T Mai
    Jun 26 '18 at 22:27
  • Maybe you should check the HTML before passing to urllib, like stackoverflow.com/questions/827557/… or stackoverflow.com/questions/3633154/python-check-html-valid
    – fatih_dur
    Jun 27 '18 at 7:31
  • Hi @TMai if this answer has solved your question please consider accepting it by clicking the check-mark. This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. There is no obligation to do this.
    – Squanchy
    Jul 2 '18 at 16:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.