I've been doing some research on using R to datamine twitter, but I haven't really found an answer or a decent tutorial for my question.

I'm interested in pulling tweets from twitter with a certain hashtag, within a certain timeframe, and plotting the location of those tweets on a map in either QGIS or ArcMap.

I know that tweets can have geolocation tied to them, but how do I extract this information in the first place?

  • This might help: mike.teczno.com/notes/streaming-data-from-twitter.html I admit I didn't read it all, but it appears they show how to get each tweets location. – ianbroad Oct 27 '14 at 18:44
  • 1
    it seems like you could lose the product tags "r", "qgis" and "arcgis" since you just need to extract the coordinates from Twitter's API. Once you have that information, you'd add the points to any product using its standard functionality – Stephen Lead Oct 28 '14 at 3:27
  • 401 error is coming on running the code. – shikhar Jan 25 '17 at 9:01
up vote 22 down vote accepted

I found a way using purely Python to get the coordinates for tweets using a word filter. It doesn't seem like many people include location with their tweets.

This might not be what you're after either because this is live streaming data. You can test it by putting a unique filter word and then tweeting that word from your Twitter account. You will see your tweet show up in Python almost instantly. This would be pretty cool to use for some huge event.

You'll need to install Tweepy.

pip install tweepy

And get a Twitter API Key.

Then you can use this script as a template:

import json
from tweepy import Stream
from tweepy import OAuthHandler
from tweepy.streaming import StreamListener

#Enter Twitter API Key information
consumer_key = ''
consumer_secret = ''
access_token = ''
access_secret = ''

file = open("C:\\Output.csv", "w")
file.write("X,Y\n")

data_list = []
count = 0

class listener(StreamListener):

    def on_data(self, data):
        global count

        #How many tweets you want to find, could change to time based
        if count <= 2000:
            json_data = json.loads(data)

            coords = json_data["coordinates"]
            if coords is not None:
               print coords["coordinates"]
               lon = coords["coordinates"][0]
               lat = coords["coordinates"][1]

               data_list.append(json_data)

               file.write(str(lon) + ",")
               file.write(str(lat) + "\n")

               count += 1
            return True
        else:
            file.close()
            return False

    def on_error(self, status):
        print status

auth = OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
twitterStream = Stream(auth, listener())
#What you want to search for here
twitterStream.filter(track=["Halloween"])

Check out this documentation from Twitter too, it shows what you can put in the filter.

Here's the result of putting the filter as "Halloween" for a few minutes:

enter image description here

And for the hell of it, here's the first 2000 tweets that mentioned Halloween!

http://i.stack.imgur.com/bwdoP.png enter image description here

Happy Halloween!

  • This would be 100% what I was looking for if it would somehow let me comb through older tweets as well. I'm going to play around with this and see what I can come up with. Thank you very much! – Bradley_Jay Oct 27 '14 at 22:00
  • @Bradley_Jay No problem. According to the following, Twitter doesn't really make older tweets available. stackoverflow.com/questions/1662151/… – ianbroad Oct 27 '14 at 22:06
  • You can also filter with a bounding box instead of 'track' e.g. locations=[-180,-90,180,90] – Matt Oct 27 '14 at 23:15

protected by Ian Turton Jan 25 '17 at 9:23

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.