By William Nee
Senior Director, Market Intelligence
In part 1 of this series (located here), we discussed why a marketer might find it worthwhile to extract and analyze data from Twitter, and then reviewed the ways that could be accomplished through third-party tools. We also laid the groundwork for how it would be possible for us to perform this function by using the Twitter API, and how that actually might be the best approach to use.
To review, through the Twitter API, programmers can access a variety of objects, the most important being of course Tweets and Users. These objects contain core attributes that describe the object – the table below gives an example of some of the data we can retrieve from the Tweet object. You’ll note that not only does the Tweet object contain interesting information (e.g. retweets, replies, even location information), but it also allows access to the Twitter User object. That access allows us to extract information about the user who sent the Tweet. In addition to the requested results, search requests automatically return a Tweet’s unique ID, along with the author and creation time/date stamp.
Data that Can Be Retrieved from a Tweet
For the example we’ll be developing in this series of articles, we’ll be using what Twitter calls its “Standard Search API” to run searches containing specific keywords against historical Tweets. For the marketer this is arguably the most useful API, since it allows us to access a vast trove of information that can be mined and analyzed for historical trend and future projection purposes.
At this point it’s appropriate to mention that Twitter also offers a “Streaming API.” While the Search API is used to query and retrieve historical Twitter data, the Streaming API gives the developer access to Twitter’s near real-team data stream. The Streaming API could be useful to the marketer in a number of ways. For example, after a competitor’s press release has been issued, the Streaming API could be used to determine what kind of immediate traction it’s been getting.
1. App Registration
The first thing we need to do to get started on our Twitter application is to apply for a developer account on the Twitter developer site https://developer.twitter.com/. Once we obtain that and log on, we’ll be greeted by this welcome screen:
From here, we’ll choose “Create an app,” where, among other things, we’ll be asked to describe the intended purpose of our application. We’ll then need to submit our proposal to Twitter for approval; keep in mind the company has tightened up its application creation process during the past year in response to data privacy and abuse concerns (e.g. spam, malicious automation).
Once you get approval to create an app (the time this takes varies, but often it’s immediate), you’ll be assigned a set of Keys and Tokens. The Consumer Key/Secret identifies the client accessing Twitter API services. The Access Tokens specify the data the user’s allowed access to. Make a note of these keys and tokens; you’ll need them to connect to the Twitter API.
Ok, great! So now we’ve set up a Twitter account and received our access tokens. In the next article we’ll begin coding our program!
William Nee is a Senior Director of Market Intelligence at Oracle Corporation. In this role, he performs analyses on a variety of software industry topics for his company’s leadership, including extracting competitive insights by applying machine learning against social media Big Data. Previously, he held senior competitive intelligence roles at Oracle and SAP, where among other functions, he performed detailed functional and market analyses of an array of enterprise tech vendors.