Cacti to monitor twitter

During the events that took place in Egypt during Jan 25, the Egyptians discovered twitter and almost overnight it became one of the most reliable sources of up to the second news, monitoring it for news became something I do several times a day. I’ve been working on cacti for a while so I decided to have cacti monitor the main twitter hashtags concerned with the status in Egypt. And so I started on this mini project, building a monitoring environment that would monitor certain twitter hashtags and hosting it on an amazon micro instance.

Cacti graphs

Some of the test graphs Im showing

First I started by getting an ubuntu LAMP micro instance ( ami : ), I then secured an elastic IP and associated it with the machine, installing cacti was as simple as issuing a

sudo apt-get install cacti

Cacti was now up and running, I modified the amazon security policy to expose port 80 to the Internet, testing it from my machine I was able to connect to it and view the built in graphs monitoring the machine hosting cacti.

The next steps include :

1. Integrating cacti.

2. Enabling guest account and limiting it to certain graphs.

In an earlier tweet I explained how tweets can be published from shell, checking around I realized that reading tweets using a shell script isn’t the most optimum option, a small java application can do it much more efficiently.

Using a Java API seemed to be the better option, so I used twitter4j, having used it before I was familiar with how it works and would be able to develop this app quickly, here is how this app works:

it gets a search phrase, goes in and counts all the tweets from the public time line that have this phrase since the last time the application was run, this means that the application retains state, this state retention could have been done using a simple mysql db, instead i decided to use a file that gets stored on /tmp, the only problem i had was that when a new search string was used the first search yields 1000 results (the maximum search target), hence I hard coded it so whenever the result is 1000 it returns 0, here is the effective part of the code.

Query q=new Query();
q.setRpp(100);
q.setQuery(args[0]);
q.setSinceId(max);

q.setPage(i);

QueryResult result = twitter.search(q);

List<Tweet> tweets = result.getTweets();

So now the result is running

java -jar fetch_tweets.jar search_strings

The next step would be integrating this into cacti, for this I used the simple instructions provided by cacti’s manual, Finally I secured an elastic IP from Amazon and I attached the machine, now the machine is up and running.

I created a new user guest and enabled it to view these graphs and limited it’s access.

Feel free to try it http://79.125.3.193/cacti/

Advertisements

One thought on “Cacti to monitor twitter

  1. […] Even though it offers an API as well as business subscription I can’t imagine that it’d make it in that area since their value proposition is quite basic and can be easily replicated by any company with access to developers. Even system admins can hack something using tools such as cacti (wrote an entry about that ages ago) […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s