Using Tweetnest with Hashtags part I

Currently I’m building a small plugin that’d work with hashtags instead of users, so that tweets with a certain phrase can be harvested and later on displayed, I’m using the tweetnest database and pages for the sake of simplicity, however the crawler its self will be written in java for efficiency, I’m building on top of the tweet_from_table application, since it includes the most up to date classes, I Ordered my netbeans to copy the project, once i had a copy i proceeded to the modification.

I’m thinking along the following lines:

  1. There will be a table, holding all the hashtags being searched for, this table will include each hashtag and its count, an additional column in the tn_tweets will be added that will include each tweet and which hashtag(s) does it relate to, the challenge here is breaking down the many to many relationship without breaking the GUI.
  2. The crawler will fire up, query these hashtags, and the last tweet in db for each, and then proceeds searching for them one by one, the result set will be inserted into the database, using the same convention used by the original tweetnest php crawler.
  3. Finally the results will be displayed using the same GUI used by tweetnest.
With any luck i believe i’ll be able to at least wrap up some version of the crawler today, the first step was downloading the database into my local machine to test.
Update, as it turns out the crawler is a lot hard to build than i first estimated, however i built the core of it, it inserts values correctly into the tweets table, however I need to populate the other tables as well.

4 thoughts on “Using Tweetnest with Hashtags part I

  1. Hi,

    Would you be ok sharing the crawler you built for hashtags? It would be a great help for me – want to archive all tweets mentioning GoUNESCO.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s