With recent boom in Twitter’s popularity, several Twitter sentiment analysis tools were released these tools search twitter public timeline and try to guess their general mood. 25trends.me is another real time Twitter sentiment analysis tool, much like Twitrratr it searches twitter public timeline based on a user inputted search query (classically a #Tag). 25trends market differentiator is the fact that it can handle Arabic tweets. However what it lacks in accuracy it makes up for in style, using the more visually appealing canvas rather than the usual tag cloud.
Natural Language Processing (NLP) is the field of computer science concerned with using AI to extract the semantics from natural human language. Available Arabic NLP tools are far from perfect and still require a lot of development.
25trends.me uses look up tables to assess the sentiment associated with harvested tweets. Look up tables include sentiment related words and their associated value (Positive or Negative). A valid approach given that there isn’t a solid AI powered NLP analysis tool that can handle Arabic, however the analysis accuracy varies tremendously based on the complexity of the tweets. Basically it lacks the ability to analyse the context of each and every word, analyzing them as several entities rather than the building blocks of the sentence.
25trends.me lists the most frequently used words with the searched query, listing them in a web-kit powered canvas in a way that visually represents both their frequency of usage as well as the sentiment associated with them. Searching Twitter’s public timeline 25trends is limited by the API thresholds which limits the yielded results to the last 1500 tweets. For heavily used #tags this wouldn’t yield realistic results. Caching commonly searched for #tags would increase the number of tweets analyzed per search however it would introduce another level of complexity to the product and may result in an even less accurate results, not to mention the added load on the backend with the database exponential growth. Generally speaking I don’t think it was built with scalability in mind, I don’t don’t it can handle 10s not to mention 100s of concurrent searches.
Twitter’s new API version makes it quite hard for such a website to function, as it doesn’t include anonymous public timeline search (which 25trends is using), Allowing 60 search requests per hour per authenticated account, it would limit the website to 60 searches per hour unless they come up with a really clever twitter hack that relies on balancing the requests across hundreds of authenticated accounts. Which is exactly what Twitter is trying to do with that new API, cutting out small Twitter analytics players who rely on public time line search rather than firehose access or gnip.
Other statistics and figures are presented as side dish to the sentiment analysis, fun statistics such as the top tweeters, the associated #tags as well as the percentage of the languages used in tweets mentioning the search query. Generally speaking 25trends is fun to use, you can waste a couple of minutes mining the web, accuracy takes a backseat when it comes to fun to use well built websites.
Even though it offers an API as well as business subscription I can’t imagine that it’d make it in that area since their value proposition is quite basic and can be easily replicated by any company with access to developers. Even system admins can hack something using tools such as pachube or cacti (wrote an entry about that ages ago), Here is the hacked cacti instance to play around with (guest/guest).
Well built, perfect for its designed purpose (Twitter analytics for the masses). Even though its an unoriginal concept that lacks the sufficient accuracy its quite fun to use. Generally speaking I’ll miss it when it gets pushed out of business next march when the current twitter API is decommissioned unless of course they manage to secure Firehose access, which is crazy expensive.
Lets see if it makes it out of beta.