Update: An Amazon guy sent me a mail because I used the old SQS API. Please use the gems from RightScale instead of the one presented below, it supports the new SQS WSDL, which is cheaper.
Thanks to the first part of this article, you master Flex and ActionScript 3 like nobody. Don’t you ?
It’s now time to retrieve some data. You could need a realtime updated graph to monitor a bunch of rackmount servers, your database, your mailbox, or the activity of your friends of Facebook, who knows. After taking a look at the Technorati top blogs list, I wondered what was the activity of readers on those blogs, ie. check how many comments are posted on their last articles.
This article will focus on the data provider, coded in Ruby. And remember, the retrieved data will be sent to an Amazon SQS queue.
I know, why would we need to use Ruby and Amazon SQS when we can do all those tasks directly in ActionScript 3 ? Like I wrote before, you could use it to monitor something which is not directly accessible by Flash, as the ActionScript code is executed on the client computer. Data providers could also be CPU/Network expensive and/or spreaded in a computing cloud. Hey, finally, don’t you want to read some Ruby code ?
Let’s parse those RSS feeds. You’ll need one gem, the “Flexible RSS and Atom reader for Ruby”: simple-rss.
require 'open-uri' # To retrieve data from the word wide web
require 'rubygems' # Needed to load installed gems
require 'simple-rss'
feeds_url = %w{
http://feeds.feedburner.com/Techcrunch
http://feeds.feedburner.com/Mashable
... a lot more here ...
}
feeds_url.each do |feed_url|
# Parse each feed
rss = SimpleRSS.parse open(feed_url)
rss.entries.each do |entry|
begin
entry.guid << "/" unless entry.guid[-1] == 47
comments_feed_url = "#{entry.guid}feed"
rss_comments = SimpleRSS.parse open(comments_feed_url)
rss_comments.entries.each do |comment|
puts "Found a new comment #{comment.title} on '#{entry.title}'"
end
rescue
puts "There was a problem retrieving comments from #{comments_feed_url}"
end
end
endAs I want to keep the code simple, I only chose Wordpress blogs, as you can retrieve the comments feed for each entry easily: adding “/feed” to the end of the entry url.
Whis this code, you’ll get something like:
Found a new comment By: Tim-TechFruit on 'LoveFilm merges with Amazon’s DVD rentals'
Found a new comment By: Esi on 'Cuill has Irish roots'
Found a new comment By: Sam B on 'Women more likely to give passwords for chocolate? Yeah, but did they offer the men guilt-free sex?'
Found a new comment By: john b on 'Women more likely to give passwords for chocolate? Yeah, but did they offer the men guilt-free sex?'
Found a new comment By: Carl on 'Women more likely to give passwords for chocolate? Yeah, but did they offer the men guilt-free sex?'
We said earlier that we wanted to monitor the comments, right ? We will put this code in a infinite loop, and keep the found comments in a hash, just to announce only the new ones.
comments = {} # hash of found comments
loop do
feeds_url.each do |feed_url|
# Parse each feed
rss = SimpleRSS.parse open(feed_url)
rss.entries.each do |entry|
begin
entry.guid << "/" unless entry.guid[-1] == 47
comments_feed_url = "#{entry.guid}feed"
rss_comments = SimpleRSS.parse open(comments_feed_url)
rss_comments.entries.each do |comment|
if not comments[comment.guid]
comments[comment.guid] = comment.guid
puts "Found a new comment #{comment.title} on '#{entry.title}'"
end
end
rescue
puts "There was a problem retrieving comments from #{comments_feed_url}"
end
end
end
puts "... waiting 1 minute ..."
sleep 60
endWhis this code, you’ll get something like:
Found a new comment by: Buster Cherry on 'Crazy Gets Evicted!'
Found a new comment by: DONT MAKE FUN OF HOMELESS on 'Crazy Gets Evicted!'
Found a new comment by: joe on 'Crazy Gets Evicted!'
Found a new comment by: Oh, Give Me a Home Where the... on 'Crazy Gets Evicted!'
... waiting 1 minute ...
... waiting 1 minute ...
... waiting 1 minute ...
Found a new comment by: beth on 'Ben Savage Is Still Alive'
Found a new comment by: Dain Starr on 'Ben Savage Is Still Alive'
Found a new comment by: I LOVE BEN SAVAGE!! on 'Ben Savage Is Still Alive'
Found a new comment by: Chuy on 'Ben Savage Is Still Alive'
It would be a good idea to check each new feed in a new thread, as 99% of the processing time is devoted to network requests. We’ll use a new gem for this: fastthread.
require 'fastthread'
threads = []
comments = {} # hash of found comments
feeds_url.each do |feed_url|
t = Thread.new do
loop do
rss = SimpleRSS.parse open(feed_url)
rss.entries.each do |entry|
# ...
end
sleep 60
end
end
threads << t
end
# Wait for all threads to exit (never, in this case)
threads.each { |t| t.join }With this code, you’ll see the same output as earlier, but, you guess, with every messages mixed.
Finally, we can put our data in an Amazon SQS queue. In the graph, we’ll show one node per feed, add a child node when a new comment comes in. This way, we only need one message format, only containing the feed URL.
It’s time to register to Amazon SQS. You’ll get two keys: AWS access key and AWS secret access key. You’ll also need to install a new gem: SQS.
Just require ’sqs’ as the other gems, set your credentials, and create a queue named “feeds”.
require 'rubygems'
require 'sqs'
SQS.access_key_id = "my key here"
SQS.secret_access_key = "my secret key here"
queue = SQS.create_queue "feeds"
Each time a comment is detected, we can put a message in the queue.
rss_comments.entries.each do |comment|
if not comments[comment.guid]
comments[comment.guid] = comment.guid
puts "Found a new comment #{comment.title} on '#{entry.title}'"
queue.send_message "#{feed_url}"
end
endYou can download the full code here.
When you’ve finished playing, you can delete the queue with the following piece of code.
SQS.get_queue("feeds").delete!You’re now ready to animate this with our Flex graph. It will be the subject of the next and last part of this article. (actually, there will be a fourth post to illustrate all of this, think about a racing game…).
! French readers, you can get it 
. I bought this book 2 years ago, when it still was a beta PDF. Peter Armstrong then found a paperback editor, and you’ll easily understand why if you read it.
Recent Comments