Toblog

I’m currently writing some code that fetches a file from an sftp server. Using ruby 1.8.7 / net-sftp 2.0.4 the following code triggers an exception:

Net::SFTP.start( sftpHost, sftpUser, :password => sftpPass ) do |sftp|
  $stderr.puts "Downloading: #{sftpPath}#{sftpFile}"
  xml = sftp.download!( "#{sftpPath}#{sftpFile}" )
end

The exception is:

/usr/lib64/ruby/gems/1.8/gems/net-sftp-2.0.4/lib/net/sftp/session.rb:123:in `download!': uninitialized constant Net::SFTP::Session::StringIO (NameError)

The solution is to add:

require 'stringio'

to the top of the session.rb file mentioned in the exception message.

Update: 2010-08-19 16:00 I’ve just heard back from the developer having emailed him this blog entry. He is hopefully pushing a fix this evening. Now that’s service!

Update: 2010-08-20 Last night version 2.0.5 was released which has the fix in it.

I have a query against an Oracle database that returns a decimal value for a price. Ruby’s Oracle connector returns that as a BigDecimal object.

Once I have got all the items I have to aggregate them. As part of the aggregation I run this code:

@hash[id].price = ( ( @hash[id].price * @hash[id].fill_volume ) +
                    ( item.price * item.fill_volume ) ) /
                  ( @hash[id].fill_volume + item.fill_volume )

Each time it runs this code it gets slower. After about 250 loops of this code it gets really slow. Here’s the time it takes to do the 269th loop (the times are UNIX time with microseconds):

269
1281710074.568992
1281710075.574573
270

…just over a second! This seems to happen more if the id is the same over many loops.

A very simple solution is to make the price be a Float rather than a BigDecimal. When I do that, I get the following times:

269
1281710378.886902
1281710378.886920
270

Significantly faster (and given I have 10000s of lines to parse this is a big thing) worth the potential loss of accuracy. What I’d like to know is why BigDecimal behaves like this (at least in ruby 1.8.6). Any ideas?

I’ve bought the blogroll back since it disappeared during the upgrade. It’s now generated from my google reader opml feed which I export and run though some ruby which means it’s going to be up-to-date more frequently.

Here is the code that takes the opml and outputs the html fragment:

#!/usr/bin/ruby

require 'rexml/document'
require 'pp.rb'

class FeedData
  attr_accessor :name, :url, :category

  def initialize( name, url, category )
    ( @name, @url, @category ) = name, url, category
  end
end

class FeedDataList
  def initialize()
    @array = Array::new
  end

  def add( name, url, category )
    tmpFeed = FeedData::new( name, url, category )
    @array.push( tmpFeed )
  end

  def eachCategory
    catArray = Array::new
    @array.each do |feed|
      if !catArray.member?( feed.category ) then
        catArray.push( feed.category )
      end
    end

    catArray.sort.each do |result|
      yield result
    end
  end

  def eachItemInCategory( category )
    @array.each do |feed|
      if feed.category == category then
        yield feed
      end
    end
  end
end

def parse_opml( opml_node, feeds, parents_names=[] )
  opml_node.elements.each('outline') do |element|
    if element.elements.size != 0 then
      feeds = parse_opml( element, feeds, parents_names + [ element.attributes[ 'text' ] ] )
    end
    if element.attributes['xmlUrl'] then
      feeds.add( element.attributes['title'], element.attributes['htmlUrl'], parents_names.last )
    end
  end

  return feeds
end

opml = REXML::Document.new( STDIN )
feeds = FeedDataList::new
feeds = parse_opml( opml.elements['opml/body'], feeds )

feeds.eachCategory do |category|
  print "<br /><strong>#{category}</strong> "
  feeds.eachItemInCategory( category ) do |item|
    print "<a href=\"#{item.url}\">#{item.name}</a> "
  end
  print "\n"
end

To make it work:

$ ./opml2html.rb < <opml file>

I got the inspiration for the recursive opml parsing code from the Dekstop blog, so thanks are due to them!

UPDATE 2007-09-30: You can now make the twitter facebook application update your status, so this is now redundant!

UPDATE 2007-09-09 (Cribbed from Christian’s site as I couldn’t put it any better): It is with great disappointment that I must make this announcement. Facebook has requested that I remove the code from my website. They have also contacted everyone else who has found my code and publicly mentioned that they are using it.

I am saddened at this turn of events because the idea behind the code was to extend Facebook’s current service and fill in the gap that their API had. The API still does not provide a means for updating ones status.

I’ve been wanting to update my facebook status with my twitterings for some time now. Unfortunately facebook do not provide an API for setting the status so I have been thwarted. Until now. Yesterday I accidentally found Christian Flickinger’s blog entry where he has found a way to update facebook status using php. At the end of his blog he writes:

Anyone with some experience could easily use the above code to check Twitter and (if updated) push to Facebook. Happy mashing!

I know how the twitter api works and I’m a big fan of ruby so I thought I would pick up the gauntlet and hack up a quick ruby script to do just that.

You’ll need to install the json and curb gems for it to work. Then copy and paste the below code into a file called fbTwit.rb in a suitable place on your unix box (it writes state down so you may want it in it’s own directory), edit the variables so that they are correct for your accounts, run it by hand to check it works for you and then set it to run every (say) 10 minutes via cron with something like the following in your crontab:

00,10,20,30,40,50 * * * * cd [directory]; ./fbTwit.rb 2>&1 > /dev/null

# Code removed at the request of facebook.

The quality of the code probably isn’t that great: it only took a short time to write, but it works (for me).

[2007-08-01] Yesterday this stopped working, along with the SSL login for m.facebook.com. I have changed it to use the http login which still works, although whether you want your password sent in the clear is up to you.

Iris use dropped in ID Card plans

Fusion reactors in 10 years claim

Internet Explorer unsafe for 284 days in 2006

Part of Russian rocket lands in Wyoming

The problem with OpenXML
… ‘not only must an interoperable OOXML implementation first acquire and reverse-engineer a 14-year old version of Microsoft Word, it must also do the same thing with a 16-year old version of WordPerfect.’

15 tips to choose a good text type

Quit [Smoking] Counter

Humble Little Ruby Book

Riverbend’s take on Saddam’s hanging / lynching

Lord Levy in abuse of tax payer’s money allegation

New Scientist compiles a list of fun materials