ProjectX Blog

Further to my first post on using acts_as_xapian, I have been trying to make xapian work with pagination and association proxies properly

class Lesson < ActiveRecord::Base
# Index user_id as a term in xapian

    belongs_to :user
    def self.find_with_xapian(search_term, options={})
      ActsAsXapian::Search.new([self], search_term, options).results.collect{|x| x[:model]}
    end
end

class User < ActiveRecord::Base
    has_many :lessons do  #Extend this association
      # Override the method in lessons
      def self.find_with_xapian(search_term, options={})
        ActsAsXapian::Search.new([proxy_reflection.klass],
          "{proxy_reflection.primary_key_name}:#{proxy_owner.id}  #{search_term}", options)
        .results.collect{|x| x[:model]}
      end
    end
end

This will ensure that current_user.locations.find_with_xapian will find the correct number of locations, enabling us to work with pagination, etc. What’s left to do is to get the Search object out so that we can get matches_estimated out. I’ll leave out for next time.

This post was brought to you from Software Freedom Day, New Zealand.

Jakob Nielsen has just written an article on location finders. The article outlines some behaviour changes of users in finding locations over the years.

The lastest research has found…

It’s still striking that 73% of users went to a search engine (mainly Google) when we asked them to find a nearby location for a specific company. Only 13% went straight to that company’s own website, while the remaining 13% went to a dedicated mapping service.

Given this changing behavior, we now have guidelines for search engine optimization (SEO) for the locator.

We’ve focused on making ZoomIn SEO friendly from day one so that businesses can be easily found. We constantly tune our system to ensure that we’re at the top the list for a result.

This re-inforces the need for businesses to make sure that their locations are being “found”, using ZoomIn is a good way to make sure that happens!

Runtime Page Optimiser (RPO) is now out of beta, the team from www.actionthis.com have release the automagic web optimisation tool for IIS based sites. RPO is certified for ASP.NET 2.0 and MOSS/SharePoint 2007 intranet and internet sites.

Here’s a list of benefits:





But wait there’s still more, the team are going to release shortly an online webpage optimisation report tool based on AOL Page test.

Congrats to the ActionThis team and I’m looking forward for future improvements.

We use xapian as our offline indexing solution here at ProjectX. Recently, Francis Irving from mySociety has started a fantastic Rails plugin, acts_as_xapian. I’ll leave the introductions to the aforementioned pages. Suffice to say that it is quite easy to install, and it utilizes all the power of Xapian in a very smooth way.

One thing I have not been able to crack is to utilize association proxies on searches. Locomotivation has a solution using named_scope, but that approach removes all ordering, in addition to issuing one more unnecessary SQL statement.

class Lesson < ActiveRecord::Base
    named_scope :find_with_xapian, lambda { |*args| {:conditions => ["lessons.id in (?)", ActsAsXapian::Search.new([Lesson], args.to_s).results.collect{|x| x[:model].id}]}}
end

But if we do this instead, we can now do association proxies

class Lesson < ActiveRecord::Base
    def self.find_with_xapian(search_term, options={})
      ActsAsXapian::Search.new([self], search_term, options).results.collect{|x| x[:model]}
    end
end

current_user.lessons.find_with_xapian "something"

That would automatically scope and return lessons that belong to the current user only. Awesome!

Update: The code is not showing quite as nicely. I have pasted the same code in Pastie – http://pastie.org/270114

A while back, we did some work for Gibson groups for the Te Papa Ourspace project. We created a special version of the ZoomIn Map API for a special part of the project called the Wall.

Its nice to finally see it action!

The net is ablaze with all the iPhone pricing rip off.(Although that haven’t stopped some from queuing up already)

My 2c on the whole pricing thing is Vodafone have set the prices so that they don’t marginalise the rest of their smart phone line up.

If you sell a iPhone 3Gfor $500 or $199, then why would you buy a Nokia N95 at $1399 ?

Steve Jobs set the pricing in US so that iPhone would blow away the competition. The pricing here doesn’t reflect that 🙁

Unfortunately, NZ’s probably not big enough to invoke Apple pulling the iPhone from the Canadian marketplace.

It seems that 37 signals are dropping IE6 support. (I wish!)

Our usage on ZoomIn as follows:
IE6 24%
IE7 43%
Firefox 21%
Safari 4.5%

So we have a ways to go before IE6 support is dropped from ZoomIn !

Startup Magazine issue #2 is out on the stand.

Here’s my two cents about Yahoo search.

We get a lot of traffic from search engines. So we spend a lot of time optimising our SEO and our site to make sure we have as much content in the search engine index.

When it comes down to the hard numbers of referrers, 98% of our search engine referrers come from Google and only 2% comes from Yahoo. (We have about the same number of pages in both indexes) Even though Yahoo is now powering the search for Xtra, the return for us is still so low. We’d rather spend our time and energy optimising SEO for Google we get far more return.

I’m sorry Yahoo, you don’t deliver the traffic to make it worth our while keeping our index fresh. As a result of poor click return, we restrict the rate that Yahoo crawler can access our site. (Not that they pay attention to robots.txt !)  That way Google has more of our CPU cycles and bandwidth to crawl our site.

<rant>

I’ve just got a response from Yahoo about a question I asked them yesterday and I got a “canned” response.

When will companies, learn that canned responses wastes your customers time!

Here’s my email to yahoo.

Hi

Yahoo Slurp is ignoring robots.txt for my site.

I had the rule

User-agent: Slurp
Crawl-delay: 200

and it is being ignored. (Its worked for months like this)

I have now updated our robots.txt about 12 hours ago to

User-agent: Slurp*
User-agent: Yahoo*
Crawl-delay: 200

And its still ignoring the robots.

What’s going on ?

Here’s there response.

Thank you for writing to Yahoo! Search.

We apologize for the excess traffic to your site. In a continuing effort
to improve our index freshness and comprehensiveness we are bringing
online additional crawler machines to prepare them for production.

To limit the amount of traffic your site receives, you can place a
crawl-delay in your robots.txt, specifying the number of seconds each
crawler will wait between each request. Please check out the following
pages for detailed information about this:

Thanks Yahoo for answering my question. NOT
</rant>