Archive for June, 2008

Why Yahoo search is irrelevant

Friday, June 27th, 2008

Here’s my two cents about Yahoo search.

We get a lot of traffic from search engines. So we spend a lot of time optimising our SEO and our site to make sure we have as much content in the search engine index.

When it comes down to the hard numbers of referrers, 98% of our search engine referrers come from Google and only 2% comes from Yahoo. (We have about the same number of pages in both indexes) Even though Yahoo is now powering the search for Xtra, the return for us is still so low. We’d rather spend our time and energy optimising SEO for Google we get far more return.

I’m sorry Yahoo, you don’t deliver the traffic to make it worth our while keeping our index fresh. As a result of poor click return, we restrict the rate that Yahoo crawler can access our site. (Not that they pay attention to robots.txt !)  That way Google has more of our CPU cycles and bandwidth to crawl our site.

Canned responses

Friday, June 27th, 2008


I’ve just got a response from Yahoo about a question I asked them yesterday and I got a “canned” response.

When will companies, learn that canned responses wastes your customers time!

Here’s my email to yahoo.


Yahoo Slurp is ignoring robots.txt for my site.

I had the rule

User-agent: Slurp
Crawl-delay: 200

and it is being ignored. (Its worked for months like this)

I have now updated our robots.txt about 12 hours ago to

User-agent: Slurp*
User-agent: Yahoo*
Crawl-delay: 200

And its still ignoring the robots.

What’s going on ?

Here’s there response.

Thank you for writing to Yahoo! Search.

We apologize for the excess traffic to your site. In a continuing effort
to improve our index freshness and comprehensiveness we are bringing
online additional crawler machines to prepare them for production.

To limit the amount of traffic your site receives, you can place a
crawl-delay in your robots.txt
, specifying the number of seconds each
crawler will wait between each request. Please check out the following
pages for detailed information about this:

Thanks Yahoo for answering my question. NOT

Velocity conference for web performance is in full swing

Wednesday, June 25th, 2008

The Velocity conference on web performance and operations is in full swing there are a wealth of information in the slides.

Here is a selection of the most interesting presentations:


Xlinks digest – 23 / 06 /2008

Monday, June 23rd, 2008

Xlinks is a selection of interesting links as discovered by the ProjectX staff:

    Study proves Bad boys really do get the girls
    Added on 06/23/2008 at 03:15PM
    The Pixar Touch – The Making of a company – NY Times
    Added on 06/23/2008 at 03:11PM

    Happiness as your business model
    Added on 06/23/2008 at 03:09PM

    Hacking TCP/IP to support location awareness
    Added on 06/23/2008 at 12:44PM

    Bootstrappers beware
    Added on 06/21/2008 at 01:01PM

    Ajax Avenue no exit
    Added on 06/20/2008 at 10:51PM

    Subversion 1.5.0 released – simpler branching and merging
    Added on 06/20/2008 at 04:36PM

    17 mistakes that startups make
    Added on 06/20/2008 at 08:48AM

    The power of remarkable
    Added on 06/20/2008 at 07:02AM

    Web browsers and Memory Fragmentation
    Added on 06/20/2008 at 07:01AM

    Visualisation of the Code swarm for Python: AWESOME
    Added on 06/19/2008 at 05:35PM

    The Google User experience
    Added on 06/19/2008 at 03:13PM

    Adaptive path behind Myspace re-launch
    Added on 06/19/2008 at 09:42AM

    Open Cell Id database
    Added on 06/19/2008 at 09:21AM

    Value of a College Education for Ajax Developers
    Added on 06/18/2008 at 11:24AM

    Cooking for engineers
    Added on 06/18/2008 at 10:15AM

    Web design trends 2008
    Added on 06/17/2008 at 11:28AM

    VC’s Angels or bootstrapping
    Added on 06/17/2008 at 10:19AM

    Create more value than you capture.

    Tim OReilly
    Added on 06/17/2008 at 09:23AM

    Modules.js – a standalone dynamic javascript loader
    Added on 06/17/2008 at 08:58AM

Google now giving priority to fast ads

Thursday, June 19th, 2008

Its official, Google are now giving priority to faster sites. (As mentioned a month ago) Slow sites risk their “Quality” score being reduced.

So its important to Google….

Larry and Sergey, articulated this philosophy in Ten things Google has found to be true. One of these principles is “Fast is better than slow.” We’ve found this rule to be especially applicable to the landing pages of AdWords ads. When a user clicks an ad, a landing page that loads quickly provides a better user experience than a landing page that loads slowly.

So here’s the rub.

Starting today, this load time factor will be incorporated into your keywords’ Quality Scores. Keywords with landing pages that load slowly may get lower Quality Scores (and thus higher minimum bids).

Now that your Google ads monies at risk, there’s never better a time to speed up your pages.

Exploring photos on ZoomIn

Thursday, June 19th, 2008

We’re lifting the lid on some functionality that we’ve been working for a while to make it easier to browse content on ZoomIn. For the first release (currently in “Garage Mode”) we’re focusing on browsing photos. So now you can look at all of the photos around any location, as you zoom and pan around the photos will constantly update.

To get the browse photos explorer either click on  explore photos on the home page or search for any address or place and expand the results box to reveal the Explore photos link.

We have two modes of browsing Photo bar and Thumbnails.

Photo Bar style

Browse the photos via the photo bar.

Thumbnail style

View all of the photos on the map.

Explore Photos is in Garage mode, which means we’ll be tinkering and improving over the next few weeks.

We appreciate your suggestions and feedback, so let us know what you think.

Firefox 3 built for faster web experience

Thursday, June 19th, 2008

Firefox 3 has been officially released and looking under the hood it has been tweaked for faster page loading.

Here are the key connection values in Firefox 2 and 3.

Attribute: Firefox 2 Firefox 3
Max Connections 24 30
Max Connections per server 8 15
Max persistant connections per proxy 4 8
Max persistant connections per server 2 4

As we can see the maximum number of connections and number of connections per server has been increased. So that’s good news for more parallel downloading. And they’ve also increased the number of persistance connections so that future requests will be a bit faster. Another interesting thing is that http pipelining is turned off by default.

So far Firefox 3 has been snappier and more memory efficient! Download Firefox 3 today and squeeze a bit more performance from your bandwidth!

Xlinks Digest – 17 / 06 / 2008

Tuesday, June 17th, 2008

Xlinks is a collection of interesting links as discovered by the ProjectX Team

    Steve Yegge on Server side Javascript
    Added on 06/17/2008 at 07:21AM

    The passive agressive programmer or why nothing gets done
    Added on 06/16/2008 at 02:44PM

    Added on 06/16/2008 at 02:40PM

    Now you can upload PDFs into Google Docs
    Added on 06/16/2008 at 02:40PM

    Calculus in 20 minutes
    Added on 06/16/2008 at 02:39PM

    Facebook no longer the second largest social network
    Added on 06/16/2008 at 02:39PM

    How to create keypress navigation in jQuery
    Added on 06/16/2008 at 02:37PM

    37 signals on You have to treat employees like customers
    Added on 06/16/2008 at 02:36PM

    Algorithmic ink in Javascript
    Added on 06/16/2008 at 02:34PM

    Zen Planning: 4 Simple Rules for Balancing Short Term Necessity with Long Term Direction
    Added on 06/16/2008 at 02:33PM

    NZ Broadband is trailing the rest of the world
    Added on 06/16/2008 at 02:03PM

    Runtime page optimiser
    Added on 06/16/2008 at 12:19PM

    Javascript framework usage at top sites
    Added on 06/12/2008 at 08:39AM

    37 signals link to BMW’s Gina
    Added on 06/11/2008 at 10:13PM

    Yahoo Design templates -wireframes
    Added on 06/11/2008 at 05:58PM

    Yahoo Design Patterns
    Added on 06/11/2008 at 05:57PM

    Free Rails 2.1 PDF
    Added on 06/10/2008 at 04:53PM

    How not to be viral – software adoption patterns
    Added on 06/10/2008 at 03:49PM

    iPhone NZ bound July 11
    Added on 06/10/2008 at 12:54PM

    Passenger 2.0 rc1 and ruby enterprise released
    Added on 06/10/2008 at 09:43AM

ZoomIn traffic visualisation using gltrail

Monday, June 16th, 2008

How cool is this ????

Here’s a look at some ZoomIn traffic from last Sunday. The video was based on the realtime visualisation of ZoomIn using the glTrail tool by (They also make the awesome gltail tool)

Who knew ZoomIn would build its own deathstar !

Runtime Page Optimizer

Monday, June 16th, 2008

The team from ActionThis have created a new product called Runtime Page Optimizer that will dynamically optimise your webpages at runtime. It looks like a fantastic product to solve a lot of the problems highlight by the NZ Homepage hall of shame.

Runtime Page Optimiser features:

  • Combines all javascript and css on the fly
  • Generates CSS sprites on the fly
  • No code modifications required!
  • Runs on 2 and and IIS 6/7
  • Compatible with IE6/7 , Firefox 2/3, Safari 3

The product is currently in Beta and from the demos it already looking really great.

Well done guys, I look forward to seeing this product hit 1.0!