29th July 2008

How to get search engine (Google, Yahoo, MSN) referal keywords using PHP

There are many reasons why you'd want to know what keywords your current visitor entered your website on; whether you want to show them similar content, gather opinion or maybe show a targetted piece of advertising, the bigger question is "how?". I recently (through boredom more than necessity) gave this a go, so I'll show you how I did it.

What I wanted to achieve

On Seopher.com I receive quite a lot of traffic from Google - a good 60-70% of my traffic is natural search oriented. I wanted a way to offer a richer experience to these users and try to increase the number of pages they view. So the objective was to take the keywords they entered in on and provide a list of similar articles.

PHP Functions

First off let's look at the functions I've written to get the job done.

This first function (getReferalHost) simply obtains the referal URL and uses the PHP function parse_url() to break things down. Then it's a simple set of 'if' statements to return a user-friendly hostname. So whereas the hostname might be images.google.com, I'm using "strstr" to check whether the string 'google' is in the URL, and so on. It's not foolproof but it's okay.

function getReferalHost() { $refer = parse_url($_SERVER['HTTP_REFERER']); $host = $refer['host']; if(strstr($host,'google')) { return 'Google'; } elseif(strstr($host,'yahoo')) { return 'Yahoo'; } elseif(strstr($host,'msn')) { return 'MSN'; } }

This next function (getKeywords) parses the referal URL again (not best practice is it? Shame on me). Then I've got a specific preg_match for each of the different vendors (based on the URLs that each of them uses. Google will format their query as such: http://www.google.com/search?&rls=en&q=seopher+another+keyword so I know that what I'm after starts with '&q=' with the keywords separated by '+'. So the regex used matches that. Yahoo and MSN do exactly the same, just with slightly different syntax - hence the need for the 'if' statement.

function getKeywords() { $refer = parse_url($_SERVER['HTTP_REFERER']); $host = $refer['host']; $refer = $refer['query']; if(strstr($host,'google')) { //do google stuff $match = preg_match('/&q=([a-zA-Z0-9+-]+)/',$refer, $output); $querystring = $output[0]; $querystring = str_replace('&q=','',$querystring); $keywords = explode('+',$querystring); return $keywords; } elseif(strstr($host,'yahoo')) { //do yahoo stuff $match = preg_match('/p=([a-zA-Z0-9+-]+)/',$refer, $output); $querystring = $output[0]; $querystring = str_replace('p=','',$querystring); $keywords = explode('+',$querystring); return $keywords; } elseif(strstr($host,'msn')) { //do msn stuff $match = preg_match('/q=([a-zA-Z0-9+-]+)/',$refer, $output); $querystring = $output[0]; $querystring = str_replace('q=','',$querystring); $keywords = explode('+',$querystring); return $keywords; } else { //else, who cares return false; } }

Really you can add as many different vendors to this function, I'm just focusing on the obvious 3 because they constitute about 98%+ of my search traffic.

This final function is what I'm using to write the recommendations; it takes the keywords used as a parameter and uses MySQL scoring to deliver results... So it takes the search terms and tries to find similar content on the website:

function getSearchHelperList($term, $limit, $currentPageID) { $searchterm = $term; //a function I use to sanitise all inputs $searchterm = $this->cleanInput($searchterm); /*query that takes the searchterms and scores it against the articles it also checks that the title isn't the same as the page we're already on because we don't want it showing you a link to your current page*/ $sql = "SELECT CatID, Title, Content, CleanTitle, UNIX_TIMESTAMP(TIME) AS FORMATED_TIME, MATCH (Title, Content) AGAINST ('$searchterm') AS score FROM Articles WHERE MATCH (Title, Content) AGAINST ('$searchterm') AND CleanTitle != '$currentPageID' AND Time < CURRENT_TIMESTAMP ORDER BY score DESC"; $result = mysql_query($sql); $num = mysql_numrows($result); if($num < $limit) { $limit = $num; } //old school right? for loops! //build an unordered list of related content if($limit > 0) { echo '<ul>'; for($i=0; $i<$limit; $i++) { echo '<li><a href="/articles/'.mysql_result($result,$i,'CleanTitle').'">'; echo mysql_result($result,$i,'Title'); echo '</a></li>'; } echo '</ul>'; } else { //display a message } }

Usage

Using these functions is really simple, I've commented my code so you can see the method behind the madness:

/*if there were keywords = it was a search engine referal from one of the vendors I'm scanning*/ if($s->getKeywords()) { //get the keywords $keywords = $s->getKeywords(); if($id) { //if the page has an ID set (only content pages have ID's $articleID = $id; } else { //I thought I'd be nice and creative about nulling my articleID //this is an article title I'll never use (hopefully anyway!) $articleID = 'rawr-donkey-squirrel-ballsack'; } ?> <div class="rss-promo search-helper"> <span>Can I help you <strong><?php echo $s->getReferalHost(); ?></strong> user?</span> You entered looking for <strong> <?php $helperterms = ''; foreach($keywords as $keyword) { $helperterms.= $keyword.' '; echo $keyword.' '; } ?> </strong> so maybe you'd also be interested in:<br /> <?php /*get the list of related content, using all the keywords, how many items I want returned and the articleID to avoid*/ $s->getSearchHelperList($helperterms, 6, $articleID); ?> </div> <?php }

Done! This example is pretty specific to what I'm doing but it shows you how I'm getting keyword data back. The functions should be pretty re-usable too (bar the getSearchHelperList which is bespoke to my needs). I'm not providing a download with this one because there's not enough to download, but the demo has instructions on what to do.

Tagged with php