Get a list of Google Search results RSS

• • Reply
  

  RodrigueH

  Member

  225 Points

  59 Posts

  Get a list of Google Search results

  Oct 23, 2008 06:53 PM|LINK

  Hello everyone ,

  I come to you with an interesting thing I thaught about doing for myself althought there is this already complete tool from google  : http://blogsearch.google.com/

  I am thinking of developping an web application that permits me to track new changes that the google blog search engine finds..

  Can I in any possible way retrieve in some kind of list all the search results google finds for an search?

  exemple : I want to search "myname", and i fill a list with the results that google returned for "myname"

  and then i can analyze them ..save them etc..

  So basically my question is : Is it possible to retrieve google's search results in a list on my asp.net web application ?

  google results search google search retrieve in list google search results retrieve in list

  Rod H
  • Moved by XIII on Oct 23, 2008 07:38 PM
• • Reply
  

  wortho

  Member

  579 Points

  92 Posts

  Re: Get a list of Google Search results

  Oct 23, 2008 08:00 PM|LINK

  Hi Rod,

  Yes you can do what you are asking. Surf on over to Googles CSE (Custom Search Engine) here http://www.google.com/coop/cse/ 

  Now here you can define the domain(s) (ie' your website or blog) that you want to search on.

  Then in your asp.net application you can send a request to google, with your CSE Id and google will return you an xml list of search results, pages, time for search, everything you see on the google search result. Then you can format or save it or what-have you.

  We use it in our websites to provide a search (look at the search box on the header www.cadabranewzealand.com) that just submits a search to google, we get an Xml resultset back and format it with Xslt.

  Hth,

  Steve

  Please remember to 'Mark as Answer' if this post answered your question!
  • Marked as answer by RodrigueH on Oct 24, 2008 05:56 PM
• • Reply
  

  RodrigueH

  Member

  225 Points

  59 Posts

  Re: Get a list of Google Search results

  Oct 24, 2008 05:56 PM|LINK

  Hey thanks for the reply, I already did the first step :

  1)Getting CSE.

  2)I am now searching threw google on to send a request to google...If you can lead me with an exemple would be great!..

  And your website is great by the way and thanks for your reply.
  

  Rod H.
  

  Rod H
• • Reply
  

  wortho

  Member

  579 Points

  92 Posts

  Re: Get a list of Google Search results

  Oct 24, 2008 07:37 PM|LINK

  Hey Rod,

  There is several examples on Google for using their CSE, have a look here - http://www.google.com/coop/docs/cse/cref.html

  In a nutshell, you can send a request like this;

  String requestString = "http://www.google.com/cse?cx=" + googleCse + "&client=google-csbe&output=xml_no_dtd&q=" + searchQuery;

  I use WebRequest to submit it to google and get an Xml document back (just paste the string in your browser to see the result)

  Obviously replace the variables with your Guid for the CSE and the text for the Search Query.

  Then you can do whatever with the xml document you get back.

  Cheers,

  Steve

  Please remember to 'Mark as Answer' if this post answered your question!
• • Reply
  

  RodrigueH

  Member

  225 Points

  59 Posts

  Re: Get a list of Google Search results

  Oct 24, 2008 08:16 PM|LINK

  Hehe great!..

  well thanks a lot, I somehow needed an example of something, i was getting a bad request, and then i figured it out..

  Sorry i sounded like a beginner, because I am one..I am still a student in university and I am learning .NET on my own because in uni we are still on the basic stuff!

  Thanks again!

  
  
  Rod H
  • Edited by RodrigueH on Oct 24, 2008 08:48 PM
• • Reply
  

  RodrigueH

  Member

  225 Points

  59 Posts

  Re: Get a list of Google Search results

  Oct 24, 2008 09:46 PM|LINK

  Hey Steve, last question I promiss, what I am doing now is getting the response into a string and removing with regular expressions things i dont need, is there a better way to do this?

  Can i get the results in a better form like some kind of xml ?

  Thanks again!.
  

  Rod H
• • Reply
  

  wortho

  Member

  579 Points

  92 Posts

  Re: Get a list of Google Search results

  Oct 24, 2008 10:36 PM|LINK

  Yeah it should come back as Xml then you can just run an Xsl Transform over it.

  Whats the url you are sending as the request?

  See the one I posted above, it includes the command to say I want the output in xml without the dtd - so add that to your request, so that it looks something like

  http://www.google.com/cse?cx=000527865469685173751%3Ac7-le6lc0c8&client=google-csbe&output=xml_no_dtd&q=auckland

  So replace the google cx id with the search engine you want to query (the search you set up or another) and the q= with the query you want to send.

  Then you get back some nice Xml.Note the Xml will be different depending on number of pages, spelling suggestions and so on.

  Hope that helps. Let me know if it doesnt work for you.

  Cheers,

  steve

  Please remember to 'Mark as Answer' if this post answered your question!
• • Reply
  

  RodrigueH

  Member

  225 Points

  59 Posts

  Re: Get a list of Google Search results

  Oct 25, 2008 07:14 AM|LINK

  Well i was using this query  :

  http://www.google.com/cse?cx=013193463202043133794:es0nsfnhcok&ie=UTF-8&q=\"rodrigue+hajjar\"&sa=Search   (this works fine but doesn't return XML )
  

  because if used the first one you gave me replacing with my cx ofcourse, I'd get 400 bad request error.

  and I tried the one you just gave me it works fine, but when i replace with my cx i get a bad request error.

  my cx is :  013193463202043133794

  Thanks again.

  Rod H.
  

  Rod H
• • Reply
  

  wortho

  Member

  579 Points

  92 Posts

  Re: Get a list of Google Search results

  Oct 25, 2008 09:09 AM|LINK

  Looks like you are missing part of your CX.

  Should it be like this?

  Cx = 013193463202043133794:es0nsfnhcok

  So then you would get something like this?

  http://www.google.com/cse?cx=013193463202043133794%3Aes0nsfnhcok&client=google-csbe&output=xml_no_dtd&q=rodrigue

  You know the Home page for your CSE - click on the code tab and email me the form information and I can verify it for you. It will give you some sample Html code as though you were going to embed the search.

  Cheers,

  Steve

  Please remember to 'Mark as Answer' if this post answered your question!
• • Reply
  

  RodrigueH

  Member

  225 Points

  59 Posts

  Re: Get a list of Google Search results

  Oct 25, 2008 10:03 AM|LINK

   

  Congratulations on creating the Search Specific List search engine. Below you will find some important information to assist you in the ongoing management of your Custom Search Engine:

  • For day-to-day management of your search engine visit http://www.google.com/coop/manage/cse?cx=013193463202043133794:nxqdicc2lpq
  • To incorporate your search engine into your site or blog, you can find the necessary code at http://www.google.com/coop/manage/cse/code?cx=013193463202043133794:nxqdicc2lpq
  • To add your Custom Search Engine search box to your Google Personalized Homepage, visit http://fusion.google.com/add?moduleurl=http%3A//www.google.com/coop/api/013193463202043133794/cse/nxqdicc2lpq/gadget
  • To manage all your Custom Search Engines from your Google Personalized Homepage, add the Custom Search Console by visiting http://fusion.google.com/add?moduleurl=http%3A//www.google.com/coop/cse/manage_gadget.xml

  Thank You for using Google Custom Search!

  The Google Custom Search Team

   

  This is the email i got from them when i created the Custom Search Engine.

   
  

  Rod H