I am using Futures to implement sitemap.

1) How do I split sitemap files to less than 50,000 records?

2) How do I add provider for yahoo?

Should I just write my own interaface for Sitemaps instead of using the Search provider from Futures?

 Has this been integrated to 3.5? all I see is the preview page: http://quickstarts.asp.net/Futures/services/doc/searchsitemaps.aspx

The problem I see with the current release is that it is not possible to split index files into groups of less than 50,000 (this is limit for google). You have to manually list all files in web.config, example:

<searchSiteMap enabled="true">

<add name="Navigation" type="Microsoft.Web.Preview.Search.AspNetSiteMapSearchSiteMapProvider, Microsoft.Web.Preview"/>

<add name="Category" type="CategorySiteMapData, App_Code" targetUrl="Category" pathInfoFormat="true" queryStringDataFields="Id" targetUrlseparator=""/>

</searchSiteMap>

However it is possible that in the case above AdSiteMapData will return more than 50,000 rows. It would be nice to have additional parameter specifying maximum file size and have the system automaticlly split it. I am not sure how to do it with the existing system. I guess I need to write it all on my own (which should not be that difficult after all)...

If there is any way too split this link file into several files with each containging less than 50,000 links please let me know. 

If it is not possile, do you think this will be addressed in the next release?

Another problem is how to make this working with yahoo (simple urllist.txt)?

Thanks so much!

I got this working by creating few classes (each returning less than 50,000 records). However now I am getting error that the date is invalid:

41

Invalid date An invalid date was found. Please fix the date or formatting before resubmitting.

Parent tag: Tag: Value: Found:

url lastmod 4/19/2008 12:33:02 AM Apr 18, 2008

 Here is my code:

1    using System;
2    using System.Collections;
3    using System.Collections.Generic;
4    using System.Collections.Specialized;
5    using System.Data;
6    using System.IO;
7    using System.Text;
8    using System.Web;
9    using System.Xml;
10   
11   using System.Data.Linq;
12   using System.Linq;
13   
14   
15   //using System.IO;
16   //using System.Query;
17   
18   using AspNet.StarterKits.Classifieds.BusinessLogicLayer;
19   using AspNet.StarterKits.Classifieds.Web;
20   using SubSonic;
21   using CSKDB;
22   using Microsoft.Web.Preview.Search;
23   
24   public class AdSiteMapData : DynamicDataSearchSiteMapProvider
25   {
26       public override IEnumerable DataQuery()
27   
28       {
29           //return AdsDB.GetActiveAds(DefaultValues.IdNullValue);
30           //return (IEnumerable)SPs.GetActiveAdsByRowRange(0, 39000).GetReader();
31           //LINQ
32           adDataContext ads = new adDataContext();
33           var a = ads.GetActiveAdsByRowRange(0, 49999);
34           return a;
35   
36           //var a = ads.GetActiveAdsByRowRange(0, 49999);
37       }
38   }
39   
40   public class AdSiteMapData2 : DynamicDataSearchSiteMapProvider
41   {
42       public override IEnumerable DataQuery()
43       {
44           //LINQ
45           adDataContext ads = new adDataContext();
46           var a = ads.GetActiveAdsByRowRange(50000, 99999);
47           return a;
48       }
49   }
50   
51   //public class AdSiteMapData3 : DynamicDataSearchSiteMapProvider
52   //{
53   //    public override IEnumerable DataQuery()
54   //    {
55   //        //LINQ
56   //        adDataContext ads = new adDataContext();
57   //        var a = ads.GetActiveAdsByRowRange(100000, 149999);
58   //        return a;
59   //    }
60   //}
61   
62   public class CategorySiteMapData : DynamicDataSearchSiteMapProvider
63   {
64   
65       public class Foo
66       {
67           public String Id;
68   
69       }
70   
71       public override IEnumerable DataQuery()
72       {
73   
74           //IDataReader dr = SPs.GetCategoryWithAds().GetReader();
75           //List<Foo> list = new List<Foo>();
76           //while (dr.Read())
77           //{
78           //    Foo foo = new Foo();
79           //    foo.Id = dr[0].ToString();
80           //    list.Add(foo);
81           //}
82           //dr.Close();
83           //return list.ToArray();
84           
85           //DataSet ds = new DataSet();
86           //DataSet ds2 = new DataSet();
87   
88           // THIS BELOW DOES NOT WORK. WHY????
89           //Must be strongly typed!
90           //ds = SPs.GetCategoryWithAds().GetDataSet().Tables[0].DefaultView;
91           //ds2.Tables.Add(ds1.Tables[0].Copy());
92   
93           //return (IEnumerable)SPs.GetCategoryWithAds().GetDataSet().Tables[0].Rows;  
94   
95           //LINQ
96           adDataContext ads = new adDataContext();
97           var a = ads.GetCategoryWithAds();
98           return a;
99   
100          //var a = ads.GetActiveAdsByRowRange(0, 49999);
101          //return
102          //(
103          //    from A in ads.GetActiveAdsByRowRange(0, 49999)
104              
105          //    select new Foo
106          //    {
107          //        Id = A.Id;
108          //    }
109          //)
110  
111          //return (IEnumerable);
112  
113          //Northwind db = new Northwind(connectionString);
114          //Table<Product> Products = db.Products;
115  
116          //var q =
117          //    from p in Products
118          //    where p.UnitsInStock > (p.ReorderLevel * 10)
119          //    select p;
120  
121          //return q;
122      }
123  }

Ok, I have formatted the date in the stored procedure and returned as string. Now have another problem.

Seems that if the Web.Sitemap map contains external links, the otput gets formatted in wrong way when using Microsoft.Web.Preview.Search.AspNetSiteMapSearchSiteMapProvider

The result can be seen here: http://test.zikbay.com/SearchSiteMaps.axd?sitemap=Navigation

 All the external links in web.sitemap show as follows:

1    - <url>
2      <loc>http://test.zikbay.comhttp://community.zikbay.com/blogs/features/default.aspx</loc> 
3      </url>

Here are my settings from web.config:

1    	<microsoft.web.preview>
2    		<searchSiteMap enabled="true">
3    			<providers>
4            <add name="Navigation" type="Microsoft.Web.Preview.Search.AspNetSiteMapSearchSiteMapProvider, Microsoft.Web.Preview" />
5    				<add name="Ad" type="AdSiteMapData, App_Code" targetUrl="Ad" pathInfoFormat="true" queryStringDataFields="Id" targetUrlseparator="" lastModifiedDataField="ModifiedOn"/>
6            <add name="Ad2" type="AdSiteMapData2, App_Code" targetUrl="Ad" pathInfoFormat="true" queryStringDataFields="Id" targetUrlseparator="" lastModifiedDataField="ModifiedOn"/>
7            <add name="Category" type="CategorySiteMapData, App_Code" targetUrl="Category" pathInfoFormat="true" queryStringDataFields="Id" targetUrlseparator="" lastModifiedDataField="ModifiedOn" />
8    			</providers>
9    		</searchSiteMap>
10   	</microsoft.web.preview>

This is my sitemap:

1    
2    <siteMap>
3    	<siteMapNode url="~/" title="Home"  description="">
4    		   
5    		<siteMapNode url="default.aspx" title="Home"  roles="*">
6    			<siteMapNode url="Category/0" title="All Categories"  description=""/>
7    			<!--<siteMapNode url="Category/147412" title="Buy/Sell"  description=""/>-->
8    
9          <siteMapNode url="Green.aspx" title="Green"  description=""/>
10         <siteMapNode url="Category/10542" title="Real Estate"  description=""/>
11   			<siteMapNode url="Category/147413" title="Jobs"  description=""/>
12   			<siteMapNode url="Category/316" title="Services"  description=""/>
13   			<siteMapNode url="Category/147402" title="Personal"  description=""/>
14   			<!--<siteMapNode url="Category/147447" title="Travel"  description=""/>-->
15         
16         <siteMapNode url="Category/hotels" title="Hotels"  description=""/>      
17   			<siteMapNode url="LocationSearch_All.aspx" title="All Locations"  description=""/>			
18   			
19   			
20   			<!--<siteMapNode url="Registering.aspx" title="Register"  description=""/>
21   			<siteMapNode url="Registering.aspx" title="Register"  description=""/>
22   			<siteMapNode url="Registering.aspx" title="Register"  description=""/>
23   			<siteMapNode url="Registering.aspx" title="Register"  description=""/>
24   			<siteMapNode url="Registering.aspx" title="Register"  description=""/>
25   			<siteMapNode url="Registering.aspx" title="Register"  description=""/>
26   			<siteMapNode url="Registering.aspx" title="Register"  description=""/>-->
27   			<!--<siteMapNode url="" title=""  description=""/>-->
28         
29   		</siteMapNode>
30   
31       <siteMapNode url="Category/147491" title="Coupons"  roles="*">
32       </siteMapNode>
33       
34   		<siteMapNode url="Category/Buy-Sell" title="Buy/Sell"  roles="*">
35   			<!--<siteMapNode url="" title=""  description=""/>-->
36   		</siteMapNode>
37       
38   			
39   		<!--<siteMapNode url="Category/buysell/default.aspx" title="Buy/Sell"  roles="*"/>
40   		<siteMapNode url="Category/services/default.aspx" title="Services"  roles="*"/>-->
41   
42   		<siteMapNode url="PostAd.aspx" title="Post FREE Ad"  roles="*">
43   			<siteMapNode url="" title=""  description=""/>
44   		</siteMapNode>
45   		
46   		<siteMapNode url="MyAds.aspx" title="My Ads & Profile" roles="*">
47   			<siteMapNode url="MyProfile.aspx" title="My Profile" roles="*" />
48         <siteMapNode url="user/userwebsitesettings.aspx" title="My Store/Website" roles="*" />
49         
50         <siteMapNode url="Login.aspx" title="Sign In" roles="*" />
51   			<siteMapNode url="Register.aspx" title="Create an Account" roles="*" />
52   		</siteMapNode>
53   			
54   		<siteMapNode url="Admin/Default.aspx" title="Site Administration" roles="Administrators">
55   			<siteMapNode url="Admin/Settings.aspx" title="My Profile" roles="Administrators" />
56   		</siteMapNode>
57   
58   		<siteMapNode url="http://Community.zikbay.com" title="Community"  roles="*">
59   			<!--<siteMapNode url="" title=""  description=""/>-->
60   		</siteMapNode>
61   			
62   		<siteMapNode url="Help/Help.aspx" title="Help" roles="*">
63   			<siteMapNode url="Help/Registering.aspx" title="Register"  description=""/>			
64   			<siteMapNode url="Help/PostingAd.aspx" title="Post Ad"  description=""/>
65   			<siteMapNode url="Help/PostingHotelAd.aspx" title="Post Hotel Ad"  description=""/>
66         <siteMapNode url="Help/UserStore.aspx" title="User Store/Website"  description=""/>      
67   			<siteMapNode url="Help/FAQ.aspx" title="Questions"  description=""/>
68         <siteMapNode url="Help/AvoidScam.aspx" title="Avoid Scam"  description=""/>
69         <siteMapNode url="Help/Terms.aspx" title="Terms"  description=""/>
70   			<siteMapNode url="Help/PrivacyPolicy.aspx" title="Privacy" />
71   
72   			<siteMapNode url="http://community.zikbay.com/blogs/features/default.aspx" title="What is New" roles="*"/>
73   
74   		</siteMapNode>
75   			
76     </siteMapNode>
77   </siteMap>
78   
     

I think this is a bug in SiteMapSearch provider. It just does not work with external links in web.sitemap.

The only solution I can see would be to create another class and generate entries for Web.Sitemap in Google sitemap format by using my own Class like I did with ads and categories.

Please recommend