Remove Duplicates From XML with XSLT

I have a sitemap.xml. I am trying to figure out how to remove duplicate <loc>. If <loc> is a duplicate then remove the containing <url> and all contents with the duplicate <loc>. I have my xslt somewhat working. It removes the <loc> if it is a duplicate, but leaves the leftover <lastmod>, <priority>, and <changefreq> for each duplicate. Any help would be greatly appreciated. Jake

Here is my XSLT Code:

<?startSampleFile ?>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
version="1.0"
xmlns:s="http://www.sitemaps.org/schemas/sitemap/0.9"
exclude-result-prefixes="s">

<xsl:output method="xml" omit-xml-declaration="yes"/>

<xsl:key name="kloc" match="s:loc"
use="." />
<xsl:key name="klastmod" match="s:lastmod"
use="../s:loc"/>
<xsl:key name="kpriority" match="s:priority"
use="../s:loc"/>
<xsl:key name="kchangefreq" match="s:changefreq"
use="../s:loc"/>

<xsl:template match="/*">

<xsl:for-each select=
"*/s:loc[generate-id()
=
generate-id(key('kloc',.)[1])]">
<xsl:value-of select="."/>
<xsl:for-each select="key('klastmod', .)">
<xsl:value-of select="."/>
</xsl:for-each>
<xsl:for-each select="key('kpriority', .)">
<xsl:value-of select="."/>
</xsl:for-each>
<xsl:for-each select="key('kchangefreq', .)">
<xsl:value-of select="."/>
</xsl:for-each>
</xsl:for-each>
</xsl:template>

</xsl:stylesheet>

Existing XML:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.example.com/first</loc>
<lastmod>2012-01-27T10:18:26+00:00</lastmod>
<priority>0.90</priority>
<changefreq>daily</changefreq>
</url>
<url>
<loc>http://www.example.com/second</loc>
<lastmod>2012-03-16T07:38:42+00:00</lastmod>
<priority>0.60</priority>
<changefreq>weekly</changefreq>
</url>
<url>
<loc>http://www.example.com/second</loc>
<lastmod>2012-03-16T08:36:44+00:00</lastmod>
<priority>0.60</priority>
<changefreq>weekly</changefreq>
</url>
<url>
<loc>http://www.example.com/third</loc>
<lastmod>2012-03-16T08:37:09+00:00</lastmod>
<priority>0.60</priority>
<changefreq>weekly</changefreq>
</url>
</urlset>

Desired Output:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>http://www.example.com/first</loc>
<lastmod>2012-01-27T10:18:26+00:00</lastmod>
<priority>0.90</priority>
<changefreq>daily</changefreq>
</url>
<url>
<loc>http://www.example.com/second</loc>
<lastmod>2012-03-16T07:38:42+00:00</lastmod>
<priority>0.60</priority>
<changefreq>weekly</changefreq>
</url>
<loc>http://www.example.com/third</loc>
<lastmod>2012-03-16T08:37:09+00:00</lastmod>
<priority>0.60</priority>
<changefreq>weekly</changefreq>
</url>
</urlset>

Re: Remove Duplicates From XML with XSLT

Hello

In fact I think you can just use LINQ-TO-XML to deal with the problem by checking whether the loc's Value is duplicated or notIf yesplease remove it and do outputtingSample codes look like this following

namespace MyTest
{
    class Program
    {
        static void Main(string[] args)
        {
            XDocument doc = XDocument.Load("abc.xml");
            XNamespace xname = XNamespace.Get("http://www.sitemaps.org/schemas/sitemap/0.9");
            var result = doc.Descendants(xname+"url");
            List<XElement> elements = new List<XElement>(result);

            for (int i = 1; i < elements.Count; i++)
            {
                if (elements[i - 1].Element(xname+"loc").Value == elements[i].Element(xname+"loc").Value)
                {
                    elements.RemoveAt(i);
                    i--;
                }
            }

            doc.Root.Elements().Remove();
            foreach (var item in elements)
            {
                doc.Root.Add(item);
            }
            doc.Save("c:\\try.xml");
        }
    }
}