Well its definitely time I made a post regarding the status of DNN 3.1. As many of you noticed, the dotnetnuke.com site is dogfooding the 3.1 version, and things are looking really solid. There were only a couple issues to work out over the past week ( which I will discuss in a moment ) but I think we just about ready to launch. Since there has been so little communication on the subject of 3.1, I thought it would be beneficial to provide some insight into the goals of this release version.

Immediately following the release of 3.0.13, we started work on 3.1. The goal with 3.1 was to focus on 2 major enhancement areas - Web Farm Support and Core Module Abstraction. By focussing on 2 clear objectives we hoped to reduce the amount of instability in other areas of the Core; reducing the regression testing effort, and allowing for a much shorter release cycle ( the 8 month cycle for version 3.0 was much too long ). Before embarking on this effort, both of the targeted enhancements were fully documented and reviewed by the team to ensure they specified and met the required business objectives. These documents were also posted to the community and are still available by drilling into each item on the Roadmap page ( http://www.dotnetnuke.com/Development/Roadmap/tabid/616/Default.aspx ) and clicking the link in the Resources module. Going forward, all major enhancements will undergo a standard requirements/design/review phase before inclusion on the Roadmap.

Web Farm Support is a critical enhancement for enterprise deployments. It effectively allows DotNetNuke to run in a load balanced environment. Load balancing is a technique which can be successfully used to manage both load/performance and redundancy. The initial plan was to build a web farm solution which catered to a single use case identified by the Hosting community - but I am happy to report that Dan Caron eventually abstracted the key components into a Provider model so that it can handle multiple use cases. In fact, he built both a File Cache Synchronization provider ( which relies on multiple web servers pointing at a back-end file share ) as well as Database Polling Cache Synchronization provider ( which allows for cache synchronization across independent web servers ). This architecture is incredibly flexible - and creating 2 provider implementations guarantees that the base provider API is true to the flexibility objectives ( too often we see providers with a single implementation which are doomed to fail because the base provider API has not been tested in other use cases ).

Core Module Abstraction finished the effort which was started in the 3.0 release. All of the core DesktopModules in DotNetNuke have been packaged as full Private Assembly modules ( including *.dnn file and SQL scripts ). This effectively allows them to be selectively installed/uninstalled from the application with minimal effort. For example, if you want to install DNN with only the HTML and Links modules ( and none of the other core modules ), you simply need to include the HTML.zip and Links.zip packages in the /Install/Module folder. The other benefit to this abstraction is it results in a much more minimal core - essentially just the base framework services. It also allows each module to be revised and released independent of the core application framework. We hope that this decoupling will allow the core desktopmodules to be improved and released on a much greater frequency than in the past ( ie. we have long been focussing on the body of the application and letting the limbs wither - and under the new model we are hoping these limbs can grow stronger and more powerful as time goes on ). Another thing to remember is the decoupling of the modules does not change their open source nature - they will continue to be developed and supported under the core BSD license.

In addition to the enhancements above, there were quite a number of defect corrections made in this release as the Core Team focussed on stabilizing the 3.0 product. There are too many items to note here but you can always browse our defect tracker at http://support.dotnetnuke.com to learn more.

The one major hurdle which has been holding of the release is how to deal with the MachineKeys in the web.config file.

In DNN 3.0.13 we made a change so that the MachineKey values are autogenerated during installation so that Membership information is secured for each DNN application. On a clean install, this process generates a random key value and inserts it directly into the web.config. If we did not generate a random MachineKey, and all DNN applications used the same static key, and it would be a trivial matter for a hacker to decrypt user passwords ( a major security hole ).

In 3.1.0 we evaluated the MachineKey issue again and discovered a major problem. In the past, the majority of people used XCOPY deployment to upgrade their DotNetNuke application. They would often overwrite the web.config file as part of this process and then manually open it to correct the connectionString values. This worked fine. However now that we store unique MachineKey values in the web.config, this technique would completely kill your site ( because the machinekey values would be lost - and they are not stored anywhere else ). This has already happened to a few folks with disastrous consequences.

For background clarification, we had to put the MachineKeys in the web.config because the MemberRole component from Microsoft can not access the machine.config when running under Medium Trust ( a requirement ).

So anyways, the big problem here is risk of overwrite of the web.config file. We can include upgrade instructions and take the attitude that overwriting web.config is not a supported upgrade activity - but the fact is, if it happens and a person loses their entire membership - DotNetNuke's reputation as a reliable CMS application is tarnished.

Proposed Solution:

1. We do not ship a package with a web.config file. Instead we ship a package with a release.config file ( same content as web.config ) so that it will not mistakenly overwrite the original web.config when unzipped or XCOPY deployed.

Issues:

Install - on a clean install there would be no web.config file which means the application would not run. The user would need to manually rename the file to web.config and edit the connectionString value. This could probably be automated using a script - but it would affect all users who have created DNN deployment scripts already ( ie. Hosters, Control Panel vendors ).

Upgrade - the new sections/settings in the release.config would need to be manually merged into the existing web.config. There is no simple automated method to achieve this type of XML pruning/grafting in ASP.NET 1.1. This problem exists for users right now as each new release potentially includes modifications to the web.config - and users need to upgrade their accounts manually.

At this point this seems to be the only available solution to the problem. But as you can see there are ramifications on both new installations and upgrades. We are trying to determine the best method to communicate these changes to the community in advance, to avoid a support onslaught.

Shaun,

Thank you for taking the time to update all of us on the status of DNN 3.1.  We are all sitting on pins and needles awaiting the next release.

As I was reading your post, I wondered if you had considered moving the <appSettings> section out of the web.config and putting it into its on .config file (say DotNetNuke.config), and referencing the file in the <appSettings> section.

Do instead of this:
<appSettings>
    <add key="SiteSqlServer" Value="Server=SQLSERVER;Database=DotNetNuke;uid=***;pwd=***;" />
    <add key="MachineValidationKey" value="1A0578DD60977C961BCDBD10A7D76917295C7839" />
    <add key="MachineDecryptionKey" value="DE9EF16A3E7DDC0F596FB3A305C2A551D54D1494BD2FE624" />
    <add key="MachineValidationMode" value="SHA1" />
    <add key="InstallTemplate" value="DotNetNuke.install.resources" />
    <add key="AutoUpgrade" value="true" />
    <add key="UseDnnConfig" value="true" />
    <add key="ShowMissingKeys" value="false" />
    <!-- Show missing translation keys (for development use) -->
    <add key="InstallationDate" value="6/1/2005" />
</appSettings>

you can do this in the web.config:

<appSettings file="DotNetNuke.config"></appSettings>

with the file DotNetNuke.config:

<?xml version="1.0" encoding="utf-8" ?>
<appSettings>
    <add key="SiteSqlServer" Value="Server=SQLSERVER;Database=DotNetNuke;uid=***;pwd=***;" />
    <add key="MachineValidationKey" value="1A0578DD60977C961BCDBD10A7D76917295C7839" />
    <add key="MachineDecryptionKey" value="DE9EF16A3E7DDC0F596FB3A305C2A551D54D1494BD2FE624" />
    <add key="MachineValidationMode" value="SHA1" />
    <add key="InstallTemplate" value="DotNetNuke.install.resources" />
    <add key="AutoUpgrade" value="true" />
    <add key="UseDnnConfig" value="true" />
    <add key="ShowMissingKeys" value="false" />
    <!-- Show missing translation keys (for development use) -->
    <add key="InstallationDate" value="6/1/2005" />
</appSettings>

From what I can tell, this could alleviate a number of problems, the only other section people would need to modify is the SqlDataProvider section. I may not totally understand what the actually problem is, and I am unaware if this has been discussed or not. In any cause, it is an option that could be available. It would require a few core modifications, mostly editing the new .config file instead of the web.config file.

We have and that is likely a route for the future.  From my memory (we had lots of discussion on this), after just starting the promotion of DNN with the Windows Shared Hosting promotion with MS, we did not want drastic changes for the installation scripts the hosters are using to automatically provision sites just a couple weeks after they started. They will still need minor changes with this release I think but they should be VERY minor.

I have a stupid solution, that I'm sure you've considered already, but everyone already knows I'm ignorant here, so what the heck.  Could the installation code *check* to see if there's a web.config file?  And then prompt the user of this dire event, and ask, 'If this is a new install, press yes, otherwise, ya need to talk to somebody'.  If the users answers 'yes' button, (new install), then rename the install.config or whatever, and then this issue is solved?  Of course, fuzzy's solution is probably better, but since I don't understand its consequences, I'll just add the stupid one.  The update script provided w/3.x appears to already have a way of communicating w/users the status of the install.  I don't know the repercussions of adding a clientAPI side prompt asking for a yes/no answer though...

Of course, an easier alternative is to detect the absence of the web.config, or if it sees both a web.config and an install.config, to insist that the installer (human one), either rename the install.config, explain the issue why, and/or tell 'em to read the stinkin' release notes or install.txt or somethin'...

Anyway, thanks for the note.  Shows to me you follks do a lot of thinkin' about stuff most of us have no clue *what* is going on in the background, but now know they *why*.

aaava
The app needs the web.config to exist to run. So no code can check for its existance since it cannot even be called. This will probably strengthen the call for an installer, just like the ones the Starter Kits have. Sadly, this is not a complete solution when you add third party hosting into the picture.

hooligannes97 wrote:

aaava The app needs the web.config to exist to run. So no code can check for its existance since it cannot even be called. This will probably strengthen the call for an installer, just like the ones the Starter Kits have. Sadly, this is not a complete solution when you add third party hosting into the picture.

Hmm.  I'm still stuck in the old ASP days, I guess.  Back then, I know you could write a little preloader program which would check for, say, the existence of a file, prompt the user, and then depending on the reponse, do whatever.  Even w/a bat file...which I guess is pretty much an archaic method these days.   Obviously I have much to learn.  But thanks for the heads up.  I *knew* it was stupid at least! ;-)

This might be an obvious suggestion, but is there anything from preventing the core team distributing DNN 3.1 in 2 separate distributions (one with web.config for new installations and the other with release.config for upgrades).

I'm not suggesting that every DNN release should be separated into 2 distributions, however, where possible,  if it can reduces the amount of manual intervention and possible headache it should be considered.

Moving the AppSettings into another config file was considered but it really does not solve the problem with users of 3.0.13 who mistakenly overwrite their current web.config with the new web.config ( minus the AppSettings ) and in the process lose their MachineKey values.

 

Seems simple enough to have the install/upgrade backup any existing web.config file if exists to web.config.bak...   Pretty common procedures in most application upgrades:

Do you want to make a backup of upgraded files? Y/N 

Maybe even throw up a message regarding the machine key issue...  If a file was backed up, parse and display the machine key field and database connections strings that needs to be replaced.  Seems logical to display a dialog box prompting to replace the values with these key values and connection strings.

Finally a big fat red box that warns of the dire consequences of blowing this prompt  off and the imfamous microsoft reversed cancel / approve buttons... 

Finally..  Your concerns regarding host provider installation script seems valid, however, I am not aware of any hosts that offer upgrades for existing installations..  All seem to script new installations making the upgrade issue moot.  For the hosts that want to offer upgrades, the solution may be simpler to offer a seperate upgrade script.

I believe there are a few security issues concerning the upgrade process, look for an email at security@dotnetnuke.com.

Just my 2 cents...  decisions are easy when your not accountable for support or consequences.

 

smartsoure-it,

We could do all sorts of clever things in the upgrade process if it wasn't that the problem here is with the web.config file.
It contains the key's that we want to prevent overwriting. And without a web.config the application will not run the install page.

I have been doing some tests with patching the current web.config file with this : http://apps.gotdotnet.com/xmltools/xmldiff/default.aspx

It will create a differences file which can be used to patch 3.0.13 web.config files to 3.1.0 files. This differences file can be used witch XMLPatch :

Diffgram (Xml Diff Language)

<xd:xmldiff version="1.0" srcDocHash="1316713648403303542" options="None" fragments="no" >
<xd:node match="1" >
<xd:node match="2" >
<xd:node match="1" >
<xd:node match="7" />
<xd:add>
<section name="caching" type="DotNetNuke.Framework.Providers.ProviderConfigurationHandler, DotNetNuke" />
</xd:add>
</xd:node>
</xd:node>
<xd:node match="4" >
<xd:node match="7" />
<xd:add>
<add key="InstallMemberRole" value="true" />
<!--optionally install the MemberRole scripts --></xd:add>
<xd:node match="9" />
<xd:add>
<add key="EnableWebFarmSupport" value="false" />
<!--Set to true to enable web farm suppot. See Web Farm Support documentation for additional configuration steps.-->
<add key="EnableCachePersistence" value="false" />
<!--Some cached items can persist an app recycle by serializing the object to disk. Set to "true" to enable this performance enhancement.--></xd:add>
</xd:node>
<xd:node match="7" >
<xd:node match="5" >
<xd:change match="@defaultProvider" >DBLoggingProvider</xd:change>
<xd:node match="1" >
<xd:node match="2" />
<xd:add>
<add name="DBLoggingProvider" providerPath="~\Providers\LoggingProviders\Provider.DBLoggingProvider\" type="DotNetNuke.Services.Log.EventLog.DBLoggingProvider.DBLoggingProvider, DotNetNuke.Provider.DBLoggingProvider" />
</xd:add>
</xd:node>
</xd:node>
<xd:node match="7" >
<xd:node match="1" >
<xd:node match="2" >
<xd:add type="2" name="includePageName" >true</xd:add>
</xd:node>
</xd:node>
</xd:node>
<xd:add>
<caching defaultProvider="FileBasedCachingProvider" >
<providers>
<clear/>
<add name="FileBasedCachingProvider" providerPath="~\Providers\CachingProviders\FileBasedCachingProvider\" type="DotNetNuke.Services.Cache.FileBasedCachingProvider.FBCachingProvider, DotNetNuke.Caching.FileBasedCachingProvider" />
<add name="BroadcastPollingCachingProvider" providerPath="~\Providers\CachingProviders\BroadcastPollingCachingProvider\" type="DotNetNuke.Services.Cache.BroadcastPollingCachingProvider.BPCachingProvider, DotNetNuke.Caching.BroadcastPollingCachingProvider" />
</providers>
</caching>
</xd:add>
</xd:node>
</xd:node>

</xd:xmldiff>

The "problem" with this is that you can only create a patch file between two specific web.config files. So there would have to be more than one patch file (one for every upgrade). That way you could patch the web.config from version to version.

I don't understand why this is, if it is, holding up the release.

1.  web.config exists always, we know that ..  either old or new
2.  back it up, always
3.  if the backup is not created, stop and give permissions issue


If this is the only reason..  cant you have an agree to terms (you must make a backup) ..  so we can move on with the release.  I want the 60+ bug fixes so I can move on to making some core mods that I need to make.   It has been two month since a release...  too long in my opinion. 

As mentioned, I don't know of any host that automates upgrades yet ..   plus 3.1 is not out.   Make two versions..  the standalone and one for hosts/upgrades to be released.

Just my thoughts.

Geert. 

Gotcha, I'm sure there are even more implications to this issue...   I see the issue clearer now based on the fact a web.config is required to even start the upgrade process and i like others simply ftp / xcopy over the installation to upgrade.

Scott,

Proposed Solution:
1. We do not ship a package with a web.config file. Instead we ship a package with a release.config file ( same content as web.config ) so that it will not mistakenly overwrite the original web.config when unzipped or XCOPY deployed.

"The Proposed Solution is actually the best and should be the new standard for all releases."

We normally do several tests before deploying a new version.  We always need to rename or copy the current web.config so it doesn't get overwritten.  We doubt there is anyone who has not accidently overwritten their web.config during new version testing.  Using a release.config would accomplish what most should be doing anyway.  This forces everyone to actually look in the new release.config to see what has changed.  We suspect that a lot of problems are actually caused by users NOT using the new web.config and choosing instead to use their existing version. 

Since the web.config cannot be modified, opened, downloaded, etc by the framework we do not see another solution.  Any type of installation attempt would likely fail on a hosted site since all activity is thru the browser.

If your looking for a consensus on the "Proposed Solution", then we AGREE.

thanks,
mikez