Dynamics 365 on-premises / Chrome 72 download issue and work around

Chrome 72 has changed the way it handles the Content-Disposition HTTP response header (and possibly others).  This change has affected how Microsoft Dynamics 365 delivers downloads.

When you download a file from Dynamics, it delivers a file with an HTTP response like so, for a file named "My File.pdf"

HTTP/1.1 200 OK
Cache-Control: private
Transfer-Encoding: chunked
Content-Type: application/octet-stream
Server: Microsoft-IIS/8.5
REQ_ID: a9dabf5d-ee18-40ad-aa76-a5b025d73e24
X-UA-Compatible: IE=8;IE=9;IE=10;IE=11
Content-Disposition: attachment; filename='My%2b%20File.pdf'
ContentLength: 956641
X-AspNet-Version: 4.0.30319
Persistent-Auth: true
X-Powered-By: ASP.NET
Date: Fri, 01 Feb 2019 22:02:33 GMT


Per spec, quoted-strings should be quoted using double quotes.  It looks like the issue is Chrome gravitated more towards spec and decided to stop supporting single quotes in quoted-strings.

Chrome 72 is now treating the single quotes as part of the actual file name.

If you have a comma in your filename, you'll receive an abort: ERR_RESPONSE_HEADERS_MULTIPLE_CONTENT_DISPOSITION
Chrome thinks the server is trying to deliver multiple files which it doesn't allow in this situation.

Otherwise, Chrome will just save the file with the single quotes.  In this case, the file is perfectly valid and just needs to be renamed, removing the leading and trailing single quotes.  This might prove challenging for users where the operating system will be hiding the file extensions:

That is the safe work-around until Microsoft or Chrome releases a fix.

Reading any further may cause Microsoft to not support your installation.

This work-around is only valid for Dynamics 365 on-premises.  It has been tested with 8.2.2 and 8.2.3.  It has NOT been tested with 9.x.  Online users will have to wait for Microsoft or Chrome to release a patch.

For Microsoft Dynamics 365 on-premises, a work around (to be deployed by a System Administrator) is to modify the CRM website's web.config. WARNING: This is very much unsupported and may cause issues with subsequent upgrades.  It may cause other issues.  This work around has been tested for downloads out of CRM in Chrome 71, Chrome 72, Edge, IE11, and Firefox.

1. Save a backup the web.config
2. Edit the web.config file
3. Browse to XML Path:

• 8.2.2:  the /configuration/system.webServer/rewrite node
• 8.2.3:  the /configuration/location/system.webServer/rewrite node

4. AFTER (not in) the ... element paste:

<outboundRules>
  <rule name="Content-Disposition filename" preCondition="IsSingleQuoteFileName">
    <match serverVariable="RESPONSE_Content_Disposition" 

      pattern="(.* filename=)('([^'].+)')(.*)" />
    <action type="Rewrite" value="{R:1}&quot;{R:3}&quot;{R:4}" />
  </rule>
  <preConditions>
    <preCondition name="IsSingleQuoteFileName">
      <add input="{RESPONSE_CONTENT_DISPOSITION}" pattern=".* filename='[^'].+'" />
    </preCondition>
  </preConditions>
</outboundRules>

This uses the IIS Rewrite Module to replace single quotes wrapping a filename, in the HTTP response Content-Disposition header, with double quotes.

5. Save the file. Warning: your Microsoft Dynamics 365 website will recycle.  You should apply this while users are not in the system.

If there were any issues (added , your Dynamics website may fail to load.  Restore to the backed up web.config.

I'm started an issue with the Chromium team here:  https://bugs.chromium.org/p/chromium/issues/detail?id=927913

Update 2/5/2019: Operating per spec; WontFix.

I'll be starting an issue with the Dynamics team on 2/4/2019, when partner support opens.

Update 2/5/2019:  The first line of support has reproduced and escalated.  According to the support analyst, the issue does NOT occur in their online environment.

Update 2/6/2019: 

1. Chrome team monitoring the size of the change's affect.  My case has been merged into this case: https://bugs.chromium.org/p/chromium/issues/detail?id=927366

2. I've been informed by Microsoft that this will be patched for online in "one of the next weekly releases."  Also versions 8.2 and 9.0 on-premises (didn't mention whether that is 8.2.2 or 8.2.3; no timeline yet).

Update 2/19/2019: Microsoft plans to release to on-premises, in 8.2.3.249, on March 12th, 2019.  (I assume also a 9.x version for on-premises but don't have confirmation on that.

Delete/Import performance resolved in Dynamics 365! Thanks Dynamics Development Team!

Thank you to Microsoft for quickly resolving the delete and import performance, as it relates to the cascading user defined functions, in Dynamics 365.  Our internal testing shows both perform MUCH better in Dynamics 365 on-premise (which means you can expect it Online also).

Thank you fellow Dynamics users who have helped reach out to Microsoft and push this important fix through.  This is one of the fastest turnarounds I have seen from this dev team!

Dynamics Connect has Moved

The Dynamics Connect portal has moved to the CRM Ideas Portal.  Most of my existing ideas were carried forward.  I had to re-create the delete performance idea I added last month.  It must have been added after the conversion.  You can now vote for that item in the CRM Ideas Portal!.

Dynamics CRM Slow Deletes and Imports

Update February 16th, 2017: Initial testing (and a look under the hood) show that Dynamics 365 On-Premise (8.2.0.749) has received this performance enhancement!  Go try it out now!

Original Post follows.

I've posted a new item on the CRM Ideas Portal to try to get Microsoft to look at the terrible delete and import times which occasionally occur in Microsoft Dynamics CRM.  If you commonly experience slow deletes, slow imports, or see messages like this in your event viewer (which means your users are seeing this):

Query execution time of 3183.0 seconds exceeded the threshold of 10 seconds. Thread: 22; Database: Hoven_2016051_MSCRM; Server:v2016sql; Query: select * from dbo.fn_CollectForCascadeWrapper(System.Collections.Generic.List`1[Microsoft.SqlServer.Server.SqlDataRecord], 7102, 0, 1, 0) order by i.

then I encourage you to up-vote this item on the portal.  (Yes, that is 53 minutes to import what contained some security roles!)  In my research I've found the compile time of the fn_CollectForCascadeWrapper is horrendous (in particular fn_CollectForCascadeDelete, fn_CollectForCascadeDeleteSchema, and fn_CollectForCascadeRemoveLink).  These methods are generated based on your metadata.  The more entities and relationships you add to CRM the worse these methods get.  The more organizations you have on the SQL server, the more likely you are to have the issue to occur.

In it I suggest an alternative that proved to use 28x less CPU, execute 4x faster, and use nearly 10x less memory in the SQL plan cache (on one sample data set).

This was testing against CRM 2016 with SP1 but goes back to at least CRM 2011.

Update October 03, 2016:  A great escalation engineer at Microsoft took on my case, quickly reproduced, and opened a design change request with the development team.  The Work Item is 346488.  He tells me other customers are asking to implement my proposal also.  THANK YOU dear reader for pushing for this change with me.  Please continue to do so!  Currently it may be 2 releases (about a year) before we know if this will be implemented.