I’m delighted to announce that my new full-length book on Fiddler is now available. Weighing in at 330 pages, it covers every feature in Fiddler, and provides the most comprehensive available documentation of how to develop Fiddler extensions (including Inspectors, Transcoders, and IFiddlerExtensions) and build on the FiddlerCore class library. The book is available in paperback and DRM-free PDF formats.

You can learn more about the book at http://fiddlerbook.com. The site includes a link to the final Table-of-Contents, as well as all of the code samples in the book.

Thanks for your interest in Fiddler!

-Eric

Over on my other blog, I just posted an article showing how you can use Fiddler to find bugs in applications that don't expect authenticating proxies.

Back in October of last year, I blogged the results of some surveys and announced that some telemetry features would be added to the Fiddler beta builds. Now that 10 months have passed, I’ll provide an update on how things are going.

SmartAssembly Telemetry

Microsoft products use the Customer Experience Improvement Program (CEIP, sometimes called SQM) to collect telemetry from customer computers so we can better understand the configuration of those PCs and how the software is being used. Unfortunately, unlike Windows Error Reporting (sometimes called Watson) 3rd-party applications like Fiddler can’t participate in CEIP. Fortunately, as I announced in October, I discovered an alternative. Since then, the beta builds of Fiddler have been instrumented using RedGate’s SmartAssembly tool. SmartAssembly contains a bunch of useful features, including obfuscation, automated error reporting, and the one¹ Fiddler uses-- feature usage reporting.

I elected to instrument only the beta version to keep the amount of data collected to a reasonable size—while only a small fraction of the overall user-base, the beta audience is large enough to be statistically significant for the entire Fiddler population. There’s probably a bit of a bias because Beta users tend to be more “Advanced” than the average users, but I expect the effect is slight. I probably should change my strategy here to instead use a sampling percentage (e.g. 5% of users) but I haven’t found the need to do so thus far.

A total of 31,000 beta users have opted-in to data collection; data collected from the Version Check webservice (described shortly) suggests that this is about 20% of the users of the beta.

With just one call in Fiddler’s startup routine:

  SmartAssembly.ReportUsage.PlatformData.ReportAll();

…a set of basic information about the user’s environment is collected. The results were definitely interesting; more users than I had expected have .NET4 installed, while only a small percentage of users have multiple monitors. XP usage is still common, although Windows Vista and later account for 56 (and growing) percent of the machines running Fiddler.

Despite the fact that Fiddler itself is only available in English, a majority of Fiddler users run Fiddler on machines with a non-English locale (blue and purple are the US and Great Britain; all other colors are non-English):

This data should prove useful if I ever decide to translate Fiddler into other languages.

While the environment information is useful, the most interesting information comes from feature usage reporting. Basic information, including the count of sessions-per-user-per-day and the Fiddler version, is captured automatically. Adding application-specific metrics is simple. SmartAssembly offers two easy methods to instrument your code. First, you can add an attribute to any method and when that method is called, its counter will be incremented:

        #if TELEMETRY
          [SmartAssembly.ReportUsage.ReportUsage]
        #endif
        /// <summary>
        /// Show Fiddler's Options dialog
        /// </summary>
        [CodeDescription("Show the Fiddler Options dialog")]
        public void actShowOptions()

In some cases, you’ll want to capture events at a more granular level; in that case, just call the ReportUsage method with a string specifying the feature name:

        #if TELEMETRY
          SmartAssembly.ReportUsage.UsageCounter.ReportUsage("FiddlerBoot");
          if (CONFIG.bIsBeta) {
            SmartAssembly.ReportUsage.UsageCounter.ReportUsage("IsBeta");
          }   
        #endif

Feature usage reporting allows me to quickly see which features in Fiddler are most popular, as well as which features need to have their discoverability improved. For instance, I know which of Fiddler’s tabs are used in a session (AutoResponder tab is activated in 7 times as many sessions as the Composer), and which features are getting overlooked (Fiddler’s insanely powerful QuickExec feature gets used in only 1% of Sessions).

By pairing the SmartAssembly telemetry data with surveys and community feedback, a tiny team (i.e. me) can prioritize improvements to maximize the benefits for the tool's huge user-base.

Version Check Analysis

Of course, while rich telemetry is the most valuable, I’m also interested in getting some basic statistics from the entire userbase. By default, when Fiddler starts, it pings a web service to see whether a new version is available. As a part of this request, the current Windows version and Fiddler version are provided so that the web service can check for a compatible upgrade. I picked a random day, August 1st, and analyzed the data. It appears that only about 66% of users are using the latest version of Fiddler, released on June 23rd. Nearly a quarter of Fiddler users are using builds from the last two years, and about 4% of users are using truly ancient builds. 6% of the calls didn’t provide meaningful version information, either because the calls were fiddled (something that Fiddler is quite good at :-) or the builds were so old that the version information wasn’t present.

It’s a little sad to see that so many users aren’t getting the maximum benefit out of the many hundreds of hours of investment made in Fiddler over the past two years. (Not to mention the hundreds of bugfixes!) If you’re running an older version of Fiddler, please upgrade today.

Book Survey

In last fall's survey, just over a thousand Fiddler users indicated that they would buy a printed Fiddler book, and ~600 users indicated that they would purchase an E-Book on the topic. About 900 users said they’d read the EBook if it were free.

The Fiddler book was completed in June and is now available in paperback and DRM-less PDF format ($19 in paperback and $10 in PDF). The book turned out about 33% longer than I expected (it’s 330 pages) and has received good reviews. Sales have been brisk, and while I haven’t yet surpassed the goals suggested by the survey, I’m pretty happy with the results from the first two months. Additionally, two international publishers have tendered proposals for translated versions (Korean and Chinese) and a third (Japanese) is under consideration but not yet committed. Unfortunately, my plan to make the book available for free borrowing via the Kindle Lending Library hit two snags: first, the book doesn't work well on the non-Fire Kindles (because images and source code don't reformat well in .epub/.mobi), and second, Amazon's terms for books in the lending library prohibit making the book available from any other source in electronic form. That would preclude me offering the book to everyone who prefers the PDF format (which looks great on the iPad and PCs).

-Eric

¹ Fiddler doesn’t use the obfuscation feature because most of the classes are chock full of public methods, properties, and fields which must be callable from FiddlerScript and Extensions. Similarly, Fiddler already included a (manual) exception-reporting feature, so I elected not to switch over to the SmartAssembly version.

From the mail bag...

Q:When I use IE10 on Windows 8 to visit an IIS Express instance using the following URL http://ipv4.fiddler:2468/, I get an unexpected error message.

HTTP/1.1 400 Bad Request
Content-Type: text/html;charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Wed, 22 Aug 2012 19:59:52 GMT
Connection: close
Content-Length: 334

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN""http://www.w3.org/TR/html4/strict.dtd">
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Bad Request - Invalid Hostname</h2>
<hr><p>HTTP Error 400. The request hostname is invalid.</p></BODY></HTML>

What's up with that?

A: First, it's important to understand that you don’t need to use any special “.fiddler” addresses for Fiddler to capture local traffic when using IE (since version 9). You only need to use those when your client is making its HTTP requests using System.Net (e.g. HTTPWebRequest), which unconditionally bypasses the proxy for addresses it deems "local."

Having said that, you'll find that the same request works if you use the address http://localhost.fiddler:2468/. The reason that localhost.fiddler works is that when Fiddler gets this URL, it sets the request's Host header to “localhost” before passing it to the IIS Express server. If you instead use ipv6.fiddler as the request URL's hostname, Fiddler sets the Host header to “[::1]”. If you use ipv4.fiddler, Fiddler sets the Host header to “127.0.0.1”.

IIS Express only expects the “localhost” Host header value and returns the HTTP/400 for the “127.0.0.1” Host, saying, in effect “I don’t have a website with that hostname.”

-Eric

Read more here http://fiddler2.com/r/?telerik.

I’m a long-time ThinkPad fan and owner—the keyboards are sublime, and I love the TrackPoint interface. What I don’t love is the ThinkPad driver software, which doesn’t send the standard WM_MOUSEWHEEL messages to the system. That bug means that many .NET controls don’t scroll properly. Over the years, I’ve fixed many of the controls in Fiddler to workaround the problem, but not all of them. For instance, ThinkPad users will notice that when they try to middle-click-scroll the HexView controls, they get the scroll-cursor, but no scrolling as they move the cursor:

Over on his blog, Mark Rideout explains the root cause of Lenovo’s scrolling problem and explains how .NET developers can workaround the bug by adding hidden scrollbar controls to trick the driver software. I’ll be looking at using this trick in a future version of Fiddler. For now, a cumbersome but effective configuration tweak to the Lenovo driver settings allows end-users to get scrolling working properly in Fiddler and most other applications.

Start Notepad using the Run As Administrator option. Open the file C:\program files\lenovo\trackpoint\tp4table.dat (note that the tp4table.dat file may be in another path on your computer). Add the lines in blue under the existing lines in grey.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Pass 1 rules (These rules run last)
;
; If no scrolling method was found with the builtin logic, then
; these rules are checked.
;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
; Fiddler
*,*,Fiddler.exe,*,*,*,WheelStd,1,9

Restart your system to force the driver software to reload its settings, and you should find that middle-click-scrolling works properly in Fiddler.

Until next time…

-Eric

The new Fiddler Beta has a few useful tricks. First, you can copy a DataURI to your clipboard (from Fiddler itself, or anywhere else):

and use the new Edit > Paste Files as Sessions command:

… to instruct Fiddler to parse the DataURI and create a new Session for it. You can then use Fiddler’s inspectors to examine the resulting object:

Next, the WebView Inspector has been enhanced to render previews of WOFF font, MP3 audio, and H264 video responses, if you have IE9+ installed:

Lastly, Fiddler’s AutoResponder now permits you to use Regular Expression Group Replacements to map one URL to another. For instance, this rule:

…will convert http://somesite.com/assets/Test1.gif to http://www.example.com/mockup/Test1.gif. This capability makes it even easier to replace content using Fiddler’s AutoResponder.

This new build of Fiddler has a ton of other fixes and performance improvements. I hope you enjoy it!

-Eric

With the recent kerfuffle about sites providing misleading P3P statements, I decided to throw together a quick Fiddler add-on that displays privacy information simply in the Fiddler UI.

Install the new Privacy Scanner add-on and Fiddler will gain a new top-level menu named Privacy.

The menu has two options. The Enabled option controls whether the add-on does anything at all; it’s a good practice for Add-ons to offer a simple way to turn them off when they’re not wanted.

When the add-on is enabled, it will add a Privacy Info column to the session list and will flag HTTP/HTTPS responses which set cookies. Evaluation of any P3P statements that come along with those cookies will change the session’s background color:

Sessions that send a satisfactory P3P policy are shown in green. Sessions that set a cookie without a P3P policy are yellow; in the default IE settings, these cookies will not be sent to a 3rd party context. Sessions that send a P3P policy that does not permit use of the cookie in a 3rd party context are rendered in orange. Sessions that send invalid P3P header tokens will have a red background.

When the menu option Rename P3P header if invalid is checked, if a session presents a P3P statement that is malformed, that P3P header will be renamed to Malformed-P3P to prevent the browser from interpreting it as the P3P 1.0 specification suggested (e.g. ignoring the unknown tokens). You can see the difference when you load this test page; the cookies with bogus compact policies will be dropped in a 3rd party context.

For folks looking to build their own Fiddler extension, the code is provided below.

-Eric

using System;

using System.Collections;

using System.Globalization;

using System.Collections.Generic;

using System.Windows.Forms;

using System.Text;

using Fiddler;

using System.IO;

using System.Diagnostics;

using Microsoft.Win32;

using System.Reflection;

using System.Text.RegularExpressions;

[assembly: Fiddler.RequiredVersion("2.3.9.0")]

[assembly: AssemblyVersion("1.0.1.0")]

[assembly: AssemblyTitle("PrivacyScanner")]

[assembly: AssemblyDescription("Scans for Cookies and P3P")]

[assembly: AssemblyCompany("Eric Lawrence")]

[assembly: AssemblyProduct("PrivacyScanner")]

public class TagCookies : IAutoTamper2

{

    private bool bEnabled = false;

    private bool bEnforceP3PValidity = false;

    private bool bCreatedColumn = false;

    private System.Windows.Forms.MenuItem miEnabled;

    private System.Windows.Forms.MenuItem miEnforceP3PValidity;

    private System.Windows.Forms.MenuItem mnuCookieTag;

    public void OnLoad()

    {

        /*

 * NB: You might not get called here until ~after~ one of the AutoTamper methods was called.

 * This is okay for us, because we created our mnuContentBlock in the constructor and its simply not

 * visible anywhere until this method is called and we merge it onto the Fiddler Main menu.

 */

        FiddlerApplication.UI.mnuMain.MenuItems.Add(mnuCookieTag);

    }

    public void OnBeforeUnload() {  /*noop*/   }

    private void InitializeMenu()

    {

        this.miEnabled = new System.Windows.Forms.MenuItem("&Enabled");

        this.miEnforceP3PValidity = new System.Windows.Forms.MenuItem("&Rename P3P header if invalid");

        this.miEnabled.Index = 0;

        this.miEnforceP3PValidity.Index = 1;

        this.mnuCookieTag = new System.Windows.Forms.MenuItem();

        this.mnuCookieTag.MenuItems.AddRange(new System.Windows.Forms.MenuItem[] { this.miEnabled, this.miEnforceP3PValidity });

        this.mnuCookieTag.Text = "Privacy";

        this.miEnabled.Click += new System.EventHandler(this.miEnabled_Click);

        this.miEnabled.Checked = bEnabled;

        this.miEnforceP3PValidity.Click += new System.EventHandler(this.miEnforceP3PValidity_Click);

        this.miEnforceP3PValidity.Checked = bEnforceP3PValidity;

    }

    public void miEnabled_Click(object sender, EventArgs e)

    {

        miEnabled.Checked = !miEnabled.Checked;

        bEnabled = miEnabled.Checked;

        this.miEnforceP3PValidity.Enabled = bEnabled;

        if (bEnabled) { EnsureColumn(); }

        FiddlerApplication.Prefs.SetBoolPref("extensions.tagcookies.enabled", bEnabled);

    }

    public void miEnforceP3PValidity_Click(object sender, EventArgs e)

    {

        miEnforceP3PValidity.Checked = !miEnforceP3PValidity.Checked;

        bEnforceP3PValidity = miEnforceP3PValidity.Checked;

        FiddlerApplication.Prefs.SetBoolPref("extensions.tagcookies.EnforceP3PValidity", bEnforceP3PValidity);

    }

    private void EnsureColumn()

    {

        if (bCreatedColumn) return;

        FiddlerApplication.UI.lvSessions.AddBoundColumn("Privacy Info", 1, 120, "X-Privacy");

        bCreatedColumn = true;

    }

    public TagCookies()

    {

        this.bEnabled = FiddlerApplication.Prefs.GetBoolPref("extensions.tagcookies.enabled", false);

        this.bEnforceP3PValidity = FiddlerApplication.Prefs.GetBoolPref("extensions.tagcookies.EnforceP3PValidity", true);

        InitializeMenu();

        if (bEnabled) { EnsureColumn(); } else { this.miEnforceP3PValidity.Enabled = false; }

    }

    private void SetP3PStateFromHeader(string sValue, ref P3PState oP3PState)

    {

        if (string.IsNullOrEmpty(sValue))

        {

            return;

        }

        string sUnsatCat = String.Empty;

        string sUnsatPurpose = String.Empty;

        sValue = sValue.Replace('\'', '"');

        string sCP = null;

        Regex r = new Regex("CP\\s?=\\s?[\"]?(?<TokenValue>[^\";]*)");

        Match m = r.Match(sValue);

        if (m.Success && (null != m.Groups["TokenValue"]))

        {

            sCP = m.Groups["TokenValue"].Value;

        }

        if (String.IsNullOrEmpty(sCP))

        {

            return;

        }

        // Okay, we've got a compact policy token.

        oP3PState = P3PState.P3POk;

        string[] sTokens = sCP.Split(new char[] { ' ' }, StringSplitOptions.RemoveEmptyEntries);

        foreach (string sToken in sTokens)

        {

            // Reject clearly invalid tokens...

            if ((sToken.Length < 3) || (sToken.Length > 4))

            {

                oP3PState = P3PState.P3PMalformed;

                return;

            }

            if (",PHY,ONL,GOV,FIN,".IndexOf("," + sToken + ",", StringComparison.OrdinalIgnoreCase) > -1)

            {

                sUnsatCat += (sToken + " ");

                continue;

            }

            if (",SAM,OTR,UNR,PUB,IVA,IVD,CON,TEL,OTP,".IndexOf("," + sToken + ",", StringComparison.OrdinalIgnoreCase) > -1)

            {

                sUnsatPurpose += (sToken + " ");

                continue;

            }

            // TODO: Look up the token in the complete collection and check validity

        }

        // If a cookie contains an unsatisfactory purpose and an unsatisfactory category, mark it

        // http://msdn.microsoft.com/en-us/library/ie/ms537343(v=vs.85).aspx#unsatisfactory_cookies

        if ((sUnsatCat.Length > 0) && (sUnsatPurpose.Length > 0))

        {

            if (oP3PState == P3PState.P3POk)

            {

                oP3PState = P3PState.P3PUnsatisfactory;

            }

        }

    }

    private enum P3PState

    {

        NoCookies,

        NoP3PAndSetsCookies,

        P3POk,

        P3PUnsatisfactory,

        P3PMalformed

    }

    public void OnPeekAtResponseHeaders(Session oSession) 

    {

        if (!bEnabled) return;

        P3PState oP3PState = P3PState.NoCookies;

        if (!oSession.oResponse.headers.Exists("Set-Cookie"))

        {

            return;

        }

        oP3PState = P3PState.NoP3PAndSetsCookies;

        if (oSession.oResponse.headers.Exists("P3P"))

        {

            SetP3PStateFromHeader(oSession.oResponse.headers["P3P"], ref oP3PState);

        }

        switch (oP3PState)

        {

            case P3PState.P3POk:

                oSession["ui-backcolor"] = "#ACDC85";

                oSession["X-Privacy"] = "Sets cookies & P3P";

                break;

            case P3PState.NoP3PAndSetsCookies:

                oSession["ui-backcolor"] = "#FAFDA4";

                oSession["X-Privacy"] = "Sets cookies without P3P";

                break;

            case P3PState.P3PUnsatisfactory:

                oSession["ui-backcolor"] = "#EC921A";

                oSession["X-Privacy"] = "Sets cookies; P3P unsatisfactory for 3rd-party use";

                break;

            case P3PState.P3PMalformed:

                oSession["ui-backcolor"] = "#E90A05";

                if (bEnforceP3PValidity)

                {

                    oSession.oResponse.headers["MALFORMED-P3P"] = oSession.oResponse.headers["P3P"];

                    oSession["X-Privacy"] = "MALFORMED P3P: " + oSession.oResponse.headers["P3P"];

                    oSession.oResponse.headers.Remove("P3P");

                }

                break;

        }

    }

    public void AutoTamperRequestBefore(Session oSession) { }

    public void AutoTamperRequestAfter(Session oSession){ /*noop*/ }

    public void AutoTamperResponseAfter(Session oSession) {/*noop*/}

    public void AutoTamperResponseBefore(Session oSession) { /*noop*/ }

    public void OnBeforeReturningError(Session oSession) {/*noop*/}

}

Recently, a handful of folks have emailed me complaining that some HTTPS sites cannot be reached by their .NET programs or from any program when Fiddler is configured to decrypt traffic. Notably, these users only have problems when running on Windows Vista or later.

So, what’s going on?

When I need to troubleshoot issues that occur below Fiddler, NetMon is a good choice. Taking a look at the HTTPS handshake, the problem is immediately apparent:

This server is returning a TLS Alert at the beginning of the handshake. The alert severity is 1 meaning WARNING, and the code is 112 (0x70) which is the unrecognized_name alert. With this alert, the server is indicating that it does not recognize the hostname provided by the server’s Server Name Indication TLS extension. This warning is only encountered when the client is running on Windows Vista and later because the SChannel HTTPS stack on Windows XP doesn’t support SNI and other TLS extensions.

The server’s behavior is explicitly not recommended according to the standards:

If the server understood the ClientHello extension but does not recognize the server name, the server SHOULD take one of two actions: either abort the handshake by sending a fatal-level unrecognized_name(112) alert or continue the handshake. It is NOT RECOMMENDED to send a warning-level unrecognized_name(112) alert, because the client’s behavior in response to warning-level alerts is unpredictable. If there is a mismatch between the server name used by the client application and the server name of the credential chosen by the server, this mismatch will become apparent when the client application performs the server endpoint identification, at which point the client application will have to decide whether to proceed with the communication.

The server’s behavior is problematic because the .NET Framework’s HTTPS implementation is unable to ignore this warning and it is not able to automatically “fail over” to an earlier protocol version, leading to a  lengthy delay (~80 seconds) before a connection failure in Fiddler or other .NET applications.

To enable Fiddler to workaround sites with this problematic configuration, click Rules > Customize Rules. Scroll to the OnBeforeRequest method and add the following block:

if (oSession.HTTPMethodIs(“CONNECT”) && oSession.HostnameIs(“BuggySite.com”))
{
  oSession[“x-OverrideSslProtocols”] = “ssl3”;
  FiddlerApplication.Log.LogString(“Legacy compat applied for inbound request to BuggySite.com”);
}

Save the script and this block will force Fiddler to handshake with the server using only the SSLv3 protocol, which does not send TLS extensions, avoiding the problem.

To apply the same workaround in your .NET application without Fiddler, use the same trick: ServicePointManager.SecurityProtocol = SecurityProtocolType.Ssl3;

-Eric

I’m delighted to announce that my new full-length book on Fiddler is now available. Weighing in at 330 pages, it covers every feature in Fiddler, and provides the most comprehensive available documentation of how to develop Fiddler extensions (including Inspectors, Transcoders, and IFiddlerExtensions) and build on the FiddlerCore class library. The book is available in paperback and DRM-free PDF formats.

You can learn more about the book at http://fiddlerbook.com. The site includes a link to the final Table-of-Contents, as well as all of the code samples in the book.

Thanks for your interest in Fiddler!

-Eric

Over on my other blog, I just posted an article showing how you can use Fiddler to find bugs in applications that don’t expect authenticating proxies.

Back in October of last year, I blogged the results of some surveys and announced that some telemetry features would be added to the Fiddler beta builds. Now that 10 months have passed, I’ll provide an update on how things are going.

SmartAssembly Telemetry

Microsoft products use the Customer Experience Improvement Program (CEIP, sometimes called SQM) to collect telemetry from customer computers so we can better understand the configuration of those PCs and how the software is being used. Unfortunately, unlike Windows Error Reporting (sometimes called Watson) 3rd-party applications like Fiddler can’t participate in CEIP. Fortunately, as I announced in October, I discovered an alternative. Since then, the beta builds of Fiddler have been instrumented using RedGate’s SmartAssembly tool. SmartAssembly contains a bunch of useful features, including obfuscation, automated error reporting, and the one¹ Fiddler uses– feature usage reporting.

I elected to instrument only the beta version to keep the amount of data collected to a reasonable size—while only a small fraction of the overall user-base, the beta audience is large enough to be statistically significant for the entire Fiddler population. There’s probably a bit of a bias because Beta users tend to be more “Advanced” than the average users, but I expect the effect is slight. I probably should change my strategy here to instead use a sampling percentage (e.g. 5% of users) but I haven’t found the need to do so thus far.

A total of 31,000 beta users have opted-in to data collection; data collected from the Version Check webservice (described shortly) suggests that this is about 20% of the users of the beta.

With just one call in Fiddler’s startup routine:

  SmartAssembly.ReportUsage.PlatformData.ReportAll();

…a set of basic information about the user’s environment is collected. The results were definitely interesting; more users than I had expected have .NET4 installed, while only a small percentage of users have multiple monitors. XP usage is still common, although Windows Vista and later account for 56 (and growing) percent of the machines running Fiddler.

Despite the fact that Fiddler itself is only available in English, a majority of Fiddler users run Fiddler on machines with a non-English locale (blue and purple are the US and Great Britain; all other colors are non-English):

This data should prove useful if I ever decide to translate Fiddler into other languages.

While the environment information is useful, the most interesting information comes from feature usage reporting. Basic information, including the count of sessions-per-user-per-day and the Fiddler version, is captured automatically. Adding application-specific metrics is simple. SmartAssembly offers two easy methods to instrument your code. First, you can add an attribute to any method and when that method is called, its counter will be incremented:

        #if TELEMETRY
          [SmartAssembly.ReportUsage.ReportUsage]
        #endif
        /// <summary>
        /// Show Fiddler’s Options dialog
        /// </summary>
        [CodeDescription(“Show the Fiddler Options dialog”)]
        public void actShowOptions()

In some cases, you’ll want to capture events at a more granular level; in that case, just call the ReportUsage method with a string specifying the feature name:

        #if TELEMETRY
          SmartAssembly.ReportUsage.UsageCounter.ReportUsage(“FiddlerBoot”);
          if (CONFIG.bIsBeta) {
            SmartAssembly.ReportUsage.UsageCounter.ReportUsage(“IsBeta”);
          }   
        #endif

Feature usage reporting allows me to quickly see which features in Fiddler are most popular, as well as which features need to have their discoverability improved. For instance, I know which of Fiddler’s tabs are used in a session (AutoResponder tab is activated in 7 times as many sessions as the Composer), and which features are getting overlooked (Fiddler’s insanely powerful QuickExec feature gets used in only 1% of Sessions).

By pairing the SmartAssembly telemetry data with surveys and community feedback, a tiny team (i.e. me) can prioritize improvements to maximize the benefits for the tool’s huge user-base.

Version Check Analysis

Of course, while rich telemetry is the most valuable, I’m also interested in getting some basic statistics from the entire userbase. By default, when Fiddler starts, it pings a web service to see whether a new version is available. As a part of this request, the current Windows version and Fiddler version are provided so that the web service can check for a compatible upgrade. I picked a random day, August 1st, and analyzed the data. It appears that only about 66% of users are using the latest version of Fiddler, released on June 23rd. Nearly a quarter of Fiddler users are using builds from the last two years, and about 4% of users are using truly ancient builds. 6% of the calls didn’t provide meaningful version information, either because the calls were fiddled (something that Fiddler is quite good at or the builds were so old that the version information wasn’t present.

It’s a little sad to see that so many users aren’t getting the maximum benefit out of the many hundreds of hours of investment made in Fiddler over the past two years. (Not to mention the hundreds of bugfixes!) If you’re running an older version of Fiddler, please upgrade today.

Book Survey

In last fall’s survey, just over a thousand Fiddler users indicated that they would buy a printed Fiddler book, and ~600 users indicated that they would purchase an E-Book on the topic. About 900 users said they’d read the EBook if it were free.

The Fiddler book was completed in June and is now available in paperback and DRM-less PDF format ($19 in paperback and $10 in PDF). The book turned out about 33% longer than I expected (it’s 330 pages) and has received good reviews. Sales have been brisk, and while I haven’t yet surpassed the goals suggested by the survey, I’m pretty happy with the results from the first two months. Additionally, two international publishers have tendered proposals for translated versions (Korean and Chinese) and a third (Japanese) is under consideration but not yet committed. Unfortunately, my plan to make the book available for free borrowing via the Kindle Lending Library hit two snags: first, the book doesn’t work well on the non-Fire Kindles (because images and source code don’t reformat well in .epub/.mobi), and second, Amazon’s terms for books in the lending library prohibit making the book available from any other source in electronic form. That would preclude me offering the book to everyone who prefers the PDF format (which looks great on the iPad and PCs).

-Eric

¹ Fiddler doesn’t use the obfuscation feature because most of the classes are chock full of public methods, properties, and fields which must be callable from FiddlerScript and Extensions. Similarly, Fiddler already included a (manual) exception-reporting feature, so I elected not to switch over to the SmartAssembly version.

From the mail bag…

Q:When I use IE10 on Windows 8 to visit an IIS Express instance using the following URL http://ipv4.fiddler:2468/, I get an unexpected error message.

HTTP/1.1 400 Bad Request
Content-Type: text/html;charset=us-ascii
Server: Microsoft-HTTPAPI/2.0
Date: Wed, 22 Aug 2012 19:59:52 GMT
Connection: close
Content-Length: 334

<!DOCTYPE HTML PUBLIC “-//W3C//DTD HTML 4.01//EN””http://www.w3.org/TR/html4/strict.dtd”>
<HTML><HEAD><TITLE>Bad Request</TITLE>
<META HTTP-EQUIV=”Content-Type” Content=”text/html; charset=us-ascii”></HEAD>
<BODY><h2>Bad Request – Invalid Hostname</h2>
<hr><p>HTTP Error 400. The request hostname is invalid.</p></BODY></HTML>

What’s up with that?

A: First, it’s important to understand that you don’t need to use any special “.fiddler” addresses for Fiddler to capture local traffic when using IE (since version 9). You only need to use those when your client is making its HTTP requests using System.Net (e.g. HTTPWebRequest), which unconditionally bypasses the proxy for addresses it deems “local.”

Having said that, you’ll find that the same request works if you use the address http://localhost.fiddler:2468/. The reason that localhost.fiddler works is that when Fiddler gets this URL, it sets the request’s Host header to “localhost” before passing it to the IIS Express server. If you instead use ipv6.fiddler as the request URL’s hostname, Fiddler sets the Host header to “[::1]”. If you use ipv4.fiddler, Fiddler sets the Host header to “127.0.0.1”.

IIS Express only expects the “localhost” Host header value and returns the HTTP/400 for the “127.0.0.1” Host, saying, in effect “I don’t have a website with that hostname.”

-Eric

Read more here http://ericlawrence.com/fiddler/letterfromeric.asp.

Find the new Fiddler blog here: http://blogs.telerik.com/fiddler/posts