Oct 12 2009

Getting the source window of a request

Category: Firefox, JavaScriptJonathan Fingland @ 3:35 pm

This post draws heavily on a question and answer on “An observer for URL changes (Firefox extension)” from StackOverflow.

In this post, however, I’m going to focus only on getting the window from which an http request originated.

  • What you need to know

    Yesterday’s post about XHR Listening by a Firefox Addon gives a good basis to work from so I will assume you’ve read over that and understood it (you if haven’t read it, do so now).

  • Organization and code

    In the case on StackOverflow, the logic was all in the observer, not the TracingListener. It also makes a good example of how to get selected data.

    
    
    var myObserver = {
       observe: function(aSubject, aTopic, aData){
              if (aTopic == 'http-on-examine-response')
              {
                    var oHttp = aSubject.QueryInterface(Ci.nsIHttpChannel);
                    var interfaceRequestor =   oHttp.notificationCallbacks
                                                   .QueryInterface(Ci.nsIInterfaceRequestor);
                    aSubject.DOMWindow = interfaceRequestor.getInterface(Ci.nsIDOMWindow);
              }
         }
       },
    
       QueryInterface: function(iid){
          if (!iid.equals(Ci.nsISupports) &&
              !iid.equals(Ci.nsIObserver))
            throw Components.results.NS_ERROR_NO_INTERFACE;
    
          return this;
       }
    }
    
    • How does it work?

      The three lines after our topic check very simply QueryInterface into nsIHttpChannel (like we had to with getting the URI and requestMethod), then the tricky bit is the next two steps: getting the channel’s notificationCallbacks and QueryInterface-ing into an nsIInterfaceRequestor, and then calling getInterface to get an nsIDOMWindow. nsIInterfaceRequestor provides a single method, getInterface, which is very similar to QueryInterface, but not the same (See the nsIInterfaceRequestor docs for more info).

  • Finishing up

    All that’s left is to register the observer

    
    var observerService = Cc["@mozilla.org/observer-service;1"]
        .getService(Ci.nsIObserverService);
    
    observerService.addObserver(myObserver,
        "http-on-examine-response", false);
    


Oct 11 2009

Howto: XHR Listening by a Firefox Addon

Category: Firefox, JavaScript, Pirate QuestingJonathan Fingland @ 4:00 am

The following post draws significantly from a post by Jan Odvarko at http://www.softwareishard.com/blog/firebug/nsitraceablechannel-intercept-http-traffic/ but goes a bit further. There are also some sections which were inspired by Firebug, but are heavily modified.

  • What you need to know

    Before I get into the code, understand that one of the most important things in this process to understand is that your extension’s listener is just one in a chain. It is the responsibility of every listener in the chain to pass on the information. Failure to do this has some amusing consequences…. like nothing loading in the browser.

    Just to make it really clear — Don’t drop the ball. (Edit: And while you can edit the data in the stream — don’t do it unless you have a really good reason.)

  • Convenience methods and aliases

    A lot of the Firefox internals are accessed using Components.classes and Components.interfaces. While the verbosity makes it clear, it can at times be overly repetitive and, honestly, can take a long time to write out. A fairly common shorthand is in use, Cc and Ci with a few other less common shorthands like CCIN (for creating instances of a class based on a class name and an interface name) and CCSV (similarly creating a service based on a class name and interface name)

    
    if (typeof Cc == "undefined") {
    	var Cc = Components.classes;
    }
    if (typeof Ci == "undefined") {
            var Ci = Components.interfaces;
    }
    if (typeof CCIN == "undefined") {
    	function CCIN(cName, ifaceName){
    		return Cc[cName].createInstance(Ci[ifaceName]);
    	}
    }
    if (typeof CCSV == "undefined") {
    	function CCSV(cName, ifaceName){
    		if (Cc[cName])
    			// if fbs fails to load, the error can be _CC[cName] has no properties
    			return Cc[cName].getService(Ci[ifaceName]);
    		else
    			dumpError("CCSV fails for cName:" + cName);
    	};
    }
    • What’s with all of the typeof checks?

      Firebug, gotta love it, but it declares the same things using const. Inside of an if() block, a const is still seen and conflicts, even when the if condition evaluates to false. The code above is essentially a workaround to satisfy both possibilities. If the user has firebug installed, then carry on; if the user doesn’t have firebug installed, declare those shorthands

  • The constructor

    function TracingListener() {
    }

    Above is a (very) simple constructor function for us to create objects from. The methods and properties on the prototype are below. Note that while I could have changed the structure to accommodate better data-hiding, the method below reduces the number of new functions created by making them all declared only once on the prototype. Functions in the constructor are recreated every time the constructor is called with new yourConstructor() whereas functions on the prototype are shared by all instances.

  • The prototype definition

    • Basic properties

      TracingListener.prototype =
      {
          originalListener: null,
          receivedData: null,   //will be an array for incoming data.
      

      The first part of the prototype definition is setting up some basic properties. Note that both are assigned null. These properties will exist on all instances of TracingListener, and thus not be undefined if/when checking. In the case of receivedData, do not be tempted to make it an array here. Remember that methods and properties on the prototype are shared by all instances of the same type — and we don’t want all instances to share the same array for data.

      Also worth note is that receivedData is a good candidate for data-hiding and declaring it local to the constructor… but scope and visibility limitations would mean the functions requiring access to it would either need to be in the constructor as well, or have accessor and mutator methods for it. If you’re making a Singleton or a small number of instances, declaring functions in the constructor is no big deal, but this listener will be instantiated hundreds or thousands of times and it’s important to keep the duplication to a minimum.

    • Methods on the prototype

      • Interface Requirements
            //For the listener this is step 1.
            onStartRequest: function(request, context) {
            	this.receivedData = []; //initialize the array
        
        	//Pass on the onStartRequest call to the next listener in the chain -- VERY IMPORTANT
        	this.originalListener.onStartRequest(request, context);
            },

        onStartRequest is the first thing called when the actual request processing begins. This is also the best opportunity to initialize the array on this listener.

            //This is step 2. This gets called every time additional data is available
            onDataAvailable: function(request, context, inputStream, offset, count)
            {
               var binaryInputStream = CCIN("@mozilla.org/binaryinputstream;1",
                                         "nsIBinaryInputStream");
                binaryInputStream.setInputStream(inputStream);
        
                var storageStream = CCIN("@mozilla.org/storagestream;1",
                                         "nsIStorageStream");
                //8192 is the segment size in bytes, count is the maximum size of the stream in bytes
                storageStream.init(8192, count, null); 
        
        	var binaryOutputStream = CCIN("@mozilla.org/binaryoutputstream;1",
                                         "nsIBinaryOutputStream");
                binaryOutputStream.setOutputStream(storageStream.getOutputStream(0));
        
                // Copy received data as they come.
                var data = binaryInputStream.readBytes(count);
        
                this.receivedData.push(data);
        
                binaryOutputStream.writeBytes(data, count);
        
                //Pass it on down the chain
                this.originalListener.onDataAvailable(request,
                                                  context,
                                                  storageStream.newInputStream(0),
                                                  offset,
                                                  count);
            },

        onDataAvailable essentially copies the data from the binaryInputStream to our receivedData array and to the storageStream (via the binaryOutputStream). Then we pass a new InputStream from our storageStream onto the next listener in the chain.

            onStopRequest: function(request, context, statusCode)
            {
        	try
        	{
                        //QueryInterface into HttpChannel to access originalURI and requestMethod properties
        		request.QueryInterface(Ci.nsIHttpChannel);
        
                        //this is specific to the PirateQuesting Add-on, but is left here as an example of how to modify behaviour based on the requested URL
        		if (request.originalURI
                            && piratequesting.baseURL == request.originalURI.prePath
                            && request.originalURI.path.indexOf("/index.php?ajax=") == 0)
        		{
        
        			var data = null;
        			if (request.requestMethod.toLowerCase() == "post")
        			{
        				var postText = this.readPostTextFromRequest(request, context);
        				if (postText)
        					data = ((String)(postText)).parseQuery();
        
        			}
        
                                //Combine the response into a single string
        			var responseSource = this.receivedData.join('');
        
        			//fix leading spaces bug
        			//(FM occasionally adds spaces to the beginning of their ajax responses...
                                //which breaks the XML)
        			responseSource = responseSource.replace(/^\s+(\S[\s\S]+)/, "$1");
        
                                //gets the date from the response headers on the request.
                                //For PirateQuesting this was preferred over the date on the user's machine
        			var date = Date.parse(request.getResponseHeader("Date"));
        
                                //Again a PQ specific function call, but left as an example.
                                //This just passes a string URL, the text of the response,
                                //the date, and the data in the POST request (if applicable)
        			piratequesting.ProcessRawResponse(request.originalURI.spec,
                                                       responseSource,
                                                       date,
                                                       data);
        		}
        	}
        	catch (e)
        	{
        		//standard function to dump a formatted version of the error to console
        		dumpError(e);
        	}
        	//Pass it on down the chain
        	this.originalListener.onStopRequest(request,
                                                 context,
                                                 statusCode);
            },

        The onStopRequest above has a few tricky parts. The first is the QueryInterface to nsIHttpChannel – this is critical to getting the info needed. The second tricky part is to get the posted variables. To do so, you need to check that the requestMethod was indeed post, and then we call readPostTextFromRequest which I’ll introduce in a bit. The last tricky bit is getting the Date header from the response. Date.parse() plays nicely with those (assuming the server response conforms)

            QueryInterface: function (aIID) {
                if (aIID.equals(Ci.nsIStreamListener) ||
                    aIID.equals(Ci.nsISupports)) {
                    return this;
                }
                throw Components.results.NS_NOINTERFACE;
            },

        This is pretty standard for anything fulfilling an interface contract for Firefox (or other mozilla-based browsers). QueryInterface is part of the nsISupports interface and is the only part which is scriptable. All interfaces are derived from nsISupports, so it has to be there.

      • Utility methods

        The following methods are required by our TracingListener but are not part of the interface contract. (It would also have been possible to define them globally or within a pseudo-namespace.)

            readPostTextFromRequest : function(request, context) {
                try
                {
        	        var is = request.QueryInterface(Ci.nsIUploadChannel).uploadStream;
        	        if (is)
        	        {
        	            var ss = is.QueryInterface(Ci.nsISeekableStream);
        	            var prevOffset;
        	            if (ss)
        	            {
        	                prevOffset = ss.tell();
        	                ss.seek(Ci.nsISeekableStream.NS_SEEK_SET, 0);
        	            }
        
        	            // Read data from the stream..
        		    var charset = "UTF-8";
        		    var text = this.readFromStream(is, charset, true);
        
        	            if (ss && prevOffset == 0)
        	                ss.seek(Ci.nsISeekableStream.NS_SEEK_SET, 0);
        
        	            return text;
        	        }
        		else {
        			dump("Failed to Query Interface for upload stream.\n");
        		}
        	    }
        	    catch(exc)
        	    {
        			dumpError(exc);
        	    }
        
        	    return null;
        	},

        I will readily admit that readPostTextFromRequest is mostly taken from Firebug, though there are a few changes. Basically, we have to do the same thing as before and QueryInterface into the appropriate interface. In this case we need nsIUploadChannel to get access to uploadStream. And then we QueryInterface the uploadStream into a nsISeekableStream (noticing a pattern, yet? QueryInterface is your best friend.. and worst enemy.). After that we store the original offset in the stream in prevOffset, and then seek to the beginning of the stream. Then we read the data and, if the stream was at position 0 originally, we seek to the beginning again.

        	readFromStream : function(stream, charset, noClose)	{
        
        	    var sis = CCSV("@mozilla.org/binaryinputstream;1",
                                    "nsIBinaryInputStream");
        	    sis.setInputStream(stream);
        
        	    var segments = [];
        	    for (var count = stream.available(); count; count = stream.available())
        	        segments.push(sis.readBytes(count));
        
        	    if (!noClose)
        	        sis.close();
        
        	    var text = segments.join("");
        	    return text;
        	}
        
        }

        readFromStream is also largely from Firebug with a few modifications. It is however remarkably similar to what is done in onDataAvailable and onStopRequest. Basically, we get a BinaryInputStream to work with the stream given. Then we loop through the segments of the stream (size provided by available()) and add them to an array. When finished with that, we join the segments and return the text.

        httpRequestObserver = {
        
        	observe: function(request, aTopic, aData){
        		if (typeof Cc == "undefined") {
        			var Cc = Components.classes;
        		}
        		if (typeof Ci == "undefined") {
        			var Ci = Components.interfaces;
        		}
        	    	if (aTopic == "http-on-examine-response") {
        	    		request.QueryInterface(Ci.nsIHttpChannel);
        
        			if (request.originalURI
                                    && piratequesting.baseURL == request.originalURI.prePath
                                    && request.originalURI.path.indexOf("/index.php?ajax=") == 0) {
        				var newListener = new TracingListener();
            				request.QueryInterface(Ci.nsITraceableChannel);
            				newListener.originalListener = request.setNewListener(newListener);
        			}
        		}
        	},
        
        	QueryInterface: function(aIID){
        		if (typeof Cc == "undefined") {
        			var Cc = Components.classes;
        		}
        		if (typeof Ci == "undefined") {
        			var Ci = Components.interfaces;
        		}
        		if (aIID.equals(Ci.nsIObserver) ||
        		aIID.equals(Ci.nsISupports)) {
        			return this;
        		}
        
        		throw Components.results.NS_NOINTERFACE;
        
        	},
        };

        This part is fairly straightforward. The object httpRequestObserver has to fulfill the contract for the nsIObserver interface — which only has two methods: observe and QueryInterface.

  • Observer registration

    Finally, we need to register the observer:

    var observerService = Cc["@mozilla.org/observer-service;1"]
        .getService(Ci.nsIObserverService);
    
    observerService.addObserver(httpRequestObserver,
        "http-on-examine-response", false);

    Now the observerService will call the observe method on httpRequestObserver whenever it notifies observers with the http-on-examine-response topic.

    When you want to unregister the observer, use:

    observerService.removeObserver(httpRequestObserver,
        "http-on-examine-response");

    As you can see, getting the text and post variables from an http request is non-trivial.

Note, though, that this code does not check the context to determine whether the http request is for a browser window, or from a browser window so depending on the complexity of your situation, you may want to do that as well. Perhaps, I’ll add that in another post.

(See Firebug license here. Special thanks to the Firebug team and to Jon Odvarko for providing so much useful material. The interface docs at oxymoronical are a great resource. The Mozilla Developer Center also deserves special credit for great documentation. )

Update (Jan 17, 2010): Corrected a small bug in onStopRequest (Thanks Broady!). See below for details.

Update (April 22, 2010): Corrected a bug which doesn’t occur if Firebug is installed (Thanks Harini!). See below for details.

Tags: , , ,


Oct 09 2009

Implementing an XPCOM Firefox Interface and Creating Observers

Category: Firefox, JavaScriptJonathan Fingland @ 4:05 am

There are lots of cases when it is desirable to implement one of the XPCOM interfaces in use by Firefox, or other mozilla-based browsers. There are three cases where PirateQuesting does so, but once you see the concept, it should be easy to adapt to your situation.

  • You must have a QueryInterface to enjoy this ride

    First off, all XPCOM interfaces in Firefox inherit from nsISupports (Also see details on oxymoronical.com here). Only one method is scriptable and part of XPCOM — QueryInterface — and it must be present in all implementations of XPCOM interfaces.

    
    //"implements" nsISupports
    var InterfaceImplementation = function() {
      QueryInterface: function (aIID) {
          if (aIID.equals(Components.interfaces.nsISupports))
          {
             return this;
          }
          throw Components.results.NS_NOINTERFACE;
      }
    }
    

    The above is an example of the very minimum required to support any interface. QueryInterface requires a first parameter which is an aIID from Components.interfaces.*. There is also a second, optional, parameter, but as I have never come across this in use, it’s not worth pursuing here.

  • Now what?

    A very common (and useful) use of XPCOM interface implementation is creating your own observers, for example:

    
    var myObserver = {
    
      observe: function(request, aTopic, aData){
        if (aTopic == "http-on-examine-response")
        {
          //response has come back, now what?
        }
        else if (aTopic == "http-on-modify-request")
        {
          //opportunity to modify headers on request
        }
      },
    
      QueryInterface: function(aIID){
         if (aIID.equals(Components.interfaces.nsIObserver) ||
             aIID.equals(Components.interfaces.nsISupports))
        {
          return this;
        }
        throw Components.results.NS_NOINTERFACE;
      },
    };
    

    The nsIObserver interface is fairly simple as it only adds one new method. As you can see now, though, QueryInterface now checks for both nsIObserver and nsISupports. Remember: any interface you implement must have a QueryInterface supporting all interfaces in the inheritance chain.

  • Observer registration

    If you then wanted to register your observer, it’s as easy as:

    
    var observerService = Components.classes["@mozilla.org/observer-service;1"]
                                   .getService(Components.interfaces.nsIObserverService);
    
    observerService.addObserver(myObserver,"http-on-examine-response", false);
    observerService.addObserver(myObserver,"http-on-modify-request", false);
    

    Then to unregister, you do:

    
    observerService.removeObserver(myObserver, "http-on-examine-response");
    observerService.removeObserver(myObserver, "http-on-modify-request");
    

    When I have a chance, I’ll add a more complete example of using observers to watch http requests, but in the meantime check out the list of Observer Notifications at MDC.

Tags: , ,