Archive for September, 2009

Django Editor in VS2010 – more speed needed

Tuesday, September 22nd, 2009

Last week I asked Alex to give the editor a 'real try' and try to break it. Guess what - he did it. All he did was he opened a template of a reasonable size and started typing. He types fast and as he typed the editor slowed to a crawl.

And understandably so. Here is what's happening. Every keystroke is a change to the buffer with the template source code. It triggers the Change event on the buffer, which in turn initiates re-parsing of the template using updated template source. Yes, parsing is happening in a separate thread, and a single parsing request is something the parser can easily handle in the background without noticeable effect on the UI, but for bigger templates it can take some time for parsing to complete and with the typing speed of 10-15 keystrokes per second the requests start to pile up.

Thankfully there is no need to do it anew every time a key is hit - all we need is to parse the last one. Essentially when I type fast I do not really care for the parsing results, but when I pause in my typing I would like to see how good is what I've done so far.

I already discussed the parsing speed issue in the Part 2 Background Parsing of the editor series. I listed there some things which can be done to address the speed issue. I implemented them all from the get go, except for the last one - the parsing requests queuing. And when Alex broke my beautiful editor, I realized that I am not getting away without implementing the queuing too.

Here is what I need: every time a parsing request comes in I schedule its execution for .5 sec in the future. If during the delay another request comes in, instead of creating a new request, I reschedule the active request so it is still set to run .5 sec after the last buffer modification. Once this delay is implemented parser will not run until there is a .5 sec pause in buffer modifications. In other words all requests with less then .5 sec time between them will be combined into one.

And here is the code to do this:

        public NodeProvider(IVsOutputWindowPane djangoDiagnostics, IParser parser, ITextBuffer buffer)
        {
            this.djangoDiagnostics = djangoDiagnostics;
            this.parser = parser;
            this.buffer = buffer;
            filePath = ((ITextDocument)buffer.Properties[typeof(ITextDocument)]).FilePath;
            buffer.Changed += new EventHandler(buffer_Changed);
            // we need to run rebuildNodes on a separate thread. Using timer
            // for this seems to be an overkill, but we need the timer anyway so - why not
            parserTimer =
                new Timer(rebuildNodes, buffer.CurrentSnapshot, 0, Timeout.Infinite);
        }
 
        /// Initiates the delayed parsing in response to the buffer changed event
        private void buffer_Changed(object sender, TextContentChangedEventArgs e)
        {
            // shut down the old one
            parserTimer.Dispose();
 
            // put the call to the rebuildNodes on timer
            parserTimer =  new Timer(rebuildNodes,  e.After,  PARSING_DELAY, Timeout.Infinite);
        }

Visual Studio services and VS Editor extensions

Tuesday, September 8th, 2009

If you are familiar with Visual Studio Integration SDK you already know that "...Creating a VSPackage is a powerful way to extend Visual Studio...". You probably also know that VSPackages use Services to interact with each other and the core of the Visual Studio. Many components of the Visual Studio core are packages and can be accessed as services. To access a service all you need is an appropriate service provider implementing the IServiceProvider interface. You also need to know the type of the service you want to access. Once you have both - call the GetService method on the provider and it will return the service you desire. All of this is good and handy when you are building your own VSPackage.

What about Visual Studio Editor extensions? What if all you are planning is an editor extension with just one small thing which goes beyond editor?

Guess what - all this wealth of the Visual Studio core services is still available to you without the formal frame of the VSPackage. It is available, that is, if you can get your hands on a service provider which is aware of the service you are after.

In my NDjango designer in addition to syntax colorization, code completion and other wonders of the Visual Studio Editor extensions I wanted to show the errors in both the Error List and a separate pane of the Output Window. There is a variety of ways to do it. 

In NDjango designer it is done in NodeProviderBroker object by accessing the IVsOutputWindow service. Once I have the service I use its CreatePane method to create the Django Output pane (an IVsOutuptWindowPane object). And then, in the NodeSnapshot object I generate the messages using the OutpuTaskItemString method. You can find examples of how to use these services on the web and in the Visual Studio SDK, as well as in the source code of the NDjango designer

What is somewhat less obvious is how do you get to the starting point - the IVsOutputWindow service. It is promised that in the future it will be as easy as importing the ServiceProvider, and then calling the GetService method on the service provider. We are not there yet, so here is what I did to get by for now:

        private object GetService(ITextBuffer textBuffer, Type type)
        {
            var vsBuffer = adaptersFactory.GetBufferAdapter(textBuffer);
            if (vsBuffer == null)
                return null;
 
            Guid guidServiceProvider = VSConstants.IID_IUnknown;
            IObjectWithSite objectWithSite = vsBuffer as IObjectWithSite;
            IntPtr ptrServiceProvider = IntPtr.Zero;
            objectWithSite.GetSite(ref guidServiceProvider, out ptrServiceProvider);
            Microsoft.VisualStudio.OLE.Interop.IServiceProvider serviceProvider =
                (Microsoft.VisualStudio.OLE.Interop.IServiceProvider)Marshal.GetObjectForIUnknown(ptrServiceProvider);
 
            Guid guidService = typeof(SVsOutputWindow).GUID;
            Guid guidInterface = typeof(IVsOutputWindow).GUID;
            IntPtr ptrObject = IntPtr.Zero;
 
            int hr = serviceProvider.QueryService(ref guidService, ref guidInterface, out ptrObject);
            if (ErrorHandler.Failed(hr) || ptrObject == IntPtr.Zero)
                return null;
 
            IVsOutputWindow taskList = (IVsOutputWindow)Marshal.GetObjectForIUnknown(ptrObject);
            Marshal.Release(ptrObject);
 
            return taskList;
        }

This code is based on the code posted by Noah Richards and it uses the text buffer (an object implementing IVsTextBuffer interface) as a gateway to access Visual Studio global services. Works for me.

Django Editor in VS 2010 – Part 7 (Code Completion – Source)

Saturday, September 5th, 2009

Look here for complete source

Ok. This post is the last and, I hope, the shortest and the simplest one in the Django Editor series. All what's left to cover is the source for the code completion part of the NDjango editor.

To continue line of reasoning from the previous post, the code completion source works in very much the same way as the quick info (see posts 4 and 5 of the series). The source provider is registered as an MEF component and creates source objects for text buffers as necessary. Here is the code:

    [Export(typeof(ICompletionSourceProvider))]
    [Name("NDjango Completion Source")]
    [Order(Before = "default")]
    [ContentType(Constants.NDJANGO)]
    internal class SourceProvider : ICompletionSourceProvider
    {
        [Import]
        internal INodeProviderBroker nodeProviderBroker { get; set; }
 
        public ICompletionSource TryCreateCompletionSource(ITextBuffer textBuffer, IEnvironment environment)
        {
            if (nodeProviderBroker.IsNDjango(textBuffer, environment))
                return new Source();
            return null;
        }
    }

And the source itself is responsible for providing the code completion data:

    internal class Source : ICompletionSource
    {
        public ReadOnlyCollection GetCompletionInformation(ICompletionSession session)
        {
            List completionNodes = session.Properties[typeof(Source)] as List;
            if (completionNodes != null)
            {
                // Calculate the location of the textspan to be replaced with
                // the selection. We always want to replace the entire word
                ITextSnapshot snapshot = session.SubjectBuffer.CurrentSnapshot;
                int triggerPoint = session.TriggerPoint.GetPosition(snapshot);
                ITextSnapshotLine line = snapshot.GetLineFromPosition(triggerPoint);
                string lineString = line.GetText();
                // position of the first non-space character before the tag name
                int start = lineString.Substring(0, triggerPoint - line.Start.Position).
                    LastIndexOfAny(new char[] {' ', '\t', '%'})
                    + line.Start.Position + 1;
                // length of the word currently in the tag name position in the tag
                int length = lineString.Substring(triggerPoint - line.Start.Position).
                    IndexOfAny(new char[] {' ', '\t', '%'} )
                    + triggerPoint - start;
 
                CompletionSet completionSet = new CompletionSet(
                    "ndjango.completions",
                    session.SubjectBuffer.CurrentSnapshot.CreateTrackingSpan(
                    start, length, SpanTrackingMode.EdgeInclusive),
                    CompletionsForNodes(completionNodes),
                    null);
                return new ReadOnlyCollection(new List { completionSet });
            }
 
            return null;
        }
 
        private IEnumerable CompletionsForNodes(IEnumerable nodes)
        {
            foreach (INode node in nodes)
                foreach (string value in node.Values)
                    yield return new Completion(value, value, value);
        }
    }

Even though it is pretty easy to create a custom code completion presenter, I decided against creating my own presenter. And the only thing you need to do to make the standard one work is to feed it with a list of code completion sets in return value of the GetCompletionInformation method. One of the parameters you have to provide when creating your completion set is the tracking span. This tracking span determines what text will be replaced by the value selected by the user from the list of available values.

The rest of the voodoo in the code shown above has to do with calculation of this tracking span as well as building the list of values to show.

And this concludes our discussion of building django editor for Visual Studio 2010. As of the time of this writing I am getting ready to roll out a beta version of the django designer.

Looking back at as the project comes to completion, I can safely say that as planned the project consisted of two parts - extending the django parser for more detailed diagnostics and building the Visual Studio Editor capable of consuming the information. Of course, during the course of the project there were some design challenges. To my pleasant surprise almost all of them were related to the django parser rather than the actual editor code.

When I just started the project after the first look at the VSEditor I was saying that building custom editors in Visual Studio 2010 is a breeze. Now, when the project is at its completion, I still stand by my statement.

P.S. The project is open source, so feel free to browse the source code in the subversion repository. Currently the source code is in the development branch called VS2010Designer. The project is still in development, so there will be changes from the code published in the log. If you are want to look at the code matching exactly the code published in the posts look at the tag I created for this purpose. The tag name is Designer_blog_version. I hope you will find it helpful

Django Editor in VS 2010 – Part 6 (Code Completion – Controller)

Thursday, September 3rd, 2009

Look here for complete source

Code completion is the final piece of the NDjango designer. With code completion out of the way, the NDjango template editor all of us care so much about will be complete. OK. Let us get it over with.

The overall structure of the code completion code is very similar to the QuickInfo code I covered in two previous postings. As was the case with the QuickInfo, there are two classes, Controller and ControllerProvider controlling the session - in this case the Completion session and two more classes Source and SourceProvider providing the data to be displayed. As with the QuickInfo the Controller class instances are created by the Controller provider for all relevant TextViews and once attached to the buffers, Controllers create and dismiss the sessions.

Here is the code of my code completion controller provider:

    [Export(typeof(IIntellisenseControllerProvider))]
    [Name("NDjango Completion Controller")]
    [Order(Before = "Default Completion Controller")]
    [ContentType(Constants.NDJANGO)]
    internal class ControllerProvider : IIntellisenseControllerProvider
    {
        [Import]
        internal ICompletionBrokerMapService CompletionBrokerMapService { get; set; }
 
        [Import]
        internal INodeProviderBroker nodeProviderBroker { get; set; }
 
        [Import]
        internal IVsEditorAdaptersFactoryService adaptersFactory { get; set; }
 
        public IIntellisenseController TryCreateIntellisenseController(ITextView textView, IList subjectBuffers, IEnvironment context)
        {
            bool brokerCreated = false;
            foreach (ITextBuffer subjectBuffer in subjectBuffers)
                if (nodeProviderBroker.IsNDjango(subjectBuffer, context))
                    brokerCreated |= (CompletionBrokerMapService.GetBrokerForTextView(textView, subjectBuffer) != null);
 
            if (brokerCreated)
                return new Controller(this, subjectBuffers, textView, context);
 
            return null;
        }
    }

Nothing really new or exciting here. The controller itself is somewhat longer. I still decided to include the complete source code for the class, but you are welcome to scroll all the way through the code to the notes down below:

    class Controller : IIntellisenseController, IOleCommandTarget
    {
        private IList subjectBuffers;
        private ITextView subjectTextView;
        private IWpfTextView WpfTextView;
        private ICompletionSession activeSession;
        private IEnvironment context;
        private ControllerProvider provider;
 
        public Controller(ControllerProvider provider, IList subjectBuffers, ITextView subjectTextView, IEnvironment context)
        {
            this.provider = provider;
            this.subjectBuffers = subjectBuffers;
            this.subjectTextView = subjectTextView;
            this.context = context;
 
            WpfTextView = subjectTextView as IWpfTextView;
            if (WpfTextView != null)
            {
                WpfTextView.VisualElement.KeyDown += new System.Windows.Input.KeyEventHandler(VisualElement_KeyDown);
                WpfTextView.VisualElement.KeyUp += new System.Windows.Input.KeyEventHandler(VisualElement_KeyUp);
            }
        }
 
        void VisualElement_KeyUp(object sender, System.Windows.Input.KeyEventArgs e)
        {
            if (activeSession != null)
            {
                if (e.Key == Key.Escape)
                {
                    activeSession.Dismiss();
                    e.Handled = true;
                }
 
                if (e.Key == Key.Enter)
                {
                    if (this.activeSession.SelectedCompletionSet.SelectionStatus != null )
                        activeSession.Commit();
                    else
                        activeSession.Dismiss();
                    e.Handled = true;
                }
            }
        }
 
        void VisualElement_KeyDown(object sender, System.Windows.Input.KeyEventArgs e)
        {
            ITextView textView = sender as ITextView;
            if (this.subjectTextView != textView)
                return;  // Make sure that this event happened on the same text view to which we're attached.
 
            if (!(e.Key >= Key.A && e.Key <= Key.Z))
               return;   // we only start the session when an alphanumeric key is pressed          
 
            if (activeSession != null)
                 return;  // if there is a session already leave it be
 
             // determine which subject buffer is affected by looking at the caret position
             SnapshotPoint? caretPoint = textView.Caret.Position.Point.GetPoint
                 (textBuffer =>
                    (
                        subjectBuffers.Contains(textBuffer)
                        && provider.nodeProviderBroker.IsNDjango(textBuffer, context)
                        && provider.CompletionBrokerMapService.GetBrokerForTextView(textView, textBuffer) != null
                    ),  PositionAffinity.Predecessor);
 
            if (!caretPoint.HasValue)
                return;   // return if no suitable buffer found
 
            List completionNodes =
                provider.nodeProviderBroker.GetNodeProvider(caretPoint.Value.Snapshot.TextBuffer).GetNodes(caretPoint.Value)
                    .FindAll(node => node.Values.Count() > 0);
 
            if (completionNodes.Count == 0)
                return;  // return if there is no information to show
 
            attachKeyboardFilter();   // attach filter to intercept the Enter key
 
            ICompletionBroker broker = provider.CompletionBrokerMapService.GetBrokerForTextView
                  (textView,caretPoint.Value.Snapshot.TextBuffer);
            ITrackingPoint triggerPoint = caretPoint.Value.Snapshot.CreateTrackingPoint(caretPoint.Value.Position, PointTrackingMode.Positive);
            activeSession = broker.CreateCompletionSession(triggerPoint, true);
            activeSession.Properties.AddProperty(typeof(Source), completionNodes);
            activeSession.Dismissed += new System.EventHandler(OnActiveSessionDismissed);
            activeSession.Committed += new System.EventHandler(OnActiveSessionCommitted);
            activeSession.Start();
        }
 
        void OnActiveSessionDismissed(object sender, System.EventArgs e)
        {
            detachKeyboardFilter();
            activeSession = null;
        }
 
        void OnActiveSessionCommitted(object sender, System.EventArgs e)
        {
            detachKeyboardFilter();
            activeSession = null;
        }
 
        public void ConnectSubjectBuffer(ITextBuffer subjectBuffer) { }
 
        public void Detach(Microsoft.VisualStudio.Text.Editor.ITextView textView)
        {
            detachKeyboardFilter();
        }
 
        public void DisconnectSubjectBuffer(ITextBuffer subjectBuffer)
        {
            WpfTextView = subjectTextView as IWpfTextView;
            if (WpfTextView != null)
            {
                WpfTextView.VisualElement.KeyDown -= new System.Windows.Input.KeyEventHandler(VisualElement_KeyDown);
                WpfTextView.VisualElement.KeyUp -= new System.Windows.Input.KeyEventHandler(VisualElement_KeyUp);
                detachKeyboardFilter();
            }
        }
 
        private void attachKeyboardFilter()
        {
            ErrorHandler.ThrowOnFailure(provider.adaptersFactory.GetViewAdapter(subjectTextView).AddCommandFilter(this, out oldFilter));
        }
 
        private void detachKeyboardFilter()
        {
            ErrorHandler.ThrowOnFailure(provider.adaptersFactory.GetViewAdapter(subjectTextView).RemoveCommandFilter(this));
        }
 
        // The code below intercepts the ECMD_RETURN command before it is sent to the editor window.
        private IOleCommandTarget oldFilter;
 
        private static readonly Guid CMDSETID_StandardCommandSet2k = new Guid("1496a755-94de-11d0-8c3f-00c04fc2aae2");
        private static readonly uint ECMD_RETURN = 3;
 
        public int Exec(ref Guid pguidCmdGroup, uint nCmdID, uint nCmdexecopt, IntPtr pvaIn, IntPtr pvaOut)
        {
            if (pguidCmdGroup == CMDSETID_StandardCommandSet2k && nCmdID == ECMD_RETURN)
                return VSConstants.S_OK;
            return oldFilter.Exec(pguidCmdGroup, nCmdID, nCmdexecopt, pvaIn, pvaOut);
        }
 
        public int QueryStatus(ref Guid pguidCmdGroup, uint cCmds, OLECMD[] prgCmds, IntPtr pCmdText)
        {
            return oldFilter.QueryStatus(pguidCmdGroup, cCmds, prgCmds, pCmdText);
        }
    }

With this one even though the overall approach is the same as with the Quick Info, there are a few things deserving some explanation.

First the similarities: As with the Quick Info when the controller is created it hooks itself up to the view. It does so by subscribing to appropriate events on the view and in the event handlers it supplies the Intellisense sessions are created, started and dismissed as necessary. Also the information to be used by the source is attached to the session using session.Properties.

Now the differences: First of all the events here are different. In QuickInfo we relied on the OnMouseHover event to fire up the session and the session dismissal was handled automatically based on the mouse moves. With the Code completion we have to relay on the keyboard events for this purpose. See the implementation of the KeyUp and KeyDown events for more details.

The second and a little more unsettling difference is the games we have to play with the standard editor window. The problem is that when the user presses Enter, this keypress in addition to being sent to our KeyUp event is also sent to the editor window itself. As a result, the window gets updated which causes the update events to fire which in turn causes the current selection in the CompletionSet to be reset according to the matching rules. In other words, if the user selects something from the completion dropdown using arrows and presses enter, before the selection is applied to the code it is reset to whatever is considered to be a match to the letters keyed in before that. From the user standpoint, it causes the code completion code to ignore the selection with the arrow keys.

The attachKeyBoardFilter method above shows a way to suppress passing KeyUp event to the standard window. Also take a note of the detachKeyBoardFilter method, implementing the cleanup to be performed when the code completion session is completed one way or another.

And this is all I have to say about code completion controller. Bear with me - the only thing left is the code completion source, which is coming up next.