Refactoring Tools Shoot Out

Over the past few years, I’ve been using a number of refactoring add-ons for Visual Studio.  The three refactoring add-ins I have tried are: Refactor! Pro from DevExpress, Visual Assist X from Whole Tomato and ReSharper from JetBrains. In this post, I’ll review and compare these refactoring add-ons.

I program mostly in C++ and C#, with the occasional foray into JavaScript. For me, refactoring support for C++ is the holy grail. The C++ code bases I’ve worked in are the older code bases and its the older code bases that need refactoring more often. First, let’s talk about the C++ products: Refactor! Pro and Visual Assist X. I’ve used both of these products for about a year or more, giving them a thorough workout in my daily development.

Between these two, Visual Assist X is the hands-down winner. While Refactor! Pro has many more refactorings, it slows down Visual Studio considerably. Just scrolling the source in the text editor becomes unbearably slow. While Refactor! Pro provides many more refactorings than Visual Assist X, I’ve also experienced a quality problem with Refactor! Pro. I filed over 300 bugs on their documentation and refactorings.

If DevExpress can address the slowdown and quality issues, then I would recommend its use. I enjoy the “live identifiers” it provides during refactoring where I can see the code updated with the new names as I type. I like the ability to TAB through all the code affected by a refactoring before I commit the change. I like the large catalog of refactorings that they have. However, a small catalog of reliable refactorings is better than a large catalog of unreliable refactorings. Similarly, it is just intolerable for an add-in to slow down basic operations like scrolling through code with the cursor keys. I navigate code all day long, even if I’m not refactoring it. Since the only way to turn off Refactor! Pro is to uninstall it, its just simply too prohibitive to have it installed in my daily working environment.

The bug reports I filed on Refactor! Pro are based on a test suite for refactorings that I created. The test suite was created by taking each supported refactoring technique and writing source code that can be refactored with that technique and then applying the refactoring. For many of the bugs I filed on Refactor! Pro, this was how I was able to reproduce the problem. For other bugs that I filed, I was able to supply a copy of open source code that I was refactoring. Most of the time you will be working on code where you might be reluctant to supply source to the tool vendor, so it will not be so easy to supply code that reproduces the problem. The slowness and bugginess of Refactor! Pro seemed to increase with the size of the project/solution loaded into Visual Studio. The last time I went through my test suite, I could not get even 50% of the tests to pass for Refactor! Pro and very few refactorings passed all the test cases.

For Visual Assist X, the experience so far has been very different. While it isn’t flawless in its refactorings and it has a far smaller catalog of refactorings than Refactor! Pro, it keeps Visual Studio responsive during editing and I have experienced far fewer bugs with its refactorings than I did with Refactor! Pro. Refactoring support is new to Visual Assist, having only appeared in the most current version. Visual Assist started as an improved replacement for the intellisense in Visual Studio. Intellisense has been one of the weaker areas of Visual Studio, but it is getting an overhaul for Visual Studio 2010. I do not rely inordinately on intellisense, so until Visual Assist contained refactoring support, I had been largely ignoring it. I still don’t use its intellisense support so much, but I do use its ability to navigate code, particularly the ability to bounce back and forth between a C++ header declaration and the C++ source file definition of a symbol. I prefer Visual Assist’s find references window over the one built into Visual Studio.

Here are some pros and cons for these two C++ refactoring add-ins:

Refactor! Pro Visual Assist X
Pros Cons Pros Cons
  • C++, C#, VB.NET support
  • live identifiers, particularly for rename
  • enclose in curly brace
  • C++ support is buggy
  • slows down Visual Studio considerably
  • C++ refactorings are buggy
  • refactorings aren’t well documented
  • has hard crashed Visual Studio
  • .NET support can interfere with ReSharper
  • C++, C#, VB.NET support
  • improved code navigation
  • quality is good
  • improved find references over Visual Studio
  • small catalog of refactorings
  • C# support can interfere with ReSharper

For .NET languages, there is an additional refactoring add-in that can be used: ReSharper. ReSharper is more than just a refactoring tool, it provides many quick operations that supply little pieces of code. I find that developing with ReSharper is so much more productive, that now I can’t imagine developing C# code without it.

ReSharper
Pros Cons
  • class navigation
  • optimize usings
  • highlight redundant code, provide easy means to eliminate it
  • More stable
  • Hasn’t gotten a refactoring wrong yet
  • Has Push Up/Down Method
  • No C++ support
  • Has crashed a couple times on me, sending a crash report to its master, but didn’t crash VS.NET

My recommendation is to get ReSharper if you write .NET code and to get Visual Assist X if you write C++ code. Sometimes the two add-ins fight a little bit when you have them both loaded, but you can quickly disable Visual Assist X from its menu if you don’t want it to be operational at the moment. You can also configure Visual Assist X to not process .cs files if you want to use only ReSharper on C# code.

9 Responses to “Refactoring Tools Shoot Out”

  1. Agile Code Reviews « Legalize Adulthood! Says:

    […] as a team refactoring session instead of some formal process with paperwork and log sheets. We use Visual Assist X, an automated refactoring tool, to refactor our code. Because the refactorings are applied in an […]

    Like

  2. Rob Heyes Says:

    You don’t say what versions of the above tools you tested with.

    I’m a big fan of Refactor Pro! and would agree that the C++ support isn’t as good as it could be, but it seems to have improved a lot lately. However for C# I wouldn’t want to use anything else.

    Like

  3. Ashkan Says:

    VAX can also enclose in curly braces. All you’ve got to do is to select the part of code you want enclosed and press {.

    Great blog by the way! I really like it. :)

    Like

  4. legalize Says:

    I tested with the current version of all tools. I tried various builds of ReFactor! for about a year before I finally gave up on it. Several of my coworkers tried ReFactor! and all of them reported that it slowed Visual Studio down considerably. I honestly don’t know what they’re doing that causes scrolling of a source file to become dog ass slow and spike my CPU to 100%, but they’re obviously doing something. I don’t care what refactoring support you have, but there’s no excuse for slowing down code navigation with the arrow keys.

    You’re more than welcome to download my refactor test suite and manually execute the ~625 test cases that I created for ReFactor!. I posted results for 3.0.5: 595 test cases, 225 tests passing, 370 tests failing, 4 refactorings passing all tests. At the time, they were assuring me that they had very high code quality.

    Its almost as if they didn’t believe me that it just failed miserably in many circumstances. They also tried to blame the slowness on my laptop not having enough memory, or being too old. I bought more memory for my laptop (which was generally a good thing), but it didn’t help their product run any faster. Also, my coworkers machines were much faster than my laptop and it didn’t make any difference.

    I still have 3 bug reports in ‘Active’ state, one open since November of 2007, 24 are ‘Processed’ to be acted upon at some unspecified future date, 4 are ‘Processed (By Design)’, 44 were marked as duplicates, 250 were marked fixed (some as recently as March of ’09), 7 marked ‘Implemented’ (not sure what the difference is between implemented and fixed), 2 marked as won’t fix.

    I’m sorry, but 250+ bugs filed and fixed on my behalf alone indicates low quality. Yeah, they fixed those. But letting those 250+ bugs get out to the general release indicates testing deficiency on their part. Look at my test suite. Many of the simple cases fail. Yeah, C++ syntax is hairy, but they knew this going into it. I didn’t even get around to writing test cases for all the refactorings they claim to support because the quality and performance was so low that I just gave up.

    Like

  5. Refactoring Tools Shoot Out « Jasper Blog Says:

    […] Read more: Legalize Adulthood! […]

    Like

  6. mwb1100 Says:

    Thanks for the very good review of these tools. Sounds like you put in a phenomenal amount of work into this (especially on Refactor! Pro). 300+ bugs?? they need to put you on the payroll.

    Like

  7. Mike O'Neill Says:

    Hi I agree with the review

    I have used VAX for 3 yrs + now and I use its refactoring regularly. The one big aspect to VAX is the improved intellsence and things like Auto Snippets with shortcuts and the ability to use acronyms for intellesence eg ProcessDataSet it will list it as a suggestion if you type just pds. The refactoring to me is a bonus on all teh otehr features of VAX.

    That said I have dabbled with Refactor Pro and even have a license , but still I see the performance hit and eventually uninstall it , It could benefit from an off switch . I agree its refactorings are slick but I can’t live without VAX so it loses out. If you could swith it off in every day use I may well use it again, just for refactoring

    I recently tried the Coderush Express freebie from Dev Express with the same result , dog slow. I suspect its the sheer fact that 3 parsers are vying for position . Once again I gave up and reverted to VAX only .

    As you gather I am a great fan of VAX,but I also share the sentiment that a few good uns beats a load of slow uns . A lot of the RefactorPro ones are a bit obscure and a good base set is VAX plus whatever VS 2008 gives as std

    Hope this helps

    Mike

    Like

    • Rob Heyes Says:

      I gather that the next version (9.2) of CodeRush/Refactor Pro! will have an off switch or at least can be dynamically loaded – the release candidate is available now.

      Like

  8. Automated refactoring for C++ « Our Craft Says:

    […] Refactoring Tools Shoot Out by Legalize Adulthood ( […]

    Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: