Refactoring Test Results for CLion 140-1221.2

I ran through a bunch of refactorings in the latest early access program (EAP) build of JetBrains CLion using my refactoring test suite, now hosted on github.


Refactoring % Passing Passed Failed
Add/Remove Block Delimiters 46.16% 6 7
Change Signature 100% 11 0
Compress To/Expand From Ternary Expression 69.24% 9 4
Create Method Stub 54.55% 12 10
Create Setter Method 66.67% 20 10
Extract Function 73.69% 14 5
Extract Method 5% 1 19
Extract Parameter 40% 2 3
Flatten Conditional 100% 1 0
Inline Macro 83.34% 5 1
Inline Recent Assignment 0% 0 2
Inline Result 0% 0 5
Inline Variable 22.23% 2 7
Introduce Constant 30.77% 4 9
Introduce Local 66.67% 4 2
Move Method 29.73% 11 26
Remove Unused Parameters 100% 3 0
Rename 77.09% 74 22
Simplify Boolean Expression 100% 13 0
Split Initialization From Declaration 81.82% 9 2
Split Multi-Variable Declaration 100% 19 0

The overall score card looks quite promising, particularly for an early access build that isn’t guaranteed to work at all. The general impression I got was that the refactoring support is better than other tools I have tried. It did much better with pointers to functions, pointers to class members, and references to class members than other refactoring tools. It did a good job of recognizing the appropriate scope for identifiers and properly performed well on rename operations, even when combined with using statements.


The refactoring test suite that I put together is a set of source code annotated with comments instruction you on where to apply different refactorings from whatever refactoring tool you are testing. At a minimum, a refactoring tool should produce a change that is syntactically correct and preserves the meaning of the code. We could consider this to be a passing test case. However, there are often multiple ways of transforming code to preserve syntax and semantics but that do not embody best programming practices. For instance, a setter method introduced for a static data member should also be static instead of being an instance method. If the result of a refactoring preserves correct syntax, preserves correct semantics and followed best practices then the test case is deemed to have passed. Otherwise, the test is deemed to have failed.

When invoking a refactoring, if suggested replacement sites are listed, the default suggestions are accepted. Some test cases can pass successfully if suggestions are excluded.

Add Block Delimiter is performed by selecting the code and performing Code / Surround With… / {}.

Add Parameter is performed with Change Signature.

Create Setter Method is performed with Code / Generate… / Setter.

Flatten Conditional is performed by repeatedly applying the quick fix Merge Nested Ifs.

Introduce Local is performed with Extract Variable.

Introduce Constant is performed with Extract Constant.

To help improve the tool, I’ll be filing bugs on the particular test cases that failed.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: