I ran through a bunch of refactorings in the latest early access program (EAP) build of JetBrains CLion using my refactoring test suite, now hosted on github.
Summary
| Refactoring | % Passing | Passed | Failed |
|---|---|---|---|
| Add/Remove Block Delimiters | 46.16% | 6 | 7 |
| Change Signature | 100% | 11 | 0 |
| Compress To/Expand From Ternary Expression | 69.24% | 9 | 4 |
| Create Method Stub | 54.55% | 12 | 10 |
| Create Setter Method | 66.67% | 20 | 10 |
| Extract Function | 73.69% | 14 | 5 |
| Extract Method | 5% | 1 | 19 |
| Extract Parameter | 40% | 2 | 3 |
| Flatten Conditional | 100% | 1 | 0 |
| Inline Macro | 83.34% | 5 | 1 |
| Inline Recent Assignment | 0% | 0 | 2 |
| Inline Result | 0% | 0 | 5 |
| Inline Variable | 22.23% | 2 | 7 |
| Introduce Constant | 30.77% | 4 | 9 |
| Introduce Local | 66.67% | 4 | 2 |
| Move Method | 29.73% | 11 | 26 |
| Remove Unused Parameters | 100% | 3 | 0 |
| Rename | 77.09% | 74 | 22 |
| Simplify Boolean Expression | 100% | 13 | 0 |
| Split Initialization From Declaration | 81.82% | 9 | 2 |
| Split Multi-Variable Declaration | 100% | 19 | 0 |
The overall score card looks quite promising, particularly for an early access build that isn’t guaranteed to work at all. The general impression I got was that the refactoring support is better than other tools I have tried. It did much better with pointers to functions, pointers to class members, and references to class members than other refactoring tools. It did a good job of recognizing the appropriate scope for identifiers and properly performed well on rename operations, even when combined with using statements.
Details
The refactoring test suite that I put together is a set of source code annotated with comments instruction you on where to apply different refactorings from whatever refactoring tool you are testing. At a minimum, a refactoring tool should produce a change that is syntactically correct and preserves the meaning of the code. We could consider this to be a passing test case. However, there are often multiple ways of transforming code to preserve syntax and semantics but that do not embody best programming practices. For instance, a setter method introduced for a static data member should also be static instead of being an instance method. If the result of a refactoring preserves correct syntax, preserves correct semantics and followed best practices then the test case is deemed to have passed. Otherwise, the test is deemed to have failed.
When invoking a refactoring, if suggested replacement sites are listed, the default suggestions are accepted. Some test cases can pass successfully if suggestions are excluded.
Add Block Delimiter is performed by selecting the code and performing Code / Surround With… / {}.
Add Parameter is performed with Change Signature.
Create Setter Method is performed with Code / Generate… / Setter.
Flatten Conditional is performed by repeatedly applying the quick fix Merge Nested Ifs.
Introduce Local is performed with Extract Variable.
Introduce Constant is performed with Extract Constant.
To help improve the tool, I’ll be filing bugs on the particular test cases that failed.
Leave a Reply