None of the material we used
to recruit participants mentioned
refactoring or gave specifics of the
concepts we would be asking about.
Consequently, unlike other studies,
rather than include only developers known to refactor, there should
have been no obvious bias for or
against the use of refactoring. We recruited through personal contacts,
word of mouth, and social media,
supported by our website http://se-
folklore.com and a You Tube video.
The demographic data we collected
reflected a variety of experience,
roles, languages, level of qualification, and type of development.
We set a high bar in assessing
whether a participant would agree
with placing a limit on number of
methods or class depth or would still
refactor when that limit is exceeded.
Different choices would change the
distribution of the frequencies in the
different categories somewhat, but
the same trends would be evident.
Our wording of the possible responses might have affected what responses participants chose, but the
barriers we identified came from
the free-text commentary, which
was unlikely to be affected by such
concerns. Even those who would
refactor mentioned some of the
same concerns (as reported in Table
3 “Yes” columns). Participants from
all demographics provided comments, so there is no obvious bias
due to background, as was also the
case with our target group—those
who would limit number of methods
or depth of inheritance.
We conducted the survey more
than eight years ago and IDEs have
since improved, so perhaps we would
get different results today. However,
little has changed in refactoring sup-
port, and we are not aware of any re-
search directly addressing the issues
our survey identified. More recent
surveys have identified similar issues.
For example, Kim et al.
concerns regarding lack of adequate
test suites. Chen et al.
3 noted 21% of
participants reported lack of support
from management as a reason for not
refactoring and 12.4% did no plan-
ning regarding refactoring. However,
these studies surveyed developers
who were already committed to refac-
tor, whereas ours included those who
chose not to refactor. That these more
recent surveys report findings simi-
lar to ours suggests there has been
little change in barriers to refactoring
since our survey .
We used two metrics to identify
potential design-quality issues based
on published theories. It is possible
the theories are wrong. However,
our conclusions are based on comments by participants who (rightly or
wrongly) believed the theories.
There are significant barriers preventing developers from refactoring
to remove software design-quality
issues, no matter how they are identified. Reducing or even eliminating
the barriers has the potential to significantly improve software quality.
One means is to provide refactoring
support that is goal-directed rather
than operations-directed. Another
is to provide better quantification
of the benefits, thus better informing the decision as to whether or
not to refactor.
We thank our survey participants,
many of whom made the extra effort
to provide the comments we included
and discussed here. We also thank the
anonymous reviewers for their valuable comments and suggestions.
1. Bavota, G., Oliveto, R., Gethers, M., Poshyvanyk, D.,
and De Lucia, A. Methodbook: Recommending move
method refactorings via relational topic models.
IEEE Transactions on Software Engineering 40, 7
(July 2014), 671–694.
2. Baxter, G., Frean, M., Noble, J., Rickerby, M.,
Smith, H., Visser, M., Melton, H., and Tempero, E.
Understanding the shape of Java software. In
Proceedings of the 21st Annual ACM SIGPLAN
Conference on Object-Oriented Programming,
Systems, Languages, and Applications (Oct. 2006),
3. Chen, J., Xiao, J., Wang, Q., Osterweil, L.J., and Li, M.
Perspectives on refactoring planning and practice:
An empirical study. Empirical Software Engineering
21, 3 (June 2016), 1397–1436.
4. Chidamber, S.R., Darcy, D.P., and Kemerer, C.F.
Managerial use of metrics for object-oriented
software: An exploratory analysis. IEEE
Transactions on Software Engineering 24, 8 (Aug.
5. Chidamber, S.R. and Kemerer, C.F. A metrics suite
for object-oriented design. IEEE Transactions on
Software Engineering 20, 6 (June 1994), 476–493.
6. Fowler, M. Refactoring: Improving the Design of
Existing Code. Addison-Wesley, Boston, MA, 1999.
7. Gorschek, T., Tempero, E., and Angelis, L. A
large-scale empirical study of practitioners’ use of
object-oriented concepts. In Proceedings of the 32nd
ACM/IEEE International Conference on Software
Engineering (Cape Town, South Africa, 2010),
8. Harun, M.F. and Lichter, H. Towards a technical
debt-management framework based on cost-benefit
analysis. In Proceedings of the 10th International
Conference on Software Engineering Advances
(Barcelona, Spain). International Academy,
Research, and Industry Association, 2015.
9. Kazman, R., Cai, Y., Mo, R., Xiao, L., Feng, Q., Haziyev,
S., Fedak, V., and Shapochka, A. A case study in
locating the architectural roots of technical debt. In
Proceedings of the 37th International Conference on
Software Engineering (Firenze, Italy). IEEE Press,
10. Kim, M., Zimmermann, T., and Nagappan, N. A field
study of refactoring challenges and benefits. In
Proceedings of the ACM SIGSOF T 20th International
Symposium on the Foundations of Software
Engineering (Cary, NC). ACM Press, New York, 2012,
11. Murphy-Hill, E. and Black, A.P. Refactoring tools:
Fitness for purpose. IEEE Software 25, 5 (Sept.-Oct.
12. Murphy-Hill, E., Parnin, C., and Black, A.P. How we
refactor, and how we know it. IEEE Transactions on
Software Engineering 38, 1 (Jan. 2012), 5–18.
13. Robson, C. and McCartan, K. Real World Research,
Fourth Edition. John Wiley & Sons, Inc., 2015.
14. Saldana, J. The Coding Manual for Qualitative
Researchers, Third Edition. SAGE Publications Ltd.,
15. Shatnawi, R., Li, W., Swain, J., and Newman, T.
Finding software metrics threshold values using
ROC curves. Journal of Software Maintenance and
Evolution: Research and Practice 22, 1 (Jan. 2010),
16. Szöke, G., Nagy, C., Ferenc, R., and Gyimóthy, T. A
case study of refactoring large-scale industrial
systems to efficiently improve source code. In
Proceedings of the 14th International Conference
on Computational Science and Its Applications
(Guimarães, Portugal, June 30–July 3). Springer
International Publishing, Cham, Switzerland, 2014,
17. Vakilian, M., Chen, N., Negara, S., Rajkumar, B.A.,
Bailey, B.P., and Johnson, R.E. Use, disuse, and
misuse of automated refactorings. In Proceedings
of the 34th International Conference on Software
Engineering (Zurich, Switzerland). IEEE Press, 2012,
18. Xing, Z. and Stroulia, E. Refactoring practice: How it
is and how it should be supported (an Eclipse case
study). In Proceedings of the 22nd International
Conference on Software Maintenance (Philadelphia,
PA). IEEE Press, 2006, 458–468.
Ewan Tempero ( email@example.com) is
an associate professor of computer science at the
University of Auckland, Auckland, New Zealand.
Tony Gorschek ( firstname.lastname@example.org) is a
professor of software engineering at the Blekinge
Institute of Technology, Karlskrona, Sweden.
Lefteris Angelis ( email@example.com) is a professor of
statistics and information systems in the Department
of Informatics at Aristotle University of Thessaloniki,
©2017 ACM 0001-0782/17/10
Watch the authors discuss
their work in this exclusive