the systems on every test run, which
clears all caches. That is not the right
answer for all testing, but it definitely
reduces interference from previous
Orchestrating an Automated Test Lab
The Deliberate Revolution
Kode Vicious Unleashed
George V. Neville-Neil ( firstname.lastname@example.org) is the proprietor of
Neville-Neil Consulting and co-chair of the ACM Queue
editorial board. He works on networking and operating
systems code for fun and profit, teaches courses on
various programming-related subjects, and encourages
your comments, quips, and code snips pertaining to his
Copyright held by author.
code and configurations are checked
out onto the systems in the test group.
That might work if everyone were diligent about checking in and pushing
changes from the test system. But in
my experience, people are never that
diligent, and inevitably someone upgrades a system that had crucial test
results or configuration changes on it.
Use a networked file system and it will
save whatever hair you have left on
your head. (I should have learned this
lesson sooner.) Ensure the networked
file system traffic goes across the control interfaces and not the test interfaces. That should go without saying,
but in test lab construction, much of
what I think could go without saying
needs to be said.
At this point we have fulfilled the
most basic requirements of a net-
working test system: We have control
over all the systems, and we have a
way to ensure all the systems can see
the same configuration data without
undue risk of data loss. From here it
is time to write the automation that
controls these systems. For most test-
ing scenarios, I tend to just reboot all
written code. It seems that inatten-
tion to detail is pervasive, and when
I see a poor testing setup I should be
prepared to see poor code as well.
At this point, we now know that
we have to retain control over the systems—and we have several ways to do
that via separate control interfaces—
and ultimately, we have to have control over the system’s power. The next
place that most test labs fall down is
in access to necessary files.
Once upon a time a workstation
company figured out they could sell
lots of cheap workstations if they
could concentrate file storage on a
single, larger, and admittedly more
expensive, server. Thus was born
the Network File System, the much
maligned, but still relevant, way of
sharing files among a set of systems.
If your tests can in any way destroy a
system, or if upgrading a system with
new software removes old files, then
you need to be using some form of
networked file system. Of late I have
seen people try to handle this problem with distributed version control
systems such as Git, where the test
GRADUATE STUDY AT UCF IN ORLANDO
Computer Science at the University of Central Florida (UCF) is looking for graduate
students who are motivated to conduct cutting-edge research and pursue excellence.
Successful applicants will be offered 5 years of support (assuming satisfactory
progress) in the form of either a Graduate Teaching Assistantship or Graduate
Research Assistantship. These positions will be providing financial support, including
tuition waivers, and competitive stipends. US citizens are highly encouraged to apply.
UCF is located in Orlando FL — “The City Beautiful.” Orlando has hosted several
major academic conferences in recent years, including ACM Multimedia. UCF is
the largest university in the United States by undergraduate enrollment and the
country’s second-largest by total enrollment. Both Orlando and UCF have grown
from supporting the space program to supporting a thriving high-technology industry,
including many software startups.
Computer Science at UCF is performing ground-breaking research in a wide
range of research areas, including computer vision, image and video processing,
machine learning, AI, virtual reality, HCI, computer graphics, software engineering
and systems, database, parallel computation, networking and mobile computing,
computer security, bioinformatics and systems biology, theory, algorithms, and
quantum computing. See http://www.cs.ucf.edu.
These research areas have been sponsored by federal government agencies and
local governments, as well as established close collaborations with the leading
companies such as Google, Sarnoff, Microsoft, IBM, GE, and Siemens. The
Computer Science program has internationally-recognized faculty, including 5
Fellows of the IEEE.
We are also very proud of many of our students’ achievements, for example,
National Collegiate Cyber Defense Championship, a prestigious national award
for cybersecurity in 2014, and successfully competing at the World Finals of the
Association of Computing Machinery’s International Collegiate Programming Contest.
• A Bachelor degree or the equivalent in
Computer Science or related majors
• Excellent academic credentials
demonstrated by GPA scores
• Competitive GRE test scores
Prior publications and an MS degree in CS
(or a related area) are a plus.
HOW TO APPLY:
All applications must be submitted online at