A hard, remotely proctored four-item test takes up to 24 hours. Innovative approaches to test design, job analysis, form assembly, test equating, and psychometrics were needed to prepare the program for accreditation under the ISO/IEC 17024 standard and the US Department of Defense 8140 program. A performance test with an established reputation and large community of practice requires candidates to find a vulnerability in, gain access to, and perform an “exploit” in three individual virtual machines and a small network of other such machines. These tasks coincide precisely with the job description, giving the test the highest possible face validity. The presentation will explore the following specific challenges: • Usefully measuring reliability • Maintaining a passing standard without trial items • Making meaningful use of partial scoring • Managing test security where failing candidates are allowed multiple retakes and often sit through one session for recon only