The AWS Certification team is seeking a Psychometrician with experience working with criterion-referenced assessment programs to support a large global AWS Certification and Credentialing program. In this role, you will support all psychometric aspects of exam development and operation, including job analyses, standard setting, automated test assembly, item and test analyses, optimal item bank design, quality assurance, and project planning.
You will work closely with a team of psychometricians, subject matter experts, certification exam program managers, publishing, delivery, security, and product management teams to support ongoing analyses of exam and credential data. To be successful in this position, you must be highly motivated, creative, detail oriented, and a self-starter who is able to think big, execute, ensure high quality, yet stay focused on the details.
Key job responsibilities
Conduct Job Task Analysis (JTA) workshops and post-JTA survey analyses to define the blueprint and test specifications for new certifications or updates to existing certifications
Conduct standard setting studies to set the passing score for exams and credentials
Run item analysis to evaluate quality and performance of exam items
Use automated test assembly procedures to assemble forms or item pools
Work with content development to track item bank trends and optimize the health of item banks
Support the development of a cloud-based analytics and reporting system
Partake in development and performance analysis of credentials
Interpret and clearly communicate the results of analyses to stakeholders through written and oral reports
Follow the accreditation standards set by ISO/IEC:2012 17024 and the National Council for Certifying Agencies (NCCA) as they relate to valid psychometric practices
Contribute to the development and execution of the strategic goals regarding the AWS certification and credentialing program.
Consult with leadership, internal staff, external consultants, and industry leaders regarding advancement of current offerings
Basic qualifications
PhD
At least 1 year post-doctoral direct experience working with high stakes certification or licensure programs
Experience conducting job task analysis and standard setting
Don't want to miss the next one?
Subscribe to daily email alerts for roles matching your interests.
Experience in equating, item analysis, and form assembly
Experience using item response theory
Experience using Generative AI tools
Proficiency in R or Python languages
Preferred qualifications
Experience in testing for global audiences
Experience with different exam designs and delivery (e.g. LOFT, CAT)
Experience using advanced AI methods (e.g., Machine Learning, Agentic AI)
History of conference presentations and publications
Experience with data forensics methods
Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.
Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.
The base salary range for this position is listed below. Your Amazon package will include sign-on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.
USA, OR, Portland - 136,000.00 - 184,000.00 USD annually USA, TX, Austin - 136,000.00 - 184,000.00 USD annually USA, VA, Arlington - 136,000.00 - 184,000.00 USD annually USA, VA, Herndon - 136,000.00 - 184,000.00 USD annually USA, WA, Bellevue - 136,000.00 - 184,000.00 USD annually USA, WA, Seattle - 136,000.00 - 184,000.00 USD annually
Similar roles you might like
More openings like this one — take a look before you go.
