19.5 – Use Technology in Assessment and Evaluation

Purpose & Context

Career Development Professionals (CDPs) use reputable technology in the selection, administration, scoring, and interpretation of assessment and evaluation results. CDPs adhere to professional and ethical principles of privacy and confidentiality.

Effective Performance

Competent career development professionals must be able to:

P1. Assess available technology-enabled assessment and evaluation tools, considering:
- Test publisher’s reputation
- Confidentiality of test results
- Security of data
- Accommodations available to meet needs of clients with disabilities
P2. Select assessment and evaluation tools, considering:
- Test purpose and context
- Psychometric properties, e.g. validity, reliability, fairness
- Client characteristics
- Accommodation options
P3. Evaluate own qualifications to determine if qualified to administer and interpret results:
- Contract administration and interpretation to qualified persons, where necessary
P4. Obtain informed consent, e.g. for use of assessment and evaluation tools, for use of results
P5. Select accommodations required, if appropriate
P6. Prepare for assessment and evaluation session:
- Verify technology works
- Select accommodations, as required
P7. Administer examination in compliance with test publisher’s requirements
- Explain how to use the technology
- Verify mode of testing is appropriate
- Allow client time to practice
- Monitor assessment, as appropriate
  - Note any incident that may impact test results
P8. Verify results:
- Recognize unusual or unexpected results, e.g. pattern across scales, inconsistencies with information
  collected using other measures or interview findings
- Report results that appear erroneous to the testing publisher
P9. Interpret results, considering:
- Technical documentation, e.g. limitations of test results, prescribed use and interpretation of scales, norm
  or comparison groups
- Psychometric characteristics, e.g. reliability, error of measurement, validity, fairness
- Purpose of the assessment and evaluation
P10. Present results:
- Explain how to use results to inform decisions, e.g. career decision, developmental needs
- Describe norms, raw scores or derived score scales clearly, as applicable
- Advise client on benefits and limitations of test results and interpretations
P11. Protect confidentiality of results, as required:
- Comply with legislation and regulations, e.g. Personal Information Protection and Electronic Documents Act
  (PIPEDA)

Knowledge & Understanding

Competent career development professionals must know and understand:

K1. Testing practice in the context area, e.g. assessment and evaluation measures appropriate for vocational and
career
planning purposes
K2. Testing principles and ethical use of tests, such as:
- Test construction, administration, scoring and interpretation
- Psychometrics and measurement, e.g. classical test theory
- Descriptive statistics, e.g. frequency distributions, descriptive statistics characterizing the normal curve,
  measures of central tendency, measures of variation, indices of relationship
- Scales, scores, and transformations, e.g. types of scales, types of scores, scale score equating, cut scores
- Reliability and measurement error
- Validity and meaning of test scores
- Fairness
K3. Advantages and disadvantages of different types of assessment instruments, e.g. mental and physical ability tests,
achievement tests:
- Knowledge
- Work sample or performance test
- Interest measures, e.g. career related interests
- Work and personal values measures
- Personality inventories, e.g. vocational-oriented or clinical
- Interviews
- Comprehensive career planning measures
K4. Legal rights of test takers
K5. Confidentiality of test materials and test information
K6. Descriptive statistics, e.g. frequency distributions, measures of central tendency, measures of variation, indices
of relationships
K7. Scales, scores, and transformations, e.g. types of scales, types of scores, scale score equating
K8. Reliability and measurement error
K9. Validity and meaning of test scores
K10. Fairness, i.e. ensuring test design, content and format do not result in biased test scores for particular groups

Contextual Variables

Competent career development professionals must be able to perform this competency in the following range of contexts:

Assessment and evaluation tools will be selected to meet the specific needs of each client.

Administration and interpretation of results might be delegated to authorized individual as per test publisher guidelines.

Glossary & Key References

Terms

Industry-specific terms contained in the standard defined here, where applicable.

None

Information Sources and Resources for Consideration

A Test-Taker’s Guide to Technology-based Testing. International Test Commission, 2010.

Context Rating Scales

Criticality

Q: What is the consequence of a professional being unable to perform this skill according to the standard?

Moderate risk: CRITICAL

Frequency

Q: How frequent and under what conditions is this skill performed?

Routinely, regular course of procedure

Level of Difficulty

Q: Under routine circumstances, how would you rate the level of difficulty in performing this skill?

Moderate difficulty or complexity

Time Required to Gain Proficiency

Q: What is the average length of time or number of repeated events that are minimally necessary for an individual to become proficient in performing the skill to the standard?

A career development professional requires a minimum experience of reporting assessment results to at least 40 clients with varying assessment needs.

Autonomy

Practitioners typically perform this competency without supervision, and alone.

Automation

It is unlikely that this competency will automate.

Requisite Work Aids, Tools, Equipment or Materials

None