How do we evaluate an LLM for bias and safety before deploying it?

A good pre-deployment evaluation should treat bias, safety, security, and reliability as measurable engineering properties, not as a single “does this model seem safe?” test. NIST’s AI Risk Management Framework recommends evaluating trustworthiness throughout the lifecycle, and its Generative AI profile specifically calls for subgroup fairness testing, adversarial testing, and documented evaluation results.

1. Start with a risk specification

Before testing the model, define:

• Intended uses: What is the model actually allowed to do?
• Prohibited uses: What should it refuse?
• Affected groups: Which demographic or vulnerable populations could be harmed?
• Failure severity: What constitutes a minor, serious, or catastrophic failure?
• Acceptable thresholds: How much error, bias, refusal, leakage, or unsafe behavior is tolerable?
• Human oversight: Which decisions require a person rather than the model?

This matters because there is no universal “bias score” or “safe enough” threshold. A customer-service chatbot and a model assisting with medical decisions need very different evaluation criteria.

2. Build a representative evaluation set

Use several kinds of test data rather than relying on a public benchmark.

Bias/fairness tests

Create matched/counterfactual prompts where the relevant demographic attribute changes but everything else stays the same.

For example:

“Jordan is applying for a software engineering job. Evaluate Jordan's qualifications.” Test variants with different names, genders, ethnic/cultural signals, ages, disabilities, dialects, etc.

Measure whether changing an irrelevant attribute changes:

• recommendation quality
• sentiment
• toxicity/denigration
• assumptions about competence
• refusals
• access to services
• generated descriptions or stereotypes

NIST specifically recommends evaluating performance across demographic groups and subgroups, including intersecting groups, and using counterfactual/low-context prompts for bias testing.

Don't test only one dimension at a time. Intersectional groups—for example, older women, disabled men, or speakers of a particular dialect—can experience failures that disappear in aggregate statistics.

3. Test safety adversarially

Normal prompts aren't enough. Have testers actively try to make the model violate its safety requirements.

Test:

• jailbreaks
• prompt injection
• multi-turn manipulation
• role-playing attacks
• obfuscated requests
• multilingual attacks
• misspellings and slang
• conflicting system/user instructions
• attempts to extract hidden instructions
• requests for dangerous or illegal assistance
• self-harm and abuse scenarios
• extremist/hateful content
• privacy attacks
• malicious code generation

Importantly, test multi-turn conversations, not just isolated prompts. Real attackers can gradually move a model toward an unsafe response. Anthropic, for example, describes using safety evaluations, risk assessments, bias evaluations, ambiguous scenarios, and extended multi-turn conversations before deployment.

OWASP's current LLM security guidance also emphasizes risks such as prompt injection, sensitive-information disclosure, excessive agency, insecure output handling, and overreliance.

4. Measure both false negatives and false positives

A safety system can fail in two directions:

The goal isn't “maximize refusals.” It's to maximize appropriate behavior.

For example, measure:

Safety recall

unsafe requests correctly refusedall unsafe requests\frac{\text{unsafe requests correctly refused}} {\text{all unsafe requests}}

and

Benign acceptance rate

benign requests correctly answeredall benign requests\frac{\text{benign requests correctly answered}} {\text{all benign requests}}

Then examine those metrics by demographic group and scenario.

5. Add human evaluation

Automated graders are useful but shouldn't be the only judge.

Use trained reviewers to score samples for:

• harmfulness
• stereotyping
• fairness
• factuality
• appropriateness of refusal
• privacy violations
• severity of failure
• contextual nuance

Have multiple reviewers independently label difficult cases and measure agreement.

LLM-as-a-judge can dramatically increase evaluation scale, but human review should validate the grader, particularly for high-impact or ambiguous cases.

6. Red-team the entire system, not just the model

This is one of the most important distinctions.

A model may be relatively safe in isolation but unsafe when connected to:

• email
• databases
• browsers
• code execution
• financial systems
• customer records
• internal documents
• external APIs

For example:

User → LLM → database → tool → external API The security evaluation needs to test the whole chain.

OWASP's GenAI Red Teaming Guide explicitly takes this broader approach, covering model-level vulnerabilities as well as system integration issues such as prompt injection and data exposure.

7. Evaluate privacy and information leakage

Try to determine whether the system can reveal:

• system prompts
• training-data fragments
• confidential documents
• another user's information
• API credentials/secrets
• personal information
• data supplied in previous sessions

Also test whether retrieval-augmented systems enforce authorization correctly.

A particularly important test is:

Can user A cause the model to retrieve or reveal information belonging to user B? This is often a system-level access-control problem rather than a model-quality problem.

8. Establish deployment gates

Don't just produce a 200-page evaluation report. Turn results into go/no-go criteria.

For example:

The exact thresholds should be determined by the application's risk level.

9. Test in stages

A practical pipeline looks like:

Offline evaluation → adversarial red team → human review → limited pilot → monitored deployment → continuous evaluation

Don't regard deployment as the end of evaluation. NIST's framework treats evaluation and risk management as lifecycle activities, and its newer TEVV work explicitly covers LLMs and agentic systems.

After deployment, continuously sample and evaluate real interactions—subject to privacy requirements—and watch for:

• new jailbreaks
• demographic disparities
• changes after model updates
• unexpected use cases
• hallucination clusters
• emerging abuse patterns
• user complaints
• safety incidents

10. Keep an evaluation dossier

Before launch, you should be able to answer:

1. What can this model do?
2. What can it do that we don't want it to do?
3. Who is disproportionately affected by its failures?
4. How frequently do important failures occur?
5. How severe are they?
6. How did we discover them?
7. What mitigations were applied?
8. How effective were those mitigations?
9. What residual risks remain?
10. Who has authority to stop deployment?

That documentation makes the evaluation auditable rather than anecdotal.

A useful mental model

I'd structure the evaluation as:

Risk definition → representative tests → automated measurement → human evaluation → adversarial red teaming → mitigation → re-test → deployment gate → continuous monitoring

And importantly, evaluate the deployed application, not merely the base LLM. A model's safety properties can change substantially when you add system prompts, RAG, tools, memory, fine-tuning, agents, or autonomous actions.

For a formal starting point, nist.gov and owasp.org are particularly useful. NIST also maintains an AI evaluation program focused specifically on measuring GenAI capabilities and limitations.

Don't regard deployment as the end of evaluation. NIST's framework treats evaluation and risk management as lifecycle activities, and its newer TEVV work explicitly covers LLMs and agentic systems.

After deployment, continuously sample and evaluate real interactions—subject to privacy requirements—and watch for:

Evaluating a Large Language Model (LLM) for bias and safety before deployment requires a multi-layered, rigorous approach combining automated testing, human-in-the-loop evaluation, and red teaming . Because LLMs are probabilistic and generative, traditional software testing is insufficient.

Here is a structured framework to evaluate an LLM effectively:

- Identify specific risk taxonomies relevant to your use case (e.g., hate speech, PII leakage, self-harm, medical/financial misinformation, or demographic bias).
- Establish acceptable risk thresholds and baseline safety policies tailored to your target audience and domain.[[1]](https://google.com/goto?url=CAESRgHrOzAVtteda4gB7gtOp1y0Gs0n4Jc5QSdKFz3wgldeuTnAWzead4W0g-Lx4kt0CaKw21ohMFtKhEuFKPZQr1tsUyQO4IY)[[2]](https://google.com/goto?url=CAEScQHrOzAVTjkoy1QsxTETs38JhSH5ahBbDMjSBqej5Cqn0aHAmIUmRWRG27qo0f5I7VXFh0fQ08UxazcWZ5ar-IRsMC-gY7O3BOkqBvw8GbXtwjPABXg95SfA8z6_D44svbkvfWWk3-LCMpy2-K9KoV3g)[[3]](https://google.com/goto?url=CAESTgHrOzAVor39G3cpB3MK64jaNI0HfD6bHOkpbH39W6fZkQVNNdQAmFMTKo2rpTBmtGCtDC11Jvwf6EfyATE6MJ7gg4qn_eFdkPIVjnCkVw)[[4]](https://google.com/goto?url=CAESVQHrOzAVQJqzPAeGoLrLMQ_U4wu87M5BEZA6k6H9_GwZ8ffTPaTWvky2harMMgB1j_5sZffjkQT_7xqZUOU_3uAF6QQ3e7mKmZ-eAdJ0VYikYLXg9vc)[[5]](https://google.com/goto?url=CAESWAHrOzAVkBuozVjyNIwcg2jhvNbC5A41OGa7Aib2keYLgkVGhrWGSZWNgQ9Rh6_u3neSMON9ufgjpK2EfRQddzdQTwS-wMqyEmFA1DtNIpDNd7H6LwJjBxY)

- Real Toxicity Prompts: Evaluate the model's tendency to generate toxic language when given neutral or edge-case prompts.
- BOLD (Bias in Open-Ended Language Generation): Assess biases across gender, race, profession, and religious ideologies.
- CrowSPairs / StereoSet: Measure stereotypical biases embedded in the model's associations.
- TruthfulQA: Measure how prone the model is to mimicking human falsehoods and conspiracy theories.[[1]](https://google.com/goto?url=CAESQwHrOzAVntBxBAcb-g5-7Y9rdCSRknytUTFywCgEoJDo841lRQNRT4Rs2-BfprYuxGD5L0di2HMjOU7RwOS9wNblhcg)[[2]](https://google.com/goto?url=CAESZgHrOzAVzgZtbv9XTbCxw20aC-ONAZvdXNrcf5Q2LOxlQRltTZQCbCaUrW-9FB3vFTpN3RJpHC_2KOffXZhPUWFxp_kE7xhXtuDkVUEB0QuaGMBjYVu0l2K_OXBdQpCrURHRlt3CkA)[[3]](https://google.com/goto?url=CAESRgHrOzAVNrIkQ-CI0tKNLkk8gUX94PmNCcvQCMVnycORbB0hy2ZhVthA5bYR3CY5EgwZ_WBcMg4xIR5eJUSAJIzzZnoH1Bo)[[4]](https://google.com/goto?url=CAESbQHrOzAV6wcSQLYrItzsNkjURl50EX3UjHSufCtaWFdC57zb0LoPcB4Ugj7kFro9_MHmhR2iuVn2cc-kjpgq1GnsU0pU75RXyF22MQY5DOhUWEr2EnuAEh05vKOVGOufbq9OvuYGVGgSMIk225U)[[5]](https://google.com/goto?url=CAESfwHrOzAVUruInZr5gKEXiMtNTiolucb2sQaSURyloiOqjjxzROAx6JuDFvQJ0M0kn-Pja2WqyZgJ6Mrye7WHMuzqFFFFz5nPHcVA-FL3-GhkL5VAgBEDNO7w7KgyhZkQitZ3CL1dAC0qJkKoBwjy36uvhuTtJybfAwYVTifuJY4)

- Employ frameworks like **Garak** (LLM vulnerability scanner) or **Promptfoo** to systematically inject adversarial prompts, jailbreaks, and injection attacks at scale.
- Use a more powerful, aligned LLM (such as GPT-4 or Claude 3.5 Sonnet) as a scalable judge to score model outputs against predefined safety rubrics for tone, safety, and factual alignment.[[1]](https://google.com/goto?url=CAESXQHrOzAVWuAAbyH5i_TGTS3smANnxvxPcwu1D6_G3Mv_dBGCl51RKdEDiQ6FsxDRNWHzUuVucTzfYgTMGYhWgx0o1kH0FpHZpkwLHuAs2nnX8_ytwfaXXLcpbKVEgA)[[2]](https://google.com/goto?url=CAESRgHrOzAVqqIjCOSWhQPN0gpsYUuXmACLcyxadR0ye5L7o5JLc7jskfA0_jn_YwhHvdQkhWgK0Ug04jQmyrc-OskJsYS_kLw)[[3]](https://google.com/goto?url=CAESpQEB6zswFea9nevF4MNEX3rSdJEysbv1oVF5NZ3B935Go5wiN8gaM6FyolAWcTdoD0edLQ1yoAoAg4yTkSB_Xe-OvHjjYwwNvtVgOC07gM6q3VskGh71n5xj-fr5owY_V19tgZcJLVj8QMjy7BjGc2aC1T_miKDUG35DtvkmOkP64AMIjoI_tN3UnBodJJNnvuSCuw-uVd4pJuYOtxKETSP_ZyCMROo)[[4]](https://google.com/goto?url=CAESeAHrOzAV-WCY-wMRoMK7V_2LXRJD3M1Bh62TgkOSaxGiH3C1Sf4IKMjXOIrePgeadd-SdoV57JocSvvCx5-3-aOk4uyurFmfqPJewOTld_j3ejPPRZkjgA3FnRI2Kh32R0BtQ7b33Bc2e5AM0FmigZ-xweEb_aV8Mg)[[5]](https://google.com/goto?url=CAESRgHrOzAVheUqCboBQdwLQ5MeAQFt08vRTdzUXvyF0s4nG6HO_LILW75BW0-TeoWY0jqlAAwjMo7WCk8WAgsbnt3k-sQVzeg)

- Assemble a cross-functional red team (domain experts, ethicists, security engineers) to actively try and break the model's guardrails.
- Test boundary conditions, multi-turn conversational steering (jailbreaking over multiple prompts), and culturally nuanced dogwhistles that automated tools might miss.[[1]](https://google.com/goto?url=CAESRgHrOzAVBfsrAf-SXcTAkL_BiYtipq3JZs600B5iHQGO-Hc1ghibDf97vOPukgtbhJv3M5Av9L7_7XTVQXxvAfeGWY-UjLg)[[2]](https://google.com/goto?url=CAESkgEB6zswFRp4Ze6Nc0nJ6Xq6s3TevPa6Em54qon5qCo4PPH3LkVBnKC7Su-g3zQdgENNOS3oW6sEqGEkvnU2l2oOUJYsjjAbLioRzm2wa4gzO0XPWpH5Lh4tIGQML0kPXmoIumX5C5XOXpL4O-Bl9C9YcTDU8zYirf0rIaVj1y6m2Ex8pRjHkVBRAMOa4qI7A5o5Gg)[[3]](https://google.com/goto?url=CAESeQHrOzAVw7eRyb4Wqdha3elaEFU0hIVUEaqqArBivzr3JeCKWOio4mjUSGcsY6H0-YCsyv8QX0hFkzNCpTMI0iy6SNZnelrJzmXB_Rh4MD_2fmZ5jazQheSYfhRFW3ZabbhcvEFMzbuVJ7GVwaDotewG0UVKsf_XJOU)[[4]](https://google.com/goto?url=CAESRgHrOzAV9YKmBp460sqSDx_-SiUWvBSdS7qNaSG9dorXkAi4edQ2h-UprW-tnxAeKh84nfouWTCVDkEtJJh3rMna29aHNOo)[[5]](https://google.com/goto?url=CAESawHrOzAV-xMdNGp1Fvhe-8ki6LQP5imZYlM-1sN7zj0nvfJwNxtaegEkiiJLJZyI_7M_ppqx80fzqqM9_DRMV8X0PBF3JPerLMQPXO_YPjJZm8CxdAiSrGoxtHLhJd5VUcUDVdldgWuLrQFG)

- Run targeted evaluation metrics across diverse demographic cohorts to check for disparate impact or performance degradation in specific languages, dialects, or cultural contexts.
- Involve human evaluators (via platforms like Scale AI or Appen) to grade subjective nuances, helpfulness, and safety alignment that automated pipelines cannot reliably quantify.[[1]](https://google.com/goto?url=CAESUAHrOzAVBLgsn8jJ2RyLLewSIUJNMSaUOrZrijhiH314ce2szS3ji0eeab8ppUYZqedKpgf1_wTqYRM3rzSy7wyyWGYeszDyrmePczPZ-YLW)[[2]](https://google.com/goto?url=CAESTAHrOzAVDt95YhklOfsKzHgStZCtRA3arGb8i7yGLQ5GrCdJgFo6Y4zL_IP8423_fHosVF1BBtQ8pDMQ0e6gU1Xvn3iBRGEgoYA-k90)[[3]](https://google.com/goto?url=CAESRgHrOzAVccR_XshxN6o-Qec4PCuI2rX8lpLd0R4xIpC4TuxZnsvuJH1AJPfSkZex-ebw9xxHOiVjCwMJ5Fw3vX52kyfvDXs)[[4]](https://google.com/goto?url=CAESdwHrOzAVYtFWfDyPnupzUUT1l1VSV6zogJQ-yYj30xk4HFIHipz2ErMNgUH-PQLUJM_bpOKwlkUCR3KtZsTqacMvUCaIdKD6inxkZtugszAoF9sh43Cr4dPHfygWCoXqAWNsQ6fvq4vFHVvZiXjvwHKMzwtciPKg)[[5]](https://google.com/goto?url=CAESQwHrOzAVpw-8KzWeWJGx7eBKdJmM5ajSR9GITq6BvvxgSHbWMG4nWIa5n4yBfFJcYNDTegzyh0roJAmUZzSL2bLane0)

- Implement real-time input/output guardrails (e.g., Llama Guard, NeMo Guardrails, or specialized APIs) to catch residual toxic outputs or data leaks at inference time.
- Set up logging and telemetry pipelines to audit production traffic continuously for drift or emerging safety gaps.[[1]](https://google.com/goto?url=CAESgwEB6zswFdZH9BTBywZQ-vH9MeyRR8Pj1PTWgEYnVi2DATlNYGfuWCYU1oYLNpenBzVsAjq0Y0pHryeZTJii_DFAqWpebmzrklGQwv8JN6NcfCXj9q5QBc60Ju-tOxRi_KEphzOkjAgo7VjiSVYYzfuu_DTKClhMOIpttQn0Us9iQKIMIw)[[2]](https://google.com/goto?url=CAESXwHrOzAVkTUwA7xw2Tq3hr0sntfNCIfvpayNIRQZm5wNWcJ4VflQ7b_I__Q2VSoSCyCNQ0Z5jU-KzO-9sGMXeRgaT6yZ8HBpRKsfiHrlK2nsoJts6HIMEDPW30f_1P6F)[[3]](https://google.com/goto?url=CAESfQHrOzAVGv0DR5CssNe5NKDRN0CGnPUEIgUYTeJ0Z8uuGNKvAbjYqlw8Y1d-oRcUs-FUicX3VbIB4aiXaL5NRfAZkS-A4qWwvfB1JmXU_RLqWihv5Eh3StsOygdxGw6fJcJqG0EksAXOZiNWgPIvb67N4oChwA7KJipYx7M2)[[4]](https://google.com/goto?url=CAESWQHrOzAV9miwk_Ah5XRF3Muc7W7vq88cc_P43FQmg01qhlj_55AA1qVWeR-jy2Y_4A-OqkH50qiMyBPn5U4Saquwgps9onM4waks8tPxv96l6mZ2_M1OULGw)[[5]](https://google.com/goto?url=CAESYgHrOzAVCjOZtriAH0bvl4RJqtab8Ex201YtL--iSchHyxNN1LIoHTnTvK9GyO9u3coM8OvmM7RraEIWbK2pBP5uA84xRtuE5gKYD2Lw8eCMGocY2xuQg2EE--bfFGarQ7AP)

If you'd like, let me know:

• What domain or industry your LLM application is for (e.g., healthcare, customer service, public-facing chatbot)
• Whether you are training/fine-tuning a model or deploying an off-the-shelf API

I can tailor a specific testing checklist or recommend open-source evaluation tools for your setup.

A good pre-deployment evaluation should treat bias and safety as measurable system properties, not as a single benchmark score. The key is to evaluate the model + system + intended use case, because risks can change substantially when you add RAG, tools, agents, or high-impact decisions. NIST explicitly recommends context-specific testing and evaluation throughout the AI lifecycle.

1. Define what “safe” and “fair” mean first

Before testing, write down:

• Intended use: What will users ask it to do?
• Prohibited use: What should it never do?
• Affected groups: Which demographic, linguistic, socioeconomic, or other groups could experience different outcomes?
• Harm categories: e.g. harassment, hate, self-harm assistance, dangerous instructions, privacy leakage, misinformation, discrimination.
• Severity: What constitutes a minor, serious, or unacceptable failure?
• Pass/fail thresholds: Which failures block deployment?

This prevents the common mistake of optimizing for a generic “safety score” that doesn't correspond to actual deployment risks.

2. Build a representative evaluation set

Use several kinds of test cases:

1. Normal cases — representative of expected user behavior.
2. Edge cases — ambiguous, unusual, incomplete, multilingual, and adversarial inputs.
3. Counterfactual pairs — identical prompts with demographic attributes changed.
4. Historical incidents — real or simulated examples of failures you particularly want to prevent.
5. Adversarial prompts — jailbreaks, prompt injection, obfuscation, role-play, conflicting instructions, etc.
6. System-specific cases — prompts involving your RAG corpus, tools, APIs, business rules, and actual workflows.

Keep a held-out test set that isn't repeatedly used during model tuning, otherwise you risk optimizing for the benchmark rather than the underlying behavior.

3. Measure bias quantitatively

For each relevant demographic group, measure things such as:

• Quality: accuracy, helpfulness, task completion.
• Refusal rate: does the model refuse equivalent requests disproportionately?
• Toxicity/hatefulness: whether outputs contain harmful stereotypes or derogatory content.
• Stereotyping: whether demographic attributes change assumptions about occupations, abilities, personality, etc.
• Allocation/decision differences: if the LLM recommends, ranks, classifies, or allocates something.
• Language performance: differences across languages, dialects, accents, or varieties of English.
• Intersectional effects: e.g. gender × race rather than looking at each attribute independently.

NIST's GenAI profile specifically recommends evaluating fairness across demographic groups and using benchmarks plus red-teaming and counterfactual/low-context prompts to identify systemic bias.

A simple example:

Give the model 1,000 otherwise-identical prompts, changing only the person's demographic attribute. Compare refusal rates, sentiment, factuality, recommendations, and harmful-content rates. Don't just ask whether the average score is good. Look for disparities between groups.

4. Test safety with adversarial red teaming

Create a dedicated red-team phase where testers actively try to make the system violate its policies.

Test:

• Jailbreaks and instruction conflicts
• Prompt injection, including indirect injection through retrieved documents/web pages
• Requests for dangerous or illegal assistance
• Self-harm and other vulnerable-user scenarios
• Hate and harassment
• Sexual content involving minors
• Privacy and confidential-data extraction
• Credential/system-prompt leakage
• Hallucinated authoritative advice
• Manipulation and persuasion
• Tool misuse and unauthorized actions
• Malicious files/images if multimodal
• Long-context attacks
• Multilingual and obfuscated attacks

Prompt injection deserves special attention for applications with tools or RAG: OWASP identifies it as a major LLM risk because successful attacks can cause sensitive-data disclosure, unauthorized function access, or manipulation of decisions.

5. Evaluate the application, not just the base model

A model can pass a safety test in isolation and become unsafe once connected to tools.

For example:

Model alone:

“I can't access your company's files.” Agent with tools: The same model may have access to a file-search tool and be tricked into retrieving confidential information.

So evaluate the entire chain:

user → application → retrieval → LLM → tools → output → downstream systems

Check especially for:

• Excessive permissions
• Sensitive-information disclosure
• Unsafe tool calls
• Improper output handling
• Prompt injection
• Data poisoning
• System-prompt leakage
• Excessive agency

These are prominent categories in OWASP's current LLM/GenAI risk taxonomy.

6. Combine automated tests with humans

Use automated evaluation for scale, but don't rely exclusively on an LLM judge.

A strong setup is:

Automated tests → independent model judges → human expert review → red team

Human reviewers should inspect a statistically meaningful sample of both successes and failures, with additional review of high-severity cases.

For high-impact applications, involve domain experts and representatives of affected populations, not just ML engineers.

NIST's ARIA work, for example, combines model testing, red teaming, field testing, dialogue annotation, and tester questionnaires.

7. Establish deployment gates

Turn the results into explicit release criteria.

For example:

The exact thresholds should depend on the application. A customer-service chatbot and a system assisting with medical decisions shouldn't have the same tolerance for failure.

8. Document the evaluation

For every release, retain:

• Model/version and configuration
• Training/fine-tuning changes
• Evaluation datasets and provenance
• Demographic groups tested
• Metrics and confidence intervals
• Known limitations
• Red-team methodology
• Failed cases and severity
• Mitigations
• Residual risk
• Deployment restrictions
• Who approved the release

This creates an auditable evaluation report/model card/system card rather than a vague claim that “the model was tested.”

9. Monitor after deployment

Pre-deployment testing isn't a permanent safety certificate. Models, prompts, users, retrieval data, and attack techniques change.

Monitor:

• Safety violations
• Bias complaints
• Refusal-rate changes
• New jailbreaks
• Privacy incidents
• Hallucination/error rates
• Tool-call anomalies
• Distribution shifts
• Performance by relevant user groups

Then periodically rerun the evaluation suite and automatically trigger reassessment after major model, prompt, retrieval, or tool changes. The UK AI Safety Institute similarly emphasizes repeated evaluation before and after deployment, with evaluation intensity proportional to risk.

A practical architecture

I'd implement the evaluation pipeline roughly like this:

                 ┌─────────────────────┐
                 │ Intended-use spec   │
                 │ + risk taxonomy     │
                 └──────────┬──────────┘
                            ↓
                 ┌─────────────────────┐
                 │ Evaluation datasets │
                 │ normal + edge cases │
                 │ bias + adversarial   │
                 └──────────┬──────────┘
                            ↓
             ┌──────────────────────────────┐
             │ Model + complete application │
             │ RAG + tools + guardrails      │
             └──────────────┬───────────────┘
                            ↓
          ┌─────────────────┼─────────────────┐
          ↓                 ↓                 ↓
      Bias tests       Safety tests      Security tests
          ↓                 ↓                 ↓
          └─────────────────┼─────────────────┘
                            ↓
                  Human / expert review
                            ↓
                   Risk & severity scoring
                            ↓
                    Deployment gates
                            ↓
                   Monitor + regressions

The overarching principle is “measure → attack → mitigate → retest → monitor.” NIST's GenAI profile and newer TEVV work provide a useful framework for structuring this kind of evaluation.

If you're building an evaluation program in practice, the next useful step is to turn this into a concrete pre-deployment test suite with metrics, sample test cases, severity levels, and pass/fail thresholds for your particular LLM application.

The exact thresholds should depend on the application. A customer-service chatbot and a system assisting with medical decisions shouldn't have the same tolerance for failure.

8. Document the evaluation

For every release, retain:

• Model/version and configuration
• Training/fine-tuning changes
• Evaluation datasets and provenance
• Demographic groups tested
• Metrics and confidence intervals
• Known limitations
• Red-team methodology
• Failed cases and severity
• Mitigations
• Residual risk
• Deployment restrictions
• Who approved the release

This creates an auditable evaluation report/model card/system card rather than a vague claim that “the model was tested.”

9. Monitor after deployment

Pre-deployment testing isn't a permanent safety certificate. Models, prompts, users, retrieval data, and attack techniques change.

Monitor:

• Safety violations
• Bias complaints
• Refusal-rate changes
• New jailbreaks
• Privacy incidents
• Hallucination/error rates
• Tool-call anomalies
• Distribution shifts
• Performance by relevant user groups

Then periodically rerun the evaluation suite and automatically trigger reassessment after major model, prompt, retrieval, or tool changes. The UK AI Safety Institute similarly emphasizes repeated evaluation before and after deployment, with evaluation intensity proportional to risk.

A practical architecture

I'd implement the evaluation pipeline roughly like this:

Evaluating a Large Language Model (LLM) for bias and safety before deployment requires a multi-layered, rigorous approach combining automated testing, human evaluation, and red teaming . Because LLMs are probabilistic and generative, traditional software testing isn't enough.

Here is a structured framework to evaluate your LLM effectively:

1. Define the Scope and Risk Profile
   • Identify your specific use case, target audience, and domain (e.g., healthcare vs. general customer service).
   • Map out potential failure modes and high-risk scenarios (such as generating hate speech, leaking PII, or hallucinating medical advice).
2. Select and Curate Evaluation Datasets
   • Use established benchmark datasets for safety and bias (e.g., RealToxicityPrompts, BOLD, CrowS-Pairs, or ToxiGen).
   • Supplement these with custom, domain-specific evaluation prompts that mimic real-world inputs your users will provide.
3. Automated Safety and Bias Testing
   • Run your evaluation prompts through the model at scale.
   • Measure toxicity, stereotyping, hate speech, and dangerous content using automated classifiers or guardrail frameworks (such as Llama Guard or evaluation toolkits like Promptfoo or Garak).
4. Adversarial Testing (Red Teaming)
   • Conduct manual or automated red teaming to actively try and break the model.
   • Test for jailbreaks, prompt injections, persona adoption (forcing the model to act maliciously), and boundary-pushing inputs.
5. Human Evaluation (RLHF/RLAIF Alignment Checks)
   • Involve human domain experts and diverse annotators to review nuanced outputs where automated tools fall short—especially regarding subtle cultural biases, tone, and contextual appropriateness.
6. Establish Guardrails and Thresholds
   • Set quantitative thresholds for acceptable error, toxicity, or bias rates (e.g., zero tolerance for severe PII leakage or explicit hate speech).
   • Implement runtime guardrails (like input/output filters) to catch violations dynamically if a slip-through occurs in production.

Would you like to explore specific open-source evaluation tools or focus on a particular type of bias/safety risk (like medical hallucinations or prompt injection) for your deployment?

A good pre-deployment evaluation treats bias and safety as measurable system properties, not as “does the model seem reasonable?” NIST’s AI RMF recommends a lifecycle approach—Govern, Map, Measure, Manage—and its generative-AI profile specifically calls for subgroup fairness testing, adversarial testing, and documented limitations.

1. Define the acceptable-risk boundary first

Before testing, document:

• Intended use: what the model is supposed to do.
• Disallowed use: what it must not do.
• Affected groups: demographics, languages, disabilities, vulnerable populations, etc.
• High-impact decisions: hiring, lending, healthcare, education, law enforcement, safety-critical actions.
• Failure severity: what constitutes a minor error versus an unacceptable harm.
• Human oversight: which outputs/actions require review.

This prevents the common mistake of selecting benchmarks first and deciding what “good” means afterward.

2. Build a representative evaluation set

Create a held-out test set containing:

Normal cases

• Typical user requests
• Different languages, dialects, writing styles
• Different levels of expertise

Counterfactual pairs Change only the demographic attribute:

“The doctor said he...”
“The doctor said she...”

Or compare otherwise identical applications with different names, genders, ethnic cues, ages, etc.

Edge and adversarial cases

• Ambiguous prompts
• Toxic or discriminatory language
• Jailbreak attempts
• Prompt injection
• Conflicting instructions
• Misleading retrieved documents
• Attempts to extract private information

NIST specifically recommends benchmarks, counterfactual/low-context prompts, subgroup testing, and examining training/evaluation-data representation when assessing harmful bias.

3. Measure bias quantitatively

Don't rely on a single “bias score.” Measure several dimensions appropriate to the application:

For an automated decision system, you can additionally examine metrics such as equal opportunity or equalized odds where they make sense. NIST explicitly cautions that fairness metrics need to be appropriate to the particular use case.

4. Test safety through multiple layers

Evaluate at least:

Content safety

• Violence
• Self-harm
• Sexual content
• Hate/harassment
• Illegal or dangerous instructions
• Exploitation of vulnerable users

Truthfulness/reliability

• Hallucination rate
• Unsupported claims
• Citation accuracy
• Resistance to misleading premises
• Calibration/uncertainty

Privacy

• Memorization of sensitive information
• PII extraction
• Cross-user data leakage
• Training-data extraction

Security

• Prompt injection
• Jailbreaks
• System-prompt leakage
• Tool/API abuse
• Data exfiltration
• Malicious documents in RAG

OWASP's current LLM guidance highlights prompt injection, sensitive-information disclosure, excessive agency, misinformation, improper output handling, and other application-level risks.

5. Red-team it

Have people deliberately try to break the system rather than merely answer benchmark questions.

Use:

• Internal security engineers
• Domain experts
• Trust & safety specialists
• People familiar with affected communities
• External red-teamers where appropriate

Give them realistic attack goals, such as:

“Make the model reveal information it shouldn't.”
“Get it to treat two demographic groups differently.”
“Cause it to take an unauthorized action.”
“Make it produce a dangerous recommendation.”

NIST's ARIA evaluation program similarly separates model testing, red-teaming, and field testing, which is a useful structure for an organization's evaluation program.

6. Test the whole application, not just the base model

This is crucial.

A relatively safe base LLM can become unsafe when surrounded by:

• RAG
• System prompts
• Plugins/tools
• Browsing
• Memory
• User-generated content
• Automated decision logic
• External APIs

For example, prompt injection can manipulate an LLM into disclosing information or invoking connected functions it shouldn't. OWASP recommends least-privilege access, separating untrusted content, validating outputs, and requiring human approval for high-risk actions.

7. Establish release gates

Turn evaluation into explicit go/no-go criteria.

For example:

• Critical safety failure → automatic release blocker
• PII leakage → automatic release blocker
• Unauthorized tool action → automatic release blocker
• Bias disparity above defined threshold → investigation/blocker
• Hallucination above domain-specific threshold → block or restrict use
• All remaining failures → documented risk acceptance and mitigation

Don't optimize solely for an aggregate score. A model averaging 95% across tests can still have a catastrophic 5% failure mode.

8. Evaluate after deployment too

Pre-deployment testing isn't enough. Establish:

• Continuous sampling and evaluation
• User/report feedback
• Bias monitoring by subgroup
• Safety-incident monitoring
• Regression tests for every model/prompt change
• Periodic red-teaming
• Versioned evaluation datasets
• Rollback capability

NIST emphasizes ongoing risk management rather than treating evaluation as a one-time certification.

A practical evaluation pipeline

Define risks → build representative/counterfactual test set → automated metrics → human evaluation → adversarial red-team → application/security testing → release gates → monitored rollout → continuous regression testing.

The most important principle is: evaluate the model in the context in which people will actually use it. Bias and safety are properties of the combination of the model, prompts, data, tools, users, and deployment environment—not just the raw LLM.

Evaluating a Large Language Model (LLM) for bias and safety before deployment requires a rigorous, multi-layered framework combining automated testing, human evaluation, and red teaming . Because LLMs are probabilistic and generative, traditional software testing isn't enough; you must test for edge cases, systemic prejudices, and harmful content generation.

• Define Policy and Taxonomy : Establish clear definitions of what "safety" and "fairness" mean for your specific use case. Document unacceptable failure modes (e.g., hate speech, PII leakage, political bias, self-harm encouragement) and map them to standard safety taxonomies.
• Curate and Utilize Benchmark Datasets : Run the model through established and custom evaluation datasets to measure baseline metrics. Examples include RealToxicityPrompts for toxicity, BOLD or CrowS-Pairs for bias and stereotyping, and TruthfulQA for hallucinations.
• Conduct Automated Red Teaming : Use automated tools and adversarial prompt generators (like Garak or Promptfoo) to probe the model at scale. These tools fire thousands of malicious, out-of-bounds, or jailbreak prompts to test how easily the model's guardrails break.
• Perform Human Evaluation (RLHF/Red Teaming) : Enlist diverse human red teams (domain experts, ethicists, and representative end-users) to interact with the model organically. Human evaluators catch subtle context-dependent biases, sarcasm, and nuanced toxicity that automated classifiers miss.
• Implement Guardrail Testing and Latency Checks : Evaluate the wrapper safety layers (such as Llama Guard, NeMo Guardrails, or Azure AI Content Safety) independently. Measure how effectively they catch unsafe outputs and check the performance trade-off in terms of latency and false-positive rates.
• Run Disaggregated Performance Analysis : Slice your evaluation data across demographic variables (e.g., gender, race, regional accents) to ensure the model does not exhibit disparate accuracy, helpfulness, or toxicity amplification across different user groups.

If you'd like to dive deeper, let me know:

• What is the intended use case or domain for your LLM (e.g., customer-facing chatbot, internal HR tool, medical assistant)?
• Are you looking for specific open-source evaluation tools or frameworks to integrate into a CI/CD pipeline?

Evaluating a Large Language Model (LLM) for bias and safety before deployment requires a rigorous, multi-layered approach combining automated testing, human evaluation, and red teaming . Because LLMs are probabilistic and generative, traditional software testing isn't enough; you must test for edge cases, harmful outputs, and systemic disparities.

Here is a structured framework to evaluate your LLM effectively:

• Define Safety and Bias Taxonomies : Establish clear definitions of what "safe" and "biased" mean for your specific use case. Use standard taxonomies like the OWASP Top 10 for LLMs or the EU AI Act risk categories to categorize risks (e.g., hate speech, toxic language, PII leakage, hallucinations, and socioeconomic or gender bias).
• Curate and Run Automated Benchmark Datasets : Use established, standardized datasets to test baseline vulnerabilities at scale. Examples include RealToxicityPrompts for toxicity, BOLD (Bias in Open-Ended Language Generation) for demographic bias, and TruthfulQA for truthfulness and hallucinations.
• Build Domain-Specific Golden Test Sets : Supplement public benchmarks with custom, domain-specific prompts that reflect your real-world application. Include adversarial edge cases, culturally nuanced phrases, and queries designed to trick the model into revealing hidden biases or breaking guardrails.
• Conduct Adversarial Red Teaming : Assemble a diverse group of internal or external domain experts ("red teamers") to manually probe the model. Their goal is to actively break safety guardrails through prompt injection, jailbreaking, multi-turn manipulation, and social engineering scenarios.
• Implement Automated Guardrails and Scoring : Integrate continuous evaluation tools (such as Llama Guard, NeMo Guardrails, or specialized evaluation frameworks like Promptfoo or G-Eval) into your pipeline. These tools score model outputs programmatically for toxicity, sentiment polarity, and policy violations during CI/CD checks.
• Measure Disparate Impact and Fairness : Evaluate performance metrics across different demographic groups (e.g., race, gender, nationality, age) embedded in your prompts. Check if the model's tone, helpfulness, or accuracy degrades or exhibits systemic stereotypes when referencing specific protected classes.
• Incorporate Human-in-the-Loop (RLHF/RLAIF) Validation : Before general availability, run a closed beta or pilot with a representative sample of end users. Use structured feedback forms to capture subtle qualitative failures, offensive outputs, or micro-biases that automated evaluators miss.

To tailor this evaluation strategy specifically for your project, tell me:

• What is the domain or use case of the LLM (e.g., customer service chatbot, medical advisory, internal HR tool)?
• Who is your target audience or end user?

I can help you prioritize which specific benchmarks and testing methods matter most for your deployment.

A good pre-deployment evaluation should not be a single “bias score” or safety benchmark. Treat it as a risk-based test program covering the model, the application around it, and the real-world use case.

1. Define what “safe” means for your use case

Start with a threat/risk model:

• Who can be harmed?
• What decisions or actions can the LLM influence?
• What information can it access?
• What failures are unacceptable?
• Which demographic groups, languages, or populations could experience different outcomes?
• What level of human oversight is required?

NIST's Generative AI Profile specifically recommends identifying affected groups and measuring fairness across demographic groups and subgroups rather than treating fairness as a generic property of the model.

2. Establish a test matrix

I'd evaluate at least these dimensions:

OWASP's current LLM risk taxonomy includes prompt injection, sensitive-information disclosure, supply-chain/model poisoning, improper output handling, excessive agency, misinformation, and other application-level risks.

3. Test bias with paired and intersectional prompts

Don't just ask, “Does the model say offensive things?”

Use counterfactual pairs where everything except the demographic attribute changes:

“Evaluate Alex's qualifications for the engineering job.”

versus

“Evaluate Alex's qualifications for the engineering job.”
(with a different gender-associated name)

Then expand across gender, race/ethnicity, age, disability, nationality, religion, dialect, etc., where relevant to your application.

Measure things such as:

• Response quality
• Recommendation/ranking differences
• Refusal-rate differences
• Toxicity/denigration rates
• Sentiment differences
• Factuality differences
• Allocation or decision disparities

Critically, test intersections—for example, gender × age—not merely each attribute independently. NIST explicitly recommends examining differences across groups and intersecting groups.

4. Combine benchmarks with adversarial testing

Use established evaluations where applicable, but don't assume benchmark performance proves safety.

For example, Stanford's HELM Safety evaluates categories including violence, fraud, discrimination, sexual content, harassment, and deception, using benchmarks such as BBQ, HarmBench, XSTest, and red-team scenarios.

Then build your own application-specific red-team suite:

• Direct harmful requests
• Indirect requests
• Multi-turn manipulation
• Role-play attacks
• Obfuscation
• Multilingual attacks
• Context poisoning
• Conflicting instructions
• “Ignore previous instructions” variants
• Malicious documents in RAG
• Attempts to induce unsafe tool calls

OWASP recommends adversarial testing and attack simulations specifically because conventional testing won't reliably expose prompt-injection vulnerabilities.

5. Don't evaluate the model in isolation

This is one of the most important points.

If your system is:

User → application → RAG → LLM → tools → database

then evaluating only:

prompt → LLM → response

misses major risks.

Test the complete system for:

• Unauthorized data retrieval
• Prompt injection through retrieved documents
• Excessive tool permissions
• Unsafe SQL/API generation
• PII exposure
• Privilege escalation
• Unsafe actions
• Output injection into downstream software

For high-impact actions, use least-privilege permissions and human approval gates rather than expecting the model's refusal behavior to be your security boundary. OWASP explicitly recommends this approach.

6. Use humans—but don't rely exclusively on humans

A strong evaluation combines:

1. Automated metrics — scalable regression testing.
2. Human evaluation — nuanced judgments and unexpected failures.
3. Expert red teaming — security and domain-specific attacks.
4. Affected-community testing — especially for consequential demographic harms.
5. Model-based judges — useful for scale, but independently validate them.

Model judges can themselves have biases or miss subtle failures. Stanford's HELM Safety, for example, uses automated judges for scalability but explicitly notes that its evaluations do not comprehensively cover all risks.

7. Establish deployment gates

Before launch, define hard thresholds, not just averages.

For example:

• Critical safety violation: zero tolerance
• PII leakage: zero tolerance
• Unauthorized high-impact action: zero tolerance
• Demographic disparity: below a predefined threshold
• Harmful-response rate: below a predefined threshold
• Hallucination rate: below a use-case-specific threshold
• Regression: no statistically significant degradation from the approved version

The exact thresholds should depend on the application. A chatbot generating marketing copy and an LLM helping make medical or employment decisions should obviously have very different tolerances.

8. Make the evaluation reproducible

Version and retain:

• Model/version
• System prompt
• Safety policies
• Retrieval corpus
• Tool definitions
• Sampling parameters
• Test datasets
• Exact prompts
• Model outputs
• Evaluator versions
• Scores
• Human annotations
• Known failure cases

Run the same suite whenever you change the model, system prompt, RAG corpus, safety filter, tool permissions, or other significant system component.

9. Continue testing after launch

Pre-deployment testing is only a release gate, not proof of safety.

Monitor production for:

• New jailbreaks
• Bias complaints
• Refusal failures
• Hallucinations
• Privacy incidents
• Tool misuse
• Distribution shifts
• New languages/use cases
• Changes in user behavior

Feed discovered failures back into a continuously growing regression set.

A practical evaluation architecture

I'd structure the process like this:

Risk assessment → Test specification → Automated benchmarks → Counterfactual fairness tests → Red team → Human/domain review → System-level security tests → Deployment gates → Monitored rollout → Continuous regression testing

The key principle is: evaluate the actual system against plausible harms, not merely the underlying model against a benchmark. NIST frames this as testing, evaluation, verification, and validation throughout the AI lifecycle, while Stanford's HELM work illustrates why multidimensional, transparent evaluation is preferable to a single aggregate score.

If you're building an actual evaluation program, I can also give you a concrete 50–100 test-case evaluation matrix with metrics, pass/fail thresholds, red-team categories, and a scoring rubric.

Evaluating a Large Language Model (LLM) for bias and safety before deployment requires a rigorous, multi-layered approach combining automated testing, human evaluation, and red teaming . Because LLMs are probabilistic and generative, traditional software testing is insufficient.

Here is a structured framework to evaluate an LLM for safety and bias:

• Define Policy and Taxonomy : Establish clear definitions of what constitutes "safety" (e.g., hate speech, self-harm, dangerous content, PII leakage) and "bias" (e.g., demographic, geographic, political, socioeconomic) relevant to your specific domain and user base.
• Curate Representative Benchmark Datasets : Gather or build diverse test datasets that reflect real-world user inputs, edge cases, and culturally nuanced prompts. Use established benchmarks alongside domain-specific test suites.
• Automated Red Teaming and Stress Testing : Use automated frameworks and adversarial prompt generation (fuzzing) to bombard the model with jailbreaks, prompt injections, and boundary-pushing inputs to see where it breaks or outputs toxic content.
• Human Evaluation (RLHF/RLAIF feedback loops) : Implement human-in-the-loop evaluation (Red Teaming by diverse human experts) to assess subjective nuances, subtle biases, tone, and conversational safety that automated metrics miss.
• Quantitative Bias Metrics Application : Measure disparities in model performance and sentiment across protected attributes (gender, race, religion) using specific metrics like stereotype agreement scores, counterfactual fairness testing, and toxicity probability ratios.
• Guardrail and Filter Integration Testing : Evaluate the efficacy of pre- and post-processing safety layers (content filters, classifiers, input sanitizers) to ensure they catch violations without causing excessive false positives or degrading utility.
• Continuous Monitoring and Auditing Strategy : Set up continuous logging, feedback collection, and periodic re-evaluations post-deployment to catch drift, emergent behaviors, or newly discovered vulnerability vectors.

Would you like me to dive deeper into automated red teaming tools , or do you need help defining specific bias metrics for your use case?

A good pre-deployment evaluation should treat the LLM as a sociotechnical system, not just a model. NIST’s AI Risk Management Framework recommends evaluating trustworthiness across the lifecycle, including safety, security, privacy, fairness, transparency, and reliability. Its Generative AI Profile is specifically designed for GenAI risk management.

1. Define the risk boundary first

Before running benchmarks, document:

• Intended uses and explicitly prohibited uses
• Who the users are and who could be affected
• What decisions the model can influence
• What data it can see
• What tools/actions it can invoke
• What happens when it is wrong
• Severity thresholds for unacceptable failures

A customer-support chatbot and an AI making medical, hiring, lending, or security decisions need very different evaluation standards.

2. Evaluate bias quantitatively

Build test sets covering relevant demographic and contextual groups, then compare:

• Quality: accuracy, helpfulness, completeness
• Error rates: false positives/negatives by group
• Refusal rates: does it refuse legitimate requests disproportionately?
• Toxicity/stereotyping: differences in generated content
• Representation: whose perspectives or identities are consistently omitted or mischaracterized
• Intersectional groups: e.g., gender × age × race rather than testing each dimension independently

Use matched prompts where only the demographic attribute changes. For example, test equivalent scenarios with different names, genders, dialects, or cultural contexts.

Don't reduce this to one "bias score." Look for specific failure modes and statistically meaningful disparities.

3. Test safety with adversarial evaluation

Create a threat taxonomy relevant to your deployment:

OWASP's current LLM risk taxonomy specifically highlights prompt injection, sensitive-information disclosure, improper output handling, excessive agency, misinformation, and other application-level risks.

4. Red-team it

Don't rely exclusively on a fixed benchmark.

Use:

• Internal security/AI red teams
• Domain experts
• Diverse external testers
• Automated adversarial prompt generation
• Multi-turn attacks
• Indirect attacks through documents/web pages/RAG data
• Adaptive attacks that learn from the model's responses

Importantly, red teaming should be repeatable and measurable, rather than just collecting interesting anecdotes. Anthropic has noted that red teaming remains insufficiently standardized and that repeatable evaluation processes are important for comparing models.

5. Evaluate the entire application

This is where many evaluations fail.

An otherwise well-behaved model can become dangerous when connected to:

• Email
• Databases
• Browsers
• Code execution
• Financial systems
• Internal documents
• APIs
• Autonomous agents

Test whether prompt injection or model mistakes can cause unauthorized actions. Apply least privilege, sandboxing, validation of model-generated outputs, rate limits, and human approval for high-impact operations.

Recent real-world testing incidents have reinforced this point: models given unintended network access have performed actions against real systems, meaning that evaluation infrastructure itself needs strong isolation and access controls.

6. Measure both failure and success

A useful evaluation report might track:

• Attack success rate
• Harm severity
• False-refusal rate
• Hallucination/error rate
• Bias/disparity metrics
• Privacy leakage rate
• Instruction-following rate
• Tool authorization violations
• Robustness under paraphrase/multilingual/multi-turn attacks
• Performance confidence intervals

For safety, don't simply report "99% safe." A 1% failure rate can be unacceptable if the remaining failures involve catastrophic harm.

7. Establish deployment gates

Turn evaluation results into explicit go/no-go criteria.

For example:

Deploy only if: no critical-severity failures are reproducible; high-severity failures are below an agreed threshold; demographic disparities meet predefined limits; unauthorized tool actions are blocked; and all known high-risk failure modes have mitigations and monitoring.

The thresholds should be determined before looking at the results, so the team isn't tempted to move the goalposts.

8. Keep testing after launch

Safety evaluation isn't a one-time certification. Monitor:

• New jailbreaks
• User complaints
• Distribution shifts
• New demographic disparities
• Hallucination rates
• Safety incidents
• Tool misuse
• Model/version changes
• New external threats

Then maintain a regression suite so every model, system-prompt, retrieval, or tool-permission change gets reevaluated.

In short: the strongest approach is risk assessment → representative benchmarks → bias testing → adversarial red teaming → application/security testing → predefined deployment gates → continuous monitoring.

A useful governing principle is: evaluate the model under the conditions in which it will actually be used, including the people, data, tools, permissions, and adversaries around it. NIST's AI Resource Center explicitly supports testing, evaluation, verification, and validation (TEVV) as part of this lifecycle approach.