I am planning to audit our algorithms for racial and gender bias. Who offers independent algorithmic auditing?

Independent algorithmic auditing is an emerging and specialized field, especially with new regulations (like NYC Local Law 144 for automated employment decisions) and growing oversight from the EEOC. Third-party firms specialize in evaluating datasets, model logic, and outcomes for racial, ethnic, and gender disparities.

Prominent independent organizations and consulting groups that offer algorithmic bias and responsible AI auditing include:

• ORCAA : A dedicated algorithmic auditing consultancy that evaluates software systems for safety and fairness across regulated industries. They are an inaugural member of the U.S. AI Safety Institute Consortium.
• BABL AI : Specializes in independent third-party bias audits and algorithmic impact assessments, aligning evaluations with standards like the EU AI Act and EEOC frameworks.
• Trilateral Research : Offers compliance and independent third-party bias auditing services tailored to automated employment decision tools (AEDTs) and data protection impact assessments.
• Major Accounting and Management Firms (Big Four) : Firms like Deloitte, PwC, EY, and KPMG have dedicated Responsible AI and algorithmic trust practices that perform independent third-party fairness and bias assessments.
• Specialized Technical & Legal Consultancies : Boutique AI governance firms (such as Warden AI or specialized tech-focused legal consultancies) routinely partner with internal compliance teams to run diagnostic fairness tests.

To help narrow down which type of auditor fits your needs, could you share:

• What industry or use case is the algorithm for (e.g., hiring, lending, healthcare)?
• Are you looking for an audit to meet specific regulations (like NYC Local Law 144) or for internal governance?

Yes. There is now a meaningful market of firms that perform independent algorithmic/AI audits, including testing specifically for racial, gender, and intersectional disparities.

Firms worth considering

• babl.ai — A particularly interesting option if you want a genuinely independent audit rather than a software vendor's self-assessment. BABL describes itself as an independent AI auditing firm and has conducted audits covering gender, race/ethnicity, intersectional groups, governance, and risk assessment. Its audit framework is based on an auditing model published at ACM FAccT.
• holisticai.com — Provides independent AI auditing and fairness assessments, including race/ethnicity and sex testing for employment systems. It has been used for NYC Local Law 144 audits.
• orcaarisk.com — Offers broader algorithmic audits, not just hiring bias checks. Its work covers generative AI, predictive models, automated decision systems, and facial recognition, with quantitative testing for gender and race/ethnicity bias.
• fairnow.ai — Offers third-party bias audits as well as ongoing automated fairness monitoring. Its assessments can examine multiple demographic groups and intersectional effects; it also has a synthetic-data approach when demographic data are limited.
• fairgap.com — A newer specialist that explicitly positions itself as an independent third-party auditor with no AI-tooling-vendor relationships and no remediation/consulting services attached to its audits. It says its audits cover race, gender and applicable regulatory requirements.
• https://orcaarisk.com/what-we-do — Particularly worth looking at if you need to audit a model outside hiring; it says it can perform quantitative bias testing using live or test data and can use inference methods when demographic data aren't available.

What I'd look for in an auditor

For an organizational audit, I wouldn't select a firm merely because it advertises "AI fairness." Ask prospective auditors whether they will independently perform:

1. Outcome/disparate-impact testing by race/ethnicity and gender.
2. Intersectional analysis (e.g., race × gender), rather than treating race and gender independently.
3. Statistical significance and confidence intervals, not just a pass/fail 80% rule.
4. Proxy-variable analysis—whether apparently neutral variables reproduce protected-class disparities.
5. Data and model validation, including whether the audit population is representative.
6. Subgroup analysis for smaller populations and missing demographic data.
7. Documentation of methodology and limitations sufficient for an outside party to reproduce the conclusions.
8. True auditor independence—ideally they don't sell the same system, build the model, or make their audit contingent on purchasing remediation software.

That last point matters. For example, FairGap explicitly says it does not provide consulting, design, or remediation services for the systems it audits, which is a stronger independence model than an organization that both sells an AI-governance platform and audits its customers.

If you tell me what kind of algorithms you're auditing (e.g., hiring, lending, healthcare, advertising, facial recognition, pricing, criminal justice, general-purpose ML), I can narrow this to 5–7 credible independent auditors and compare their methodologies, independence, geographic coverage, and likely fit.

Yes. There are several firms that explicitly offer independent third-party algorithmic/bias audits, including audits of racial and gender disparities.

Firms worth considering

• BABL AI — A specialist independent AI auditing firm. Its published audits include testing for disparate impact across gender, race/ethnicity, and intersectional groups, as well as governance and risk assessment. Its work has been used for NYC Local Law 144 audits.
• ORCAA (O'Neil Risk Consulting & Algorithmic Auditing) — Focuses specifically on algorithmic auditing and fairness. It audits predictive models, automated decision systems, generative AI and facial recognition, and offers quantitative bias testing for race/ethnicity and gender.
• Holistic AI — Provides independent AI assurance, governance and bias/fairness assessments, including employment-related algorithmic systems. For example, Vervoe identifies Holistic AI as its independent auditor for race/ethnicity and sex disparate-impact testing.
• FairGap — A newer specialist firm that launched an explicitly independent algorithmic-bias audit service in 2026. It says it does not have commercial relationships with AI-tool vendors and does not provide consulting/remediation for systems it audits, which is an unusually strong independence position.
• AI Audits — Offers independent AI compliance and fairness testing specifically covering gender, race and socioeconomic factors, along with transparency and governance audits.

What I'd look for in your RFP

Don't just ask whether a firm performs an "AI bias audit." For a genuinely independent audit, I'd require:

1. Independence: no development, implementation, optimization, or remediation work on the system being audited.
2. Protected-group analysis: race/ethnicity and gender individually and intersectionally (e.g., Black women), rather than only aggregate fairness.
3. Outcome-level testing: selection rates, false-positive/false-negative rates, error rates, calibration, ranking effects, and appropriate disparate-impact/statistical tests.
4. Data audit: examination of representation, missing demographic data, proxies, labeling quality, and historical bias in training/evaluation data.
5. Testing beyond the model: audit the full decision pipeline—data → model → threshold/ranking → human decision → outcome.
6. Predefined methodology: the auditor should specify metrics and pass/fail criteria before seeing the results.
7. Reproducibility: retain the test datasets, code/configuration, sampling methodology, and statistical assumptions sufficiently for another party to reproduce the analysis.
8. Publicly defensible report: methodology, limitations, subgroup results, and conflicts of interest—not simply a statement that the model "passed."

NIST's AI Risk Management Framework similarly emphasizes independent assessors, documentation of test sets and metrics, and involvement of external experts and affected communities in appropriate evaluations.

One important distinction: NIST itself has an AI Technology Evaluation program, but its current AITE program is primarily a research/evaluation testbed rather than a conventional commercial bias-audit service.

If you tell me what kind of algorithm you're auditing (hiring, lending, insurance, healthcare, criminal justice, advertising, generative AI, etc.), I can narrow this to the 3–5 strongest independent auditors for that particular use case and compare their methodologies, independence, certifications, and likely fit.

When planning an independent audit for racial and gender bias in your algorithms, several specialized third-party firms, consultancies, and standard-setting organizations focus explicitly on algorithmic fairness, AI ethics, and compliance.

Specialized Algorithmic Auditing & AI Ethics Firms

• ORCAA (O'Neil Risk Consulting & Algorithmic Auditing)
  • Focus Areas: Automated decision systems, hiring algorithms, predictive scoring, and generative AI.
  • Services: Comprehensive algorithmic audits, bias audit reports assessing live deployments or test datasets, and risk mitigation frameworks . Founded by mathematician and author Cathy O'Neil.
• • Focus Areas: HR tech, legal compliance (such as NYC Local Law 144 for employment tools), and emerging global AI regulations.
  • Services: Independent algorithmic impact assessments (AIAs), EEOC AI bias audits, and compliance validation against frameworks like the NIST AI RMF and EU AI Act using certified third-party auditors.
• • Focus Areas: Independent audit programs, governance, bias mitigation, and privacy certification.
  • Services: Offers a rigorous, standardized independent audit framework for AI and autonomous systems (AAIS) aimed at establishing trust, ethics, and equity compliance.

Major Professional Services & Consultancies

If you require enterprise-scale governance alongside technical bias testing, major global accounting and consulting firms have dedicated Responsible AI practices:

• • Offers framework evaluations, algorithmic impact assessments, and bias mitigation strategies integrated into broader enterprise risk management.
• • Provides assessment tools and advisory services focused on fairness, transparency, accountability, and regulatory compliance for machine learning models.
• • Delivers diagnostic tools and third-party validation to test automated models for disparate impact and demographic skew.

If you'd like to narrow this down, please let me know:

• What industry or use case is your algorithm for (e.g., hiring, lending, healthcare, marketing)?
• Are you aiming for regulatory compliance (like NYC Local Law 144 or EEOC guidelines) or a general fairness/ethics evaluation?

Yes. There is now a meaningful independent-auditing market, although “independent” can mean different things, so I’d vet the auditor’s conflicts of interest and methodology carefully. A 2023 field scan of the audit ecosystem specifically cautioned that AI-audit claims can be difficult to compare and recommended stronger standards and auditor accreditation.

Firms worth considering

• O'Neil Risk Consulting & Algorithmic Auditing (ORCAA) — A specialist algorithmic-auditing firm. It audits automated decision systems, predictive models, generative AI, and facial-recognition systems, with explicit attention to discrimination, fairness, and performance. It also offers quantitative race/ethnicity and gender bias testing.
• BABL AI — Particularly relevant if you need a formal third-party bias audit. Its published work includes independent audits covering gender, race/ethnicity, intersectional groups, governance, and risk assessment; its audit framework was presented at FAccT.
• Holistic AI — Provides independent bias/fairness auditing, including employment-system audits under NYC Local Law 144.
• FairGap — A newer specialist firm explicitly positioning itself as an independent third-party bias auditor, without AI-tooling-vendor relationships or remediation consulting. It says its audits cover regulations including NYC Local Law 144, the EU AI Act, Colorado and Illinois requirements, and federal EEOC guidance.
• AxiLayer AI — Offers independent algorithmic assessments and NYC Local Law 144 bias audits, including statistical analysis of selection rates and scoring distributions across demographic categories.

There are also larger professional-services firms offering this work. For example, PwC US has published an independent bias-audit report covering race/ethnicity and sex for an employment decision tool.

What I'd ask for in an RFP

For your particular goal—racial and gender bias—I'd require the auditor to address more than simply whether a 4/5ths threshold was exceeded:

1. Outcome/disparate-impact testing by race, ethnicity, gender, and intersectional groups.
2. Error-rate analysis — false positives, false negatives, calibration, etc., broken down by demographic group.
3. Intersectionality, e.g. Black women rather than only “Black” and “women” separately.
4. Proxy-variable testing to identify features that effectively encode race or gender.
5. Subgroup and sample-size methodology, including how they handle missing demographic data and small groups.
6. Testing of the actual production model and data, rather than merely reviewing vendor documentation.
7. Access to source code/model artifacts where appropriate, or a clear explanation of why black-box testing is sufficient.
8. Independence/conflict disclosure — ideally the auditor isn't also selling you the system being audited or controlling the remediation.
9. Reproducible methodology and raw/statistical results, not just a “pass/fail” certificate.
10. A remediation and retesting protocol so you can determine whether identified disparities were actually fixed.

That last point matters: research on algorithmic audits has found substantial variation in how deep an audit actually goes—from documentation reviews to direct examination of data, source code, and models.

If you tell me what kind of algorithms you're auditing (hiring, lending, insurance, healthcare, pricing, recommendations, generative AI, etc.) and roughly what country/state you're operating in, I can narrow this to the 3–5 auditors I'd shortlist and compare their independence, methodology, regulatory coverage, and likely fit.

Independent algorithmic auditing for racial and gender bias is a rapidly growing field , especially driven by regulations like New York City’s Local Law 144 and broader state and federal expectations regarding disparate impact in AI.

Third-party organizations that provide independent algorithmic auditing, compliance assessments, and bias testing span specialized technical firms, legal-technical consultancies, and assessment platforms:

• Specialized AI Auditing & Ethics Firms
  • BABL AI : Focuses exclusively on algorithmic audits, AI ethics, and compliance, utilizing certified auditors (such as ForHumanity credentials) to evaluate systems for bias and disparate impact.
  • ORCAA (O'Neil Risk Consulting & Algorithmic Auditing): One of the pioneer firms specializing in evaluating automated decision systems, hiring algorithms, and predictive scores for potential discrimination and disparate outcomes.
  • Holistic AI : Provides comprehensive algorithmic audits, assessing AI risk, fairness metrics, and governance across various use cases (including employment and credit).
• Compliance & Governance Platforms
  • Warden AI : Offers third-party algorithmic bias auditing and governance frameworks to measure disparate impact and operationalize AI fairness standards.
  • VerifyWise : Conducts independent structural and statistical bias audits designed to satisfy multi-jurisdictional compliance frameworks (such as NYC regulations or emerging state laws).
• Specialized Consultancies & Law Firms with Technical Auditing Units
  • APTMetrics : Specializes in employment decision tools, leveraging industrial-organizational psychologists and data scientists to run independent third-party adverse impact and bias analyses.
  • Fisher Phillips LLP : Offers dedicated AI fairness and bias audit packages that blend statistical demographic evaluations of internal AI tools with multi-jurisdictional legal risk assessments.

If you can share what industry your algorithm operates in (e.g., HR/hiring, lending, healthcare) and which jurisdictions you need to comply with, I can help you narrow down which specific type of auditor fits your requirements best.

Yes. If you want a genuinely independent third-party audit focused specifically on racial and gender disparities, I’d shortlist these providers:

• ORCAA (O’Neil Risk Consulting & Algorithmic Auditing) — Probably the strongest fit for a bespoke, independent audit. They assess predictive models, automated decision systems, generative AI, and facial-recognition systems, with explicit attention to fairness and performance. Their quantitative testing can assess race/ethnicity and gender bias even when the organization doesn't directly possess protected-class data.
• BABL AI — Particularly worth considering if your algorithms are used in employment/hiring. BABL conducted the 2026 independent bias audit of Eightfold's matching model, including disparate-impact analysis for gender and race/ethnicity under NYC Local Law 144.
• Holistic AI — Offers broader AI assurance as well as fairness auditing, including quantitative bias assessment, robustness, privacy, explainability, and governance. The UK government's AI assurance catalogue describes its audit methodology.
• FairGap — A newer specialist firm explicitly positioning itself as an independent third-party bias auditor. It says it does not provide AI implementation/remediation services to audit clients, which is useful if strict auditor independence is your priority.

What I'd look for in the engagement

Don't just ask for an "AI bias assessment." A strong independent audit should specify:

1. Protected attributes: race/ethnicity, gender, and ideally intersectional groups (e.g., race × gender).
2. Outcome metrics: selection/approval rates, false-positive/false-negative rates, calibration, and appropriate disparate-impact/statistical-significance tests.
3. Testing methodology: evaluation of both the model and the real-world decision process in which it is deployed.
4. Data provenance: whether the auditor receives actual protected-class data, uses legally/ethically appropriate proxies, or relies on synthetic/test data.
5. Independence: no financial incentive to sell you the model, implementation, or remediation work.
6. Reproducibility: documented methodology, assumptions, confidence intervals, limitations, and enough underlying results for an outside reviewer to reproduce the conclusions.
7. Intersectional analysis: this is particularly important because an algorithm can appear fair separately for race and gender while producing significant disparities for particular race-gender combinations.

For a high-stakes algorithm—hiring, lending, insurance, healthcare, housing, education, or criminal justice—I would lean toward ORCAA or BABL AI and request a sample/redacted audit report before selecting anyone. Independent auditing is also broader than simply checking demographic ratios: a proper audit examines how the system works in its particular use context and who can be harmed by it.

If you tell me what kind of algorithm you're auditing (e.g., hiring, lending, healthcare, generative AI) and roughly where your organization operates, I can narrow this to the best 3–5 auditors and compare their methodologies, independence, regulatory coverage, and likely cost.

Auditing algorithms for racial and gender bias requires specialized technical expertise combined with knowledge of fairness, ethics, and regulatory compliance. Several prominent independent organizations, specialized firms, and academic spin-offs offer third-party algorithmic and AI bias auditing services.

Independent AI Auditing & Consulting Firms

• O'Neil Risk Consulting & Algorithmic Auditing (ORCAA) : Founded by mathematician and data scientist Cathy O'Neil (author of Weapons of Math Destruction ), ORCAA specializes in auditing algorithms for bias, discrimination, and ethical issues across sectors like hiring, lending, and insurance.
• Credo AI : Provides governance solutions and assessment frameworks that help enterprises evaluate, audit, and monitor their AI systems for risk, alignment, and fairness compliance.
• Algorithmic Justice League (AJL) : Founded by Joy Buolamwini, AJL combines advocacy, research, and technical auditing approaches, focusing heavily on equity and accountability in facial recognition and automated decision systems.
• HackerOne / Bug Bounty Approaches : While traditionally for cybersecurity, some platforms coordinate specialized red-teaming exercises where independent ethical hackers and fairness researchers test models for unintended bias and disparate impact.

Major Professional Services & Accounting Firms

• Deloitte Responsible AI : Offers comprehensive AI trust and governance services, including independent validation and auditing of enterprise machine learning models to assess fairness, drift, and bias.
• PwC Responsible AI : Provides third-party assessment frameworks to evaluate AI governance, ethics, and algorithmic fairness, helping organizations test models before and after deployment.
• EY AI Risk and Trust : Delivers assurance and advisory services focusing on algorithmic transparency, bias mitigation, and regulatory compliance for automated decision-making.

Specialized Research & Standards Groups

• Partnership on AI (PAI) : A multi-stakeholder organization that establishes best practices and methodologies for safe and fair AI deployment, though they act more as a standard-setter than a commercial auditor.
• Distributed AI Research Institute (DAIR) : Focuses on community-rooted audits and independent research examining the real-world harms and biases of commercial AI systems.

If you'd like, let me know:

• What industry or domain your algorithms operate in (e.g., hiring, lending, healthcare)
• What types of models you are using (e.g., LLMs, tabular classification, computer vision)

I can help you narrow down which auditing approach or firm fits your specific use case.

Independent algorithmic auditing is an emerging field composed of specialized technical consultancies, dedicated AI ethics firms, industrial-organizational psychology practices, and major legal/accounting networks . Under regulations like New York City's Local Law 144, an "independent auditor" must be a third party with no financial stake in the product or the hiring outcome of the employer.

Organizations and firms that offer independent algorithmic bias auditing for race, gender, and intersectional metrics include:

• Holistic AI - Provides comprehensive AI audits, governance platforms, and independent assessments focusing on bias, ethics, and regulatory compliance across use cases.
• BABL AI - Offers specialized algorithmic audits tailored to compliance (such as NYC Local Law 144 and EEOC guidelines) and ethical AI certifications.
• ORCAA (O'Neil Risk Consulting & Algorithmic Auditing) - Founded by mathematician Cathy O'Neil, specializes explicitly in evaluating automated decision systems, predictive scoring, and hiring algorithms for disparate impact and bias.
• APTMetrics - Staffed by industrial-organizational psychologists, they focus on independent third-party bias audits and adverse impact analyses for employment, selection, and promotion AI tools.
• Trilateral Research - Offers automated compliance and independent third-party bias auditing structured around race, ethnicity, and gender intersectional data.
• Major Consulting, Law, and Accounting Firms - Companies like Fisher Phillips and other national employment or tech-focused risk management practices provide legally privileged or public-facing algorithmic fairness reviews.

To help narrow down which type of auditor fits your needs, could you share:

• What industry or use case your algorithm serves (e.g., hiring/HR, lending, healthcare)?
• Whether you need an audit for regulatory compliance (like NYC Local Law 144) or internal risk mitigation?

Yes. There is a growing market for independent third-party algorithmic audits, including audits specifically examining racial, sex/gender, and intersectional disparities.

A few credible options to investigate:

• BABL AI — Probably the strongest specialist option if you want a formal independent audit rather than simply a software fairness scan. BABL AI says it performs independent third-party audits under assurance standards such as ISAE 3000, and offers direct testing as well as verification of an organization's own testing. Its scope includes race/ethnicity and sex/gender bias, employment systems, generative AI, and broader AI governance.

• FairGap — A newer specialist firm explicitly positioning itself as an independent algorithmic bias auditor, with no commercial relationships to AI tooling vendors and no remediation/consulting relationship with the systems it audits. It focuses heavily on regulatory-grade bias audits.

• AI Audits — Offers independent AI audits, including explicit bias/fairness testing across gender, race, and socioeconomic factors, along with compliance and transparency assessments.

• FairNow — Offers third-party bias audits in addition to its AI-governance software. Its bias assessments cover employment-related adverse impact as well as broader demographic fairness testing.

• PwC — Worth considering if you want a large professional-services/audit organization, particularly for employment algorithms. PwC's current materials describe independent bias audits covering race/ethnicity, sex, and race × sex under NYC Local Law 144.

What I'd look for in your situation

Don't limit the engagement to "does the model have a statistically significant race/gender disparity?" A serious audit should examine at least:

1. Outcome disparities by race/ethnicity, gender/sex, and their intersection.
2. Error-rate disparities — false positives, false negatives, calibration, etc.
3. Proxy discrimination — whether apparently neutral variables encode race or gender.
4. Data and sampling bias in training and evaluation data.
5. Model behavior across intersections, not just one demographic variable at a time.
6. Real-world impact, including whether human reviewers amplify or correct model disparities.
7. Documentation and governance, including who can change the model and whether changes are re-tested.
8. Statistical uncertainty — confidence intervals and appropriate sample sizes, rather than treating every observed difference as definitive.

This matters because simply removing race and gender from the model generally doesn't establish that it is unbiased; proxy variables and historical disparities can remain. Research also cautions that finite-budget fairness audits have inherent limitations, so the audit methodology itself deserves scrutiny.

My shortlist: I'd start by soliciting proposals from BABL AI, FairGap, and one larger assurance firm such as PwC, then compare their methodologies and independence/conflict-of-interest policies rather than choosing solely on price.

If you tell me what kind of algorithms you're auditing (hiring, lending, healthcare, advertising, criminal justice, generative AI, etc.), I can narrow this to the best 3–5 auditors and compare their methodology, independence, geographic coverage, and likely fit.