Platform Selection

Enterprise AI Platform Checklist

The 12 must-have capabilities for any healthcare AI governance platform

AI Platform Evaluation Guide

Not all AI platforms are created equal. Some are consumer tools with a healthcare logo slapped on. Others are enterprise-grade governance platforms designed specifically for regulated industries. This checklist helps you evaluate platforms and separate real healthcare AI governance from marketing hype.

The Complete Checklist

6 categories, 23 capabilities

Shadow AI Discovery

Discovery & Visibility

Must Have

Platform can inventory all unauthorized AI usage across the organization.

Why it matters: You can't govern what you can't see. Shadow AI discovery is the foundation.

Centralized Dashboard

Discovery & Visibility

Must Have

Single pane of glass showing all AI usage, models, departments, and data flows.

Why it matters: Leadership needs visibility. Scattered tools create blind spots.

Usage Analytics

Discovery & Visibility

Must Have

Real-time reporting on who's using AI, for what, how often, and what data is involved.

Why it matters: You need metrics to demonstrate ROI and identify risk patterns.

PHI Protection & Compliance

Automatic PHI Detection

PHI Protection & Compliance

Must Have

Real-time identification of all 18 HIPAA identifiers before data reaches AI models.

Why it matters: Manual PHI removal doesn't scale and has 100% error rate.

Automatic PHI Redaction

PHI Protection & Compliance

Must Have

Cleansing PHI from prompts while maintaining context for AI responses.

Why it matters: Blocking PHI entirely makes AI useless. Redaction preserves utility.

Data Rehydration

PHI Protection & Compliance

Nice to Have

Re-inserting redacted PHI into AI responses so users get actionable output.

Why it matters: De-identified responses aren't useful for clinical workflows.

Complete Audit Logs

PHI Protection & Compliance

Must Have

Immutable logs of every AI interaction with timestamp, user, model, and data shared.

Why it matters: OCR and auditors will ask 'prove PHI wasn't exposed.' You need evidence.

BAAs with All AI Vendors

PHI Protection & Compliance

Must Have

Business Associate Agreements with OpenAI, Anthropic, Google, and any model provider.

Why it matters: HIPAA requires BAAs for any vendor that might access PHI.

Multi-Model AI Access

GPT-4 / GPT-4o Access

Multi-Model AI Access

Must Have

Latest OpenAI models for general-purpose tasks, documentation, research.

Why it matters: Most requested model by staff. If you don't provide it, shadow usage continues.

Claude Access (Anthropic)

Multi-Model AI Access

Must Have

Claude 3.5 Sonnet and other Anthropic models for analysis and complex reasoning.

Why it matters: Different models excel at different tasks. Staff need options.

Gemini Access (Google)

Multi-Model AI Access

Nice to Have

Google's Gemini models for research, data analysis, and multimodal tasks.

Why it matters: Model diversity prevents vendor lock-in and optimizes for use cases.

Model Selection by Use Case

Multi-Model AI Access

Must Have

  • Ability to choose the right model for each task (documentation vs.
  • analysis vs.
  • coding).

Why it matters: One model doesn't fit all needs. Platform should guide optimal selection.

Policy & Governance Controls

Role-Based Access Control

Policy & Governance Controls

Must Have

  • Different AI permissions by department, role, or user (clinical vs.
  • admin vs.
  • leadership).

Why it matters: Not everyone needs access to all models. Controls prevent misuse.

Model-Level Restrictions

Policy & Governance Controls

Must Have

Ability to limit which models are available to which users or departments.

Why it matters: Some models cost more or have different risk profiles.

Usage Quotas

Policy & Governance Controls

Nice to Have

Set limits on AI usage per user, department, or organization-wide.

Why it matters: Cost control and preventing abuse or overuse.

Content Filtering

Policy & Governance Controls

Nice to Have

Block certain types of prompts (e.g., requests to generate clinical advice without oversight).

Why it matters: Some use cases are too high-risk even with PHI protection.

Staff Enablement

Onboarding & Training

Staff Enablement

Must Have

Guided setup, prompt engineering basics, and appropriate use case education.

Why it matters: Staff won't adopt tools they don't understand or trust.

Prompt Templates

Staff Enablement

Nice to Have

Pre-built, approved prompts for common healthcare tasks (discharge summaries, appeal letters, etc.).

Why it matters: Reduces cognitive load and ensures consistency.

Support & Help Resources

Staff Enablement

Must Have

Documentation, FAQs, and access to support when staff have questions.

Why it matters: Unsupported tools get abandoned. Support drives adoption.

Security & Infrastructure

SOC 2 Type II Certification

Security & Infrastructure

Must Have

Platform provider has completed SOC 2 Type II audit for security controls.

Why it matters: Demonstrates serious security posture and third-party validation.

HIPAA Compliance

Security & Infrastructure

Must Have

Platform architecture is designed for HIPAA compliance (not just 'HIPAA-ready').

Why it matters: HIPAA-ready ≠ HIPAA compliant. You need actual compliance.

Data Residency Controls

Security & Infrastructure

Nice to Have

Ability to specify where data is stored and processed (US-only, specific regions).

Why it matters: Some BAAs require data to stay in specific jurisdictions.

SSO / SAML Integration

Security & Infrastructure

Nice to Have

Single sign-on integration with your existing identity provider.

Why it matters: Reduces password fatigue and improves security posture.

Must-Have vs. Nice-to-Have

How to prioritize when evaluating platforms

16 Must-Haves

  • Shadow AI discovery
  • Centralized dashboard
  • Usage analytics
  • Automatic PHI detection
  • Automatic PHI redaction
  • Complete audit logs
  • BAAs with AI vendors
  • Multi-model access (GPT-4, Claude)
  • Role-based access control
  • Training & support
  • SOC 2 Type II certification
  • HIPAA compliance

These are non-negotiable for healthcare AI governance. A platform missing any of these is not enterprise-ready.

7 Nice-to-Haves

  • Data rehydration
  • Gemini access
  • Usage quotas
  • Content filtering
  • Prompt templates
  • Data residency controls
  • SSO / SAML integration

These add value but aren't dealbreakers. Prioritize based on your specific needs and maturity level.

Red Flags to Watch For

Warning signs that a platform isn't enterprise-ready

No Shadow AI Discovery

Red Flag

If the platform can't inventory existing shadow AI usage, it's just adding another tool to the chaos — not solving the problem.

'HIPAA-Ready' Instead of 'HIPAA Compliant'

Red Flag

HIPAA-ready means 'we could be compliant if you configure it correctly.' You need actual compliance, not homework.

Single Model Access Only

Red Flag
  • If they only offer one AI model (usually their own), it's vendor lock-in disguised as governance.
  • You need multi-model access.

No PHI Protection Layer

Red Flag

If they rely on staff to 'remember to remove PHI,' it's not a governance platform — it's just a wrapper around ChatGPT.

No Audit Logs

Red Flag
  • If you can't prove what data was sent where, you can't demonstrate compliance.
  • Audit logs are non-negotiable.

No BAAs with Model Providers

Red Flag

The platform vendor might have a BAA with you, but if they don't have BAAs with OpenAI, Anthropic, etc., PHI is still exposed.

How to Use This Checklist

1

Score Each Platform

Give 1 point for each must-have, 0.5 points for each nice-to-have. A platform needs at least 12/12 must-haves to be viable.

2

Ask for Proof

Don't take marketing claims at face value. Ask for SOC 2 reports, sample BAAs, audit log exports, and customer references.

3

Test Shadow AI Discovery

Ask the vendor to demonstrate how they would inventory your existing shadow AI usage. If they can't, they're not solving your biggest problem.

4

Validate PHI Protection

Test the PHI detection with real (de-identified) patient scenarios. See if it catches all 18 HIPAA identifiers automatically.

See How AuthenTech AI Measures Up

Book a Shadow AI Risk Check and we'll show you how our platform delivers on all 12 must-haves

Get Platform Selection Help Learn About AuthenTech AI

Concerned about Shadow AI in your organization?

3 Minutes. 16 Questions. No Fluff.

Quick Online Risk Check