Picture this: Your AI model just made a decision that cost your company $50,000. When your boss asks "Why did it do that?" you stare at the black box and shrug. That's the moment you realize explainable AI isn't just nice to have—it's essential for survival.
This guide shows you how to implement explainable AI with Ollama models. You'll learn to interpret model decisions, build transparent systems, and debug AI behavior like a detective solving a mystery.
Why Ollama Model Interpretation Matters
AI models make thousands of decisions daily. Without interpretation, you're flying blind. Ollama's local deployment makes explainability both possible and practical.
Consider these scenarios:
- A medical AI recommends treatment without explanation
- A hiring algorithm rejects candidates for unknown reasons
- A financial model denies loans with zero transparency
Each represents a lawsuit waiting to happen. Explainable AI transforms these black boxes into transparent, accountable systems.
Core Concepts of Explainable AI
Local vs Global Explanations
Local explanations interpret individual predictions:
# Local explanation for single prediction
explanation = explain_prediction(
model=ollama_model,
input_text="Patient shows symptoms of...",
prediction_id="case_001"
)
Global explanations reveal model behavior patterns:
# Global explanation for model behavior
global_insights = analyze_model_behavior(
model=ollama_model,
dataset=training_data,
feature_importance=True
)
Attention Mechanisms in Ollama
Ollama models use attention weights to focus on input parts. These weights reveal decision-making processes:
import requests
import json
def get_attention_weights(prompt, model_name="llama2"):
"""Extract attention weights from Ollama model responses"""
payload = {
"model": model_name,
"prompt": prompt,
"stream": False,
"options": {
"temperature": 0.1,
"top_p": 0.9
}
}
response = requests.post(
"http://localhost:11434/api/generate",
json=payload
)
if response.status_code == 200:
result = response.json()
return {
"response": result["response"],
"eval_duration": result["eval_duration"],
"prompt_eval_duration": result["prompt_eval_duration"]
}
else:
raise Exception(f"API call failed: {response.status_code}")
# Example usage
prompt = "Analyze this medical report: Patient presents with chest pain and shortness of breath."
attention_data = get_attention_weights(prompt)
print(f"Model focused on: {attention_data['response']}")
Setting Up Explainable AI with Ollama
Installation and Configuration
First, install required dependencies:
# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh
# Install Python packages
pip install ollama requests numpy pandas matplotlib seaborn
pip install lime shap interpretml
Model Preparation
Download and configure an Ollama model:
# Pull a model optimized for explanation
ollama pull llama2:7b-chat
ollama pull codellama:7b-instruct
# Verify installation
ollama list
Basic Explanation Framework
Create a foundation for model interpretation:
import ollama
import json
from typing import Dict, List, Any
class OllamaExplainer:
def __init__(self, model_name: str = "llama2"):
self.model_name = model_name
self.client = ollama.Client()
def explain_decision(self, prompt: str, context: str = "") -> Dict[str, Any]:
"""Generate explanation for model decision"""
explanation_prompt = f"""
Original prompt: {prompt}
Context: {context}
Explain your reasoning step by step:
1. What key information did you identify?
2. What factors influenced your decision?
3. What alternatives did you consider?
4. Why did you choose this specific response?
Provide a clear, structured explanation.
"""
response = self.client.chat(
model=self.model_name,
messages=[
{"role": "system", "content": "You are an AI that explains its reasoning clearly and systematically."},
{"role": "user", "content": explanation_prompt}
]
)
return {
"original_prompt": prompt,
"explanation": response['message']['content'],
"model_used": self.model_name,
"timestamp": response.get('created_at', 'unknown')
}
def compare_decisions(self, prompts: List[str]) -> Dict[str, Any]:
"""Compare explanations across multiple prompts"""
comparisons = []
for prompt in prompts:
explanation = self.explain_decision(prompt)
comparisons.append(explanation)
return {
"comparisons": comparisons,
"total_prompts": len(prompts),
"model_consistency": self._analyze_consistency(comparisons)
}
def _analyze_consistency(self, explanations: List[Dict]) -> float:
"""Analyze consistency across explanations"""
# Simplified consistency metric
# In production, use semantic similarity measures
explanation_lengths = [len(exp['explanation']) for exp in explanations]
avg_length = sum(explanation_lengths) / len(explanation_lengths)
variance = sum((length - avg_length) ** 2 for length in explanation_lengths) / len(explanation_lengths)
return 1.0 - (variance / (avg_length ** 2)) # Normalized consistency score
# Initialize explainer
explainer = OllamaExplainer("llama2")
# Example usage
prompt = "Should we approve this loan application for a startup?"
explanation = explainer.explain_decision(prompt, "Startup has 2 years history, $50K revenue, good credit")
print(json.dumps(explanation, indent=2))
Advanced Interpretation Techniques
Token-Level Analysis
Analyze how models process individual tokens:
def analyze_token_importance(prompt: str, model_name: str = "llama2"):
"""Analyze importance of individual tokens in decision making"""
tokens = prompt.split()
token_scores = {}
# Baseline response
baseline_response = ollama.generate(
model=model_name,
prompt=prompt
)['response']
# Test each token's impact by removal
for i, token in enumerate(tokens):
modified_tokens = tokens[:i] + tokens[i+1:]
modified_prompt = ' '.join(modified_tokens)
modified_response = ollama.generate(
model=model_name,
prompt=modified_prompt
)['response']
# Calculate similarity score (simplified)
similarity = calculate_response_similarity(baseline_response, modified_response)
token_scores[token] = 1.0 - similarity # Higher score = more important
return sorted(token_scores.items(), key=lambda x: x[1], reverse=True)
def calculate_response_similarity(response1: str, response2: str) -> float:
"""Calculate similarity between two responses"""
words1 = set(response1.lower().split())
words2 = set(response2.lower().split())
intersection = words1.intersection(words2)
union = words1.union(words2)
return len(intersection) / len(union) if union else 0.0
# Example usage
prompt = "Recommend investment strategy for retirement planning"
token_importance = analyze_token_importance(prompt)
print("Token Importance Rankings:")
for token, score in token_importance[:5]:
print(f"{token}: {score:.3f}")
Decision Tree Visualization
Create visual representations of model decision paths:
import matplotlib.pyplot as plt
import networkx as nx
from matplotlib.patches import Rectangle
class DecisionTreeVisualizer:
def __init__(self, explainer: OllamaExplainer):
self.explainer = explainer
def create_decision_tree(self, base_prompt: str, decision_points: List[str]) -> None:
"""Create visual decision tree for model reasoning"""
fig, ax = plt.subplots(figsize=(12, 8))
# Create decision tree structure
tree_data = []
for i, point in enumerate(decision_points):
modified_prompt = f"{base_prompt} Consider: {point}"
explanation = self.explainer.explain_decision(modified_prompt)
tree_data.append({
'level': i,
'decision_point': point,
'explanation': explanation['explanation'][:100] + "...",
'x': i * 2,
'y': len(decision_points) - i
})
# Plot decision nodes
for node in tree_data:
# Draw decision box
rect = Rectangle((node['x'] - 0.8, node['y'] - 0.3), 1.6, 0.6,
facecolor='lightblue', edgecolor='black')
ax.add_patch(rect)
# Add decision text
ax.text(node['x'], node['y'], node['decision_point'],
ha='center', va='center', fontsize=10, weight='bold')
# Add explanation below
ax.text(node['x'], node['y'] - 0.6, node['explanation'],
ha='center', va='top', fontsize=8, wrap=True)
# Connect nodes with arrows
for i in range(len(tree_data) - 1):
ax.arrow(tree_data[i]['x'], tree_data[i]['y'] - 0.3,
tree_data[i+1]['x'] - tree_data[i]['x'],
tree_data[i+1]['y'] - tree_data[i]['y'] + 0.6,
head_width=0.1, head_length=0.1, fc='gray', ec='gray')
ax.set_xlim(-1, max(node['x'] for node in tree_data) + 1)
ax.set_ylim(0, len(decision_points) + 1)
ax.set_title(f"Decision Tree: {base_prompt[:50]}...")
ax.axis('off')
plt.tight_layout()
plt.savefig('decision_tree_visualization.png', dpi=300, bbox_inches='tight')
plt.show()
# Example usage
visualizer = DecisionTreeVisualizer(explainer)
decision_points = [
"Risk tolerance",
"Investment timeline",
"Current financial situation",
"Market conditions"
]
visualizer.create_decision_tree("Investment recommendation", decision_points)
Implementing LIME for Local Explanations
LIME (Local Interpretable Model-agnostic Explanations) provides local interpretability:
from lime.lime_text import LimeTextExplainer
import numpy as np
class OllamaLimeExplainer:
def __init__(self, model_name: str = "llama2"):
self.model_name = model_name
self.lime_explainer = LimeTextExplainer(class_names=['negative', 'positive'])
def predict_proba(self, texts: List[str]) -> np.ndarray:
"""Predict probability for LIME compatibility"""
probabilities = []
for text in texts:
# Get model response
response = ollama.generate(
model=self.model_name,
prompt=f"Analyze sentiment: {text}\nRespond with: POSITIVE or NEGATIVE"
)['response']
# Convert to probability
if "POSITIVE" in response.upper():
prob = [0.2, 0.8] # [negative, positive]
else:
prob = [0.8, 0.2] # [negative, positive]
probabilities.append(prob)
return np.array(probabilities)
def explain_prediction(self, text: str, num_features: int = 10) -> Dict[str, Any]:
"""Generate LIME explanation for text prediction"""
explanation = self.lime_explainer.explain_instance(
text,
self.predict_proba,
num_features=num_features,
num_samples=100
)
# Extract feature importance
feature_importance = explanation.as_list()
return {
'text': text,
'feature_importance': feature_importance,
'explanation_html': explanation.as_html(),
'prediction_confidence': explanation.predict_proba[1]
}
# Example usage
lime_explainer = OllamaLimeExplainer()
text = "This product exceeded my expectations and delivered amazing results!"
lime_explanation = lime_explainer.explain_prediction(text)
print("LIME Feature Importance:")
for feature, importance in lime_explanation['feature_importance']:
print(f"{feature}: {importance:.3f}")
Building Interpretable Pipelines
Multi-Step Explanation Pipeline
Create comprehensive explanation workflows:
class ExplainablePipeline:
def __init__(self, model_name: str = "llama2"):
self.model_name = model_name
self.explainer = OllamaExplainer(model_name)
self.lime_explainer = OllamaLimeExplainer(model_name)
def comprehensive_explanation(self, prompt: str, context: str = "") -> Dict[str, Any]:
"""Generate comprehensive multi-level explanation"""
# Step 1: Basic decision explanation
decision_explanation = self.explainer.explain_decision(prompt, context)
# Step 2: Token-level analysis
token_importance = analyze_token_importance(prompt, self.model_name)
# Step 3: Comparative analysis
similar_prompts = self.generate_similar_prompts(prompt)
comparison = self.explainer.compare_decisions(similar_prompts)
# Step 4: Confidence assessment
confidence_metrics = self.assess_confidence(prompt)
return {
'decision_explanation': decision_explanation,
'token_importance': token_importance[:5], # Top 5 tokens
'comparative_analysis': comparison,
'confidence_metrics': confidence_metrics,
'interpretation_summary': self.generate_summary(decision_explanation, token_importance)
}
def generate_similar_prompts(self, original_prompt: str) -> List[str]:
"""Generate similar prompts for comparison"""
variation_prompt = f"""
Create 3 variations of this prompt that test different aspects:
Original: {original_prompt}
Provide variations as a simple list, one per line.
"""
response = ollama.generate(
model=self.model_name,
prompt=variation_prompt
)['response']
variations = [line.strip() for line in response.split('\n') if line.strip()]
return variations[:3] # Return top 3 variations
def assess_confidence(self, prompt: str) -> Dict[str, float]:
"""Assess model confidence in decision"""
# Generate multiple responses
responses = []
for _ in range(5):
response = ollama.generate(
model=self.model_name,
prompt=prompt,
options={'temperature': 0.3}
)['response']
responses.append(response)
# Calculate consistency
consistency = self.calculate_response_consistency(responses)
return {
'consistency_score': consistency,
'confidence_level': 'high' if consistency > 0.8 else 'medium' if consistency > 0.5 else 'low',
'sample_size': len(responses)
}
def calculate_response_consistency(self, responses: List[str]) -> float:
"""Calculate consistency across multiple responses"""
if len(responses) < 2:
return 1.0
similarities = []
for i in range(len(responses)):
for j in range(i + 1, len(responses)):
sim = calculate_response_similarity(responses[i], responses[j])
similarities.append(sim)
return sum(similarities) / len(similarities) if similarities else 0.0
def generate_summary(self, decision_explanation: Dict, token_importance: List) -> str:
"""Generate human-readable summary of interpretation"""
top_tokens = [token for token, _ in token_importance[:3]]
summary = f"""
Model Decision Summary:
- Key decision factors: {', '.join(top_tokens)}
- Reasoning approach: {decision_explanation['explanation'][:100]}...
- Model confidence: Based on token analysis and consistency checks
This interpretation provides transparency into the model's decision-making process.
"""
return summary.strip()
# Example usage
pipeline = ExplainablePipeline()
prompt = "Should we invest in renewable energy stocks given current market conditions?"
comprehensive_explanation = pipeline.comprehensive_explanation(prompt)
print(json.dumps(comprehensive_explanation, indent=2))
Debugging Model Decisions
Error Analysis Framework
Identify and diagnose model decision errors:
class ModelDebugger:
def __init__(self, model_name: str = "llama2"):
self.model_name = model_name
def debug_decision_error(self, prompt: str, expected_response: str, actual_response: str) -> Dict[str, Any]:
"""Debug why model made incorrect decision"""
debug_prompt = f"""
Analyze this decision error:
Input: {prompt}
Expected: {expected_response}
Actual: {actual_response}
Identify:
1. What went wrong in the reasoning?
2. What information was misinterpreted?
3. What would correct the decision?
4. How can similar errors be prevented?
Provide detailed analysis.
"""
debug_response = ollama.generate(
model=self.model_name,
prompt=debug_prompt
)['response']
return {
'error_analysis': debug_response,
'error_type': self.classify_error_type(expected_response, actual_response),
'correction_suggestions': self.generate_corrections(prompt, expected_response),
'prevention_strategies': self.suggest_prevention_strategies(debug_response)
}
def classify_error_type(self, expected: str, actual: str) -> str:
"""Classify the type of error made"""
if len(actual) < len(expected) * 0.5:
return "insufficient_response"
elif "not" in actual.lower() and "not" not in expected.lower():
return "negation_error"
elif any(word in actual.lower() for word in ["sorry", "cannot", "unable"]):
return "refusal_error"
else:
return "reasoning_error"
def generate_corrections(self, prompt: str, expected: str) -> List[str]:
"""Generate suggestions for correcting the decision"""
correction_prompt = f"""
How can we modify this prompt to get the expected response?
Original: {prompt}
Expected: {expected}
Suggest 3 specific modifications:
"""
response = ollama.generate(
model=self.model_name,
prompt=correction_prompt
)['response']
corrections = [line.strip() for line in response.split('\n') if line.strip()]
return corrections[:3]
def suggest_prevention_strategies(self, error_analysis: str) -> List[str]:
"""Extract prevention strategies from error analysis"""
strategies = []
if "context" in error_analysis.lower():
strategies.append("Provide more context in prompts")
if "example" in error_analysis.lower():
strategies.append("Include relevant examples")
if "specific" in error_analysis.lower():
strategies.append("Be more specific in requirements")
if "step" in error_analysis.lower():
strategies.append("Break down complex requests into steps")
return strategies
# Example usage
debugger = ModelDebugger()
error_debug = debugger.debug_decision_error(
prompt="Analyze market trends",
expected_response="Detailed analysis of current market conditions with specific data points",
actual_response="Markets are generally positive"
)
print("Debug Analysis:")
print(json.dumps(error_debug, indent=2))
Performance Monitoring and Metrics
Explanation Quality Metrics
Monitor the quality of explanations:
class ExplanationMetrics:
def __init__(self):
self.metrics_history = []
def evaluate_explanation_quality(self, explanation: Dict[str, Any]) -> Dict[str, float]:
"""Evaluate quality of generated explanation"""
metrics = {
'completeness': self.measure_completeness(explanation),
'clarity': self.measure_clarity(explanation),
'accuracy': self.measure_accuracy(explanation),
'consistency': self.measure_consistency(explanation),
'usefulness': self.measure_usefulness(explanation)
}
# Calculate overall quality score
metrics['overall_quality'] = sum(metrics.values()) / len(metrics)
self.metrics_history.append(metrics)
return metrics
def measure_completeness(self, explanation: Dict[str, Any]) -> float:
"""Measure how complete the explanation is"""
explanation_text = explanation.get('explanation', '')
# Check for key explanation components
components = [
'reasoning' in explanation_text.lower(),
'factor' in explanation_text.lower(),
'decision' in explanation_text.lower(),
'consider' in explanation_text.lower(),
len(explanation_text) > 100 # Minimum length
]
return sum(components) / len(components)
def measure_clarity(self, explanation: Dict[str, Any]) -> float:
"""Measure clarity of explanation"""
explanation_text = explanation.get('explanation', '')
# Simple clarity metrics
sentences = explanation_text.split('.')
avg_sentence_length = sum(len(s.split()) for s in sentences) / len(sentences) if sentences else 0
# Optimal sentence length is 15-20 words
clarity_score = 1.0 - abs(avg_sentence_length - 17.5) / 17.5
return max(0.0, min(1.0, clarity_score))
def measure_accuracy(self, explanation: Dict[str, Any]) -> float:
"""Measure accuracy of explanation (simplified)"""
# This would require ground truth data in production
# For now, return a placeholder based on consistency
return 0.85 # Placeholder
def measure_consistency(self, explanation: Dict[str, Any]) -> float:
"""Measure consistency of explanation"""
# Check if explanation is consistent with previous similar explanations
if len(self.metrics_history) < 2:
return 1.0
recent_scores = [m['overall_quality'] for m in self.metrics_history[-5:]]
variance = np.var(recent_scores) if recent_scores else 0
return 1.0 - min(variance, 1.0)
def measure_usefulness(self, explanation: Dict[str, Any]) -> float:
"""Measure usefulness of explanation"""
explanation_text = explanation.get('explanation', '')
# Check for actionable insights
actionable_terms = ['should', 'recommend', 'suggest', 'consider', 'improve']
actionable_count = sum(1 for term in actionable_terms if term in explanation_text.lower())
return min(1.0, actionable_count / 3.0) # Normalize to 0-1
def generate_quality_report(self) -> Dict[str, Any]:
"""Generate comprehensive quality report"""
if not self.metrics_history:
return {"error": "No metrics available"}
latest_metrics = self.metrics_history[-1]
# Calculate trends
if len(self.metrics_history) >= 5:
recent_quality = [m['overall_quality'] for m in self.metrics_history[-5:]]
quality_trend = (recent_quality[-1] - recent_quality[0]) / recent_quality[0]
else:
quality_trend = 0.0
return {
'current_quality': latest_metrics,
'quality_trend': quality_trend,
'total_evaluations': len(self.metrics_history),
'average_quality': sum(m['overall_quality'] for m in self.metrics_history) / len(self.metrics_history),
'recommendations': self.generate_improvement_recommendations(latest_metrics)
}
def generate_improvement_recommendations(self, metrics: Dict[str, float]) -> List[str]:
"""Generate recommendations for improving explanation quality"""
recommendations = []
if metrics['completeness'] < 0.7:
recommendations.append("Include more comprehensive reasoning steps")
if metrics['clarity'] < 0.7:
recommendations.append("Use clearer, more concise language")
if metrics['usefulness'] < 0.7:
recommendations.append("Provide more actionable insights")
if metrics['consistency'] < 0.7:
recommendations.append("Maintain consistent explanation patterns")
return recommendations
# Example usage
metrics_evaluator = ExplanationMetrics()
# Evaluate explanation quality
sample_explanation = {
'explanation': 'The model decided to recommend investment based on three key factors: market stability, growth potential, and risk assessment. The decision process considered current economic indicators and historical performance data.'
}
quality_scores = metrics_evaluator.evaluate_explanation_quality(sample_explanation)
print("Quality Metrics:")
print(json.dumps(quality_scores, indent=2))
# Generate quality report
quality_report = metrics_evaluator.generate_quality_report()
print("\nQuality Report:")
print(json.dumps(quality_report, indent=2))
Production Deployment Strategies
Real-time Explanation API
Build production-ready explanation services:
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import Optional
import uvicorn
app = FastAPI(title="Ollama Explainable AI Service")
class ExplanationRequest(BaseModel):
prompt: str
context: Optional[str] = ""
model_name: Optional[str] = "llama2"
explanation_type: Optional[str] = "comprehensive"
class ExplanationResponse(BaseModel):
explanation: dict
confidence: float
processing_time: float
@app.post("/explain", response_model=ExplanationResponse)
async def explain_decision(request: ExplanationRequest):
"""Generate explanation for model decision"""
try:
import time
start_time = time.time()
# Initialize pipeline
pipeline = ExplainablePipeline(request.model_name)
# Generate explanation based on type
if request.explanation_type == "comprehensive":
explanation = pipeline.comprehensive_explanation(request.prompt, request.context)
else:
explainer = OllamaExplainer(request.model_name)
explanation = explainer.explain_decision(request.prompt, request.context)
# Calculate processing time
processing_time = time.time() - start_time
# Assess confidence
confidence = explanation.get('confidence_metrics', {}).get('consistency_score', 0.5)
return ExplanationResponse(
explanation=explanation,
confidence=confidence,
processing_time=processing_time
)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@app.get("/health")
async def health_check():
"""Health check endpoint"""
return {"status": "healthy", "service": "Ollama Explainable AI"}
@app.get("/models")
async def list_models():
"""List available models"""
# This would integrate with Ollama's model list
return {"models": ["llama2", "codellama", "mistral"]}
if __name__ == "__main__":
uvicorn.run(app, host="0.0.0.0", port=8000)
Monitoring and Logging
Implement comprehensive monitoring:
import logging
from datetime import datetime
import json
class ExplanationLogger:
def __init__(self, log_file: str = "explanation_audit.log"):
self.log_file = log_file
self.logger = self.setup_logger()
def setup_logger(self) -> logging.Logger:
"""Setup structured logging for explanations"""
logger = logging.getLogger("ExplanationAudit")
logger.setLevel(logging.INFO)
# File handler
file_handler = logging.FileHandler(self.log_file)
file_handler.setLevel(logging.INFO)
# Console handler
console_handler = logging.StreamHandler()
console_handler.setLevel(logging.INFO)
# Formatter
formatter = logging.Formatter(
'%(asctime)s - %(name)s - %(levelname)s - %(message)s'
)
file_handler.setFormatter(formatter)
console_handler.setFormatter(formatter)
logger.addHandler(file_handler)
logger.addHandler(console_handler)
return logger
def log_explanation(self, prompt: str, explanation: dict,
quality_metrics: dict, user_id: str = "anonymous"):
"""Log explanation for audit trail"""
log_entry = {
"timestamp": datetime.now().isoformat(),
"user_id": user_id,
"prompt": prompt,
"model": explanation.get("model_used", "unknown"),
"explanation_length": len(explanation.get("explanation", "")),
"quality_score": quality_metrics.get("overall_quality", 0.0),
"confidence": quality_metrics.get("consistency", 0.0),
"processing_time": explanation.get("processing_time", 0.0)
}
self.logger.info(f"EXPLANATION_AUDIT: {json.dumps(log_entry)}")
def log_error(self, prompt: str, error: str, user_id: str = "anonymous"):
"""Log explanation errors"""
error_entry = {
"timestamp": datetime.now().isoformat(),
"user_id": user_id,
"prompt": prompt,
"error": error,
"error_type": "explanation_failure"
}
self.logger.error(f"EXPLANATION_ERROR: {json.dumps(error_entry)}")
# Example usage
audit_logger = ExplanationLogger()
# Log successful explanation
audit_logger.log_explanation(
prompt="Investment recommendation",
explanation={"explanation": "Based on market analysis...", "model_used": "llama2"},
quality_metrics={"overall_quality": 0.85, "consistency": 0.9}
)
Best Practices for Explainable AI
Prompt Engineering for Explanations
Design prompts that encourage clear explanations:
class ExplanationPromptTemplates:
@staticmethod
def decision_explanation_template(domain: str) -> str:
"""Template for decision explanations"""
return f"""
You are an expert {domain} advisor. When making recommendations:
1. State your recommendation clearly
2. Explain the key factors you considered
3. Describe your reasoning process
4. Mention any assumptions you made
5. Acknowledge limitations or uncertainties
Be specific and provide evidence for your conclusions.
"""
@staticmethod
def step_by_step_template() -> str:
"""Template for step-by-step reasoning"""
return """
Think through this step by step:
Step 1: Analyze the situation
Step 2: Identify key factors
Step 3: Consider alternatives
Step 4: Make recommendation
Step 5: Explain confidence level
Be explicit about your reasoning at each step.
"""
@staticmethod
def comparative_analysis_template() -> str:
"""Template for comparative analysis"""
return """
Compare the options by:
1. Listing pros and cons for each
2. Ranking by importance
3. Identifying deal-breakers
4. Explaining your final choice
Show your work clearly.
"""
# Example usage
templates = ExplanationPromptTemplates()
# Use template for financial advice
financial_template = templates.decision_explanation_template("financial")
prompt = f"{financial_template}\n\nQuestion: Should I invest in tech stocks now?"
response = ollama.generate(
model="llama2",
prompt=prompt
)
print("Structured Explanation:")
print(response['response'])
Explanation Validation Framework
Validate explanation quality automatically:
class ExplanationValidator:
def __init__(self):
self.validation_rules = {
'has_reasoning': self.check_reasoning_present,
'has_evidence': self.check_evidence_present,
'addresses_prompt': self.check_prompt_addressed,
'appropriate_length': self.check_appropriate_length,
'clear_structure': self.check_clear_structure
}
def validate_explanation(self, explanation: str, original_prompt: str) -> Dict[str, Any]:
"""Validate explanation quality"""
results = {}
for rule_name, rule_func in self.validation_rules.items():
results[rule_name] = rule_func(explanation, original_prompt)
# Calculate overall score
passed_rules = sum(1 for result in results.values() if result['passed'])
overall_score = passed_rules / len(results)
return {
'validation_results': results,
'overall_score': overall_score,
'passed': overall_score >= 0.8,
'improvement_suggestions': self.generate_improvements(results)
}
def check_reasoning_present(self, explanation: str, prompt: str) -> Dict[str, Any]:
"""Check if explanation contains reasoning"""
reasoning_keywords = ['because', 'therefore', 'since', 'due to', 'as a result']
has_reasoning = any(keyword in explanation.lower() for keyword in reasoning_keywords)
return {
'passed': has_reasoning,
'details': f"Reasoning indicators found: {has_reasoning}",
'keywords_found': [kw for kw in reasoning_keywords if kw in explanation.lower()]
}
def check_evidence_present(self, explanation: str, prompt: str) -> Dict[str, Any]:
"""Check if explanation provides evidence"""
evidence_keywords = ['data', 'evidence', 'research', 'studies', 'analysis', 'statistics']
has_evidence = any(keyword in explanation.lower() for keyword in evidence_keywords)
return {
'passed': has_evidence,
'details': f"Evidence indicators found: {has_evidence}",
'keywords_found': [kw for kw in evidence_keywords if kw in explanation.lower()]
}
def check_prompt_addressed(self, explanation: str, prompt: str) -> Dict[str, Any]:
"""Check if explanation addresses the original prompt"""
prompt_words = set(prompt.lower().split())
explanation_words = set(explanation.lower().split())
# Remove common words
common_words = {'the', 'and', 'or', 'but', 'in', 'on', 'at', 'to', 'for', 'of', 'with', 'by'}
prompt_words -= common_words
explanation_words -= common_words
overlap = len(prompt_words.intersection(explanation_words))
overlap_ratio = overlap / len(prompt_words) if prompt_words else 0
return {
'passed': overlap_ratio >= 0.3,
'details': f"Prompt-explanation overlap: {overlap_ratio:.2f}",
'overlap_ratio': overlap_ratio
}
def check_appropriate_length(self, explanation: str, prompt: str) -> Dict[str, Any]:
"""Check if explanation is appropriate length"""
word_count = len(explanation.split())
appropriate = 50 <= word_count <= 500 # Reasonable range
return {
'passed': appropriate,
'details': f"Word count: {word_count} (target: 50-500)",
'word_count': word_count
}
def check_clear_structure(self, explanation: str, prompt: str) -> Dict[str, Any]:
"""Check if explanation has clear structure"""
# Look for structural elements
has_paragraphs = '\n' in explanation or len(explanation.split('.')) > 1
has_transitions = any(word in explanation.lower() for word in ['first', 'second', 'next', 'finally', 'however'])
clear_structure = has_paragraphs and has_transitions
return {
'passed': clear_structure,
'details': f"Structure indicators: paragraphs={has_paragraphs}, transitions={has_transitions}",
'has_paragraphs': has_paragraphs,
'has_transitions': has_transitions
}
def generate_improvements(self, results: Dict[str, Any]) -> List[str]:
"""Generate improvement suggestions"""
suggestions = []
for rule_name, result in results.items():
if not result['passed']:
if rule_name == 'has_reasoning':
suggestions.append("Add explicit reasoning with 'because', 'therefore', or 'since'")
elif rule_name == 'has_evidence':
suggestions.append("Include supporting evidence, data, or research")
elif rule_name == 'addresses_prompt':
suggestions.append("More directly address the original question")
elif rule_name == 'appropriate_length':
suggestions.append("Adjust explanation length (aim for 50-500 words)")
elif rule_name == 'clear_structure':
suggestions.append("Improve structure with clear transitions and paragraphs")
return suggestions
# Example usage
validator = ExplanationValidator()
sample_explanation = """
I recommend investing in renewable energy stocks because the sector shows strong growth potential.
Market data indicates a 15% annual growth rate over the past five years. Additionally, government
policies favor clean energy adoption, creating favorable regulatory conditions. However, consider
the higher volatility compared to traditional energy stocks.
"""
validation_results = validator.validate_explanation(sample_explanation, "Should I invest in renewable energy?")
print("Validation Results:")
print(json.dumps(validation_results, indent=2))
User Interface for Explanations
Create user-friendly explanation interfaces:
def create_explanation_dashboard():
"""Create HTML dashboard for explanation visualization"""
dashboard_html = """
<!DOCTYPE html>
<html>
<head>
<title>Ollama Explainable AI Dashboard</title>
<style>
body { font-family: Arial, sans-serif; margin: 20px; }
.container { max-width: 1200px; margin: 0 auto; }
.explanation-card {
border: 1px solid #ddd;
border-radius: 8px;
padding: 20px;
margin: 20px 0;
background: #f9f9f9;
}
.confidence-bar {
width: 100%;
height: 20px;
background: #eee;
border-radius: 10px;
overflow: hidden;
}
.confidence-fill {
height: 100%;
background: linear-gradient(to right, #ff4444, #ffaa44, #44ff44);
transition: width 0.3s ease;
}
.token-importance {
display: flex;
flex-wrap: wrap;
gap: 10px;
margin: 10px 0;
}
.token {
padding: 5px 10px;
border-radius: 5px;
font-size: 14px;
}
.high-importance { background: #ff6b6b; color: white; }
.medium-importance { background: #ffa500; color: white; }
.low-importance { background: #95a5a6; color: white; }
.metrics-grid {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 20px;
margin: 20px 0;
}
.metric-card {
background: white;
padding: 15px;
border-radius: 5px;
border-left: 4px solid #3498db;
}
.decision-tree {
background: white;
padding: 20px;
border-radius: 5px;
margin: 20px 0;
}
</style>
</head>
<body>
<div class="container">
<h1>Ollama Explainable AI Dashboard</h1>
<div class="explanation-card">
<h2>Model Decision Explanation</h2>
<div id="explanation-content">
<p><strong>Query:</strong> <span id="query-text">Loading...</span></p>
<p><strong>Decision:</strong> <span id="decision-text">Loading...</span></p>
<p><strong>Reasoning:</strong> <span id="reasoning-text">Loading...</span></p>
</div>
<h3>Confidence Level</h3>
<div class="confidence-bar">
<div class="confidence-fill" id="confidence-fill" style="width: 0%"></div>
</div>
<p id="confidence-text">Loading...</p>
</div>
<div class="explanation-card">
<h2>Token Importance Analysis</h2>
<div class="token-importance" id="token-importance">
<!-- Tokens will be populated by JavaScript -->
</div>
</div>
<div class="metrics-grid">
<div class="metric-card">
<h3>Completeness</h3>
<p id="completeness-score">Loading...</p>
</div>
<div class="metric-card">
<h3>Clarity</h3>
<p id="clarity-score">Loading...</p>
</div>
<div class="metric-card">
<h3>Consistency</h3>
<p id="consistency-score">Loading...</p>
</div>
<div class="metric-card">
<h3>Usefulness</h3>
<p id="usefulness-score">Loading...</p>
</div>
</div>
<div class="decision-tree">
<h2>Decision Process Flow</h2>
<canvas id="decision-canvas" width="800" height="400"></canvas>
</div>
</div>
<script>
// JavaScript for dynamic content loading
async function loadExplanationData() {
try {
const response = await fetch('/api/explain', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
prompt: "Sample investment recommendation query",
explanation_type: "comprehensive"
})
});
const data = await response.json();
updateDashboard(data);
} catch (error) {
console.error('Error loading explanation data:', error);
}
}
function updateDashboard(data) {
// Update explanation content
document.getElementById('query-text').textContent = data.explanation.original_prompt || 'N/A';
document.getElementById('decision-text').textContent = data.explanation.decision || 'N/A';
document.getElementById('reasoning-text').textContent = data.explanation.explanation || 'N/A';
// Update confidence
const confidence = data.confidence * 100;
document.getElementById('confidence-fill').style.width = confidence + '%';
document.getElementById('confidence-text').textContent =
`Confidence: ${confidence.toFixed(1)}% (${getConfidenceLevel(confidence)})`;
// Update token importance
updateTokenImportance(data.explanation.token_importance || []);
// Update metrics
updateMetrics(data.explanation.quality_metrics || {});
// Draw decision tree
drawDecisionTree(data.explanation.decision_flow || []);
}
function updateTokenImportance(tokens) {
const container = document.getElementById('token-importance');
container.innerHTML = '';
tokens.forEach(([token, importance]) => {
const span = document.createElement('span');
span.className = 'token ' + getImportanceClass(importance);
span.textContent = `${token} (${importance.toFixed(3)})`;
container.appendChild(span);
});
}
function updateMetrics(metrics) {
document.getElementById('completeness-score').textContent =
(metrics.completeness * 100).toFixed(1) + '%';
document.getElementById('clarity-score').textContent =
(metrics.clarity * 100).toFixed(1) + '%';
document.getElementById('consistency-score').textContent =
(metrics.consistency * 100).toFixed(1) + '%';
document.getElementById('usefulness-score').textContent =
(metrics.usefulness * 100).toFixed(1) + '%';
}
function getConfidenceLevel(confidence) {
if (confidence >= 80) return 'High';
if (confidence >= 60) return 'Medium';
return 'Low';
}
function getImportanceClass(importance) {
if (importance >= 0.3) return 'high-importance';
if (importance >= 0.1) return 'medium-importance';
return 'low-importance';
}
function drawDecisionTree(decisionFlow) {
const canvas = document.getElementById('decision-canvas');
const ctx = canvas.getContext('2d');
// Clear canvas
ctx.clearRect(0, 0, canvas.width, canvas.height);
// Draw simple decision flow
ctx.strokeStyle = '#3498db';
ctx.lineWidth = 2;
ctx.font = '14px Arial';
const steps = ['Input', 'Analysis', 'Reasoning', 'Decision'];
const stepWidth = canvas.width / steps.length;
steps.forEach((step, index) => {
const x = stepWidth * index + stepWidth / 2;
const y = canvas.height / 2;
// Draw step circle
ctx.beginPath();
ctx.arc(x, y, 30, 0, 2 * Math.PI);
ctx.fillStyle = '#3498db';
ctx.fill();
// Draw step text
ctx.fillStyle = 'white';
ctx.textAlign = 'center';
ctx.fillText(step, x, y + 5);
// Draw connection line
if (index < steps.length - 1) {
ctx.beginPath();
ctx.moveTo(x + 30, y);
ctx.lineTo(x + stepWidth - 30, y);
ctx.strokeStyle = '#3498db';
ctx.stroke();
}
});
}
// Load data when page loads
window.addEventListener('load', loadExplanationData);
</script>
</body>
</html>
"""
return dashboard_html
# Save dashboard to file
with open('explanation_dashboard.html', 'w') as f:
f.write(create_explanation_dashboard())
print("Dashboard created: explanation_dashboard.html")
Advanced Use Cases
Multi-Model Explanation Comparison
Compare explanations across different models:
class MultiModelExplainer:
def __init__(self, models: List[str]):
self.models = models
self.explainers = {model: OllamaExplainer(model) for model in models}
def compare_explanations(self, prompt: str, context: str = "") -> Dict[str, Any]:
"""Compare explanations across multiple models"""
model_explanations = {}
for model in self.models:
try:
explanation = self.explainers[model].explain_decision(prompt, context)
model_explanations[model] = explanation
except Exception as e:
model_explanations[model] = {"error": str(e)}
# Analyze differences
comparison_analysis = self.analyze_explanation_differences(model_explanations)
return {
"prompt": prompt,
"model_explanations": model_explanations,
"comparison_analysis": comparison_analysis,
"consensus": self.find_consensus(model_explanations),
"divergent_points": self.identify_divergent_points(model_explanations)
}
def analyze_explanation_differences(self, explanations: Dict[str, Any]) -> Dict[str, Any]:
"""Analyze differences between model explanations"""
valid_explanations = {k: v for k, v in explanations.items() if "error" not in v}
if len(valid_explanations) < 2:
return {"analysis": "Insufficient valid explanations for comparison"}
# Compare explanation lengths
lengths = {model: len(exp["explanation"]) for model, exp in valid_explanations.items()}
# Compare reasoning approaches
reasoning_keywords = ["because", "therefore", "since", "due to", "analysis", "evidence"]
reasoning_scores = {}
for model, explanation in valid_explanations.items():
text = explanation["explanation"].lower()
score = sum(1 for keyword in reasoning_keywords if keyword in text)
reasoning_scores[model] = score
return {
"explanation_lengths": lengths,
"reasoning_complexity": reasoning_scores,
"most_detailed": max(lengths.items(), key=lambda x: x[1])[0],
"most_structured": max(reasoning_scores.items(), key=lambda x: x[1])[0]
}
def find_consensus(self, explanations: Dict[str, Any]) -> List[str]:
"""Find common themes across explanations"""
valid_explanations = {k: v for k, v in explanations.items() if "error" not in v}
if len(valid_explanations) < 2:
return []
# Extract common words/phrases
all_words = []
for explanation in valid_explanations.values():
words = explanation["explanation"].lower().split()
all_words.extend(words)
# Find words that appear in multiple explanations
word_counts = {}
for word in all_words:
if len(word) > 4: # Skip short words
word_counts[word] = word_counts.get(word, 0) + 1
# Return words that appear in most explanations
threshold = len(valid_explanations) * 0.6
consensus_words = [word for word, count in word_counts.items() if count >= threshold]
return sorted(consensus_words, key=lambda x: word_counts[x], reverse=True)[:10]
def identify_divergent_points(self, explanations: Dict[str, Any]) -> List[str]:
"""Identify points where models disagree"""
valid_explanations = {k: v for k, v in explanations.items() if "error" not in v}
if len(valid_explanations) < 2:
return []
divergent_points = []
# Check for contradictory keywords
positive_keywords = ["recommend", "suggest", "good", "positive", "beneficial"]
negative_keywords = ["avoid", "not recommend", "negative", "risky", "problematic"]
positive_models = []
negative_models = []
for model, explanation in valid_explanations.items():
text = explanation["explanation"].lower()
positive_score = sum(1 for keyword in positive_keywords if keyword in text)
negative_score = sum(1 for keyword in negative_keywords if keyword in text)
if positive_score > negative_score:
positive_models.append(model)
elif negative_score > positive_score:
negative_models.append(model)
if positive_models and negative_models:
divergent_points.append(f"Conflicting recommendations: {positive_models} vs {negative_models}")
return divergent_points
# Example usage
multi_explainer = MultiModelExplainer(["llama2", "mistral", "codellama"])
comparison = multi_explainer.compare_explanations(
"Should a startup invest in AI infrastructure?",
"Early-stage startup with limited funding but high growth potential"
)
print("Multi-Model Comparison:")
print(json.dumps(comparison, indent=2))
Domain-Specific Explanations
Create specialized explanations for different domains:
class DomainSpecificExplainer:
def __init__(self, domain: str, model_name: str = "llama2"):
self.domain = domain
self.model_name = model_name
self.domain_templates = self.load_domain_templates()
def load_domain_templates(self) -> Dict[str, str]:
"""Load domain-specific explanation templates"""
templates = {
"medical": """
As a medical AI assistant, provide explanations that include:
1. Clinical reasoning based on symptoms/data
2. Differential diagnosis considerations
3. Risk factors and contraindications
4. Evidence-based recommendations
5. When to seek professional consultation
Always emphasize the need for professional medical advice.
""",
"financial": """
As a financial advisor, structure explanations with:
1. Risk assessment and risk tolerance
2. Market analysis and economic factors
3. Diversification considerations
4. Time horizon implications
5. Regulatory and tax considerations
Include appropriate disclaimers about financial advice.
""",
"legal": """
As a legal information provider, include:
1. Relevant legal principles and precedents
2. Jurisdiction-specific considerations
3. Potential legal risks and benefits
4. Procedural requirements
5. When to consult a qualified attorney
Emphasize that this is information, not legal advice.
""",
"technical": """
As a technical expert, provide explanations with:
1. Technical specifications and requirements
2. Implementation considerations
3. Performance and scalability factors
4. Security and maintenance implications
5. Alternative approaches and trade-offs
Include practical implementation guidance.
"""
}
return templates
def explain_with_domain_context(self, prompt: str, context: str = "") -> Dict[str, Any]:
"""Generate domain-specific explanation"""
domain_template = self.domain_templates.get(self.domain, "")
enhanced_prompt = f"""
{domain_template}
Context: {context}
Query: {prompt}
Provide a comprehensive explanation appropriate for the {self.domain} domain.
"""
response = ollama.generate(
model=self.model_name,
prompt=enhanced_prompt
)
return {
"domain": self.domain,
"original_prompt": prompt,
"domain_enhanced_prompt": enhanced_prompt,
"explanation": response['response'],
"domain_specific_elements": self.extract_domain_elements(response['response']),
"compliance_check": self.check_domain_compliance(response['response'])
}
def extract_domain_elements(self, explanation: str) -> List[str]:
"""Extract domain-specific elements from explanation"""
domain_keywords = {
"medical": ["symptoms", "diagnosis", "treatment", "clinical", "patient", "medical"],
"financial": ["risk", "return", "investment", "portfolio", "market", "financial"],
"legal": ["law", "legal", "court", "statute", "regulation", "jurisdiction"],
"technical": ["system", "architecture", "performance", "scalability", "security", "implementation"]
}
keywords = domain_keywords.get(self.domain, [])
found_elements = [keyword for keyword in keywords if keyword.lower() in explanation.lower()]
return found_elements
def check_domain_compliance(self, explanation: str) -> Dict[str, Any]:
"""Check if explanation complies with domain standards"""
compliance_checks = {
"medical": {
"has_disclaimer": "medical advice" in explanation.lower() or "consult" in explanation.lower(),
"evidence_based": "research" in explanation.lower() or "studies" in explanation.lower(),
"risk_mentioned": "risk" in explanation.lower() or "side effect" in explanation.lower()
},
"financial": {
"has_disclaimer": "not financial advice" in explanation.lower() or "consult" in explanation.lower(),
"risk_disclosed": "risk" in explanation.lower(),
"diversification": "diversif" in explanation.lower() or "portfolio" in explanation.lower()
},
"legal": {
"has_disclaimer": "not legal advice" in explanation.lower() or "attorney" in explanation.lower(),
"jurisdiction_noted": "jurisdiction" in explanation.lower() or "local" in explanation.lower(),
"precedent_cited": "precedent" in explanation.lower() or "case law" in explanation.lower()
},
"technical": {
"specs_included": "specification" in explanation.lower() or "requirement" in explanation.lower(),
"alternatives_discussed": "alternative" in explanation.lower() or "option" in explanation.lower(),
"implementation_guidance": "implement" in explanation.lower() or "step" in explanation.lower()
}
}
checks = compliance_checks.get(self.domain, {})
results = {check: result for check, result in checks.items()}
return {
"checks": results,
"compliance_score": sum(results.values()) / len(results) if results else 0.0,
"compliant": all(results.values())
}
# Example usage
medical_explainer = DomainSpecificExplainer("medical")
medical_explanation = medical_explainer.explain_with_domain_context(
"Should I be concerned about chest pain?",
"35-year-old male, no previous heart issues, pain started after exercise"
)
print("Medical Domain Explanation:")
print(json.dumps(medical_explanation, indent=2))
Conclusion
Explainable AI transforms black-box models into transparent, accountable systems. With Ollama's local deployment capabilities, you can implement comprehensive explanation frameworks that provide real-time insights into model decisions.
The techniques covered in this guide enable you to:
- Interpret individual model decisions with detailed explanations
- Debug model behavior and identify decision errors
- Build production-ready explanation systems with monitoring
- Create domain-specific explanations for specialized use cases
- Validate explanation quality automatically
Remember that explainable AI is not just about technical implementation—it's about building trust, ensuring accountability, and enabling better decision-making. Start with basic explanations and gradually add more sophisticated interpretation techniques as your needs evolve.
The future of AI depends on transparency. By implementing explainable AI with Ollama, you're not just building better systems—you're contributing to responsible AI development that benefits everyone.
Ready to implement explainable AI in your Ollama projects? Start with the basic explanation framework and expand based on your specific use case requirements. The key is to begin with simple, clear explanations and build complexity as needed.