I Built an AI-Powered SNAP Eligibility Detection System to Make Benefits Access Faster, Fairer, and Easier
Building an AI-Powered SNAP Eligibility Detection System: Tech Stack, OCR Pipeline, AI Architecture, and Development Process
I built an AI-powered SNAP eligibility detection system to help applicants understand whether they may qualify for benefits, upload required documents, and receive a clear eligibility screening recommendation before human review.
The system was designed as more than a simple form. It combines a guided intake experience, OCR-based document processing, AI-assisted field extraction, a rules-based eligibility engine, a caseworker review dashboard, and an explainability layer that turns complex eligibility logic into plain language.
The goal was not to build an automated approval or denial machine. The goal was to build a responsible decision-support platform that helps users submit better information and helps caseworkers review applications faster and more consistently.
The Problem I Wanted to Solve
Applying for SNAP can be confusing because eligibility depends on many factors: household size, income, expenses, rent, utilities, disability status, student status, work requirements, state-specific policies, and supporting documentation.
A user may enter the wrong income amount, upload an unreadable pay stub, forget to include a household member, or misunderstand what document is required. These issues can delay applications and create more work for caseworkers.
I wanted to solve three major problems:
First, applicants need clearer guidance while completing the application.
Second, caseworkers need a faster way to review documents and eligibility factors.
Third, the system needs to explain why it reaches a recommendation instead of acting like a black box.
That led me to build an application where AI helps with document understanding, missing-information detection, and plain-language explanations, while a deterministic rules engine handles eligibility logic.
High-Level System Architecture
The application is built around five major layers:
- Applicant intake interface
- Document upload and OCR pipeline
- AI extraction and explanation layer
- Rules-based eligibility engine
- Caseworker review dashboard
The architecture looks like this:
Applicant
↓
Guided Intake Form
↓
Secure Document Upload
↓
OCR + Document AI
↓
Structured Case Data
↓
Rules-Based Eligibility Engine
↓
Eligibility Recommendation
↓
Caseworker Review
↓
Final Human Decision
I intentionally separated the AI layer from the eligibility engine. AI is useful for reading documents, identifying missing information, and generating explanations, but the actual eligibility logic needs to be deterministic, auditable, and based on approved rules.
That separation became one of the most important technical decisions in the project.
Tech Stack
For the frontend, I used a modern component-based web architecture. The applicant portal and caseworker dashboard were designed with responsive layouts, reusable UI components, and clear form validation.
The frontend stack includes:
Frontend:
- React / Next.js
- TypeScript
- Tailwind CSS or custom CSS
- Component-based dashboard UI
- Responsive applicant and caseworker views
The backend is API-driven and split into services. I used a backend structure that separates user management, intake, document processing, rules evaluation, recommendations, and audit logs.
The backend stack includes:
Backend:
- Node.js / Express or FastAPI
- REST API endpoints
- PostgreSQL for structured data
- Redis for background job queues
- Object storage for uploaded documents
- Authentication and role-based access control
For AI and document processing, I used a Python-based service layer because Python has strong support for OCR, image preprocessing, natural language processing, and machine learning workflows.
The AI/OCR stack includes:
AI and OCR:
- Python
- OCR engine for text extraction
- Image preprocessing pipeline
- Document classification model
- Field extraction model
- LLM-based explanation generation
- Retrieval-augmented policy assistant
For infrastructure, the system is designed to support secure deployment with encrypted storage, background processing, logging, monitoring, and auditability.
Infrastructure:
- Encrypted database storage
- Secure object storage for documents
- Background workers for OCR processing
- Audit logging
- Error monitoring
- Role-based access
- Environment-based configuration
Building the Frontend
The frontend has two main experiences: the applicant portal and the caseworker dashboard.
The applicant portal is designed to feel simple and guided. Instead of showing a long government-style form, the app walks the user through one section at a time.
The applicant flow includes:
1. Account creation
2. State selection
3. Household information
4. Income details
5. Expenses and deductions
6. Document upload
7. Eligibility screening result
8. Next steps
The intake flow is dynamic. That means the next question depends on the user’s previous answers.
For example, if the user says someone in the household is elderly or disabled, the system asks additional questions about medical expenses and special deductions. If the user reports employment income, the system asks about pay frequency and requests income verification documents.
This makes the experience easier for users because they only see questions that apply to their situation.
On the caseworker side, I built a dashboard that organizes each case into a reviewable summary.
The caseworker dashboard includes:
- Applicant profile
- Household composition
- Income breakdown
- Expense breakdown
- Uploaded documents
- OCR extraction results
- AI confidence scores
- Missing document alerts
- Eligibility recommendation
- Rule-by-rule explanation
- Caseworker notes
- Manual override controls
The dashboard is important because the system is not supposed to make final decisions by itself. It gives the caseworker a clear, organized view of the case so they can verify information and make the official determination.
Backend Design
The backend is organized into services so that each part of the system has a clear responsibility.
The main backend services are:
Auth Service
- Handles login, sessions, user roles, and permissions.
Intake Service
- Stores household information, income, expenses, and user responses.
Document Service
- Handles uploads, file validation, secure storage, and document metadata.
OCR Service
- Converts uploaded documents into machine-readable text.
AI Extraction Service
- Classifies documents and extracts structured fields.
Eligibility Engine
- Applies rules to structured household data.
Recommendation Service
- Generates likely eligible, more information needed, or likely ineligible.
Explanation Service
- Converts technical rule results into plain-language explanations.
Audit Service
- Records every major user, AI, and caseworker action.
This modular structure makes the application easier to test and easier to scale. If the OCR service needs improvement, I can update that service without rewriting the rules engine. If SNAP policy rules change, I can update the rules engine without changing the document upload system.
Data Model
The data model was one of the most important parts of the project because SNAP eligibility depends heavily on household structure and verified information.
The core entities include:
User
Household
HouseholdMember
IncomeSource
Expense
Resource
Document
ExtractedDocumentField
EligibilityCase
EligibilityRule
RuleEvaluation
EligibilityRecommendation
CaseworkerReview
AuditLog
Each uploaded document is stored separately from the fields extracted from it.
For example, when a user uploads a pay stub, the system stores the original file in secure document storage. Then the OCR and AI pipeline extracts fields like gross pay, pay date, employer name, and pay period. Those extracted fields are stored in the database with confidence scores.
A simplified extracted field might look like this:
{
"document_type": "pay_stub",
"field_name": "gross_pay",
"field_value": "520.00",
"confidence": 0.96,
"source_page": 1,
"requires_human_review": false
}
If the confidence score is low, the system does not treat the field as verified. Instead, it flags the field for caseworker review.
How the OCR Pipeline Works
OCR stands for Optical Character Recognition. It is the process of converting text inside an image or scanned document into machine-readable text.
In this application, OCR is used to read documents such as pay stubs, rent receipts, utility bills, IDs, lease agreements, and medical expense records.
The OCR pipeline has several steps.
1. File Upload
The user uploads a document through the applicant portal. The system checks the file type, file size, and upload validity.
Accepted files may include:
- PDF
- PNG
- JPG
- JPEG
The file is stored securely, and the document metadata is saved in the database.
2. Preprocessing
Before OCR runs, the document is cleaned up to improve text recognition.
Preprocessing may include:
- Converting the file to an image
- Rotating the image if it is sideways
- Deskewing tilted text
- Removing noise
- Increasing contrast
- Converting to grayscale
- Cropping unnecessary borders
- Detecting page layout
This step matters because many users upload phone photos, screenshots, or scans that are not perfectly clear. A blurry or tilted pay stub can cause incorrect OCR results, so preprocessing improves accuracy.
3. Text Detection
The OCR engine detects where text appears on the page. It identifies blocks, lines, and words.
Instead of only returning plain text, the OCR engine can also return bounding boxes. A bounding box tells the system where a word or field appears on the page.
Example OCR output:
{
"text": "Gross Pay",
"bbox": [122, 430, 205, 452],
"confidence": 0.98
}
This is useful because the system can understand not only what text was found, but also where it appeared.
4. Text Recognition
After detecting text areas, the OCR engine converts the image text into actual characters.
For example, a pay stub image might become:
Employee: Maria Garcia
Employer: Metro Foods
Pay Date: 05/24/2026
Gross Pay: $520.00
Net Pay: $438.21
At this stage, the OCR output is still raw. It may contain errors, missing characters, or formatting problems. That is why the next step is document classification and field extraction.
5. Document Classification
The system then classifies the document type.
It determines whether the uploaded file is likely a:
- Pay stub
- Rent statement
- Utility bill
- ID document
- Lease agreement
- Medical bill
- Childcare receipt
- Unknown document
This classification matters because each document type has different fields.
A pay stub needs gross pay and pay date. A rent receipt needs rent amount and address. A utility bill needs service address and billing period.
6. Field Extraction
Once the document type is known, the AI extraction service searches for key fields.
For a pay stub, the system tries to extract:
- Employee name
- Employer name
- Pay date
- Pay period
- Gross pay
- Net pay
- Hours worked
- Year-to-date earnings
For a rent document, it tries to extract:
- Tenant name
- Address
- Rent amount
- Payment date
- Landlord name
For a utility bill, it tries to extract:
- Provider name
- Service address
- Billing period
- Amount due
- Due date
The extraction service uses a combination of pattern recognition, document layout, and AI-based interpretation.
For example, the system may look for labels like “Gross Pay,” “Current Earnings,” or “Total Gross.” It then identifies the value near that label.
7. Confidence Scoring
Every extracted field receives a confidence score.
A high-confidence extraction might look like this:
{
"field": "gross_pay",
"value": "520.00",
"confidence": 0.96,
"status": "verified_by_ai"
}
A low-confidence extraction might look like this:
{
"field": "rent_amount",
"value": "unclear",
"confidence": 0.48,
"status": "human_review_required"
}
This is one of the most important safety features. The system does not blindly trust AI extraction. If confidence is low, the field is routed to the caseworker for verification.
How the AI Works
The AI layer is designed to assist, not replace, the eligibility decision.
It performs five main functions:
1. Classify uploaded documents
2. Extract structured fields
3. Compare user-entered data with document data
4. Detect missing or inconsistent information
5. Generate plain-language explanations
The AI does not make the final SNAP decision. It prepares the case for review and helps the rules engine receive cleaner structured data.
Document AI
The document AI service takes OCR text and layout data, then turns it into structured information.
For example, raw OCR text from a pay stub may look messy:
METRO FOODS
Employee Maria Garcia
Pay Date 05/24/2026
Current Gross 520.00
Deductions 81.79
Net Pay 438.21
The AI extraction service converts that into:
{
"document_type": "pay_stub",
"employee_name": "Maria Garcia",
"employer_name": "Metro Foods",
"pay_date": "2026-05-24",
"gross_pay": 520.00,
"net_pay": 438.21
}
This structured data can then be used by the eligibility engine.
Inconsistency Detection
The AI also compares what the user entered with what the documents show.
For example:
User entered weekly gross income: $500
Pay stub gross income detected: $520
Difference: $20
Status: Acceptable difference or review depending on threshold
If the difference is small, the system may simply note it. If the difference is large, the case is flagged for review.
The system does not accuse the applicant of fraud. It only says that the information needs verification.
That distinction is important because benefit applications often contain honest mistakes.
Missing Information Detection
The AI also checks whether required documents or answers are missing.
For example:
Reported income: Yes
Uploaded pay stub: No
System action: Request income verification
Or:
Reported rent: Yes
Uploaded rent proof: Yes
OCR confidence: Low
System action: Request clearer document or caseworker review
This helps applicants correct problems earlier and reduces the amount of back-and-forth with caseworkers.
Explanation Generation
The explanation service converts technical results into plain language.
The rules engine may produce a technical result like this:
{
"rule": "income_verification_required",
"status": "pending",
"reason": "earned_income_document_missing_or_low_confidence"
}
The explanation service turns that into:
We need a clearer document showing your income or expenses before your application can be fully reviewed.
For caseworkers, the explanation can be more detailed:
Income was entered as weekly employment income. The uploaded pay stub was processed successfully with 96% confidence. Rent was entered as $1,250, but the rent document could not be read clearly and requires manual verification.
This makes the system useful for both applicants and reviewers.
The Rules-Based Eligibility Engine
The eligibility engine is separate from the AI layer.
This engine receives structured household data and applies rules to generate a recommendation.
The rules engine evaluates:
- Household size
- Gross income
- Net income
- Income frequency
- Earned income
- Unearned income
- Shelter expenses
- Utility expenses
- Medical expenses
- Childcare expenses
- Resource rules
- Elderly or disabled household members
- Student status
- Work-related conditions
- State-specific requirements
The output is not just a result. It is a rule-by-rule evaluation.
Example:
{
"recommendation": "MORE_INFORMATION_NEEDED",
"rules": [
{
"name": "household_composition",
"status": "passed",
"explanation": "Household size was provided."
},
{
"name": "income_screening",
"status": "likely_passed",
"explanation": "Income was entered and verified from uploaded pay stub."
},
{
"name": "shelter_expense_verification",
"status": "pending",
"explanation": "Rent amount requires a clearer document."
}
]
}
This makes the system explainable and auditable.
Why I Did Not Use AI for Final Eligibility Decisions
I intentionally avoided using AI as the final decision-maker.
A machine learning model might be able to predict whether someone is likely eligible based on previous cases, but that is not the same as applying official eligibility rules. Public benefits decisions must be explainable, reviewable, and tied to policy.
That is why the system uses:
AI for assistance
Rules engine for eligibility logic
Human review for final decisions
This design keeps the system safer and more transparent.
Building the Recommendation System
The recommendation service combines information from the rules engine, OCR pipeline, and AI extraction layer.
It returns one of three statuses:
LIKELY_ELIGIBLE
MORE_INFORMATION_NEEDED
LIKELY_NOT_ELIGIBLE
A typical recommendation object looks like this:
{
"status": "MORE_INFORMATION_NEEDED",
"confidence": "MEDIUM",
"summary": "The household may qualify, but rent verification is incomplete.",
"missing_items": [
"Clear rent statement"
],
"human_review_required": true
}
This result is then shown to both the applicant and the caseworker in different levels of detail.
The applicant sees a simplified explanation. The caseworker sees the full rule breakdown, extracted fields, confidence scores, and audit trail.
Security and Privacy
Because the application handles sensitive personal and financial data, security was a major part of the build.
I designed the system around several security principles:
- Encrypt data in transit
- Encrypt data at rest
- Store documents securely
- Use role-based access control
- Limit access by user role
- Log all major actions
- Validate files before processing
- Avoid collecting unnecessary information
- Separate applicant and caseworker permissions
Uploaded documents are stored in secure object storage. The database stores document metadata and extracted fields, but access to the original file is restricted.
Caseworkers can only access cases they are authorized to review. Applicants can only access their own application. Admin users can manage rules and system settings, but their access should also be controlled and logged.
Audit Logging
Audit logging is essential for this type of system.
Every major action is recorded:
- User created application
- Household information submitted
- Document uploaded
- OCR processing started
- OCR processing completed
- Field extracted
- AI confidence score assigned
- Eligibility recommendation generated
- Caseworker opened case
- Caseworker changed a value
- Caseworker requested more information
- Caseworker made final decision
A simplified audit log entry looks like this:
{
"case_id": "SNAP-2026-0418",
"event": "FIELD_EXTRACTED",
"field": "gross_pay",
"value": "520.00",
"confidence": 0.96,
"timestamp": "2026-05-24T14:42:00Z"
}
This makes the system accountable. If someone needs to understand why a recommendation was made, the audit trail shows the exact steps.
Background Jobs and Processing Queue
Document processing can take time, especially for large PDFs or low-quality images. To avoid making users wait, I used a background job system.
The flow works like this:
1. User uploads document
2. API stores document
3. API creates processing job
4. Background worker runs OCR
5. AI service extracts fields
6. Results are saved
7. User interface updates document status
This makes the app feel faster. The user can continue filling out the application while the document is being processed in the background.
Testing the System
Testing was a major part of the build because the system needs to be reliable.
I tested the system at multiple levels:
Frontend testing:
- Form validation
- Dynamic question logic
- Mobile responsiveness
- Upload experience
- User guidance messages
Backend testing:
- API endpoints
- Authentication
- Role permissions
- Data validation
- Error handling
Rules engine testing:
- Household scenarios
- Income calculations
- Deduction calculations
- Missing information cases
- State-specific rule behavior
OCR testing:
- Pay stubs
- Rent receipts
- Utility bills
- Blurry images
- Rotated images
- Multi-page PDFs
AI testing:
- Document classification accuracy
- Field extraction accuracy
- Confidence scoring
- Explanation quality
For the rules engine, I created test households with known expected outputs. This allowed me to check whether the system was calculating recommendations correctly.
For OCR, I tested with different document formats and image qualities because real users do not upload perfect documents.
Challenges I Faced
One of the biggest challenges was document variability.
Pay stubs, rent receipts, utility bills, and IDs all come in different formats. Some users upload PDFs. Others upload phone photos. Some documents are clear. Others are blurry, tilted, cropped, or poorly lit.
That made OCR preprocessing very important.
Another challenge was income normalization. Users can be paid weekly, biweekly, monthly, irregularly, or from multiple sources. The system needed to convert these income types into a monthly estimate while preserving the original values for caseworker review.
Another challenge was designing the AI safely. It would have been easier to let a model generate a final answer, but that would not be responsible for a benefits eligibility system. Instead, I made the AI produce structured support data and explanations while leaving eligibility logic to rules and final decisions to humans.
What I Learned
This project taught me that responsible AI is not just about choosing a model. It is about system design.
The real value came from combining multiple technologies carefully:
OCR reads the documents.
AI extracts and explains information.
The rules engine applies eligibility logic.
The dashboard supports human review.
Audit logs create accountability.
Each part has a specific role.
The AI does not need to do everything. In fact, the system is better because the AI does not do everything.
Final Architecture Summary
The final system works like this:
Applicant completes guided intake
↓
Applicant uploads documents
↓
OCR extracts raw text
↓
AI classifies documents and extracts fields
↓
System assigns confidence scores
↓
Structured data is saved
↓
Rules engine evaluates eligibility factors
↓
Recommendation service creates result
↓
Explanation service creates plain-language summary
↓
Caseworker reviews the case
↓
Human decision is made
This architecture allowed me to build an application that is useful, explainable, and responsible.
Final Thoughts
Building this AI-powered SNAP eligibility detection system showed me how AI can improve public service applications when it is used carefully.
The system does not replace people. It helps people.
It helps applicants understand what information they need. It helps caseworkers review cases faster. It helps agencies organize data more consistently. And it keeps the final decision in human hands.
The most important technical decision was separating AI assistance from eligibility logic. OCR and AI make the process smarter, but the rules engine and human review make it trustworthy.
That balance is what makes the application powerful.
It is not just an AI app. It is a full eligibility support platform built with document intelligence, structured rules, explainable recommendations, secure data handling, and responsible human oversight.
Posted Jun 5, 2026, 4:57 AM UTC