How do you handle fine-grained authorization at scale ?

I’m building a personal project mainly for learning, but also as a reusable backend template. The architecture is a multi-tenant system with a dedicated IAM provider (Keycloak).

Authentication seems relatively straightforward:

* SPA uses Authorization Code Flow + PKCE
* User authenticates through Keycloak
* SPA receives access token
* Token is attached to API requests
* Resource server validates JWT

(Or use BFF + http-only cookies instead.)

Where I’m getting stuck is authorization.

My initial approach was:

* Define static fine-grained permissions in the API (e.g. `invoice:read`, `invoice:write`)
* Mirror those as non-composite client roles in Keycloak
* Allow tenant admins to create composite roles from those permissions
* Assign composite roles to users

Then use protocol mappers to include permissions in the JWT and validate them with policies like:

```csharp
[Authorize(Policy = "invoice:read")]
```

At first this seemed clean, but then I started thinking about scale.

If the system grows to hundreds of permissions across many domains/features, embedding all permissions in JWTs starts feeling problematic:

* token bloat
* larger headers on every request
* permission changes requiring token refresh
* potentially ugly role/permission management in Keycloak

So now I’m questioning whether pushing fine-grained authorization into the token is even the right approach.

The two alternatives I’m considering are:

### Option 1 — Keycloak for AuthN only

Use Keycloak strictly for authentication/identity.

Handle authorization entirely inside the application:

* roles/permissions stored in app DB
* many-to-many role/permission/user tables
* cache authorization data
* middleware/policies check DB/cache

This feels more flexible and scalable, but also means rebuilding a large part of authorization logic myself.

### Option 2 — Use Keycloak Authorization Services

Use Keycloak for both AuthN and AuthZ:

* scopes
* policies
* permissions
* authorization engine

But this seems like it could introduce:

* extra latency
* coupling business authorization to IAM
* operational complexity
* harder debugging

My main question:

How do large-scale systems typically solve fine-grained authorization without exploding JWT size or turning the IAM into a bottleneck?

Do most teams:

* keep JWTs coarse-grained and resolve permissions server-side?
* use RBAC only in tokens?
* combine RBAC + ABAC?
* use a policy engine like OPA/Casbin?
* centralize authz or keep it inside each service?

Would really appreciate hearing how people structure this in production systems.