spent some time this year turning a manual workflow into an automated one. the core problem was intent classification.

the task: read a Reddit post and determine how close the person is to making a purchase decision. not just topic matching. actual intent scoring based on language, context, specificity of the problem described.

keyword matching does not work well for this. a post containing the word "tool" or "software" tells you almost nothing. what matters is whether the person is comparing solutions, has already tried something that failed, or is asking for a direct recommendation.

the model needs to distinguish between someone venting and someone with budget ready to move. that distinction is where the signal is.

I tested several models on this classification task. the variance in output quality was significant. some handle ambiguous phrasing poorly. some are inconsistent at the threshold cases which are the ones that matter most.

still refining the scoring logic. curious if anyone here has worked on similar classification problems and what approaches produced reliable results at the edges.

Leadline btw if you want to see the implementation.