Should working as a machine learning engineer feel like a lot of trial and error?
A work project for me usually goes something like:
| Percentage of Time | Task |
|---|---|
| 10 | Researching problem |
| 20 | Implementing solution |
| 60 | Changing things via trial and error to get better results |
| 10 | Final testing |
The trial and error isn't random. I usually look at my results and see what I can do to improve accuracy, but the changes I make in my code don't always work out as expected, then I have to try other methods to bump up the accuracy of my results.
It might be helpful to mention most of the work I'm doing involves NLP, specifically information extraction from raw text data.
This is my first internship, so I wanna know if a job with high amounts of NLP is usually like this, or if I just really suck at predicting outcomes. It's hard when I'm going to multiple files, each with differences is formatting (some information is in a table, others are hidden in text, some important headers only have the first word capitalized making it difficult to distinguish from other text, etc)
It's frustrating when I usually have to try so many different solutions to a problem to see which ones are best, or more generalized.
Posted Jul 12, 2017, 11:36 PM UTC · edited
Sounds like (a portion of) the scientific method to me.
How does one build intuition without experimentation?
Keep plugging away, taking notes, recording results. You will improve.
Thanks man, I needed to hear that