What is the ubiquitous way to do “embarrassingly parallel” computing on AWS with a large amount of data?

I’ve looked into a number of different AWS services that can massively distribute parts of a large data set to many machines for computation. So far I have looked into Lambdas, Elastic MapReduce, and just managing EC2 instanced on my own.

Managing my own instances is not appealing as I do not want to incur the development costs of handling faulty instances, retrying, etc. Lambdas initially appeared promising but the size of my problem is too large, so that’s a no-go. EMR looks like the way to go, but I’ve heard mixed things about Hadoop MapReduce (main criticisms being its slow and hard to program).

I want to understand what the industry standard service is for my use case. For context, I have a huge amount of data over a twenty year period, and I want to do independent statistical analysis (regression) on every single day using R. Data size within each day is large, but could maybe fit into memory on large EC2 instances. Think of a data frame with 12000 variables and 120000 observations.

Any direction from those in the know would be greatly appreciated.