DynamoDB should make it easier to export backups. Data Pipeline/EMR is very inefficient and cumbersome.
Why can't DynamoDB backups be made available for download? This is one of our biggest fiction point with DynamoDB:
At the moment, it is very inefficient, complicated and cumbersome to export all data from a DynamoDB table with Data Pipeline/EMR.
To add insult to injury, the current Data Pipeline template does not even support On-Demand Capacity Tables.. You're either forced to:
- Change the provisioning mode for the table, which can only be done once per day and risks impacting production workloads.
- Build emr-dynamodb-connector from source, deploy and configure it, without official support or documentation.
As it is, it feels DynamoDB tries its best to keep our data hostage, which is starting to negatively impact its attractiveness for our future projects.
Does anyone have other ways to achieve this? We've used dynamodump.py in the past, but again, it is very inefficient to scan the whole database.
Posted Oct 25, 2019, 4:52 PM UTC
ugh... dealing with this right now.. Data Pipeline is the worst AWS Service I've come across so far.
Even worse than Cognito?
I was at least able to get Cognito to work after much frustration. Data Pipeline not so much.