Hi y'all,
I came across a volunteer data engineer position. They'd be willing to let me learn everything I need to satisfy the following task:
"Data Engineering -- two CRMs w/ backend SQL mirrors that we need to create automated syncs between"
What would this entail? What would I need to learn to create that? Python? Which concepts?
Posted Dec 8, 2023, 9:33 PM UTC
Ngl, this sounds like a company that would prefer to fuck around and find out instead of just paying someone that knows what they're doing. If this system contains customer data, one of the volunteers with no experience and no financial incentive to not fuck this up could cause a serious gdpr issue
This may be a good deal. Strong emphasis on may.
Are they giving you tools? (laptop, network connection, accounts, access to systems, budget for software / training?)
Start with a picture.
Who is the admin for their CRMs? Because you'll want to talk to them. Get a picture or two.
What's the geography of the CRMs and databases? Same physical location? (in the tool shed?) Same building? Different cities? Same Cloud? Different Clouds?
The part about "backend SQL mirrors" could mean several things. Get a network diagram.
What CRMs & Versions?
What database(s)? Same? Different?
Test environments? (doubtful but you should ask)
How do they do backups today? (If the answer is "Whats a backup"then Hard No, run away fast)
How many users of each CRM?
What are their operational hours? 24-7? Weekdays 8am-5pm local time?
Ps. What country? If you're in the USA you'll want to sign a liability waiver saying "Ap3340 is strictly a volunteer with zero experience, so if anything bad happens to TheCompany it is 100% not AP3340's fault. TheCompany acknowledges that AP3340 promises nothing." Otherwise if you nuke one of their CRMs, or it gets hit with ransomware, they're going to lawyer up and sue you. Actually you'll probably want them to sign such a waiver wherever you are.
Got it. Thank you for the in-depth answer. Would this task require the use of Python? They know that I don’t have any experience with data engineering. What technical tools would I be learning?
Would this task require the use of Python?
Is Python required? No, I don't think so.
Could you use Python? Sure.
Would Python be helpful? Maybe.
You'll have to learn more about their environment & goals before you start choosing tools. Or at least before you start choosing an approach.
Really you don't know anything except "they want to synchronize some thing(s) between two CRM systems."
CRM = Customer Relationship Management.
So probably customer data?
Exactly what kinds of customers does a "volunteer only" organization actually have?
CRM is a salesperson tool, so what is their organization selling?
I'd start by getting an example of what synchronizing means to them. Actually how do they synchronize things today? Whoever does that is another person you need to talk to.
Do they want to "synchronize" just names? Names and contact info? "Customer" outreach activity?
Copy new stuff one-way? e.g. from CRM.A into CRM.B? Copy new things from CRM.A into CRM.B? And anything new in CRM.B over to CRM.A ? Weekly copy? Nightly copy?
CRM.A: user adds "Alice Jones". sync: Check CRM.B and add "Alice Jones" if not already there. What do they do if Alice is already in CRM.B ?
It’s a political organization. The customers may be referring to residents.
Ok. Good luck to you!
How much are they paying you? That's a lot of work, only worth it if they're paying
I’m using this experience to get a better job. Would require no more than a few hours a week
Be skeptical. That's a huge amount of work that they should absolutely be compensating you for. If they dont, it'll be bad for both of you
All of their positions are volunteer. Do you know what skills I’d be learning if I happened to take it on?
None, it sounds like they're expecting you to do the work without any resources. This is a HUGE red flag. This is going to be expensive any way you dice it. If they're looking for free labor at this point you're not going to succeed