Work situation advice - need to migrate to DB (complicated)

This might be lengthy and I'm trying to be a little vague about some things since I don't know how much I can reveal without getting into trouble.

I'm working as a data management/Jr dev (very junior - mostly VBA due to environment limitations and lack of personal experience and need for simple working tools in the immediate) at a small telecom company.

Before I started, a lot of the design layout for the projects were done on paper by hand, then things moved to messy Excel workbooks with no macros and separate sheets. At some point each project was converted to a pretty slick functional Excel format which was probably 90% automated.

Each workbook represents a project which represents cabling for a large physical area (like miles by miles). Within each workbook are 50-150 sheets which represent objects and the individual cabling for each device. Again, I'm trying to be vague here, but if you guessed the industry, great!

We use Smartsheets as a pseudo-DB where field techs use a Smartsheet form to report data back. There are individual sheets per project just like there are individual workbooks for each project due to limitations in Smartsheets. Through some Python trickery the data is pulled back into each workbook and only represents about half of the data - the other half is granular detail about each cable and where it sits in the physical project on each device. Nobody here knows how to represent this very granular data in any other way except excel cell references. This data has to be manually adjusted and interpreted from the field tech data which is a nightmare.

The smartsheets web input forms are extremely limited technically and provide very little data validation so entries cannot be checked against existing data and can be spoofed by lazy or unintentional mistakes. This creates a lot of issues down the line.

I've suggested the Smartsheets ODBC connector license since that would allow more direct interaction with Excel and Smartsheets, but we've got data duplicated in so many places still, so I believe we need a real relational DB solution.

There are over 100 of these projects active at a time so managing x workbooks, x * x worksheets is just not feasible.

We need a database solution with some sort of customizable front end web tools that can act like Excel workbooks since each sheet has complex formulas. GIS web software does exist on the market in this industry but it just doesn't seem like that's going to happen for us due to cost and / or the time needed to migrate to completely different environment (we have no hosted servers or client authentication).

We also don't have access to a qualified front/backend dev/DB expert to help us with this.

Projects are continually a moving target with no breaks so interruption is hard to consider.

I just don't know where to start handling this type of project (I'd like to, because I like creating solutions, but I'm NOT a programmer and NOT a DB expert, just someone who has a bit of support background in IT). I do NOT want to start creating a DB not knowing what I'm doing, with the wrong table/schema.

Any advice?

​

​

Posted Nov 1, 2023, 4:01 PM UTC

16
Loading comments…