Can Justin Trudeau Fix Canada’s Broken Government IT System?
opensourcewriter.com/can-justin-trudeau-fix-c...
opensourcewriter.comPosted Sep 18, 2016, 11:27 PM UTC
2530
opensourcewriter.comThere's a whole IT culture in Ottawa that is downright toxic.
Half of the people signing off on these massive projects simply don't understand them and the other half are skimming money out of them somehow.
You need smart people both on the systems side and in management, and the Government of Canada simply doesn't pay competitively for this kind of talent. So these people end up working as consultants. There are a lot of really smart and talented people in Ottawa working as consultants. Lots of times they get paid kings ransoms to do stupid things, because nobody thought to ask them if there was a better way to do things.
3
The Government of Canada plays plenty for talent. The problem is simply bad organization from those highest up. But really, any government job has a great salary
1
The Government of Canada plays plenty for talent. The problem is simply bad organization from those highest up. But really, any government job has a great salary
Speaking in general terms when we're talking about something extremely specific is meaningless. The government does not pay its employees nearly as much as contractors with similar levels of experience or expertise, which is why there are a lot of projects dominated by contractors, and a lot of contractors that are really, really good. Sure, the government gets to get rid of them easily after the project is done, but they also lose all that expertise.
6
I'd also add that the Government doesn't actually target talent. They rely on a process to hire people, and not recruiters. They don't even really look at organisational culture/fit when hiring candidates. A shame, because organisational knowledge is important. You can read your lessons learned documentation from each project, but it doesn't capture the nuances of the project properly or the personality management issues that can arise from doing business within Government.
3
I'd also add that the Government doesn't actually target talent. They rely on a process to hire people, and not recruiters.
You're right, and it's worth mentioning here that the reasoning behind it is to stop even the vague appearance of the possibility of patronage. They literally can't hire someone they want to hire unless they jump through 17 hoops which takes months. I've seen promising students who did great work as FSWEP students fail to get bridged in simply because of how long the process took. Their supervisor and manager can want the candidate as much as they like, but it doesn't matter. You can't expect a kid with student loans and rent to pay to sit around for 4 months with no guarantees, so of course they lose good people.
And this isn't even touching on "fake" recruitment processes. They have a person in mind who they want to hire, but of course they can't just hire them, so they put up a job posting tailor-made to their CV, get a few candidates to apply, and sometimes even interview them, knowing full well who they want to hire in the first place. It's a ridiculous waste of everyone's time and money. Whenever I see hiring processes with exactly one position available, I get suspicious.
5
One positive thing that came out of the US's healthcare.gov fiasco were the two groups that came out of it.
18F - https://en.wikipedia.org/wiki/18F USDS - https://en.wikipedia.org/wiki/United_States_Digital_Service (people working for this are more temporary)
I think we could use something like these agencies. Talent pool is smaller in Canada, but it seems like we need some disruption to get past the classic government IT bureaucracy.
3
we need some disruption to get past the classic government IT bureaucracy.
My understanding is that the IT work that's giving problems was all done, and is being done, by private sector companies. Is that not true?
1
Kind of true. Private sector companies are making the product, the Project Managers in Government are not phasing it in properly or even testing it in some cases. I don't actually think it is the fault of the Project Manager, but more a fault of the system in place the puts pressure on them to hit unrealistic targets.
1
[deleted]
1
[deleted]
-3†
The government needs to adopt NoSQL in a big way.
Uhhh.. no. You don't just pick a technology and make it fit with the org. Especially not when you'd have to retrain everyone who currently uses/manages the system.
19
The government needs to adopt NoSQL in a big way
Definitely not.
But I will tell you when dealing with large amount of data, building our cube took 1/16th the time after we switched from SQL Server to NoSQL
Attitudes like this are pretty common in the tech industry, where young folks without a lot of hard experience make sweeping, generalist statements about how all of technology should change to use a new flavour of something, and then they back it up with a single lifestory about how once a project they were connected with was improved by making this change. Usually, they're unequipped to describe exactly how it was better because they're not familiar enough with computer science to actually describe or quantify the improvement or assign it to any specific reason.
So you'll get statements like this - "well, our cluster is webscale now that we have buzzword, so no one should use anything that isn't buzzword. I once read a web article that said query precache planners take up too much time, so I will cite that as the reason why it's faster at compiling."
They're basically useless because they don't contain real information, are very jumbled and use the wrong words.
15
I've repeatedly come up against complaints about a particular simulation technology which are based only on the default settings of particular implementations - even when every implementation could avoid the problems if people configured them properly and overrode the defaults. You can't declare a whole technology inferior because you're too lazy to change the defaults after installation. I suspect the same for SQL vs. NoSQL to some extent, but I'm not a database guy (more a middleware guy).
1
Also, database technology has changed a lot. The government needs to adopt NoSQL in a big way. Legacy SQL Server and Oracle are outdated
That statement is way too broad to be accurate. There are certainly applications that necessitate NoSQL databases, but to say RDBMS are obsolete or otherwise outdated is simply incorrect. In many situations a relational database makes much more sense than a NoSQL one.
8
[deleted]
1
Of course. For petabytes of data needing to be available for tens of thousands of concurrent sessions, NoSQL is a clear winner. But that represents a minority of systems; and there are real drawbacks to be considered.
6
[deleted]
1
Perhaps o should have said outdated for many typical databases uses.
This really depends on what your definition of "typical" is.
If you're doing run-of-the-mill accounting and payroll, a relational database is typically what is used. I'd be surprised to find accounting software that uses MongoDB, for example.
On the other hand, it wouldn't surprise me to see a system that aggregates data from multiple source systems for reporting and analysis to use some kind of NoSQL system.
3
What? Relational Databases aren't outdated. A great deal of data is still pretty heavily relational and continues to be well represented by relational databases. The fact most NoSQL databases are pretty piss poor at data integrity is also a strike against them.
I'll grant Oracle is usually a terrible idea, but mostly becomes it's expensive as hell to administer and should only be kept around for legacy reasons because converting giant systems to a new DB to do what's hip is insane. I'd probably recommend Postgresql for most general purpose DB tasks as it's a solid relational database that has support for document store stuff like most of the NoSQL databases do for when you don't need structured data.
7
Also, database technology has changed a lot. The government needs to adopt NoSQL in a big way. Legacy SQL Server and Oracle are outdated. (Vertica is what some of our big banks are using now)
Nope, they need to use the appropriate database technology for the job at hand. If the payroll system that best meeds the Government's needs runs on Oracle, they will use Oracle for that system.
When using off-the-shelf software, which is what should be used for already solved problems like accounting and payroll, the business needs will drive the choice of software and the choice of software will drive the choice of database.
When developing new things, you can pick the database that meets the needs of the system that's being developed. I'm sure that what CRA uses to process incoming tax returns, which is not something that they can get off-the-shelf, will need a different database than the one that lets you view the Parliamentary Business.
5
Actually the problem is more complicated.
Someone has take the view for instance that database X meets the Governments needs. That's a stupid naive view - and its unfortunately the government view. Because the needs of government are not generalized. In projects A, B, C, D and E, the database the government needs for the most effective projects may be different for each one. What has happened is that database X is approved for all so with small exceptions you can have Y, but Z1 and Z2 are ruled out despite being best for the job. The small exceptions are phased out as much as possible over time. Database X rules the day because its the government approved database, not because it suits the job. Its the database they've been talked into because X is a big company with an aggressive sales force, they will train everyone (for a fee), offer support (for a fee), it will take some responsibility even (for a fee) .... This is why shared infrastructure as implemented in the government is a failure. You are prohibited from using the solution that best suits your goal because there is a general agreement with super big company X that says you can't, and besides we don't train in that, we don't support that ,we don't want to be responsible for that, ....
Off the shelf doesn't work in government much of the time. I've seen them buy products not suited to their purpose because it is cheap. They shoehorn something into the product that was never meant to do this or that. The manor in which accounting and payroll are done in the Canadian government is not the same as private business, not the same as other governments, these aren't solutions already existing. A large part of the projects i have worked on are making something that worked for other countries successfully work for Canada and it doesn't because the Canadian government has rules even people who run the department don't understand. I've seriously had to deal with change request at government to change functionality A to B, then change it to C, than change it back to A, and then have to wait to see if the government can actually make sense of their own policy before proceeding. I've been in requirement analysis sessions where requirements conflict - they shouldn't conflict - it means two groups were handling the same thing differently, following their own rules! That's what our government is truly like.
So full time employee Joe who has been trained by company X to do his job as best he can is understandably resistant to change, he supports company X, he advises his superiors that these other solutions are doable with company X and inadvertently may be helping X with their stranglehold on shared infrastructure. Since he's trained in X to the hilt, he sees all other solutions as a threat to his job. He has to make sure he not only disapproves but that in his role as adviser that his policy decision making overlord disapproves.
But this is not just a database issue, its a server issue, its everything IT issue - this is the problem when a large org adopts a standard in IT - they handcuff themselves and are prone to project failure because the square peg doesn't fit the round hole.
And of course the standardization came about because it promises saving money, you aren't buying servers for each project, you aren't buying a software licence or support for each project, you aren't doing allot of things on an individual basis. In theory standards save money. The unquantifiable is the outcome of failed projects that are related to troubles with the standards.
2
The government needs to adopt NoSQL in a big way. Legacy SQL Server and Oracle are outdated.
4
No one needs to adopt NoSQL. It's fine for a simple blog application, but we are talking about massive amounts of relational data. There is no way I would trust the CRA data to something like Mongo.
1
Actually, I think a lot of the problem is that the people signing off on the projects don't actually understand much about computers.
A shocking number of senior public servants honestly think that the box on their desk is nothing more than an electronic typewriter and filing cabinet.
So you put them in a room with IT folks and it's like dropping a Muggle into Hogworts. Everything is magic, and the folks selling the wands get to tell them what the rules are.
So there are a lot of databases around, many of which are being used incredibly poorly to do things that no database ever should. There are also an awful lot of people who use Excel spreadsheets like paper ledgers.
I honestly don't know how to get people off of these pre-conceptions without actually teaching them some computer science, and executives don't have the time or attention span to listen long enough.
There needs to be a culture change about how the Public Service interacts with IT. It's not the PM's job to decide when we should use SQL or NoSQL, but I think it could be his job to hit reset, and say that it's every Public Servant's job to make good use of all the tools given to them, including computers. I don't know whether this PM will decide whether he wants to do that. I do think he is capable of doing it if he chooses to though.
1
No, Justin Trudeau can’t fix it, because JT is not an IT professional, nor should he be expected to fix it. Shit needs to roll down from above onto the head of whichever project manager is responsible for this screw-up. The accountable party is not a politician: it is a public service employee or contracted expert who made some very poor decisions.
In this case it's really both. There is no single project manager in charge of this. You can try to find a name but you won't succeed, not really. The foundation of SSC itself was botched. The proper way to do this was to hire more people to found SSC, get it going, and then to transition IT staff and IT tasks from other departments. But spending that much money and temporarily hiring more federal employees posed a political problem that was never dealt with head on.
Instead they simply told other departments that they would have to downsize their IT staff and send those downsized to SSC. Do you think they got the best people? Of course not, they got the people the other departments didn't need or didn't want.
This wasn't just botched on an operational level. The whole rationale behind how this department was founded was fundamentally flawed. Trying to narrow it down to a project manager to anyone below the political level will be futile.
But for some messed up reason, TBS doesn't understand that. Their SOP when told that implementing X will allow reductions in costs or staff, is to take them away from you, and then tell you to implement the plan. Since it usually requires a temporary augmentation to change the system to the point it can realise these savings, these transformations often fail,
What? No. Software version control is an IT decision. Rolling up and Centralizing individual ministry IT depts? That's a policy decision.
Fair enough. We have several primarily-responsible folk at fault:
the main contact for the private industry: an IT software architect supported by a QA tech, who is probably an executive-level manager who represents the industry that implemented the software. S/he blew smoke up everyone’s ass.
the main contact for the public (Government) industry: an Executive Project Manager (possibly but not certainly a public union member) who is responsible for realizing the Government’s wishes. S/he blew smoke up everyone’s ass.
a Government Executive (ie. probably not a member of the Public Union, probably appointed or Senior Executive) who is responsible for deciding that rolling up IT could be cost-effective. S/he blew smoke up everyone’s ass.
an Elected Government Official who decided that there should be cutbacks in the first place. S/he started the smokey fire in the first place.
Oh, and we have at least one more Government Official who decided to run with a sunk-cost fallacy instead of calling it all to a halt and reverting back to old systems that work. So that people would get paid.
A whole lot of heads should be rolling.
Nah, good IT is not just an IT job. It's heavily a leadership, organizational and administrative issue as well. If the gov't is hiring inadequate staff to do this very important job, or failing to provided the needed resources, it's not the fault of the staff.
This is exactly the sort of situation that can be solved by someone with little to no IT knowledge, someone with the authority to make calls and allocate resources and consult the right people. That's the job of an upper manager or a politician.
[removed]
Rule 3