We brought in a dept of data scientists when we bought a company.
I never witnessed the worse code ever.
I don't know if they were all 1y bootcamp grads or something. I thought math formulas was supposed to be elegant. But everything was written top-down, no reusability, like coding excel formulas that feed one cell into another and takes a hour or two to calculate. I nope'd so fast out of one "system" they built which was just daisychaining SQL queries into a new file that is fed into another SQL query repeat 7 times.
It's kind of like hiring a bunch of physicists to design a bridge. It'll probably be a great bridge, but also probably overly complex, expensive and late. And for some reason made out of a novel alloy.
All programming teams are constructed by and of crazy people
Imagine joining an engineering team. You’re excited and full of ideas, probably just out of school and a world of clean, beautiful designs, awe-inspiring in their aesthetic unity of purpose, economy, and strength. You start by meeting Mary, project leader for a bridge in a major metropolitan area. Mary introduces you to Fred, after you get through the fifteen security checks installed by Dave because Dave had his sweater stolen off his desk once and Never Again. Fred only works with wood, so you ask why he’s involved because this bridge is supposed to allow rush-hour traffic full of cars full of mortal humans to cross a 200-foot drop over rapids. Don’t worry, says Mary, Fred’s going to handle the walkways. What walkways? Well Fred made a good case for walkways and they’re going to add to the bridge’s appeal. Of course, they’ll have to be built without railings, because there’s a strict no railings rule enforced by Phil, who’s not an engineer. Nobody’s sure what Phil does, but it’s definitely full of synergy and has to do with upper management, whom none of the engineers want to deal with so they just let Phil do what he wants. Sara, meanwhile, has found several hemorrhaging-edge paving techniques, and worked them all into the bridge design, so you’ll have to build around each one as the bridge progresses, since each one means different underlying support and safety concerns. Tom and Harry have been working together for years, but have an ongoing feud over whether to use metric or imperial measurements, and it’s become a case of “whoever got to that part of the design first.”
One thing that helped: separating research code from production.
Research is to answer a decision problem and much of the optimization problems that are in software engineering do not, and should not, apply.
Once the research problem is answered. Reproducing that answer with production quality systems should be it's own project. This also serves as a reproduction of the science in the research. Satisfying that hallmark of the scientific method.
Course, getting a company to agree to such an arrangement is near impossible. Especially if they have never been crippled by the mismatch expectations of putting research code in production.
As that is an organizational problem not just an engineering problem, good luck convincing management.
An alternative, if I can't get such an arrangement is the building a platform that supports integration of research code. That can be… uh… hard. Aside from the people challenges not all tech can support such a platform.
Prayers man.
We brought in a dept of data scientists when we bought a company.
I never witnessed the worse code ever.
I don't know if they were all 1y bootcamp grads or something. I thought math formulas was supposed to be elegant. But everything was written top-down, no reusability, like coding excel formulas that feed one cell into another and takes a hour or two to calculate. I nope'd so fast out of one "system" they built which was just daisychaining SQL queries into a new file that is fed into another SQL query repeat 7 times.
Mathematicians and scientists are notoriously awful programmers. They get shit done but with absolutely 0 regard to good practices and reusability.
Would have thought mathematicians would be slightly better at writing functions
It's kind of like hiring a bunch of physicists to design a bridge. It'll probably be a great bridge, but also probably overly complex, expensive and late. And for some reason made out of a novel alloy.
https://www.stilldrinking.org/programming-sucks
I've experienced similar.
One thing that helped: separating research code from production.
Research is to answer a decision problem and much of the optimization problems that are in software engineering do not, and should not, apply.
Once the research problem is answered. Reproducing that answer with production quality systems should be it's own project. This also serves as a reproduction of the science in the research. Satisfying that hallmark of the scientific method.
Course, getting a company to agree to such an arrangement is near impossible. Especially if they have never been crippled by the mismatch expectations of putting research code in production.
As that is an organizational problem not just an engineering problem, good luck convincing management.
An alternative, if I can't get such an arrangement is the building a platform that supports integration of research code. That can be… uh… hard. Aside from the people challenges not all tech can support such a platform.