How important is enforcing Pythonic?

Discussion

I had just delivered a product where the pipeline was broken into a few parts, then given to developers to work on due to the short timeline.

I finally had a chance to look at my colleague's code and was taken aback by how un-pythonic it is. Some of the main offenders include:

  • not using "if __name__ == __main__"
  • everything is in classes where most classes have only one function (no "init" function in it, if that matters)
  • if it's a big class, the "main" function is really really really really long
  • No comments, no tests but I suppose those can be attributed to rushing to deliver

I just don't know if I should care. I don't need to maintain his code because my pipeline starts from his program output. However, I feel uneasy at the fact that...it doesn't look like Python programming at all???

I guess I'm just confused. I'm not even sure if I'm ranting or asking for guidance. The code works so...should I even do anything?

Posted May 24, 2022, 11:49 PM UTC

646
koolaidman123•

Code reviews? Why is poor quality code being merged?

Theres plenty of things that can be automated eg precommit hookd, github actions for test coverage etc. Put in place proper processes to ensure poor code isnt merged

16
monkeyunitedOP•

We're on separate team which code review is only done on my team.

Just joy of working with people with diverse background I suppose. I'm going to push for at least documentation and let the rest slide.

1
EasyLie4013•

I don't need to maintain his code

You sure about that? If so, who cares. Treat it like a black box.

If there's even a 1% chance you are wrong and you will have to maintain it then either make the guy refactor it to conform to acceptable style or at least get him to document it properly.

8
monkeyunitedOP•

at least get him to document it properly

Thank you for the suggestion. This is what I'm going to propose to the PM.

The chance of me maintaining the code is low but it's definitely not zero.

1
TheMathProphet•

Your company doesn’t have a style guide?

6
monkeyunitedOP•

heh we don't. I'll look into what that is.

4
SwitchOrganic•

Have you heard of PEP 8 before? You can use linters like Black or Flake8 to enforce certain code standards as well.

https://peps.python.org/pep-0008/

https://flake8.pycqa.org/en/latest/

https://github.com/psf/black

6
monkeyunitedOP•

Thank you for the resources! I will share with the team.

I forgot about PEP 8 but that's how I developed my coding style when learning Python!

2
skillshocked•

This response from OP says it all.

4
monkeyunitedOP•

Not gonna lie. We have a PM dominant culture so delivery is always the top priority.

The DS lacks standard processes such as a proper code review.

1
Your_Data_Talking•

Not sure you have much of a DS team if you don’t do code review, this thread has a treasure trove of advice

2
irismodel•

once you look that up, I suggest you explore a linter (https://realpython.com/python-code-quality/)

2

I took over from somebody that wrote the worst code I’ve ever seen. No comments, tests, or common sense in 4 years worth of code. I’m currently re-writing everything….

Ya, it really can matter.

4
monkeyunitedOP•

Yea...I'm sorry you have to go through that.

In my earlier days I also passed on a lot of poorly written VBA and SQL scripts. It's just a part of this profession I suppose.

2

Haha thanks for your sympathy. At the end of the day it’s work, and I’m okay with that. But I’m also trying my best to enact standards in my team so this doesn’t happen again in my organization.

2
[deleted]•

[deleted]

1
hockey3331•

Ok I've seen the above before, but where do you learn it? I learned introductory Python and coding along my math degree, but it was just a mean to an end and not the main focus.

At work, I learned "on the go", and try to deliver better and more maintainable code every time I learn something new. But, overall, it always seems like when I encounter classes and such it's assumed you know about it. "Pythonic" way too.

I'm definitely saving this thread as a reference

1
[deleted]•

[deleted]

0
HmmThatWorked•

If you don't maintain it document out the wazoo what the problem and who caused it. Send an offical.dissent letter or something along that vein. Shitty code comes back to bite everyone eventually. You need to place the blame squarely where it lies,. Tis a documentation game.

-4
the_brown_saber•

I wish people didn't use classes at all for DS

-7
jah_broni•

Why? Classes at a minimum allow you to collect similar functions into a namespace. When used effectively they are extremely important to writing good code.

10
SynbiosVyse•

I've never used classes, just functions. I don't understand the point. For algorithm work it's all about imperative line by line manipulation, functions for when you need to do things more than once or parametrize it. What are classes really for unless you're making low level stuff like new object types (eg pandas data frames).

10
analyzeTimes•

I find classes important when building out utilities such as API connectors and database connections with stored queries. It helps keep the code clean and allow for reusability across different projects. Yeah in theory you could just retype out the code in every script you write, but in my opinion that's not best practice. I guess to each their own.

9
the_brown_saber•

That's the point " when used effectively" most of the time it ain't

-1
sandmansand1•

It’s very useful when you’re writing reusable code. When you can instantiate an object with your usual functions, that store memory, allow you to train a model, write an inference script, etc.

They make less sense when you’re scikitting, but once you’re writing hundreds or thousands of lines of code, they’re indispensable. There’s a real reason every major application uses OOP. You need it to manage a large code base and it simplifies reusability.

Lots of “data scientists” train a model using a notebook and scikit then never have to write methods to turn it into a pipeline, or create tools, or write frontends around inference, or have deployment systems. Different tools for different jobs.

4
grae_n•

I do think we've gotten to a point with Python where every time you think about creating a class you should stop, reflect and think about importing someone else's code instead. One of the advantages of OOP is having a standardised set of objects. If everyone creates their own object definitions, things get very confusing.

Too many people have rewritten reusable code.

-1
sandmansand1•

At least in my industry, proprietary problems generally require proprietary solutions. Not to mention, much NLP stuff (my domain) is still early stages - not too many good high level implementations out there on my problems, but plenty of bits and pieces to cobble together a tool.

In my experience, having internal templates is usually good practice as well. Maybe even full packages maintained by the DS team.

2
grae_n•

I guess I'm thinking more of people constantly rebuilding wheels not people doing novel things.

Python doesn't need another matrix class that is ten time slower than numpy.

1
jah_broni•

Can you point me to a package that has a class that represents a user in my system? How about one that is for a page in a PDF export?

1
grae_n•

If you decide to create a custom user management system that is eventually going to be handed off to someone else you probably should be using something standardised off of github. I don't get the second one PDF export packages exist.

My point isn't that class should never be used (although looking back that does seem to be this threads origin). It's that if everyone uses their own custom user management system then migrations, mergers, or analysis can be super difficult.

Like if you and me both had websites and were using django's user management system then merging user bases would be much easier.

1
speedisntfree•

They have their place but I feel like there needs to be a real justification for the overhead above functions in modules.

3
monkeyunitedOP•

Maybe not at all, but I agree it should be used cautiously.

And especially not when there are multiple classes, each with one function inside...

2
ShadowShedinja•

It's helpful for data cleaning and for hyperparameter tuning, especially when using lambda functions.

0
the_brown_saber•

How so? Curious how's it better than a function.

5
ShadowShedinja•

Sorry, I was thinking of functions my bad.

1
PryomancerMTGA•

I've used a large variety of languages over the years and I wouldn't be surprised if Python gets replaced 5+ years from now.

So if you're not his boss and it doesn't mess your part up; I would suggest you don't sweat the small stuff.

-17
juhotuho10•

Python is a duct tape language that excels at using packages made in python and other languages while being very readable and good at variety of things

It's the Leatherman multi tool of languages.

It may be slightly better to have a individual purpose built knife and individual purpose built pliers and individual purpose built bottle opener, but why bother when you can have it all in a neat and surprisingly effective package.

I don't see python being replaced unless there is a language that does pretty much the same thing as python but better

3
kaumaron•

With what?

1
PryomancerMTGA•

If SAS can get replaced, Python can be replaced. Progress happens.

-8
ghostofkilgore•

SAS is shit and costs money though.

3
PryomancerMTGA•

Yet 20 years ago when I started; it dominated the market. All I'm saying is things change over time. The only constant language has been SQL.

-1