[SOLVED] Choosing a style when referring to modules in Python

Turbocapitalist · 04-25-2020, 02:56 AM

When writing in Python, which style is prefereable when using modules?

In one style you could use them like this:

Code:

. . .
from multiprocessing import Process, Value, Array
. . .
w=Value('i')
. . .

With another style you could use them like this:

Code:

. . .
import multiprocessing
. . .
w=multiprocessing.Value('i')
. . .

Is either style preferable? I'm guessing the second example would be preferable because it is less ambiguous which module is in use. What kind of difference is there in regards to cluttering either source code or memory? Maybe it makes more of a difference in longer scripts as opposed to shorter ones.

boughtonp · 04-25-2020, 04:46 AM

Preferable is subjective - I can't tell you which one you prefer, (nor which one I would prefer without a more concrete example).

Generally...

If you've got multiple different "Value"s then disambiguating the references obviously make sense.

If "Value" is unique within your script/project then doing so would be needless verbosity that distracts from what the code is doing.

The relevant Python docs have a third option of aliasing (import multiprocessing as mp) which balances reduced verbosity with being clear on what is being identified.

teckk · 04-25-2020, 07:49 AM

I did a little test.

Test1.py

Code:

#!/usr/bin/python

#Module loading test

def t1():
    import multiprocessing
    
def t2():
    from multiprocessing import Process, Value, Array
    
def t3():
    from PyQt5.QtGui import QFont
    from PyQt5.QtCore import Qt, QUrl, pyqtSignal
    from PyQt5.QtNetwork import QNetworkCookie
    from PyQt5.QtWidgets import QWidget, QHBoxLayout, QApplication
    from PyQt5.QtWebEngineCore import QWebEngineUrlRequestInterceptor
    from PyQt5.QtWebEngineWidgets import (QWebEnginePage, 
                                QWebEngineView, QWebEngineSettings)
                                
def t4():
    import PyQt5

Clear RAM before and between loading modules.

Code:

sudo sysctl vm.drop_caches=3

python -mtimeit -s 'import Test1' 'Test1.t1()'
1 loop, best of 5: 523 nsec per loop
:0: UserWarning: The test results are likely unreliable. The worst time (2.34 usec) was more than four times slower than the best time (523 nsec).
python -mtimeit -s 'import Test1' 'Test1.t1()'
1000000 loops, best of 5: 293 nsec per loop
python -mtimeit -s 'import Test1' 'Test1.t1()'
1000000 loops, best of 5: 288 nsec per loop
python -mtimeit -s 'import Test1' 'Test1.t1()'
1000000 loops, best of 5: 297 nsec per loop

sudo sysctl vm.drop_caches=3

python -mtimeit -s 'import Test1' 'Test1.t2()'
1 loop, best of 5: 1.94 usec per loop
:0: UserWarning: The test results are likely unreliable. The worst time (7.84 usec) was more than four times slower than the best time (1.94 usec)
python -mtimeit -s 'import Test1' 'Test1.t2()'
200000 loops, best of 5: 1.47 usec per loop
python -mtimeit -s 'import Test1' 'Test1.t2()'
200000 loops, best of 5: 1.47 usec per loop
python -mtimeit -s 'import Test1' 'Test1.t2()'
200000 loops, best of 5: 1.45 usec per loop

sudo sysctl vm.drop_caches=3

python -mtimeit -s 'import Test1' 'Test1.t3()'
1 loop, best of 5: 6.14 usec per loop
:0: UserWarning: The test results are likely unreliable. The worst time (26.3 usec) was more than four times slower than the best time (6.14 usec).
python -mtimeit -s 'import Test1' 'Test1.t3()'
50000 loops, best of 5: 5.82 usec per loop
python -mtimeit -s 'import Test1' 'Test1.t3()'
50000 loops, best of 5: 5.62 usec per loop
python -mtimeit -s 'import Test1' 'Test1.t3()'
50000 loops, best of 5: 5.69 usec per loop

sudo sysctl vm.drop_caches=3

python -mtimeit -s 'import Test1' 'Test1.t4()'
1000000 loops, best of 5: 301 nsec per loop
python -mtimeit -s 'import Test1' 'Test1.t4()'
1000000 loops, best of 5: 292 nsec per loop
python -mtimeit -s 'import Test1' 'Test1.t4()'
1000000 loops, best of 5: 303 nsec per loop

And those result are backwards from what I thought that I would get.
I thought that loading just what you needed was faster.

Someone else comment on why that test is invalid.

teckk · 04-25-2020, 10:42 AM

Code:

sudo sysctl vm.drop_caches=3

python -X importtime -c "from tkinter import *"
import time: self [us] | cumulative | imported package
import time:      1118 |       1118 | _frozen_importlib_external
import time:      8003 |       8003 |   time
import time:       486 |       8489 | zipimport
import time:       161 |        161 |     _codecs
import time:     43445 |      43606 |   codecs
import time:      8588 |       8588 |   encodings.aliases
import time:     82135 |     134328 | encodings
import time:     12792 |      12792 | encodings.utf_8
import time:       333 |        333 | _signal
import time:      2758 |       2758 | encodings.latin_1
import time:       107 |        107 |     _abc
import time:     22423 |      22530 |   abc
import time:     38396 |      60925 | io
import time:       133 |        133 |       _stat
import time:     10139 |      10271 |     stat
import time:     29884 |      29884 |     _collections_abc
import time:     14903 |      14903 |       genericpath
import time:     15936 |      30839 |     posixpath
import time:     16825 |      87817 |   os
import time:      8787 |       8787 |   _sitebuiltins
import time:       405 |        405 |   sitecustomize
import time:        77 |         77 |   usercustomize
import time:     43197 |     140281 | site
import time:     17778 |      17778 |     types
import time:     22371 |      40149 |   enum
import time:    429193 |     429193 |   _tkinter
import time:     24785 |      24785 |   tkinter.constants
import time:       164 |        164 |       _sre
import time:     15774 |      15774 |         sre_constants
import time:      8786 |      24560 |       sre_parse
import time:     23706 |      48428 |     sre_compile
import time:       282 |        282 |           _operator
import time:     11059 |      11341 |         operator
import time:     20694 |      20694 |         keyword
import time:     11215 |      11215 |           _heapq
import time:     18275 |      29490 |         heapq
import time:       200 |        200 |         itertools
import time:     16630 |      16630 |         reprlib
import time:       146 |        146 |         _collections
import time:     22241 |     100738 |       collections
import time:        90 |         90 |       _functools
import time:     20384 |     121211 |     functools
import time:       153 |        153 |     _locale
import time:      1808 |       1808 |     copyreg
import time:     16394 |     187992 |   re
import time:     51050 |     733167 | tkinter

sudo sysctl vm.drop_caches=3

python -X importtime -c "from tkinter import Frame, Tk, Menu, Label, Scrollbar"
import time: self [us] | cumulative | imported package
import time:      1827 |       1827 | _frozen_importlib_external
import time:      6646 |       6646 |   time
import time:       455 |       7101 | zipimport
import time:       139 |        139 |     _codecs
import time:     35580 |      35718 |   codecs
import time:     16538 |      16538 |   encodings.aliases
import time:     73783 |     126038 | encodings
import time:     12386 |      12386 | encodings.utf_8
import time:       821 |        821 | _signal
import time:     10277 |      10277 | encodings.latin_1
import time:       109 |        109 |     _abc
import time:     22458 |      22566 |   abc
import time:     38658 |      61224 | io
import time:       142 |        142 |       _stat
import time:     18670 |      18812 |     stat
import time:     29582 |      29582 |     _collections_abc
import time:     14877 |      14877 |       genericpath
import time:     15925 |      30802 |     posixpath
import time:     17439 |      96633 |   os
import time:      8233 |       8233 |   _sitebuiltins
import time:      7753 |       7753 |   sitecustomize
import time:       102 |        102 |   usercustomize
import time:     55290 |     168009 | site
import time:     17791 |      17791 |     types
import time:     22409 |      40199 |   enum
import time:    404174 |     404174 |   _tkinter
import time:     24817 |      24817 |   tkinter.constants
import time:       176 |        176 |       _sre
import time:     15777 |      15777 |         sre_constants
import time:      9026 |      24803 |       sre_parse
import time:     24043 |      49020 |     sre_compile
import time:       283 |        283 |           _operator
import time:      5071 |       5354 |         operator
import time:     18302 |      18302 |         keyword
import time:     11206 |      11206 |           _heapq
import time:     18289 |      29494 |         heapq
import time:       231 |        231 |         itertools
import time:     16621 |      16621 |         reprlib
import time:       145 |        145 |         _collections
import time:     22223 |      92368 |       collections
import time:        89 |         89 |       _functools
import time:     19854 |     112310 |     functools
import time:       144 |        144 |     _locale
import time:      1756 |       1756 |     copyreg
import time:     16365 |     179592 |   re
import time:     81546 |     730327 | tkinter

To the OP, I thought that is was faster importing just what you need from a module instead of the whole module.
I'm going to have to check that more when I get time. Then it depends on the module size if it makes any first launch time difference.
Python takes a while to load into RAM first instance anyway.

Someone on the arch forum pointed out to me.

Code:

>>> import dis
>>> dis.dis('import multiprocessing')
  1           0 LOAD_CONST               0 (0)
              2 LOAD_CONST               1 (None)
              4 IMPORT_NAME              0 (multiprocessing)
              6 STORE_NAME               0 (multiprocessing)
              8 LOAD_CONST               1 (None)
             10 RETURN_VALUE
>>> dis.dis('from multiprocessing import Process, Value, Array')
  1           0 LOAD_CONST               0 (0)
              2 LOAD_CONST               1 (('Process', 'Value', 'Array'))
              4 IMPORT_NAME              0 (multiprocessing)
              6 IMPORT_FROM              1 (Process)
              8 STORE_NAME               1 (Process)
             10 IMPORT_FROM              2 (Value)
             12 STORE_NAME               2 (Value)
             14 IMPORT_FROM              3 (Array)
             16 STORE_NAME               3 (Array)
             18 POP_TOP
             20 LOAD_CONST               2 (None)
             22 RETURN_VALUE

Is that more info than you asked for?

teckk · 04-25-2020, 11:08 AM

In case that is confusing. You'll need to call it depending on how you loaded it.

Code:

>>> from time import sleep
>>> for i in range(0, 5):
...     print(i)
...     sleep(1)
... 
0
1
2
3
4

>>> import time
>>> for i in range(0, 5):
...     print(i)
...     time.sleep(1)
... 
0
1
2
3
4

>>> from time import *
>>> for i in range(0, 5):
...     print(i)
...     sleep(1)
... 
0
1
2
3
4

>>> import time as t
>>> for i in range(0, 5):
...     print(i)
...     t.sleep(1)
... 
0
1
2
3
4

Turbocapitalist · 04-25-2020, 11:27 AM

It's all useful. I'm primarily wondering if there are any differences in performance for any of the three methods.

boughtonp · 04-25-2020, 12:46 PM

Quote:

Originally Posted by teckk

Someone else comment on why that test is invalid.

You're using code that doesn't exist in the wild with a testing methodolgy that is not remotely similar to live behaviour (at least for a GUI toolkit application).

Quote:

Originally Posted by Turbocapitalist

I'm primarily wondering if there are any differences in performance for any of the three methods.

Readability is important - make your code easy to understand and easy to maintain for future developers.

The slow parts of your code is not importing - if an application is behaving slowly, use a profiler against an actual running application to determine where the bottlenecks are.

pan64 · 04-25-2020, 01:37 PM

There are [at least] two approaches: write "sort" code, I don't like long lines. That means we need to use your first example.
The other one: I would like to see where is a function coming from, that makes the code maintainable (a bit better).
You don't need to agree, you prefer what you like.

I think there will be no difference in speed, because the compiled code will be identical (also the time required to compile will be similar)

Turbocapitalist · 04-26-2020, 08:11 AM

I also prefer to keep lines to 78 characters or less. Shorter is better. Yet I also usually need to be constantly reminded where functions have come from, to keep the code as clear and understandable as possible.

So I think the third way will be the most practical for now:

Code:

. . .
import time as t
import multiprocessing as mp
. . .

teckk · 04-26-2020, 09:03 AM

Quote:

I also prefer to keep lines to 78 characters or less

You can shorten lines so they fit into your editor, that's quite common. You have to break them at the correct place.

Working Example:

Code:

from tkinter import *

font = ('Monospace', 30)

root = Tk()

#All one line
button1=Button(root, text="Start", font=font, bg='green', highlightcolor='blue', highlightthickness=3)
button1.grid(row=0, column=1)

#Same thing
button2=Button(root, text="Stop", font=font, bg='red', 
    highlightcolor='blue', highlightthickness=3)
button2.grid(row=1, column=1)

#Same thing
button3=Button(
    root, text="Pause", 
    font=font, bg='yellow', 
    highlightcolor='blue', 
    highlightthickness=3, 
    )
button3.grid(row=2, column=1)

root.mainloop()

pan64 · 04-27-2020, 12:33 AM

there is a tool named black which can format the python code for you.