[SOLVED] Why use slow Python for machine learning?
ProgrammingThis forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I still think that the parts that do the heavy lifting are written in some lower level language, but with - how do you call it - python bindings? - so they can easily be deployed in python scripts.
I still think that the parts that do the heavy lifting are written in some lower level language, but with - how do you call it - python bindings? - so they can easily be deployed in python scripts.
This is definitely something I have seen - I worked on a product where the main code was compiled C#, but the code could learn and write new functions and code stored as Python and executed within.
I still think that the parts that do the heavy lifting are written in some lower level language, but with - how do you call it - python bindings? - so they can easily be deployed in python scripts.
Yes, the parts that would suffer most from being done in Python are implemented in something faster and called from Python-- Python is the "just do what I want" tool that holds the other stuff together.
When people don't understand why Python is used for machine learning, ask them why anybody would run their GNU/Linux applications from a slow interpreted shell language like Bash. (It doesn't matter, as the Bash interpreter isn't doing the work.) One of the slowest aspects of Python is the way it handles dynamic typing. It's not necessarily a slow implementation of that task, but a task that is notably less efficient than static typing.
Distribution: openSUSE, Raspbian, Slackware. Previous: MacOS, Red Hat, Coherent, Consensys SVR4.2, Tru64, Solaris
Posts: 2,800
Rep:
Quote:
Originally Posted by pan64
that is not true always. Sometimes an interpreter evaluates only the current line, nothing more. python, java and perl [for example] are compiled into byte code which is executed by an interpreter-like way. That means [more or less] the VM will execute this byte code one by one using their native implementations. These are never fully compiled to the native machine language, therefore the execution will be slower than that.
I thought that Perl was fully compiled before being executed---if it was only line-by-line interpreted I'd have seen more time between pressing return and seeing syntax errors in some of my bigger Perl scripts. :^) Once that's done, execution speed was only slightly slower than compiled C. There are Perl compilers out there. I don't think I've ever seen any that were OSS but they do (or did) exist). If a compiler doesn't exist for Python, I suspect it won't be long before it does.
Distribution: openSUSE, Raspbian, Slackware. Previous: MacOS, Red Hat, Coherent, Consensys SVR4.2, Tru64, Solaris
Posts: 2,800
Rep:
Quote:
Originally Posted by Firerat
yeap that would be my guess
the portability and the fact that modules can be "dropped in" to easily add features`
Maybe easily dropped in but first you have to find them.
Out of curiosity... does Python have anything like:
CPAN -- Comprehensive Perl Archive Network
CRAN -- Comprehensive R Archive Network
CTAN -- Comprehensive TeX Archive Network
Googling has been helpful in locating some interesting Python modules -- for example, back when I needed to find something to do "fuzzy" pattern matching -- but that doesn't seem as efficient as the Comprehensive Archive Networks and I haven't found anything quite like those for Python. (Just looking to avoid re-inventing the occasional wheel.)
Argh! Brain fart, I guess. I went through a marathon session a while back trying to merge several collections of bookmarks, organizing them (shees what a hassle that turned out to be), filling in blanks in some topics (esp. in my "languages" folder) and could have sworn I had saved that one. Again, thanks.
That second link sure would have been better than the way I finally stumbled onto the fuzzy pattern matching module I located ("fuzzywuzzy") for a "crash" effort this past summer. I cannot recall how I wound up locating it though I it may have been via googling and finding the author's project page. (It is listed on PyPI now, though.)
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.