Linux required for Data Engineering field

Hi

I am a newbie and Linux is a vast field. I am interested in Data Engineering field. This is an upcoming field in tech and there are lots of jobs in this area. One needs to know some Linux in this area. Now I want to know how much Linux is enough to get started in this field. Because Linux is a vast field, I will just spend hours and hours to study Linux. So I want to know how much Linux will be enough so that I can work in this field. I can pick up the rest of Linux knowledge on the job. Currently, I am taking some Linux beginners course on Udemy.

Thanks

Originally Posted by issacnewton Hi I am a newbie and Linux is a vast field. I am interested in Data Engineering field. This is an upcoming field in tech and there are lots of jobs in this area. One needs to know some Linux in this area. Now I want to know how much Linux is enough to get started in this field. Because Linux is a vast field, I will just spend hours and hours to study Linux. So I want to know how much Linux will be enough so that I can work in this field. I can pick up the rest of Linux knowledge on the job. Currently, I am taking some Linux beginners course on Udemy.

Your question is unanswerable as it is stated, and honestly sounds pretty bad. Almost sounds like you're saying "I only want to study JUST ENOUGH so I can get some sort of nebulous job, but not any more".

If you want a job (ANY job, in any field), you need to show effort and work hard...not only do a minimum. And you also need to keep in mind that the job at one company may be TOTALLY DIFFERENT at another company, even though they have the same titles. There are no shortcuts; either work hard, study hard, and LEARN/THINK for yourself, or you will just not succeed anywhere, at anything.

Thanks for advice TB0ne

Think of it this way, Isaac: "Linux is just one of many tools that you can expect to use in this (vast ...) field which now interests you." It is certainly not the only operating system that you would ordinarily expect to use on a very regular basis, and "basic competency" in it is probably what you'd need.

"Data Engineering" is just as vague a term as "Civil Engineering." You should explore further and get specific as to what, specifically, does and does not interest you. Then, when it's time, try to find a job in that field where you get to be "a fly on the wall," even if that means preparing other people's cups of coffee, each just the way they like it, while you watch what is going on around you.

Here's some articles which I was recently reading, about about skills.

https://cloud.google.com/certification/data-engineer doesn't seem to mention Linux at all!!!
But reading a book, like "How Linux Works", might take a truly skilled engineer only days!!!

I'd also suggest using the LQ search to find more posts by #4, about skills for a successful career.

To me, a subject title like "Data Engineering" implies that it focuses upon the data-oriented design (and re-design, and maintenance and maintainability) of production data systems. Certainly an interesting topic.

But, you must realize that you might walk into almost any company and find what the is that fossil doing here? all kinds of computer systems in service, all at the same time. IBM still sells a lot of AS/400's, for instance. Ditto mainframe computers that are still compatible with the operating environments of the 1960's and 70's. Linux is certainly well-represented there, too.

Windows and OS/X and Linux and AS/400 and big-iron, all at the same company and part of the same system? "Sure! Why not? Hey, we make this stuff look easy!"

(Oops? Did I just persuade you to become a biologist or a truck-driver? ...)

I would think that a good data engineer would need to have an "omnibus" portfolio of boots-on-the-ground functional experience, and the willingness to dive in and learn something new about some new (or, very old) system and to somehow enjoy doing so. (As I do ...) You'll need to understand how very-different computer systems can "talk" to one another and can work with one another. (But you might not necessarily be expected to know how to write the code that does it.) The knowledge-base of such an engineer would need to be able to "see the forest for the trees."

And I definitely think that one's practical training might well begin by getting really good at fixing cups of coffee, just the way that the "old hands" like it. (Hey, my career started with tearing pages off a line-printer, back when "micro-"computers were still a toy. I'm still here, and I'm still making money and having fun. I consider myself very lucky to have made a living from a life-long hobby.)

(And for having done a lot of good and helpful things for a lot of people who trusted and relied on me. It just don't get better than that, and I wish the same for you, wherever your career takes you.)

Coincidentally, the Humble Data Science Books Bundle is a current promotion.

You might want to buy it.

Just saw this story about getting a first job in data engineering:

http://peopleofcolorintech.com/artic...data-engineer/

My current job uses Windows to interface with IBM mainframe systems, in my dept. It's for a large health insurance company. I just recently interviewed with a company that's looking for people with AS/400 & Vax/VMS experiences. The company I interviewed with bought another company that still uses AS/400 & Vax/VMS. It's just to show that companies are wanting someone with lots of different experiences too.

"Big Iron" (and "Old Iron") is still used, because it still works. (The highly-redundant construction of a so-called mainframe computer, in particular, is much superior in certain applications to the current trends of binding together hundreds of smaller machines with miles of Cat-5 and Optical cable ...)

You still need to approach this business with the expectation that no(!) "single operating environment" will ever be sufficient. You will, quite routinely, encounter setups that use (and, that exploit ...) many different types of hardware. You need to be conversant in many things – and always willing to learn more. You need to be willing to jump from your comfort zone, straight into the (as yet ...) unknown.

"Old hand" that I am by now, I happen to find this to be one of the most interesting aspects of the business ... "you're doing what?! ... with that thing?!?! ... and it actually works? ..."