Updates and Separate Partitions for 1st level directories
 

1, If I have some 1st level directories (such as /usr /tmp or /opt (etc.) on a separate partition, then when I download and install an update, how does Linux know where all the directories are?



2. I am assuming these partitions need to be "mounted" when Linux is run? (I know little at this point) Can this mounting of necessary directories be automated?


Thanx, AnneF

All partitions which are listed in your /etc/fstab file are mounted as part of the boot process.

So if you wanted a partition called cat to be mounted automatically all you'd need to do is list it, and the other relevant details, in the /etc/fstab file.

This link https://www.howtogeek.com/444814/how...file-on-linux/ will explain a bit more about the fstab file

Thank you, quickbreakfast. I think I see how that works. I guess I would mount all such 1st level directories in different partitions / where they would normally be? That would just be a pointer/link to the mounted directory, right? The actual data would be stored on the drive/ partition/directory pointed to? The link was quite helpful (and kept leading me on to more adventures).

If I use GParted to create partitions (on my empty HD2 of 500GB)(p1-System, p2-Swap, p3Extended: (pe1-/root,
pe2-/home, pe3-/var,/tmp ...etc. for the directories that I want on different partitions, then you're saying that to get a Ubuntu or Debian/KDE install, to get Linux to recognize any directories on different partitions, I would have to use the command line to put their parameters into fstab? I'm still not sure EXACTLY HOW (syntax) to get the information INTO /etc/fstab, but I understand the principles and understand the elements/columns/output of the lsblk command.

Different topic:
Can Linux be set up as a single-user installation, such that I could use /home as my home directory and not have to insert a redundant username (branch to nowhere) following /home? Like the PERSONAL computer it is, and not encounter any complications (including duplicated sub-directory names)that a business or family computer might encounter?

I've got backup/image questions too, but I'm not there yet, well, other than what directories should be backed up together on the same schedule. I'm still reading about that.

Thanks again, I really appreciate you sharing your knowledge with me. --AnneF
.

An common analogy is to compare the linux filesystem with a tree. The trunk is the /root filesystem and that all other filesystems attach to the trunk as branches. Other filesystems attach to /root as mount points which are essentially directories i.e /var or /home. There is a standard to which by convention most distributions implement.

https://en.wikipedia.org/wiki/Filesy...archy_Standard

The system is is /root, /home, /var /tmp together so there is not a need for a separate "System" partition and for someone just making the switch I would keep things simple. I would suggest just /root, /home and swap. Some distributions also automatically create a /boot. You could have a single /home partition for a single user but installers automatically create /home/username and again as a someone just beginning to use linux keep it the same because anyone that would help you on this site would assume that your home is /home/username. By the way there are several command line short cuts so you don't ever need to type in /home/username.

~ is a short cut for /home/username
. is a short cut for PWD (current working directory)
Entering just the command cd will always return you back to your home directory.

There's nothing stopping you from using "/home" as your personal home directory. "/home/anne" would be a more typical configuration. I'd urge you to follow the convention.

A good reason to use something like "/home/anne" would be if you were to install a software package that uses some kind of administrative login for configuring/managing the application. Example: a database package. It's pretty common to want a database administrator account. If you've commandeered "/home" for your personal use then the administrative account for that application has to go somewhere else. Where would you put it if could couldn't use "/home/dba" (or whatever) for it?

There are some long-held UNIX/Linux conventions that make it easier to use. "/home" as a parent for user account home directories is one. I once worked in an environment where a collection of administrators had set up user accounts under "/home" on some systems, under "/users" or "usr/home" on others ... it was a real mess. (You wouldn't believe the way Oracle RDBMS had been installed on those systems.)

Good luck...

Thank you both for the good advice. I will create a username under /home - it makes more sense to me now. I didn't think anyone would write to /home but me.

Thanks, Michael, for the command line shortcuts. I programmed on Dos & Unix (a bit in C) MANY years ago, but I have used Windows for so long now, I've forgotten nearly everything. At least it's not completely foreign, I guess I should be grateful for that.

I do like separated directory types for making backups and for easier defragmentation and disk maintenance. I'm trying to figure out what makes sense to put where. In Windows, I kept the System (OS, Registry, Installed Pgms) and Data on separate partitions on one HD, and Data backups and System partition image files in separate partitions on a Backup HD. It's never set up easy, though, you have to cherrypick through C (System) for configuration files to add to the Data backups, for apps that don't let you choose where you want profile/configuration files to be kept, so they won't be clobbered (or can be restored) by a restored image file.

I installed Ubuntu 20.04, and it's a bit confusing, because the "folders" don't relate to "directories" and I don't know where things really are, physically or logically. I haven't been able to get my Firefox profile data moved over yet - like 500 passwords and bookmarks. I've got a lot to do and figure out before I can ditch windows.

Thanks again, guys, for the great help, --Anne

One more comment that linux is not Windows. linux filesystems do not fragment like Windows and having a separate home partition and data does make sense for maintenance. The link I posted shows where program files are located. System configuration files are in /etc/, users configuration files are in their home directories, no registry and nothing spread out like Windows.

Folders are the same as directories but there are no separate c: or d: drives.

The easy way is the best way for most people. When you do the install all the needed directories (also known as folders) are created and populated by appropriate system files. Newer installs create / (the root partition) and /home (the users home directory home partition), some create a /boot (boot partition), and most create a swap partition. Usually nothing else is needed. When a user is added (as is done during install or the first login after the install) that users home directory automatically is created as /home/username so you should not need to create the directory, and every time you log in you start there.
Remember this: In Windows a directory is called a folder. In Linux it most often is called a directory but may be referred to as a folder as well. The two words are synonymous for computer users.

Moving data from windows can be problematic because they don't use the clean functional layout that is done in linux. With Chrome you can import most user data including bookmarks and contacts from firefox (although I don't know about the account/password database). Then, because you probably have a google account if you use chrome, that data can easily be transferred to another machine with a chrome browser.

Overthinking things and doing stuff without a real need can cause problems. Use the defined repositories (repos) to download and install packages so things are always compatible with your system. Having the /home/username directory makes it easy to do backups of your data and since all your personal data in whatever form is there that is the only thing that needs backed up for the great majority of users.

The OS and applications can easily be reinstalled if needed. Personal data is easy to backup and restore since it is kept separate from system areas. All is much easier than the M$ environment.

If I knew how to move this to a different thread (Firefox profile) I would, but I don't know how yet. I just figured out how to use the "QUOTE" mechanism properly.


Point taken on your great suggestions and I will follow them - thanx.

I'm not using Chrome; I'm using Firefox in Win7, so I'm wanting to transfer my Firefox profile in Win7 to the (identical looking) Firefox profile in Linux (Ubuntu 20.04). Or at least my passwords database (2 files in the profile), which is the most important thing. I have not yet tried replacing JUST those two files, yet.

It seems very odd to me that Firefox has no easy way to import it's own profile.

I've tried a number of methods that don't seem to work, but I wonder if it could be because everything is in RAM since I'm running from a "live" flashdrive. I just thought since I updated something and it looked like the data was there, that it was, although Firefox did not find it when I looked for the password database via Firefox.

The thing is, to me, ALL configuration/profile, etc - ANY files into which I have set preferences, etc., - and which might be clobbered (lost) by an update or backup restoration, wherever they are located, are "MY personal data."

My current backup app (2nd Copy) maintains an uncompressed duplicate of my data in a partition on a different HD. In my backup profile, the element that runs first, cherry-picks any config files that reside outside my data HD and copies them to my data HD, so they are then backed up with my personal data and can be restored if an update destroys them.

How could I pick up ALL Linux configuration-type files, wherever they are, to backup with my data? Since I won't have 2nd copy to automate it, am I lucky enough that ALL configuration-type files (system or app) are kept in one (or two) directories, that I could back up along with /home??

--Anne

.
Again, if I knew how to move this to a new thread "Partitions and Directories" I would do so.

So I could keep /etc in the partition with /home & and backup that partition to be sure I had all the configuration files in the backup?
Let me see if I understand this (new concepts to me). Pointers to the partitions are mounted as /dir, right into RAM? so you don't need to refer to anything but the /dir? When you "save" a file, is it "saved" to the actual HD, using the mounted RAM pointer? I'm probably off-the-wall and showing my ignorance - how does this really work, under the hood?

How does Linux backup software work? Does it just back up the source /dir to the mounted target /dir without reference to the partition? I think I'm still confused about this part. If you wanted to restore one specific file from the backup, how would you do that? What does the backup look like and how could you access it?

In windows, if you don't IMAGE the entire partition, you won't get everything, so much is hidden or "in use". You can't reproduce the entire, complete OS partition itself with only File/Folder backups.

But in Linux, to backup or restore a partition, you do not have to IMAGE or CLONE the partition to pick up everything you need to duplicate that partition somewhere else? or to restore an earlier copy of it? If so, yay, that's one of the things I hate about Windows - all the hidden and repeated, meshed together stuff in the registry, requiring a complete IMAGE as a backup. In Linux, the "registry" is visible - it is the /dirs? Nice.
And, to "uninstall" something? Is it pretty much all in one place - or you know where the pieces are? Or the system knows where they are?
How would I backup everything, on all Linux partitions, using the command line? or just one /dir?

Sorry, I'm probably asking too many questions in one place (there is SO MUCH to know, my mind starts racing). If you could just please set me straight where I'm obviously thinking wrong, or point me to relevant references, I would be grateful. Thanks. --Anne

No, /etc can not be moved from /root

What and how you backup depends on your needs.

The blessing of Linux is that there are so many choices. The curse of Linux is that there are so many choices. There are many ways to back up a Linux installation. I use a program called timeshift, in the Debian repositories and I believe in most of the major distro repositories. There is also BackInTime, as well as the venerable tar and rsync, and many more. Which to use is your choice, depending on your preferences and needs. It can take some time and effort to learn about all the choices, and what each does, and how, but it's essential that you do, because no one else knows exactly what your needs and preferences are. You've invested years in learning Windows, so it's not unreasonable to expect to spend some time learning about Linux.

Here and here are a couple places to learn the file system structure and what goes where. Moving things around is likely to break your system. Let the directory tree exist as it is installed and work with your personal data within your home directory (or folder, if you choose to call it that)

As far as backups, and partitions, Linux when it is running has no concept of partitions, only file system structure and content. Mounted partitions are seen as part of the filesystem so except for specific administrative tasks they are irrelevant to everyday operations. Windows treats each partition as a separate thing, linux does not.

One very simple tool that can do backup copies for you is rsync which is a better way to copy files than the old workhorse cp. The man pages for both will tell you the differences. One way to do a full backup of system configs and personal data could be as easy as "sudo rsync -av /etc /home/username /your/backup/directory/". This would copy the /etc directory as well as the /home/username directory and all their content to /your/backup/directory.

There are many other ways to do the backups and rsync is only one of the tools that some use. You can search on the internet to find one that meets your needs and then establish your own backup process. Keeping in mind that system config files are in and under /etc, and personal data and configs are in and under /home/username. All files in the entire system except those in your home directory are related to operation of the OS and installed apps. System configs, once set up to your liking, experience infrequent changes. Personal data, OTOH, changes depending on how you use the system.

As said by michaelk you cannot move things around because the structure is fixed except for what you place under your personal home directory.

I am definitely spending TIME learning Linux, but then I expected to - I don't like flailing about - I like to have some idea what I'm doing, and without doubt, that takes Time. I am so appreciative of all of you Linux masters who are so generous with YOUR time to help folks like me when we get stuck on a wicket. --Anne
.

Thank you. It's so easy to make a non-bootable mistake in a Linux installation. I've been reading about the 1st level directories and most of them should stay in /, There are just a few that should have their own partition. Not done yet, but I'm working on it. --Anne
.

Your entire post was extremely helpful. I'm beginning to get a picture. Something like the /home directory and /backups could be on separate partitions, or even different HDs as long as they were mounted correctly (I wouldn't want backups on the same HD as the source). /etc is in / and /home could be mounted in /, but how would I name/mount the backup? mount: /home/username/backups/.... into / also? Or mount /username/backups... into the /home directory? If so, how to make a mount point there?

What I would end up with in the backup is just a duplicate of /etc, and /home - just those two directories and their contents? Does rsync actually SYNC the target with the source - that is, overwrite any changed files, add new files, and delete files from the /backup directories that have been deleted from the source - to bring the backup into an exact match to the source, or does it just overwrite each entire target directory with the source directory? My 'backup' directory would just be a duplicate of all the directories in my system (except things like /swap of course)? Assuming I backed up all the directories to /backup. /backup would need to be on a quite large partition by itself, assuming it is uncompressed. Can it be encrypted? OMG - this is SO much cleaner than Windows!
.

/home can be on a separate partition or drive, and that's not uncommon. It's actually very handy, allowing you to save your files if you have do do a reinstall. Naming is your choice, as is the mountpoint for everything.

Rsync overwrites changed files and adds new files, but it doesn't delete anything unless told to do so. It will if you tell it to. Backup options are very much dependent on the wants and needs of the user. You can back up everything, or just some things, depending on what you want. As I've said, I like timeshift for backing up my system. It runs at night, and backs up everything except what I've told it to ignore, which is a lot. In /home, I don't back up .cache and some other folders I already have backed up elsewhere. Under /, I don't back up /run, /var, /tmp, /mnt, /media, /boot, /boot/efi, nor /root/timeshift. Everything else is backed up to an external drive. Timeshift uses links, so the backups don't grow so much, and keeping 7 or so backups takes little more space than one. I currently have 8 backups, including a monthly and two weeklies, and the total is ~220GB. It uses an external 1TB drive as needed, no need for a special partition. If you prefer rsync, that's certainly an option, and it doesn't need a special partition either, just a drive/directories. Of course you're free to make a partition just for backups if you want.

Anything can be encrypted. You can encrypt your entire system if you want, or individual drives or partitions. Just be aware that encrypting destroys all data on the drive/partition/folder, irrevocably and absolutely. Anything you don't want to lose needs to be backed up first and then restored after encryption. There are also multiple ways to encrypt - dm-crypt, LUKS, veracrypt, and probably others. Each has its advantages and disadvantages. Encryption is a very big subject, and needs careful investigation before beginning.

Your entire post is so helpful in filling in blanks in the picture I'm building. I like Linux more everytime I learn something new. Would it be better to mount in / (with a longer path) or to mount in /home (with a shorter path)? How exactly would I create a mount point in /home for a backup partition on a different HD?

I looked up rsync and also found grsync, which would be less error prone to typos for me with script creating/editing. I watched one video, but there are more. Thank you for that direction. Q. So we don't talk partitions here either, just directories - so nice & clean. A backup directory on an external drive could look like /BKUPS/date/sys/ and (/(BKPS/date)/MyData/ ??? The last slash would include sub-directories?

I looked at TimeShift too. So you just keep adding incremental backups by date? Is there a way to consolidate them into the base backup at some time? Or do you have to completely copy everything again to start a new base? You said you keep 7 or so, so there must be a way to add all those changes into the base at some point, unless the after-base backups are differential backups, instead of incremental backups, so you'd just need the first (base) and the last differential backup & could delete any in-between differentials that became irrelevant. I just assumed incremental - maybe I was wrong? Differential would actually be better, because you wouldn't have to keep EVERYTHING, just those that represented significant restore points. (I keep a diary file about every change/install/configuration that I make that affect the system (not data), with separators where I make backups, so if I restore a backup at a point in time, I know exactly what to do to get back to where I was. That would work best with differential backups. I would just delete or grey out the deleted differential's heading, like I never made it. With the differential backup, you only need 2 files to restore, the base and any ONE appropriate differential. Sorry, carried away & thinking out loud.

I'll look all these up. Thank you. These can be used to encrypt directories, right? What do you use to zip/encrypt individual files? I got a nasty surprise when I found out that WinZip didn't clean out it's work folders when it finished zipping/unzipping encrypted files. I thought the files were safe, but they were just sitting (unzipped/unencrypted) out there, somewhere else - temp or work folder, whatever WinZip uses. I discovered this when they said their "new version" cleaned up all it's work files, which apparently it had NOT been doing all these years while I didn't know where they were so I couldn't clean them up either.

It sounds like it might be best to encrypt individual files (7-zip?) so they are also encrypted in the backup, but nothing is encrypted that doesn't need to be.

Thank you, thank you, thank you - for helping me fill in the holes in my slowly emerging picture. Linux is SO much cleaner and understandable than Windows. --Anne
.

Might not need to be said but if this is for a desktop, don't overcomplicate things. A separate /home partition, or even drive as sgosnell suggested is an excellent idea. I always ran with a separate drive for /home. I also used whatever default partitions the OS installer went with and never made any of them separate partitions. Did that for 20 years and never had an issue. Requirements for servers are vastly different so separate partitions can be very useful there.

I can't speak to encryption because I have never had an encrypted Linux system so others can better advise you there.

I recall having a similar problem when I migrated from Firefox to Chrome. (I've since moved back to Firefox.) I never found a way -- if one actually exists -- to migrate passwords from one browser to the other. I resorted to doing screen dumps of the passwords in FF and re-keying them into Chrome. I half-suspect that making it a major pain-in-the-neck to migrate your passwords is the browser folk's way of locking you into their software. After doing that process once and vowing to find a way from having to do it again, I wound up downloading KeyPassXC (recommended to me by a co-worker some years ago) and standardized on that as my password vault instead of relying on the browsers. I did re-key the passwords one more time but into KeyPassXC's database---then I deleted all the passwords in the browser. Now the only hassle I might encounter if I were to switch browsers is how to migrate bookmarks and, so far, none have made that an impossible task. Making a copy of the KeyPassXC database from my desktop to my laptop involved a simple sftp command. I Just have to remember to keep the two in sync.

HTH...

Sorry Michael, I meant /opt, not /etc, but the more I learn, the more directories I see that should stay right where they are, except maybe for /swap /boot /home /opt and /var, for maintenance (backup, fragmentation, etc.) reasons. Does that sound reasonable? Thanks. --Anne

Is your last name Kinney, by any chance? I know a super-smart electronic engineer / systems programmer named Michael Kinney who worked for Siemens when I did a bit of free-lance C programming for them back in the past century. He told me that his boss carried my documentation around to his employees, saying "THIS is what documentation should look like". Made my day. That wouldn't be you, would it?
.

Nope. I am an electronic engineer but not super smart...

Forget fragmentation, it isn't a problem with linux filesystems. It depends on what software you install but /opt is rarely used so a separate partition is not necessary. In my opinion you do not need a separate /opt or /var but it depends on what software you install.

The convention is to mount filesystems not associated with the OS to /mnt or /media. In reality you can mount a filesystem almost anywhere.

Thank you for letting me know about KeyPassXC - that is a good reason to have a browser-independent Password Manager.

The reason I'm trying to keep Firefox is that I think it is the safest, least intrusive browser (Google/Chrome scare me with their excessive tracking and personal data collection/sharing), and I really like the way the Password Manager works (except for migration, of course), but I just found out that Firefox has supposedly fixed that problem in the latest version (80+) by now IMporting .csv files as well as exporting them. (I haven't tried it yet, so I don't know if it works - or if it applies to Firefox Linux). I'll be back to post when I find out.

The biggest reason I use Firefox for passwords is that you can open Firefox, enter your Master Password and it will fill in your logins automatically, but will do NOTHING ELSE! You cannot VIEW or COPY or EDIT any passwords until you open a browser tab for Password Maintenance and ENTER the password AGAIN. You can then View, etc. passwords UNTIL you close that TAB - HOWEVER, it will still continue to fill in your login passwords until you close the browser itself (but your passwords are now still hidden from view).

I haven't found another PM that will do that (Roboform used to, but sadly, no more), they just suggest automatically logging out of the PM every few seconds/minutes, which means when you want to just fill in a login, you have to enter the Master Password again, nearly every time, which is pretty inconvenient, or leave the PM wide open with all your passwords in full view, if you want it to fill in logins automatically & conveniently.

Does KeyPassXC make you open a maintenance tab and enter another password if you want to actually SEE the passwords? I might change over if I could find another PM that would do that. I don't like the difficulty, in Firefox, of getting a pdf /printout to keep in my safe or encrypted in my backups. How do you like KeyPassXC compared to Firefox? --Anne
.

Keep in mind that browser based password managers are accessible by the browser application process while running. If a nefarious web site can access the browser password DB, they can grab your passwords. I am not saying this is possible as I have not researched it. I never use browser based password storage because as a paranoid security guy, it makes me nervous.

Separate password databases such as KeepassXC keep everything neatly separated. I always copy/paste passwords into sites but having said that, some applications have access to the copy/paste buffers as well. Without going down that rabbit hole, I would say overall, separate password managers such as KeepassXC are safer overall than browser based password management.

Security is never convenient.

I would have posted yesterday, only the thread is marked solved. So I thought you had moved on.

Your, or anybody's, home partition can be on a separate partition or HDD ...... To have it mount during the boot process the partition need to be included in the /etc/fstab file.

The backup HDD should be on a separate (external) drive.

If it is not on an external drive and the computer's HDD dies, then your backup also rests in peace.

You can use the mount command to mount your external/backup HDD, or when connected, clicking on it in the file manager usually mounts the backup. And the address of the backup, is often to be found in the address box of the file manager.

My data are in folders called /dog and /elephant and /cat (all of which are listed in the /etc/fstab file) so mount at boot time...... two of the three are on separate drives. The third is on a separate partition of the /root drive (HDD)...... and I'll explain more after you install a distro.

My external back up HDD adress is something like /run/media/mine/olga

What you backup is entirely your choice. If you tell your electronic marvel backup your /home folder, then that is, usually, exactly what it does and nothing more. If you want to back up your /etc and /home folders then using two commands that is exactly what the electronic marvel does.

BTW it is usually quicker to create the original back up using the copy command than using rsync.

Yes your backup is a copy of what you last backedup. So keep the backup current. (I recommend at least weekly).

This here link https://www.howtogeek.com/427480/how...-linux-system/ will tell you more about how to back up your data.

Of course if you want more detail than is provided by the above link type type q when you have read enuff..... you might need to have a distro installed to read a man page (a live distro might give you the man page).

Yesterday or the day before you asked about not having the drivers for your electronic marvel. Unless the hardware is new there is often a way to get the hardware to run. Just pop across to the hardware sub forum of this site and ask. Someone there is likely to be able to help.

As mentioned, you should not (and in most cases cannot without breaking things) try to relocate any of the directories the installer creates -- especially the subdirectories of /. You can add new directories in /, (most people don't) but should never change anything already there.

FWIW, you can create a mount point almost anywhere you choose to use for mounting your backup disk to. Commonly people use /mnt which the system creates automatically or a subdirectory there (such as /mnt/backup) which they create for that purpose, but it literally can be almost anywhere you choose. A mount point is nothing more than a directory that you create for attaching a device to the filesystem. If you are backing up your users home directory (/home/username) and you have created a subdirectory there (/home/username/backups) that is used to mount your backup disk it is quite easy, but you will have to exclude that subdirectory from the backup or it might either error or get locked into a loop constantly trying to backup the ever changing backup.

One thing you always need to be aware of is that linux is case sensitive and most people use lower case and no spaces in file and directory names because the space causes other problems. I have yet to see a single command in Linux that is anything other than lower case. User names also MUST start with a lower-case letter. Windows from the beginning was not case sensitive (earlier versions always used upper-case for file and directory names) and users have gotten used to having things in upper-case and using spaces so that is one more thing that has a learning curve when you switch.

Firefox profiles is a right mess. Before it was easy to manage bookmarks and settings and use these across systems. Trying to copy profile content is just messy and often does not work properly.

It's terrible frankly. But that's the fault of Mozilla.

I actually use /opt.. Guess that makes me one of the odd ones out.. I even have it as an own partition, this is how "important" my /opt usage is..

I would advice you try to keep things simple for now.. The only thing you REALLY need to backup is /home! And the way to do backup, is to back it up to a system which is not the same, ergo, disks which are not the same, and not on the same system. (ex, lightening hits your system and breaks all the disks, backup breaks..)

Basically this means that you backup your /home folder to a USB disk of some sort. There are so many different methods to do backup, but it really comes down to how important your data is, how confidential it is and how convenient you want your backup process to be.

Regarding encryption there are also many ways to go about this. But a convenient way is to have a fully encrypted external disk as a backup disk. But then again, a safe backup should not be encrypted, unless data is highly confidential. If the disk or partiton table breaks, there is no way to recover encrypted backup. That's I guess where raid comes in handy. It really depends on your needs. Pretty much endless options are available, but you need to find the right ones for you.