I have an Acer laptop with a 1TB Toshiba HDD, 2.5" 5400 rpm.
I am now running Xubuntu 16.10

For months I have experienced problems with the hard drive. I have run the SMART tests and they do not reveal any defects. At random times, a few days to a few weeks apart, I suddenly find that I have a "read-only file system", and therefore everything stops working. I cannot do a normal shut down because of the unwritable disk, so I kill it and of course it will not boot. In the past I had to boot from a CD, but now it starts a Busybox shell when it finds it can't read the disk, and so I can do fsck and watch it find hundreds of orphaned nodes and lots of other problems, all of which it eventually manages to fix. After that I can just boot the thing normally and all is well - until next time.

question 1: what could possible be causing this? can it be fixed? is my data at risk (I assume yes)

question 2: assuming it is not fixable, I need to replace it. I plan to buy a new hdd of equal capacity and an external USB drive caddy in order to copy every byte of the internal drive to the new one - that is to "clone" it. In the past I think dd has been recommended for this. I know that I have done it before using cp, but had a lot of problems because it did not copy the hidden files. Do I have to partition the new drive and then copy each partition individually? Is there a way to clone the whole drive with a single command (and then go to sleep while it does it). Can someone please advise the best way to do this?
Thanks!

First, a note: Don't use dd to copy a mounted system, boot from a live distro on a usb stick

Assuming /dev/sda is your source and /dev/sdb is your destination

This will make a bit by bit copy.

Things to note: Those issues sound indicative of "ungraceful" shutdowns e.g. unexpected power loss. A new disk will do exactly the same thing if it faces the same conditions.

If SMART doesn't find anything wrong with it, and you have the same problems with a new disk, I'd check your power source.

Also you can always try to

Unexpected power loss? It's a laptop. It has a battery so it doesn't lose power. But I have many times had to just force a shutdown because the system will not let me log out without writing to disk. Usually I don't shut it down because it takes a very long time to boot; I just suspend it.

I think there is something wrong with the disk because it is also extremely slow to read files. 3-5 times slower than it used to be when it was new, and the slowdown was abrupt, not gradual.

Yes I have tried a remount, but since the disk is full of errors it just goes back to read-only again as soon as it tries to write anything.

Thanks for the dd command. I will be using it when I get the new disk.

Tests.
Run the OEM hard drive diags.
Run Memtest for a few days.

Cloning is a tricky issue. We don't know the state of your hard drive anymore. It may be worth it to use a backup for data. Maybe for tips on what was installed. I tend to just reload. Makes it more likely in my opinion that all is correct.

A number of ways to clone also. Many people use clonezilla, redobackup, (programs in your repos), gparted, partimage, g4u, rsync, cpio, tar and others along with dd command. Each has good and bad to them.

whereas X is the original disk, and Y is the disk being copied to, use lsblk to get the correct /dev/sd? information.
I've heard that dd'ing a bad drive makes an exact copy including the errors even though the new drive isn't "bad" but don't know if that is true and anyway, the type of errors mentioned here should be fine with a dd copy. Be prepared for it to take a few hours, be patient, the is your friend
I always use dd for a new drive and rsync for backups

source -> destination, everything is source -> destination. good rule of thumb to remember.

dd is the absolute worst option. Maybe ddrescue in really dire circumstances, but I would never use dd.

I would use a filesystem-aware tool - any of the above mentioned would suffice. That way you get to know about which file(s) may be suspect.
Much better option would be to simply restore your latest (known good ?) backup. I guessing that is a forlorn hope.

I think I have not been clear enough about this problem. Right now there is nothing whatever wrong with my file system - everything is normal. But sometime soon, in a few days or weeks, seemingly quite at random, I will suddenly find again that I have a read-only file system. The only thing I can do then is shut down, and then boot up without mounting any partitions. Then I run can fsck and tell it to fix every error automatically, and after a few minutes it completes and tells me that all partitions are clean. At that stage I can boot the system normally and all is well - until next time, whenever that will be.

I don't know if there is anything physically wrong with the disk. Testing has not revealed anything, and it works normally until the the next event. But what can be the cause, other than the disk itself? I must find the problem and fix it, or replace the computer - an expensive one. My plan is to replace the drive with a new one. That is the sole purpose of cloning it. I want to replace the disk with minimal disruption, and if I can clone the disk so that byte-for-byte identical with the old one that should achieve my objective.

so if your disk is really ok, you can try to dd (see post #2) the content from this one onto another (bigger?) one and try to boot from the new disk. Probably it will work, and if not you can still use your old disk. Later you can resize the partitions to use the full disk.

Wouldn't rsync also work? It is used for back ups
Because it basically just makes a copy of whatever it is working on.

rsync is unable to copy data outside of a filesystem, like partition table, MBR and similar.

I question this.

If OP is going to clone his system. Then Why? The most important items that will need to be taken off the hdd that is failing is his data. The OS itself is not all that important. A simple installing of the same OS onto a different HDD would be the simplest and probably the fastest method to use. He'd only have to ensure that he gets all of his modified conf files out of the system side so he can avoid having to modify them again.

Not sure why dd is worst option, I've been using it without issues for the past 6 yrs - seems to work very well, but takes patience to let it do what it does and it is byte for byte, its actually bit for bit.
Installing a new OS can take as long or longer than DD considering adding all the extra applications and peripherals back in. DD gives you everything. It even copies the disk UUID, keep that in mind if you want to use the new disk UUID as two of the same may have problem being connected to same computer when finished. You can always change the UUID, but get it before you DD the new disk. DD before going to bed, have an OS in the morning without having to do anything else aside from removing old drive - just pray the power doesn't cycle during the process
I've dd'd Windows XP disk and booted new disk without issue - W7 different story, it worked but had to do a lot of system tweaking...