In linux, the floppy won't be known as the "A" drive. Commonly it will be /mnt/floppy or /mnt/fd0.
Your floppy will have a file system on it. For windows to read it, the file system has to be the FAT file system, but if you formatted it in Linux it could be that Linux formatted it with EXT2 or EXT3. Windows will not natively read this.
You can either reformat the floppies using FAT (which Linux can read/write) or you can obtain the EXT2 file system for Windows from here: