Windows certainly doesn't have that capability itself, but a third-party utility called Explore2fs is supposed to be able read ext2fs (and possibly ext3) partitions, although it doesn't support writes.
Check out the
home page, and be sure to read the info carefully.
I haven't heard of any other utilities which do what you want, although they might be out there.