Virtualization and disk management

[thEsp]

The one thing that keeps me far from attempting to write a proper operating system is disk management (reading & writing) through a virtual machine, such as QEMU. I have little to no knowledge about file systems and am not sure how disk images work. So my question is, how do I manage the files in a virtual disk and read through it, in the most basic way possible?

~ regards, thEsp

[Octocontrabass]

It's pretty common to mount the disk image so it appears like a physical disk under your host OS. On Linux (and other Unix-likes), you can do that with a loopback device. On Windows, you can use diskpart.

This method can be difficult to automate, and it requires your host OS to support the filesystem you're using, so you may want to find or write specialized tools to work directly with disk images.

[PeterX]

The one thing that keeps me far from attempting to write a proper operating system is disk management (reading & writing) through a virtual machine, such as QEMU. I have little to no knowledge about file systems and am not sure how disk images work. So my question is, how do I manage the files in a virtual disk and read through it, in the most basic way possible?

One way is to not use a disk image but a normal directory. For example in my makefile I have:

Code: Select all

qemu-system-x86_64 -hda fat:rw:mountdir [more options...]

where mountdir is a normal directory on my Linux host OS.

Greetings
Peter

[iansjack]

I'm not quite clear. Are you asking how to read the virtual disk from your host system or from your operating system? In the former case, see the above answers. In the latter case, you read it just as you would read a physical disk.

[bzt]

The one thing that keeps me far from attempting to write a proper operating system is disk management (reading & writing) through a virtual machine, such as QEMU.

You just specify a disk image file on the host and QEMU will show its contents as a disk to the guest.

I have little to no knowledge about file systems and am not sure how disk images work.

Disks contains sectors. When you save all sectors into a file, one after another, that's a disk image. That's all. Some VMs use their own format (VMDK, VDI, etc.) but those are the same with a header. To add such a header you can use qemu-img (separate package) or "VBoxManage convertfromraw" (part of VirtualBox) commands. But QEMU understands raw disk images too:

Code: Select all

qemu -drive file=(yourimagefile),format=raw

So my question is, how do I manage the files in a virtual disk and read through it, in the most basic way possible?

Under Linux you can create a loopback device as others said. Under Windows you can use some utility like DaemonTools, BenLunt's fs tool or FUSE. That latter has a big advantage that you can later reuse ca. 90% of the code in your kernel, and you can also implement your own FS with it (Windows version requires cygwin though).

Cheers,
bzt

[thEsp]

@iansjack: Yes sir, I'm looking for the latter. I don't know how to read the contents of a disk drive from *my OS and how to write to it beforehand (so I obviously have what to read).

[bzt]

I don't know how to read the contents of a disk drive from *my OS

You can't. You need to read sectors, and then parse the file system on your own. The former is implemented as a device in your kernel, while the latter as VFS abstraction (if you want to support more file systems).

how to write to it beforehand (so I obviously have what to read).

See all the other answers. For FAT, you can also use mtools.

Cheers,
bzt

[bloodline]

@iansjack: Yes sir, I'm looking for the latter. I don't know how to read the contents of a disk drive from *my OS and how to write to it beforehand (so I obviously have what to read).

If you use MacOS use the command line: “hdiutil attach <diskimage>” and that will mount the disk image as normal drive on your machine. Then you can build the image as needed.

Reading a disk in your OS is not easy. There are two aspects to reading disks. Firstly the physical disk, where you read the data in 512k blocks (called sectors), secondly you need a filesystem to make sense of the data in those blocks.

In my initial stages of OS development my ATA device driver handled the filesystem directly, this allowed me to learn how the FAT filesystem works without worrying about how the disk and its partitions fit into the filesystem namespace.

If any has noticed that I’ve not been very active on here recently it is because I’ve been struggling to fit the filesystem concept into my device model.

I’ve finally settled on a two layer approach, I’m treating the filesystem as a “virtual device driver”, which instead of accessing real hardware, sits on top of the ATA device driver. That way the filesystem can be treated as though it is an ordinary device.

I should note that I’m using the TripOS filesystem namespace model (each “device” is the root of its own tree) rather than UNIX (everything is mounted in a single tree), which suits my design better.