Offsets at kernel-level

[vhaudiquet]

Hi everybody !
I recently saw that my whole VFS/File handling kernel architecture was actually pretty bad ; i implemented that in the beginning of the development and now i need to do it again in a better way. I wanted to discuss about an important point of that.
In the UNIX world, files are represented on the file system with inodes (file data) and dirents (links to inodes). But when you open a file, the kernel gives you a "file descriptor", which basically contains a pointer to the inode (so you can call read() or write() with the file descriptor as argument), but also contains an offset and a mode (read, write, ...).
I don't understand why the offset/mode is present at kernel level. That requires to add a system call to seek(), which is slow and not needed.
Why couldn't the C Library holds in the FILE structure : something that represent the file for the kernel, the offset, and the mode. In that way, seek() would just be a normal userspace function, and the kernel space would not be bothered by offset/file mode, and could keep the same structure for all open() calls to a file.
Do you think that this is a good design idea ? and why was it implemented at kernel-level in UNIX in the first place ?

Thanks for reading,
Valou3433

[~]

Probably it's because the kernel needs file system access without any external help, so it needs to include its own file system code and drivers.

The system could probably be configured later to use a user-space file system, and the kernel file system could be left for emergency, for more critical operations or for restoring from a failure of the user-space file system.

But I am sure that if they are used for different cases, they would need to be finely synchronized to make data corruption impossible. A lot of data would probably need to be synchronized on the disk itself instead of in code, but those are already design decisions to investigate.

[vhaudiquet]

If the kernel needs to read at a specific offset, it just calls read() with offset as an argument.
Read prototype would be like : int read(int file, char* buffer, unsigned int length, unsigned int offset);
On read() syscall, the offset contained in the FILE struct of C library would be passed like other arguments.

[~]

You should probably reserve this whole year 2018 to implement well your file system, if you feel that you will enjoy it more than other things for this year.

I think I've proven to myself that to really learn how to do things like this, one would spend 1 year to advance a project on a topic. At the end of that year, you would have to have implemented a minimally usable program, library, project. I think that if you have no help and no experience, you will do well to spend 1 year for each thing you want to add to your kernel (1 year for the kernel core, for the user input/output, for the file system, for multitasking, for memory managemet, for a compiler toolchain, for user programs, for network, for audio).

You will find that you will have spent around 9 or 10 years, but I can assure you that what you produce will be much more abundant than the 99.99% of the rest of people who tries kernel development and related difficult matters.

This year 2018 is the second time I've formally reserved the whole year for a main project, for a C compiler totally written by me to truly learn the C language and a bit of C++, to leave space for it, as sort of a third tongue (it doesn't mean that I won't do a little of other things to make things funnier). In 2015 I reserved the year to implement a text recorder, an HTML5 program that allows me to play back keystrokes of what I write, very useful to see how a program is made, I added plain text and syntax highlight, generation of TAR archives, a floating window with tree view of the code, all of that in 2015.

So you only need to pick what you will feel funnier for the year, so that you are able to reserve it for that project and work the whole year to learn about it and implement it. You could work without a set project per year, it might feel easier, but I can tell you that having a main project per year that you need to leave minimally functional at the end of that year, you will advance probably what you would otherwise need 10 years to advance without concentrating effort and goals for a single thing in the same year to implement well.

[davidv1992]

Hi everybody !
I recently saw that my whole VFS/File handling kernel architecture was actually pretty bad ; i implemented that in the beginning of the development and now i need to do it again in a better way. I wanted to discuss about an important point of that.
In the UNIX world, files are represented on the file system with inodes (file data) and dirents (links to inodes). But when you open a file, the kernel gives you a "file descriptor", which basically contains a pointer to the inode (so you can call read() or write() with the file descriptor as argument), but also contains an offset and a mode (read, write, ...).
I don't understand why the offset/mode is present at kernel level. That requires to add a system call to seek(), which is slow and not needed.
Why couldn't the C Library holds in the FILE structure : something that represent the file for the kernel, the offset, and the mode. In that way, seek() would just be a normal userspace function, and the kernel space would not be bothered by offset/file mode, and could keep the same structure for all open() calls to a file.
Do you think that this is a good design idea ? and why was it implemented at kernel-level in UNIX in the first place ?

Thanks for reading,
Valou3433

The reason for unixy kernels to keep around an offset has to do with I/O redirection. When in the commandline you redirect output to a file, and then the process forks, it is desirable that when those different processes all write to the output, they don't overwrite each others output. By keeping the offset in the kernel, this offset can be shared between the various processes without those processes needing to know about this use case.

[Solar]

I don't understand why the offset/mode is present at kernel level. That requires to add a system call to seek(), which is slow and not needed.

To the contrary, it's faster that way.

Upon opening the file, you have all the information, courtesy of the arguments to open(). You know the mode, you know the initial offset. (Beginning of the file, or end of it for O_APPEND in which case you need to seek() anyway.) All it costs you to keep track of it are the couple of bytes required to store the information.

Updating the offset merely requires addition / subtraction and storing the result, which is definitely faster than querying the file structure about it.

The standard library -- mine at least, and the one showcased by Plaugher's book on its implementation -- do hold their own version of offset / mode... you might notice that there is no complete direct mapping of standard options to POSIX options and vice versa. Plus, the library has to handle its own kind of buffering. (Implementing ungetc() is a real PITA if you designed your buffer handling without having that particular function in mind from the beginning...)