Single source file

Programming, for all ages and all languages.
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Single source file

Post by Antti »

iansjack wrote:[...]
How can that be? You have an empty file with "a lot of meta-data with a file" and you upload it to some random ftp servers. After a while, I decide to download it. What I get? Nothing.

Now the "non-KISS" version: the file you get from me contains all the information. It goes without saying that you do not want to install a special program. However, you can open the file with vi and even understand roughly what happens.

Believe me or not, I am starting to believe that it probably is not a good idea to implement "single-source file" with C programs. I have not changed my mind about all the advantages the concept has. However, now it is starting to look like that even the single-file concept in general is too much for many programmers.
User avatar
iansjack
Member
Member
Posts: 4706
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Single source file

Post by iansjack »

You certainly seem to have pinpointed a problem with single-source files and meta-data. Myself, I do it the conventional way. I have downloaded the source for thousands of programs over the years, consisting of multiple source files (albeit encapsulated in a single .tar file). And you know what - I've never lost any meta-data necessary to turn those source files into programs.

It's simple, and it just works. No need for meta-data, it's all contained in the simple textfiles that comprise the source. Why would I need special formats that require particular IDEs/toolchains to produce the same result?
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: Single source file

Post by Brendan »

Hi,
bluemoon wrote:The thing that matter to most people is how to toolchain work, almost all toolchain and IDE assume file-based organisation, and store meta-info on project preference; there is obvious lack on the other choice for any sane comparison.
I'd say the thing that matters most to programmers is the user interface/s that they use to get work done. Source code is just data, and a user interface is just a way of viewing (and manipulating) that data. Let's all have a quick look at the wikipedia page for data visualisation to get some ideas; and then think about what the best way of viewing source code might be.

I don't see any real benefit in exposing "files" in the user interface/IDE. I'd rather see a navigation box containing a list of classes than a list of files; but then I'd rather see something more graphical than a list (e.g. a class diagram that shows relationships between classes, instead of a list of classes that tells you nothing about the relationships between them). Of course this is theoretically possible regardless of how source code is stored; it's just easier for IDE developers if they don't need to do so much work to hide the underlying storage format (e.g. and they don't need to maintain their own list of classes and which file/s they're in using a separate "project file").

Ironically, as iansjack has mentioned, for more modern programming environments there's currently a push towards "one file per class"; and in that case there's little difference between a list of classes and a list of files. That's a small step in the right direction at least (we get a list of classes!); but a good inventor would wonder about the next small step, or if a large step in the right direction is possible, or if they can make a giant leap.

Sadly, "compatibility with current tools" is like a huge brick wall preventing progress. Storing source code as plain text and storing source code as multiple files are 2 of the largest bricks in that wall.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Single source file

Post by Antti »

Dennis Ritchie wrote:C is quirky, flawed, and an enormous success.
Now I am thinking more about the "learners", "followers" and "inventors". I have some thoughts I want to share:

If we think about Linux, like already mentioned, it follows the work done by previous "inventors". In this case, the inventing era was roughly the years 1950-1980. Perhaps the most productive era was late sixties and early seventies if thinking about the concepts we are relying on in Unix-like systems. Although there are other OSs nowadays, they seem share the main principles and are "more or less Unix-like". For example, one invention the early inventors made is "the files". We have persistent objects, they're called files. - Ken? These files can be considered having at least one portable property: the name. The modification time is also a quite portable property.

All right, can we forget the files for a while? What do you think the early inventors might be thinking now (if still alive)? I have two examples:

a) "We were very wise and created a perfect system that is going to last forever. We knew how the computer industry is going to develop and how the hardware capabilites increase. We also knew how people would behave."

b) "Hey hey hey let's hold on. What we did back then seemed to be good idea and we did best we could. We have quite limited resources but we wanted to get something done and we were paid for that. It turned out that our ideas were quite good. However, I am terrified that those ideas are so high-valued among the people in this field. There were many temporary solutions that were supposed to last just until we got more hardware resources or found other way to do that more elegantly. Now it turned out that we are so stuck in this. The whole computer industry depends on these assumptions. Please try to break this and try to search new ways of doing things you consider self-evident. We won't mind. I think some of the ideas were bad at the very beginning."

Does that latter make any sense? I think it does.
User avatar
dozniak
Member
Member
Posts: 723
Joined: Thu Jul 12, 2012 7:29 am
Location: Tallinn, Estonia

Re: Single source file

Post by dozniak »

It does.
Learn to read.
User avatar
iansjack
Member
Member
Posts: 4706
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Single source file

Post by iansjack »

Please try to break this and try to search new ways of doing things you consider self-evident.
The thing is, I don't see any sign that attempts by you, or anyone else, have broken the concept of a file as a unit of data storage. And there is a saying, "if it ain't broke ...".
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: Single source file

Post by Combuster »

I have seen databasestorage systems and someone who saved disk storage by merging equal files on the filesystem and storing diffs rather than actual files.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Single source file

Post by Antti »

In a forum like this, a saying "if it ain't broke..." is not so valid than normally. Should we change the forum name? I have a good one: "Here you learn how existing operating systems work."
User avatar
iansjack
Member
Member
Posts: 4706
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Single source file

Post by iansjack »

In a forum like this, a saying "if it ain't broke..." is not so valid than normally.
I couldn't disagree more. In OS design (or in this case development tool design), as in anything else, the most important thing is to address those things that are broke or aren't implemented at all, not to think up new ways of doing things that currently work very well. Why introduce extra complexity for it's own sake? You are not even proposing any alternative to the file as a concept, just asking us to use one when many is a more appropriate way of working.
Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction.
Antti
Member
Member
Posts: 923
Joined: Thu Jul 05, 2012 5:12 am
Location: Finland

Re: Single source file

Post by Antti »

I do not understand why you think I am trying to make things more complex. I do everything I can do to simplify the mess. It seems that "what I can do" is not always enough. I agree that. If my proposition ends up being bad, it does not mean the problems cease to exist. Neither it does mean I should stop trying. It is apparent that you like very much the current way of doing things. That is why we have different views about "problems".

Why do you do "operating systems development"? What I try to do here? I am not here earning my living and I think neither anyone of us are. I do not have to "get things done" with software anyhow related to this topic: OSDev. Of course it is good to "get things done" but it is not a strict requirement. That is one of the reasons why we are able to do experiments. It seems that we do not use that opportunity very much.

Of course, I hope that at some point those experiments end up being good enough to be something more.
Kevin
Member
Member
Posts: 1071
Joined: Sun Feb 01, 2009 6:11 am
Location: Germany
Contact:

Re: Single source file

Post by Kevin »

Antti wrote:If we think about Linux, like already mentioned, it follows the work done by previous "inventors".
That's only true to a certain degree. Sure, in some fundamental prinicples like files it just follows. But there's more to a kernel (even more a monolithic one) than just one part of the userspace API.
We have persistent objects, they're called files. - Ken?
Alright, so this is what you want to overcome. Great, but the way to do this isn't to tell us to keep doing the same, but just use less of these files. You need to come up with a new concept that fulfills the same requirements (and probably some more, because otherwise there would be no motivation to make a switch from the familiar concept) in a better way. This is the part that's completely missing in your proposal yet, you don't have an alternative.

So what part exactly do you want to get of? "persistent objects"? Probably not, people wouldn't like to lose persistency. I have a strong suspicion that it's the "they're called files" part, and you're really just after renaming things. Or, if it's neither, your definition of a file wasn't complete.

Another thing that the most common file systems provide is a hierarchical directory structure to actually find files/persistent objects. This is where you could change things if you found a good new way of presenting/addressing the objects. I must say that I like tree-like structures not too bad (symlinks turn it into a directed graph and then things already start to become a bit more confusing) and organise things in hierarchies even outside the file system, so I'm not unhappy with what we have today. Occasionally I've been thinking about two- (or multi-)dimensional file systems, but it turned out a bit confusing and I didn't consider it worth thinking more about it given that hierarchical file systems just work fine.

(This is all simplified a bit by ignoring that files aren't atomic objects, but rather contain many subobjects. Therefore your proposal doesn't really make a difference for this part of the discussion, because you're merely changing the granularity at which the file system vs. the user application manages the objects in a given set of objects. Using the file system is common code, doing it in the application can easily lead to reinventing the wheel everywhere.)
b) "Hey hey hey let's hold on. What we did back then seemed to be good idea and we did best we could. We have quite limited resources but we wanted to get something done and we were paid for that. It turned out that our ideas were quite good. However, I am terrified that those ideas are so high-valued among the people in this field. There were many temporary solutions that were supposed to last just until we got more hardware resources or found other way to do that more elegantly. Now it turned out that we are so stuck in this. The whole computer industry depends on these assumptions. Please try to break this and try to search new ways of doing things you consider self-evident. We won't mind. I think some of the ideas were bad at the very beginning."

Does that latter make any sense? I think it does.
Can you name the temporary solutions made for the limited hardware resources that we're still stuck with? We're not using the exact same file system any more. What we do use is the abstraction of files and the tree-like structure. They seem pretty universal and not tied to hardware capabilities.
Developer of tyndur - community OS of Lowlevel (German)
Prochamber
Member
Member
Posts: 100
Joined: Wed Mar 13, 2013 2:27 am

Re: Single source file

Post by Prochamber »

This certainly is an interesting idea and great for the end user but would be a huge pain for developers to support everything. I use a wide range of editors on different operating systems. I can't expect them all to implement tools to support the management of these massive source files. Everyone would need learn the new standard format.

It would be great if I could have my entire operating system in one file rather a trillion. It would be really easy to simply send someone a single file and the recipient would just be able to click double click to run it in an emulator. It sounds like a compatibility dream and it would certainly make it easier to try out projects made by others.

However this would require an impossible level of standardization between operating systems. Operating systems are like spoken languages, you can never get an exact translation between two different implementations. If you just want to say "Hello" you can translate that into any language but if you want to translate a speech you'll have to settle with something 'roughly equivalent' or at best 'very similar'.

Don't get me wrong, I wish it were possible to create a single source file and not worry about build system, compatibility, etc. it's simply not possible. Perhaps build systems are a necessary complication.

PS Stop bashing Microsoft, there are plenty of other companies that produce terrible close source software. If Smart Phones/Tablets take over, Apple will be the new evil proprietary software corporation.
TachyonOS - Violates causality on 95% of attempts. Runs at approximately 1.5c.
Kevin
Member
Member
Posts: 1071
Joined: Sun Feb 01, 2009 6:11 am
Location: Germany
Contact:

Re: Single source file

Post by Kevin »

Prochamber wrote:It would be great if I could have my entire operating system in one file rather a trillion. It would be really easy to simply send someone a single file and the recipient would just be able to click double click to run it in an emulator. It sounds like a compatibility dream and it would certainly make it easier to try out projects made by others.
Aha, having only one file isn't your goal, but what you think is the right way to achieve your real goal. You're making the following assumptions:
  1. It's inherently hard to send multiple files, but it's easy to send a single file
  2. Double clicking a single file can launch an emulator, making a double click on a collection of files (e.g. a directory) launch an emulator is fundamentally impossible
  3. Users are even interested in source code files and don't want binaries, because... don't know... because they just do?
It might be a good idea to solve the first two points and thus address the real issue instead of pushing everything into a single file to evade the control of the OS and implement the improvements locally in the IDE.

The third item is really just a detail that I don't agree with and doesn't matter much to the discussion - even with binaries you usually have multiple files, because there's not only source, but also data. You can of course push data into the binary and probably also into your source file archive, so it doesn't really make a big difference conceptually.
Developer of tyndur - community OS of Lowlevel (German)
User avatar
dozniak
Member
Member
Posts: 723
Joined: Thu Jul 12, 2012 7:29 am
Location: Tallinn, Estonia

Re: Single source file

Post by dozniak »

Going back to Brendan's idea.

Replace "file" with "interface".

Now you can do many interesting things with it. You don't need to know what's behind the interface - be it file, a collection of files, an active object compiling files to execute your interface requests or something entirely different.

You cannot access the byte contents of something behind the interface, but hopefully you won't need to do that at all. This gives some flexbility and can still keep those objects persistent.
Learn to read.
Prochamber
Member
Member
Posts: 100
Joined: Wed Mar 13, 2013 2:27 am

Re: Single source file

Post by Prochamber »

Kevin wrote:It's inherently hard to send multiple files, but it's easy to send a single file
Sure, if you have multiple files you need to put it in an archive and that requires additional software to setup. If you're using Windows, you'll probably want to create a .zip. On Linux a .tar.gz is more often preferred, which isn't supported out of the box on Windows.
Kevin wrote:Double clicking a single file can launch an emulator, making a double click on a collection of files (e.g. a directory) launch an emulator is fundamentally impossible
On a well documented project this is easy, just run './configure, make, make install, <emulator>' or run the right shell script or whatever the documentation wants you to do. For poorly documented projects this can be very confusing. This is a task that a developer can figure out, however what if an end user wants to run a program that isn't available as a binary, what can they do? It would be great to just to double click on a file.
Kevin wrote:Users are even interested in source code files and don't want binaries, because... don't know... because they just do?
Source code is portable and binaries are not. Theoretically a user would simple double click on the source file and it would build and run a binary for their corresponding platform. A user doesn't always know if they need the 32-bit or 64-bit version of the binary or the version of their operating system. Binaries will always be platform specific but source code can be platform independent.

Simplicity is the goal. Having a complex file system doesn't help.
TachyonOS - Violates causality on 95% of attempts. Runs at approximately 1.5c.
Post Reply