Program data storage.

B.E · Post by **B.E** » Sat Jul 21, 2007 6:03 pm

Here's my idea,

A file is like non-volatile memory (i.e the data isn't lost when power off or program termination). Why not make each program have it's own "non-volatile" piece of memory. It would act like basically a page of memory.

This would make programs start up quicker, and would have no overhead when reading the data.

Is there any problems with this approach

Brynet-Inc · Post by **Brynet-Inc** » Sat Jul 21, 2007 6:24 pm

But RAM isn't non-volatile... I can't find your logic, is it missing?

pcmattman · Post by **pcmattman** » Sat Jul 21, 2007 6:26 pm

Hard drive access is much slower than the RAM, which is why programs take a while to load.

frank · Post by **frank** » Sat Jul 21, 2007 6:39 pm

It would waste hard drive space for one thing and programs usually have configuration files and save files for non-volatile storage. I am assuming you mean non-volatile storage like when a computer saves everything in memory to disk like when you hibernate the computer, except that you are planning on only saving the programs memory image.

Brynet-Inc · Post by **Brynet-Inc** » Sat Jul 21, 2007 6:47 pm

Not to mention it would use an excessive amount of resources... power failures would also make this idea a failure, and what about running multiple instances of the same application? - I'm assuming that when a process is terminated it can resume it's position when restarted as well? - How would you manage this? - Security concerns? proprietary API's?

Even if this was possible, which I highly doubt.. you would be insane to try it..

B.E · Post by **B.E** » Sat Jul 21, 2007 7:16 pm

Brynet-net wrote: But RAM isn't non-volatile... I can't find your logic, is it missing?

It would basically be a saved paged (if that makes since).

pcmattman wrote: Hard drive access is much slower than the RAM, which is why programs take a while to load.

it would only be as slow as when read a page from disk.

frank wrote: It would waste hard drive space for one thing and programs usually have configuration files and save files for non-volatile storage. I am assuming you mean non-volatile storage like when a computer saves everything in memory to disk like when you hibernate the computer, except that you are planning on only saving the programs memory image.

The program could save it's data structures (i.e logs, Binary trees, linked lists, configuration data, program state, etc). Edit: Also it could act as a shared memory location to.

Brynet-Inc wrote: Not to mention it would use an excessive amount of resources...

Like the paging system, the resources would only be used when needed

power failures would also make this idea a failure, and what about running multiple instances of the same application?

I'm assuming that when a process is terminated and can resume it's position when restarted? - How would you manage this? - Security concerns? proprietary API's?

These are problems I've yet to work out. Although, A MD5(or some sort of hash) could be used to manage data, this also would help prevent viruses.

Even if this was possible, which I highly doubt.. you would be insane to try it.

it'd be very simple to implement, it would be incorporated into the paging subsystem. which when a program reads a particular address, the paging system loads the data from disk.

pcmattman · Post by **pcmattman** » Sat Jul 21, 2007 7:27 pm

What you're talking about sounds a lot like page swapping, where pages are swapped out to a pagefile when they haven't been used for a while (and then swapped back in when they're needed).

Crazed123 · Post by **Crazed123** » Sat Jul 21, 2007 9:53 pm

Actually, it sounds a lot more like orthogonal persistence.

Kevin McGuire · Post by **Kevin McGuire** » Sun Jul 22, 2007 4:50 am

It seems to be the same as:

.. program startup ..
LoadSettingsFromFile(SETTINGS_FILE);
SaveSettingsToFile(SETTINGS_FILE);

The only different is that you are making the system perform this operation instead of the application.

(i.e logs, Binary trees, linked lists, configuration data, program state, etc). Edit: Also it could act as a shared memory location to./

Linked Lists And Trees
Take for instance CAD software, it might consist of large list linked structures holding objects, geometry arrays, and such other things. The problem is these change when the user loads a new drawing. Why not just let the application handle this data? This is the only major type of information I could figure would be stored in linked list and trees.

Configuration Data
This could be used with the idea since should consist most of GUI settings (default windows placements, styles, blah and blah).

Program State
When you say program state I think of all the above. This would be very similar to hibernation -- which is an idea. Why not save the entire program state when the user closes the application? The only draw back is you would need to provide two close buttons or methods so that the user can completely restart a application or hibernate it. The reason is bugs can slowly start to show over time which is why a very common solution to fixing bugs is to restart the computer or application -- anyway this might be an idea.

Try It?
I can see an idea that should work by providing two methods to (close a window in a GUI environment) where one completely terminates the application and the other hibernates it. The hibernation would allow the application to start much faster next time it is used.

I would try.

pcmattman · Post by **pcmattman** » Sun Jul 22, 2007 4:23 pm

All you'd need to do would be write the entire address space to a file, and then when it needs to be loaded again there should be no problems with writing it all back (starting with the page directories and tables might help)

piranha · Post by **piranha** » Sun Jul 22, 2007 5:33 pm

So when the program finishes loading, the OS would save a 'snapshot' of the programs memory to the harddrive and just load that the next time the program loads?

JamesM · Post by **JamesM** » Mon Jul 23, 2007 6:49 am

What you're talking about is precaching the resources a program needs, right?

Well, generating linked lists, trees, etc, are (1) fast, compared to disk access and (2) depend heavily on pointers, which change with every process instance. (This has been mentioned above)

The main loading time for many programs (especially graphics-intensive programs like games) comes from the resources they load (from disk).
Windows Vista actually has a nice feature (!!!) that caches the filenames that a process accesses on startup, then, next time that process is run, it precaches those files in RAM (in a highly efficient manner, certianly a lot more efficient than the random-access most programs use).

Also, there is a lot of talk about (has it been implemented yet?) having a small flash drive in the computer, that gets saved with the kernel/important startup files. This is persistent storage, and much faster than an HDD. It speeds up bootup time by <insert statistic>.

Just my 2-pennies worth,

JamesM

AndrewAPrice · Post by **AndrewAPrice** » Mon Jul 23, 2007 7:50 am

This has given me a few interesting ideas. How about hibernating programs? Like, if you need to close a program, you have a choice of saving it's state.

AJ · Post by AJ » Mon Jul 23, 2007 8:07 am

MessiahAndrw wrote:This has given me a few interesting ideas. How about hibernating programs? Like, if you need to close a program, you have a choice of saving it's state.

Some potential problems:

* What if your program requires dynamic linking? There is a high chance that when it wakes up, the dll could be in a different place.
* What if your program makes use of something like its Program ID or Window ID which are likely to be assigned dynamically?
* How do you solve the problem of several instances of the program running / hibernating concurrently.

I think it's a good idea in some cases, but just wanted to throw some thoughts in to the melting pot

I have heard of an OS storing the disk sectors required to be loaded each time a program starts, and going from that list. I guess that causes problems for defragmenters though!

Cheers,
Adam

JamesM · Post by **JamesM** » Tue Jul 24, 2007 1:34 am

I have heard of an OS storing the disk sectors required to be loaded each time a program starts, and going from that list. I guess that causes problems for defragmenters though!

As I mentioned above, Vista does that. Although it stores the filenames instead of the disk sectors so it can shimmy sectors around on disk to it's heart's content.