[Sputnik-list] versium (was blueprint)

[Yuri Takhteyev]

I _think_ we are on the same page here.  Perhaps there would have been
less confusion if versium documentation was more complete.

There are two very different parts to versium, which I almost
separated into two projects and two rocks, but then didn't.  In my
(yet to be released) documentation I refer to them as "versium core"
and "smart repository" (= "versium.smart").

Versium Core is a simple API to versioned storage.  It defines 8
functions and there is a 264 lines of a wrapper on top of them
(versium.lua).  I am sure it will grow a little over time despite my
best intents, but the idea is to keep Versium Core simple and small.
Versium core explicitly just stores "data".  Each Versium node is a
string.  Versium does not look into the data itself, though it
provides a hook for "inflators".  Basically, you can give it a module
to use for turning node's payload into a Lua table and back.  I could
be convinced to remove the hook and put it outside versium.lua
(though, please look at the actual code first).  Apart from this (and
a few small issues discussed below), I don't see Versium Core changing
much.

Then there is Versium's "Smart Repository".  SR implements a whole
bunch of weird stuff on top of Versium, starting with prototype
inheritance.  SR makes assumptions about the payload that versium
nodes carry: it assumes them to be key-value pairs represented as Lua
code.  SR will probably do more weird stuff later.  Anything fancy
like the stuff you are proposing with "Metasium" would either go into
SR or form a layer above it.

About indexing: I don't think Versium Core needs to know about
indexing at all.  It's job is to provide simple read-write interface
to "dumb" S3-like storage.  My current thinking is that indexing
should be implemented as a separate module and then used by SR, which
will be responsible for updating the index when the content is
changed.

Now, one thing that I've been thinking about (and perhaps this is one
of the things you are suggesting) is making Versium Core a bit more
generic, and allowing it to represent storage backends that may or may
not allow saving and may or may not maintain history.  Obviously, we
would need to be careful here, but the idea is that each storage
implementation would report a table of "capabilities" which would
(optionally) have fields like capabilities.saving, capability.history,
capability.author.  I am also open to getting rid of "extra" fields.

André proposed a while back to that I treat versium nodes as
"composites."  In other words, to allow them to have children, and
assume that any node may represent a collection of nodes.  With your
suggestions of stacked storage (if I understood it correctly), we get
an new twist to this.  Suppose we say that each node (and we are
talking about "Smart" nodes, here, not Versium Core) would not only be
allowed to have children, but can also decide where and how to store
them.  Which is to say that each SR node can configure it's own
versium core.  An extra benefit is that in this scheme Versium Core
could stay oblivious of node nesting.

We could have something like this:

1. Root Node ("/") - itself "stored" as a table defined in
"sputnik.cgi", storing it's children in wiki-data directory with
versium.storage.simple  (BTW, don't confuse "/" with @Root.)
2. A bunch of children of "/" stored in wiki-data, most of them child-less.
3. Some child of "/", say, "Section_1" gets configured to store
children in some other directory, say "wiki-data-section-1", also
using versium.storage.simple.
4. Another child, say "Temp" gets configured to store it's children in
memory, using versium.storage.in_memory.  Those will just disappear
when the process dies, but let's assume we don't care for some reason.
5. Yet another node, perhaps "Section_1/Gigs_of_Data" gets configured
to store it's children in S3.

Is that sort of what you were suggesting?

In this case indexing could be done hierarchically, in the sense that
each node will decide whether to pass a query to children and how to
filter the results.

Now, this is not a promise to implement any of this.  But I suppose it
would be good to come to some consensus as to where this could or
should go when someone has time to write it.

 - yuri

On Tue, Feb 12, 2008 at 11:43 AM, pierre pracht <pierre.pracht at gmail.com> wrote:
>
>