Re: Passing LUA script via python rados execute

[Nick Fisk <nick-ksME7r3P/wO1Qrn1Bg8BZw@public.gmane.org>]

> 
> On 02/19/2017 12:15 PM, Patrick Donnelly wrote:
> > On Sat, Feb 18, 2017 at 2:55 PM, Noah Watkins <noahwatkins-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
> wrote:
> >> The least intrusive solution is to simply change the sandbox to allow
> >> the standard file system module loading function as expected. Then
> >> any user would need to make sure that every OSD had consistent
> >> versions of dependencies installed using something like LuaRocks.
> >> This is simple, but could make debugging and deployment a major
> headache.
> >
> > A locked down require which doesn't load C bindings (i.e. only load
> > .lua files) would probably be alright.
> >
> >> A more ambitious version would be to create an interface for users to
> >> upload scripts and dependencies into objects, and support referencing
> >> those objects as standard dependencies in Lua scripts as if they were
> >> standard modules on the file system. Each OSD could then cache
> >> scripts and dependencies, allowing applications to use references to
> >> scripts instead of sending a script with every request.
> >
> > This is very doable. I imagine we'd just put all of the Lua modules in
> > a flattened hierarchy under a RADOS namespace? The potentially
> > annoying nit in this is writing some kind of mechanism for installing
> > a Lua module tree into RADOS. Users would install locally and then
> > upload the tree through some tool.
> 
> Using rados objects for this is not really feasible. It would be incredibly
> complex within the osd - it involves multiple objects, cache invalidation, and
> has all kinds of potential issues with consistent versioning and atomic
> updates across objects.
> 
> The simple solution of loading modules from the local fs sounds way better
> to me. Putting modules on all osds and reloading the modules or restarting
> the osds seems like a pretty simple deployment model with any configuration
> management system.
> 
> That said, for research purposes you could resurrect something like the
> ability to load modules into the cluster from a client - just store them on the
> local fs of each osd, not in rados objects. This was removed back in 2011:
> 
> https://github.com/ceph/ceph/commit/7c04f81ca16d11fc5a592992a4462b3
> 4ccb199dc
> https://github.com/ceph/ceph/commit/964a0a6e1326d4f773c547655ebb2a5
> c97794268
> 

Thanks Josh. They look like they refer to loading the actual rados classes themselves, whereas I just want the LUA rados class which runs a LUA script passed via JSON, to be able to load extra lua scripts from the local FS. Or have I misunderstood the contents of those commits?

I'm trying to put some examples together for a book and so wanted to try and come up with a more out of the box experience someone could follow. I'm guessing some basic examples in LUA and then come custom rados classes in C++ might be the best approach for this for now?

> Josh