Ganymede Release 1.0.11
December 9, 2002
THINGSTODO
---------------------------------------------

0) Database rework to support full history tracking

Right now, the Ganymede server maintains in its database
only the current state of the server's data.  No historical
information is retained in a form which would allow the
builder tasks to easily handle the reporting of
differentials to external processes.

That is, we would like to have Ganymede be able to write out a file
that says something like 'since the last time this builder process was
run, 3 new users were created, one user was added to group labusers, 2
groups were deleted, and one user was renamed', rather than simply
dumping out the current state of the database and just expecting that
to be bulk-loaded into the environment.  Things like LDAP and NT
maintenance could be supported a lot better if Ganymede was able to
provide limited delta records rather than forcing a complete reload
from the Ganymede server's state.

11-Oct-2001:

Note that Thomas Reith has published a fantastic tool called ldapdiff
at http://webtomware.rhoen.de/.  With ldapdiff and some appropriate
LDIF-generating Java code for a Ganymede builder task, you could
easily build an LDAP management tool out of Ganymede.

1) DNS Support

The biggest point to make about Ganymede right now is that it is as
much a framework for you to build management software with as it is a
finished product.  In particular, while the gasharl, linux, and bsd
schemas are in decent shape, you have to either already be using GASH
or be willing to settle for simple user/group management if you aren't
willing to do some work to develop a schema that is useful for you.

What is really needed is for someone to develop a generic DNS schema
kit that includes a good general parser for DNS files.  The gasharl
schema is designed to support a single comprehensive NIS domain and a
single DNS domain broken into Class-C sized subnets, but it is
designed to work with files generated/maintained by GASH.

Bootstrapping a DNS management system from zone files needs to be
supported, but Ganymede's objects and links model makes this process
of doing this a bit unobvious.  The gasharl schema uses system
information where all the interfaces and system information for
individual systems is grouped together in objects.  For our network,
this information was collated and assembled by hand.  Hopefully
there's a good way of avoiding that work.

Right now the gasharl schema kit is the most fully developed and
tested schema, but the gasharl schema kit does not include a full
and working NIS/DNS build system.

We are currently developing enhanced DNS support through the use of an
XML layer to replace the old gash hosts_info file.  We intend to
support DNS fairly comprehensively with this software, but it is not
yet ready for release.

2) Scalability issues

Right now, the Ganymede server loads all data managed into memory.
This is reasonably efficient with a good generational garbage
collector, but it's possible that using a random access binary file
for the database, when combined with an in-memory cache system using
the JDK 1.2 weak references API could give better scalability.  Memory
is cheap these days, so the Ganymede design isn't too bad for our
purposes and size, but it might be good to experiment with the basic
engine to see whether things can be made more scalable with 1.2.

In addition, the client has some scalability issues.  In particular,
there are places where the client will generate pop-up choice lists
with hundreds if not thousands of choices in them.  This is not
workable from a user interface perspective, and may be a real burden
on the client's memory.  For real scalability it might be better to
require the user to manually type in items when the lists get too big.

3) Documentation/Help

The Ganymede client includes support for a basic HTML help system, but
we don't have the HTML help web written.  Given our resources and
priorities, it isn't clear that we'll be able to do this ourselves
anytime soon, but it really should be done.

4) Internationalization

It would be a really good thing for the client and server to be
internationalized.  After getting the Ganymede announce posted in
Linux Weekly News and seeing all of the hits from Japan, Italy,
Austria, Russia, China, the Netherlands, France, Spain, Portugal,
etc., my fantasy that internationalization was something I could just
sort of pretend I'd never heard of seems to be evaporating.  I don't
think I'll be able to dedicate time to it anytime in the foreseeable
future, but someone is going to have to at some point.

5) Two phase commit for client access

The Ganymede server internally uses a two-phase commit protocol for
handling its transactions, but it does not provide a two-phase
interface for clients that may be coordinating transaction commits for
an external system.  As a result, while Ganymede can be used to master
cross-database transaction commits using the DBEditObject
commitPhase1() and commitPhase2() methods, it doesn't support a client
attempting to verify transactional consistency without performing an
actual commit.

No one has asked about this yet, but at some point I may need to
expand the client API to allow some sort of 'verify ok to commit'
method call which, if it returns a successful result, will guarantee
that a later commit would proceed to completion successfully.  This
will involve a good bit of reworking of the server's internal commit
operations, and there may be issues of deadlock and timeouts to be
considered as well.  At present, transactions lock all portions of the
database involved in the transaction while the commit logic is being
processed, but if there might be an indefinite delay between phases 1
and 2 of an externally-accessible two phase protocol, we have to
either risk unlocking the server and then hoping we can re-lock the
server in a timely fashion later, or risk leaving those portions of
the server locked until such time as a phase 2 order arrives.

I'm not sure how folks handle this problem, generally.

6) Integration of CVS into the backend

Right now, Ganymede is doing a lot of stuff in the server to handle
archiving of emitted files.  Things like the automatic saving of
backup copies of a file when a file is emitted by a builder task, and
the nightly zipping of the day's backup files.

Really, this would be better done by having the backend scripts assume
the presence of CVS, and to do CVS commits every time new files are
written out.  This would allow for much more efficient storage of the
emitted files, and would allow for easy retrieval of the emitted files
for any given moment in time.

Schema kits should be developed in this direction.

7) Eliminate the 'objects owned' field in the owner group object [bugzilla #23]

It might be a good idea to make the object ownership linkages
non-symmetric, with the invid's in each object pointing to its owners
through a non-symmetric relationship.  This would allow simultaneous
adding or deleting of objects from the owner group by distinct
transactions, would reduce the size of the database on disk, and would
simplify further the XML representation of the Ganymede database.  The
cost would be that the once-per-login getOwnerGroups() method in
GanymedeSession would have to be either reworked to perform discrete
queries, or an alternative recursive algorithm that started from the
leafs and traversed up the ownership tree piece-wise to try and
construct a reverse-transitive closure would have to be implemented.

In addition, the client's objects owned panel would have to either be
abandoned, making mass object adds or deletes for an owner group more
difficult, or implemented using completely custom logic to handle the
process of editing all objects thus added/deleted and modifying the
owner field.

Finally, there would need to be custom logic that would enforce the
current permission logic, allowing only users who can edit the
'Objects Owned' field for an owner group to add objects to that list.

8) Move build system to Ant

We may do this for the next release.  Moving to Ant would make it
possible to build Ganymede on Windows, although the whole system is
really designed to manage directory services on a Unix-like system.

9) Support server targeting with common client

Right now at ARL, we are running two instances of the Ganymede server,
one for our internal information, and one for accounts in the DMZ.  At
present, the Ganymede client and admin console are 'pre-loaded' with
properties specifying where the Ganymede server is.  It would be nice
to have a way to start the client or admin console, have the client
code get a list of local Ganymede servers, and present the user with a
choice of server to log into.

This may well not be worth the effort to code it, but it might be nice
to do.  Letting a single admin console monitor multiple Ganymede
servers might be particularly nice.

10) Rename Ganymede

We long since past need to ditch the Ganymede name.  We have a name
that we're thinking of going to, we just need to create the new logo,
graphics, and rework all of the documentation, ugh.

---

Jonathan Abbey, jonabbey@arlut.utexas.edu