[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [dist-obj] CORBA implementation ideas

Would be glad to go into as much detail as I know.  I will start here, and
go where further questions lead ...

Steve Vinoski writes:
>What expertise? Distributed systems expertise? C++ expertise? Java
>expertise? UNIX expertise? NT expertise? You get my drift. BTW, no matter
>how you approach it, distributed computing is hard -- there's no two ways
>about it.

CORBA expertise, starting with C++ (no Java on all but one project), but
going more into the details of ORBs, ORB agents, and such.  The projects use
either of the two main vendors: Iona or Visigenix.  Items include
registration, multithreading, instance lifecycle management, and OO design.
Nothing beyond what you/we already know, but well beyond what most standard
Unix programmers know, even if they have done some C++.

And I should emphasize expertise.  Not just familiarity.  CORBA can blow off
toes easily.  Deadlocking.  (Object A calls procedure in Object B, which
calls procedure in Object C, which decided to satisfy the request must call
procedure in Object A ... if each Object is single threaded ...).
Distributed memory management.  (Object A has the data.  Object B calls
procedure to get data.  When can Object A delete the data, and it is safely
in B's hands?  Not during the call.  Another call by B to tell A to forget,
after B stores it for safety?  What about a chain of these?)  Have to be
careful in design and implementation.

Motherhood, but I think we all find in distributed systems - with developers
on the different parts - that circumstantial problems arise from lack of
coordinating issues that the developers may not even know are present.  Each
developer writes an object, and each looks at the family of objects for what
they can call to satisfy calls made to them.  That simple, and in a large
enough program it is nearly a statistical certainty that deadlocks will
occur.  (I joke with friends that when IIOP is used in large amounts on the
Internet it will only be a matter of months before we have a global
deadlock, traced down to a simple circular deadlock of CORBA procedure
calls, and that a month later Time magazine will have a side-article
explaining to Joe Public what a deadlock is).

It would not be a challenge for people on this board perhaps, but the
average programmer is probably several notches below this board.  And it is
not beyond their abilities, eventually.  But it is a whole lot of learning
for a large project which is on a budget and schedule.  One of the project's
customers even bit into the entire training program.  But we all know that
coming out of your first class on a technology is not the same as being an
expert.

The promise - one you may well laugh at - was this technology was supposed
to make it easier to write flexible distributed systems.

Something the projects seem to learn and adapt is to shuffle the software
design so that the CORBA work is hidden from most programmers, and a few of
the better guys do all of it.  On a project of 150 developers (one of the
cases), the CORBA team averaged around 5.


>Is this a people issue? A skills issue? A management issue? How do you know
>that it's a technology issue, or more specifically, a CORBA issue?

The C++ was new to most of the developers.  As long as they stick to careful
programming standards that has not been the problem.  (It has not been the
answer either ... not the finest moment in programming language design,
IMHO).  It is the areas where the CORBA work happens that has defeated
estimates - later and more expensive.  The metrics are moving to suggest the
most difficult programing, line for line, on these projects is the CORBA,
and by a significant factor.

The areas of the system closest to CORBA have also been a source of nasty
recurring bugs (even as the rest of the system settles into fewer know
bugs).  Now some are vendor bugs, to be fair.  But many have been
development bugs.

To complete the answer, I imagine one could find the very best CORBA
technicians on the planet and do better.  Two of the projects did bring in
outside consultants, but I imagine there are even better.  And perhaps
expectations were unrealistic.

But to become a mainstream way of doing distributed systems, it has to match
up with mainstream developers and practices at some point.  That has not
been reached on those projects I know of.

I do know of smaller projects, of 10 or fewer developers, where the problems
do not exist to the same degree.  Perhaps some management techniques could
improve the large project performance to closer match the small project
performance.  This I just don't know, nor have I heard of many suggestions
beyond the one to isolate the CORBA to a few developers (mentioned above).

>>d) IIOP performance, to put it delicately, sucks.
>
>Compared to what? A hardware backplane? Two tin cans joined by a string?


Compared to the underlying sockets.  The seminal benchmarks were between two
top of the line HP servers on an ATM.  Sockets moved the data from process
to process, on each server, across the ATM, at 20MB/s.  The best performance
they saw on IIOP was 3.x MB/s.  This was changing granularity and twisting
almost every knob on the technology they could reach.

Don't know if the work was ever published.  Probably not.

Working with the vendor - Iona - it was narrowed down to two things causing
the difference.  The parameter marshalling and the protocol overhead.

This was two years ago and with a poor ATM interface.  But even in less
formal investigations now we find it better to pass large data in sockets
and use IIOP as a control medium.  (BTW, we find this with HTTP and RMI as
well).


>Why does Doug Schmidt's considerable investigation into this area seem to
>contradict your claims?

Pointer, please.  I am unfamiliar with this reference.  I will look and see
what I can deduce.

>Can you name a system where all these problems are handled automatically
>without requiring any thought on the part of the developers?

Actually many of the problems are dealt with in Active's technology.  For
example, distributed memory management.  One program has data.  Sends data
as message.  Assuming a persistent queue, you can delete the data as soon as
the call to pass is accepted.  No more coordination than that.  And no
coordination with the receiver.

Further, there could be multiple receivers ... say a full publish and
subscribe cycle.  Program A publishes data.  When can it delete?  Right
away.  Other programs that subscribe can get data at their leisure.

All the problems?  No, of course not.  But mere mortal programmers can makes
working distributed systems without much of the lower level pains I see on
CORBA-based projects.  All the flexibility?  Not close.  And experts such as
yourselves may argue long and hard on the choices.  But it is one way, it
works, and people are developing real world distributed systems without
nearly as much expertise as CORBA appears to demand.


>"Project DOE" -- need I say more? I'm not trying to be a smart aleck -- I'm
>simply pointing out that one does not normally think "success" when one
>thinks of Project DOE. Indicting CORBA based on Project DOE is like
>indicting operating systems based on DOS.
>
>What were the Active folks trying to do with CORBA? Was CORBA best suited
>for the problem they were trying to solve? Lots of people claim to
>understand CORBA, but they really don't -- are you sure these guys really
do?

I don't have a world of interest in DOE.  Never saw the value in it.  My
reference is for where Jim Green and that team came from.

Actually the use of CORBA was pretty sexy.  Used for keeping the internal
engines up, talking to one another, and communicating configuration data.
The core team at Active were part of the gang who wrote the specifications,
and they worked with CORBA continuously for years.  Jim is the one who send
Sun funds to back Iona, and was on the BoD for a while.  WebOrbix was
written mostly at Active so they could use a Java ORB in their product, and
sent back to Iona.

The reason it was phased out is too many problems traced back to Iona, and
they took too long to revise their product.  Further, Active got better
performance doing the ORB functions directly.

Active came from writing software to make the CORBA less painful, to deal
with issues such as distributed memory management, deadlock, and such.  Take
the hard problems - well, some at least - and make it easier to write
distributed systems ... and in particular, to integrate systems, to build
systems of systems, if you will.  Now the hard problem is the nuts and bolts
of what that integration means, and designing the higher level system.  But
at least I am no longer counting sockets, worrying when to delete something,
and a number of other things that I would have to deal with in CORBA.

>>My Conclusion.  I would rarely tell someone to use CORBA.  Certainly I
>>hesitate in the real world.  Perhaps it might be a good path to learning
in
>>an academic environment.  But I would first ask the participants to read
>>Hoare's CSP book, to get a grasp on issues central to distributed systems,
>>and to give some perspective on CORBA vs. alternatives.
>
>Oddly enough, I would give roughly the same advice. Charging off into CORBA
>or any other technology without first understanding it to some degree and
>investigating alternatives is foolish and irresponsible.
>
>Again Mark, this is not intended as a personal attack against you, Jim
>Green, or anyone else, so I sincerely hope I didn't come across that way.
>It's just that I read your posting and found that it contained nothing more
>than Fear, Uncertainty, and Doubt, with no technical content. I think
>readers of this list would be better served if you posted concrete details
>and and some technical analysis. I'd certainly like to know more details.
>Generally, however, in distributed object computing or any other computing,
>it is my opinion and experience that only poor practitioners blame their
>tools.


No, it was opinion piece with no meat.  I should not have posted it in that
form.  Not here.  Hopefully with this and perhaps follow ups my points
become substantiated.

I don't want to say that CORBA should be erradicated.  But it is not the
tool that makes distributed object systems easy and natural to evolve by the
standard software developer.  If it has a long term place, I would think it
is somewhere under the hood of better tools.

Mark

============================================================================
To unsubscribe and other requests, mailto:dist-obj-help@unity.cs.caltech.edu 
Archives, FAQ, etc.       http://www.distributedcoalition.org/mailing_lists/