The Ghosts in My Machine, Chapter 3

Visit the High Level Logic (HLL) Website
— Home of XPL (eXtensible Process Language)
It's like XML, but you can store source code in it.
Chapter 1
Chapter 2

Prepare yourself for a surprise ending. Do that now to avoid confusion later.

Around 1990, I met with an industrial engineering professor who had been working for years with artificial intelligence technology. We had a long chat about the possibility of completely automated factories. This was still a decade before frequent online purchasing and customer management systems. But it seemed reasonable to contemplate a future in which everything from initial customer contact, sales, accounting, instructions to the factory floor, robotic manufacturing on demand, packaging, right out to the shipping dock would be fully automated.

Even if you've never considered designing such a system and are unfamiliar with any such work in that direction, it isn't long before the thought occurs that such a system should be broken up into departments and operational segments. Each operation has its own specialized “knowledge” and processing to contend with, and it seems likely that interfaces between operational segments might need only handle common data exchanges. Jumping ahead, one might consider that an agent system could provide loose coupling between operational components as well as the possibility of moving pieces around from time to time for special types of interdepartmental operations.

It isn't a big jump to consider that breaking the application into “departments” is at least in part, an exercise in modeling the framework of the human organization that complete factory automation would replace. The fact that this model already exists, that it's completely common, may be the most important reason that the idea of breaking the software design up this way is so obvious. Compartmentalization is not a concept invented by computer scientists. Humans do it all the time, and have been doing it for ages. If we're going to think more about obvious common ideas, then this idea easily qualifies on that ground.

The concept of an “object” was also not invented by computer scientists. (I merely need to open this idea so that I can be flexible about its use without confusing anyone – i.e. a fair warning against thinking “simply” about common current day object oriented programming structures and techniques.) Objects exist in the world around us and we can ourselves be described as objects and operate as such. They have characteristics and some have capabilities. They interact in various ways and react in various ways to interaction. Complex (compound) objects can be composed of simpler objects; the whole sometimes being equal to more than the simple sum of its parts. The concept of an object or “entity” (in certain modeling models) is fundamental and useful in describing pretty much everything – solid ground for high level logic.

Dwelling on what at this point may seem quite obvious; we're – very generally speaking – looking to design a set of organized objects. One way or another the individuals and / or the organization will be structured to carry out tasks and solve problems, answer questions and whatever. The important distinction here is that the intent is to build a general processing engine; not a specific application that could be described exactly the same way – as a set of organized objects. The expected practical effect, if successful, is that some portion of “common” logic in a wide range of applications can be moved into the general processing engine, where it does not need to be repeatedly rebuilt in the application phase.

Another important distinction is that the High Level Logic Project attempts to do this by searching for “high level logic” rather than by “bottom-up” development. This is why I've chosen such a long-winded approach to introducing HLL. Ideas that are common and fundamental are useful and enduring. They don't need to be derived - "bottom-up" - from the most recent set of application programming frustrations.

So let's take the next step in thinking about departments in human organizations. There are various kinds of organizations, but I didn't mind leaning on my familiarity with the typical American corporate structure, and particularly with project oriented groups. Projects begin by defining a task or problem and end with a specified result. There is at least a rough analogy between this process and the general problem solver outlined in chapter 2. Project organizations are made up of individual people with specialized roles and responsibilities that form sometimes simple and sometimes complex task hierarchies.

Certainly our task cannot be to describe the individual characteristics and operations of all existing human organizations, or even of one large organization like IBM. Organizations change, and a detailed description of any specific organization is not a general model. A general high level logic engine needs entities that can be characterized by application developers or another process. The system for describing organizations needs to be easily extended by adding, defining, and organizing entities. The relationship between entities needs to be “loosely coupled” so that they may move around, help others by use of their special knowledge and capabilities, and form sub-groups to perform tasks as needed.

There is a difference between general and generic. The description above remains pretty generic. Certainly, organizations as “organized objects” seems so. It is possible to add more specifics to a general model (applying to all or most members of a category or group), or one postulated as being general (not specialized or limited to one class of things) without fear of limiting an application developer's creativity and adaptations to meet specific requirements. Allowing developers (and other processes) to customize, add, delete, define, redefine, and organize – i.e. to completely control the specific definition of the generic engine's application does that. (It's something like bringing entity-relationship diagrams to life.)

But still, it plagued me that there is apparently at least a rough analogy between the workings of a project organization and the general problem solver outlined in chapter 2. What minimal organization could assure complex task completion and be highly relevant to a wide range of software applications?

I had spent years working in project groups, and it was that experience that I most thought about when making the analogy. Generalizing a bit, the organization started at the top with an executive (like a VP of Engineering), who reigned over one or more (project) managers, who organized and managed the activities of groups of experts (in most of my personal experience – engineers). In the human world, such a simple three-layered organization has been responsible for carrying out a very large range of very simple to extremely complex tasks.

It was time to describe the characteristics and capabilities needed for a basic three-layer organization model, one that made sense generally, and would be highly relevant to a wide range of software applications. That part turned out to be not particularly difficult, and led to detailed design requirements and specifications (chapter 4, some of which are already hinted at in this chapter).

Minimal description of the three-layer HLL general organization model:

Executive(s): Interacts with external and internal “interests” to formulate, offer, accept, or reject proposals or demands. Decides whether or not to make commitments. Is concerned about the general allocation of resources within an area of responsibility. Assigns complex tasks to managers.

Manager(s): Plan an implement tasks that have been assigned to them, using available resources.

Expert(s): Carry out work details for which specialized knowledge and skills are needed, often in cooperation with other members of a group.

Very clean model so far, perfectly obvious in many respects. On the good side, it could easily be taken as a generalization of many processes including many, many software applications. Then I felt a great discomfort with the human organizational analogy. I and a great many others often feel discomfort with being a cog in the wheel of an organization. It's not quite natural. Something is wrong. (Here comes the surprise ending.)

Don't we all, individually, commonly carry out tasks requiring all three levels described above in our daily lives? What's the difference between a large distributed organization and an individual (in the context of this discussion) other than that large distributed organizations can get a lot more done in a specific amount of time? There isn't necessarily a difference between the structure and function of a multi-person organization (model) and an individual.

A friend of mine who happens to be a cook recently asked me to explain that to him. A cook, interacts with the outside world (outside the kitchen); a good example being the communication of an order from a customer via a waiter (an agent). The cook may accept or reject the order. (If for example, the order is not complete or the kitchen has run out of an ingredient required to make a dish.) If the order is accepted, he commits to the task, then assembles resources (ingredients, cooking implements) according to a plan (complete dish specification and recipes). He then prepares the food and the dish, using specialized knowledge and skills.

The fundamental high level logic had not collapsed, but it became apparent that application developers should be provided with the greatest flexibility in interpreting, mixing and applying its pieces. Large organizations like IBM are human creations, sometimes necessarily physically distributed, that can accomplish more in a particular amount of time. Otherwise (in the context of this discussion) nothing more than a reflection of ourselves and the way we – necessarily – accomplish tasks that require a higher level of intelligence.

From the High Level Logic (HLL) Open Source Project blog.