This is a summary of the e-mail, which has transpired over the data-re.org reflector since 
the last meeting of September 25, 1998.  These items will be addressed and incorporated 
into the document once the group arrives at a position of agreement.  This summary
Was written by James Lebak, MIT/Lincoln Labs and is unedited beyond this point.

Darwin asked me to edit the document based on some of the email discussion that 
occurred. I did make some fairly minor structural revisions to the document and added a 
list of assumptions based on the email. 
As I looked over the email, I saw a number of open issues that I thought should be 
resolved in open debate before changing the document. What I’ve done is to look at some 
of the basic questions that I think we have to answer and summarize the major points in 
the email discussion over these questions. Hopefully this can serve as an introduction to 
these topics for the next meeting.

[Author’s opinion] I have tried to be fair in these notes, but I think it is important for the 
reader to know my bias up front. My assumption has always been that the user explicitly 
describes the partition of the data set with some set of well-understood parameters (e.g. 
block-cyclic with size n, possibly including overlap of size v) and then explicitly 
describes to the API how those partitions are distributed (“mapped”) to processors. I 
believe this explicit description by the user is necessary given that the data re-org API has 
no idea about the computations that I intend to perform. Therefore, I think both of these 
problems, the partitioning and the mapping, have to be handled by the API that we 
define. In reviewing the email discussion, it seemed to me that some of the discussion 
assumed there was still some sort of automatic mapping mechanism which I do not 
understand. This bias may have kept me from summarizing the discussion fairly, and if so 
I apologize.[End author’s opinion.]

Email summary:
Does the programmer describe the data partition explicitly, or is the interface 
somehow intelligent enough to decide the partition based on some guidelines?
Consensus seems to be that the user describes the data explicitly in our first version of 
the API. This is referred to as the “rigid API”, as opposed to the “flexible API” that is 
intelligent enough to decide the partition for the user. No one seems to disagree with 
the idea of the flexible API, but there seems to be consensus that the complications 
involved make it too much for a first implementation.
 Dennis Cottel proposed the idea, which seems to have been accepted, of including 
data partitioning information in the data object (and eliminating process topologies — 
see below).

Once a partition of the data has been described, how is that mapped to 
processors?
Dennis states, “There are two "geometry-related" issues.  The first is a 
mental picture of what the application data looks like.  We already handle 
this completely by allowing the user to define a virtual data cube.  The 
second is the mapping of data partitions to actual hardware processors, 
something that is beyond the scope of Data Reorg. [Emphasis added] When 
thinking about dividing up the data, don't think of which data goes to which 
process (that's something for the library and OS to figure out), but instead 
think of dividing the data up into bunches which will be kept together for the 
algorithm's benefit.” 
An alternative (proposed by the author) is for the user to explicitly assign the 
described data partitions to specific processors by means of a simple linear array. 
How, in mathematical terms, are distributions described?
There was some feeling that an initial implementation should be targeted at a block 
distribution of data. Others felt that this was acceptable if it sped up implementation as 
long as that implementation did not preclude eventual support for block-cyclic 
distributions. [The author notes that a block-cyclic distribution of data may be 
described with a single extra parameter for each dimension of the cube. Therefore, any 
API that we come up with which can describe a block distribution can probably also 
support a block-cyclic distribution without too many additional arguments.]
Dennis proposed, in conjunction with the process topology proposal he made, to 
create an API which describes the data without formulas. Some of the voices raised in 
support of this proposal still seem to be describing data distribution with formulas. 
Should we include process topologies?
Some people felt that process topologies were not appropriate, because they give the 
user false expectations about underlying network topology.  However, if a vector is 
distributed, it is helpful to have a parameter which describes the number of processes 
over which that dimension will be distributed. This parameter was originally in the 
process topology object. Dennis proposed to eliminate the references to a “process 
topology” and to move this parameter into an object associated with the data 
distribution. 
Who allocates memory?
Dennis wrote that memory should be allocated explicitly by users and provided to the 
API when needed. Arkady Kanevsky objected to this, advocating instead the MPI/RT 
or VSIP model where the implementation allocates memory to satisfy user requests. 
Ken Cain suggested that we could support both models by providing a mechanism by 
which the implementation can allocate data at user request but making the use of this 
mechanism optional. 
Are data re-org calls blocking or non-blocking?
Dennis noted this question as unresolved, but it does not seem to have generated 
much debate.