DRI_OVERLAP_PAD_TRUNCATE
DRI_OVERLAP_PAD_TOROIDAL
DRI_OVERLAP_PAD_ZERO
DRI_OVERLAP_PAD_REPLICATE

note - pad could be used in cases when you're not doing overlap

consider overlapping based on other value than 0
 

*** Topic: using rank parameter, and group_size instead of an actual group parameter
we decided to add these params to DRI_Distribution_create
this should resolve the "More trouble" email subject after 9/19 meeting

*** Topic: Even More Trouble email

DRI_Distribution_get_numblocks returning # of blocks
But,  block cyclic distributions can be thought of multidimensional. Need to have a way to index the blocks (i,j,k)

need a convention for how the multi-d blocks are actually indexed

seems to need to be tied either to gdo or layout dimension specs

Taxonomy:

state-based iterator thru slices
random access with linear index number
random access with multi-d index number

the last 2 approaches need to be specified better
for now, we keep the 1st approach (already in api), and say that actual ordering of multi-d blocks accessed with linear index is determined internally by the impl.
 

*** Topic: DRI_Distribution_create function - layouts parameter should be an array
 

*** Topic: getting meta out of DRI Core email chain

• remove data spec from global_data_create (DONE)
• added name parameter to global_data_create (to support config file based environments, possible profiling libraries)
• Myra: raises issue of whether we need to name objects everywhere, not just global_data
• Jon: suggests allowing a NULL name parameter
• group consensus - naming parameter should be applied to all objects, or none (with exception of the channel, which needs it for rendezvous at connect time)
• decision: this is something that needs to be voted on later
• BUT: before voting, some offline email/thought must be done to consider object naming (all types) vs. not
• change group to group_size in DRI_Distribution_create (DONE)
• HOWEVER, change local_size output parameter to local_count
• Remove elem_size from DRI_Blockinfo structure (STILL NEEDS TO BE DONE)
• Need to specify #bytes to bufferset creation (DONE)
• Arkady: we're also missing alignment input parameter (for the system buffer create routine)
• Option to specify zero-initialized buffer on return (for sys buffer create routine)
• Need function to convert dataspec to number of bytes (DONE)
• Channel create needs dataspec parameter (DONE PARTIALLY)
• Listed in PARAMETERS, but not in SYNOPSIS
• DRI_Distribution_create - specifying layouts on input
• 2 approaches
• Approach 1: 1 DRI_Distribution_create call with array of DRI_Layout objects (1 per dim)
• Approach 2: 2 different DRI_Distribution_create calls (one simple with no layout spec, another actually giving a full blown array of DRI_Layout objects)
• Approach 2 will be followed
• Need Layout_packed_create (int order)
• Need Layout_uniform_create (int order)
• Need pre-defined layout objects (LAYOUT_PACKED_0, LAYOUT_PACKED_1, ..., up to some reasonable maximum)
• Need pre-defined layout objects (LAYOUT_UNIFORM_0, LAYOUT_UNIFORM_1, ...)
• RELATED ISSUE: need to have very consistent english descriptions about array argument positions and what they refer to (i.e., they all tie back to the GDO dimsizes array), and also need to clarify the actual layout "order" parameter meaning (value indicates the # of dimensions that run "faster")
• Separate issue: what about having equal _memory_ use in buffers associated with a partitioning of data (even if #elements truly is different resulting from the partitioning of the data)
• this is an implementation concern in some cases - it can be better to have same size buffers, even if those buffers aren't fully populated with real data
• this specification is really part of the Layout spec.
• Suggestion is to call this a "uniform" layout option
• SUMMARY BELOW
• So, we have "packed" and "uniform" descriptions for a single dim layout
• Also, we have higher dimension concept of packing order of multi-dimensions
• Then, there is the even higher level of scalable layout specification described in Mercury documentation presented at 9/19 meeting

TOPIC: Dennis suggests starting a documentation trail for decisions on particular features

• decision: feature X has been deferred to later version of DRI spec
• decison: feature Y, to this level, has been secified in DRI spec - further enhancements such as Y.1 and Y.2 will be included in later versions

TOPIC: Change global_data to virtual_data object (global indicates that it physically exists somewhere)

• group in attendance seems to like the idea, but...
• James: global/local index is important, so let's keep global data object naming
• so, it seems that global is the better term

EMAIL REVIEW DONE (above)

TOPICS: Arkady list of issues (before he leaves for the day)
 

1. What happens with unused processes
• when specifying group_dims and group_size to DRI_Distribution_create (product doesn't add up to the sum total of processes in he group)
• another condition: more procssors than data elements to divide
• another case: N/M (M>N) group_dims positions specified, (M-N) positions unspecified (b/c we want impl to do the reasonable thing in those dimensions), what is the error detection in this case? (need to make sure that there are numbers that _could_ be used with the specified numbers, such that the product equals group_size)
• we need further clarity about what constitutes an error condition
• group agrees that if group_dims is completely specified by user, it is an error if the product doesn't equal the group_size value
• another scenario: min and modulo parameters
• modulo parameter = 17 rows
• but, there are only 16 total rows in the global data object
• so, we can't possibly satisfy modulo (so this should be an error)
2. DRI_PARTITION_WHOLE and whether or not to have overlap also specified
• concensus seems to be to not allow overlap specs to go with WHOLE
• question was whether BLOCK partitioning with overlap (in the 1 processor case) is equivalent to WHOLE with the same overlap spec
• final answer: just don't allow overlap with WHOLE partitioning
3. Is overlap=0 legal?
• no
4. DRI_Blockdim structure:  would be useful to have included in the structure contents the packing order associated with that dimension
5. Left and right overlap DRI_Blockdim structure
• if I specified "pad" at the edges of the global data, which value comes back in the lov and rov DRI_Overlap objects
• for processes not on the "edge" of the global data, they'll get the overlap value specified at partitioning time
• for processes on the edge of global data, the value 0 is returned (because you have 0 overlapped rows of real data)
• Jon: suggests maintaining info on #pad positions AND #real overlap positions (based on real data)
• Jon: some troublesome cases cause the need for this information.
• ****** KEN: Work with Arkady, Myra, and Jon to get this straight

DRI_Partition_block_create ====> DRI_Partition_create_block (create is the verb referring to DRI_Partition object)
DRI_Distribution_create_with_layout

MPI-like rules should be adopted

DRI_Object_verb_adverb

DRI_Layout_create_packed
DRI_Layout_create_uiform