Ken on the fly agenda:
 

• carrying gdo dimensions through to rest of dimension-based specifications in API

Issue 1: carrying gdo dimensions through everywhere
Jon: why can't we have an interface that violates this rule? Problem is that when you do a DRI_Distribution_get_blockinfo it is strictly ordered so that each entry corresponds to global data dimensions specified earlier. This could be very inconvenient for traversing through the local memory, because the Blockdim structure for the most contiguously organized dimension could end up being anywhere in the array of Blockdim structures.

Possible solution is to have another output array of integers[ndims] returned to provide a mapping back to the layout specification (dims[] order does not change from what it is today)

Second possible solution is to change the dims[] order (so that dims[0] corresponds to most contiguous dim, ...) and then have a mapping back to global data dimension.
Ken: in this second approach the gdo map essentially is your original layout specification, right? Yes, but we'll return it in an integer array form, instead of the object form
Decision: let's call the new array layout_order
 

local dimensions of the buffer (in an array), but whose elements do not correspond to the earlier global data dimensions and the layout array specification. Reason for doing this is so it is not awkward to access the "most contiguous" dimension of data
 

Issue 2: Arkady
We can't provide a beginning offset parameter (seems to be a memory layout type of input) that defines the first place in the linear memory buffer where real data starts. Problem is that this input quantity would be dimensionless, but DRI_Layouts can only be created on a per-dimension basis

Issue 3: DRI_Blockinfo structure and handling "pathological cases" (e.g., 5 columns, split over 4 processes, specify left and right overlap of 2, and pad zeros)
Process 2 (where process ranks are 0,1,2,3) will have 1 real column of overlap on the right, and 1 pad column of overlap on the right.

Need to have more information coming back than the lov and rov (which are simply what the user specified originally) - because the situation is a little more complex than that. You need to know (for left and right) the following information:

• actual overlap positions
• actual padding positions
• padding type from the original overlap specification

• global_begin_ix
• length
• stride
• two instances of a new structure type (DRI_Blockdim_Overlap) - names are lov and rov
• DRI_Overlap_type otype
• int data
• int pad

Candidates for tomorrow's continuation:

• Distribution_get_local_count + data type info needed to create bufferset - but data types don't seem to be used anywhere in bufferset creation process

1. In change summary, number 2 DRI_Distribution_get_nublocks

2.  need to change DRI_OVERLAP_PAD_ZEROS to ZERO, REPLICATED to REPLICATE (globally)

page 14/30 orderngs

15/30 caculates

change undivided to indivisible (globally)

15/30 casesthat (no sp)
15/30 dimensionalitythat(nosp)

17/30 add "simple" onto signature of DRI_Distribution_create_simpl
 

Significant discussion about changing communication granularity to be on blocks, not on buffers. Issues include:

• in place vs. out-of-place processing - changes concept of operations