Block configuration proposal (for discussion)

[jsturdy]

Quick summary

• This issue is really cross-repo (depending on V2/V3 electronics, but the implementation will be the same in the end, modulo address differences and register differences)
• It is also backwards incompatible, significantly moreso than the recent address table only changes
• A healthy discussion and pro/con of various implementations needs to be done in order to fleshed out
  the best solution

State of the art

Currently, there are some registers that will make a total block type configuration impossible

• A simple implementation (tested for read) would be to do (in uhal syntax), e.g.,

<node id=GEB.VFATS  address=0xwhatever>
  <node id=BLOCK  address=0x0  mode="block" size=24*N VFAT rw regs permissions="rw"/>
  <node id=VFAT0 address=0x0>
    <node id=REG1 address=0x0 permissions="rw"/>
---
    <node id=VFATChannels address=0xchanoff permissions="rw"/>
      <node id=BLOCK  address=0x0  mode="block" size=24*N VFAT rw regs permissions="rw"/>
    </node> <!-- ends VFATChannels section -->
...
</node> <!-- closes GEB.VFATS node -->

• One can further move the BLOCK node (or add multiple appropriately named and placed nodes) inside the VFATX node, as the top node, do block read/write settings for just that VFAT
• However, due to the current organization as above, this will make the BLOCK node fail
• The GEB.VFATS.VFATX node contains both rw and r type registers
• The non-optimal solution is to add BLOCKS of the correct size in each of the rw permission registers, excluding the r only registers
• Better would be to move the r only registers to their own block and separate the rw into a different address space, but this requires a redesign of the firmware, which should definitely be done for both V2b and V3
• On VFAT2 there are also the "extended registers" and the "extended register pointer" which is how one accesses most of the VFAT registers, and they are currently part of the address table and probably should be removed from the user accessible address space

Proposal for improved implementation

In the "most optimal" way, I would do something like:

<node id=GEB.VFATS  address=0xwhatever>
  <node id=STATUS  address=0x0  description="Contains all readonly VFAT registers">
    <node id=VFAT0 address=0x0>
      <node id=ChipID0 address=0x0 mode="r"/>
---
    </node> <!-- ends VFAT0 readonly node -->
...
    <node id=VFAT23 address=0xvfat23off>
      <node id=ChipID0 address=0x0 mode="r"/>
---
    </node> <!-- ends VFAT23 readonly node -->
  </node>
  <node id=CONFIGURATION  address=0xconfoff>
    <node id=BLOCK  address=0x0  mode="block"  size=24*N VFAT rw regs  permissions="rw"/>
    <node id=VFAT0 address=0x0>
      <node id=BLOCK  address=0x0  mode="block"  size=N VFAT rw regs  permissions="rw"/>
      <node id=REG1 address=0x0 permissions="rw"/>
---
      <node id=VFATChannels address=0xchanoff permissions="rw"/>
        <node id=BLOCK  address=0x0  mode="block" size=24*N VFAT rw regs permissions="rw"/>
      </node> <!-- ends VFATChannels section -->
---
    </node> <!-- closes VFAT0 node -->
...
    <node id=VFAT23 address=0xvfat23off>
      <node id=BLOCK  address=0x0  mode="block" size=N VFAT rw regs permissions="rw"/>
      <node id=REG1 address=0x0 permissions="rw"/>
---
      <node id=VFATChannels address=0xchanoff permissions="rw"/>
        <node id=BLOCK  address=0x0  mode="block" size=24*N VFAT rw regs permissions="rw"/>
      </node> <!-- ends VFATChannels section -->
---
    </node> <!-- closes VFAT23 node -->
  </node> <!-- closes CONFIGURATION node -->
</node> <!-- closes GEB.VFATS node -->

Caveats

Done in this way, the firmware is still responsible for doing the remote register access correctly (e.g., using the extended register and pointer invisibly from the user), but it would be possible to do a block configuration of all VFATS with one transaction, and also of all configurable registers on a single VFAT, if desired (in the case that maybe it is not possible to write to a given VFAT, as I'm not sure what the behaviour is in this case), or even just the VFATChannels node (but only for a given VFAT in this case, and moving this to a new place would need to be yet another discussion)

Anecdotal information

I tested a block read of all the VFAT channel registers (block write was failing, possibly due to ipbus implementation on the CTP7 as the transaction reply did not match the IPBus expectation) and it worked splendidly.

[bdorney]

Sorry forgive my ignorance here. I'm not really sure how this block issue plays out. For some register that gets the same value for all chips it seems like a bcast read/write (e.g. MSPL/CFG_PULSE_STRETCH).

But what happens for those rw registers that need to be set per VFAT and cannot be the same value across the link?

Such registers would be:

• VThreshold1 (v2b),
• CFG_IREF (v3),
• Channel registers/masks/trims/etc... (both)

Probably just my lack of knowledge on what blocks are here.

Additionally what impact does this have on sw development? Is this something that we would be doing to stick with ipbus and uhal or does this have implications for rpc and xhal (are we calling xhal this new repo reg-utils now...?)? Basically what is the architecture we are trying to develop for?

[mexanick]

One register is a 32-bit long block of memory starting from the certain address. But you can also write a block of few registers in one go. Example: you have channel registers, 128 registers of 32 bit long each, located in memory one after another. Instead of doing 128 read or write transactions 32-bit each, you can do a single read or write transaction operating a 128x32bit data block at the address of the first channel register.
Since every VFAT is a hardware-defined, it is quite natural to represent it as a single block (eventually with smaller sub-blocks for e.g. channel registers - see @jsturdy example above) and write its configuration as a whole in one transaction. Then in order to set all the VFAT to certain configuration you will need only 24 write transactions instead of over 3k if you do it individually for each register.

[bdorney]

I see, so rather than:

for node_name,val in dict_nodeValuePairs.itermitems():
    writeReg(node_name, val, ...)`

Which is len(dict_nodeValuePairs) transactions you have:

channelVals = ... # size (cuint * 128)  
writeReg(channel.block, channelVals)

So it seems the difficulty is passed to setting up the channelVals, e.g. the thing to be written so it is properly interrupted.

So this increases speed of configuration/register access because there's much less transactions to be performed? When using ipbus I clearly see how this is a considerable advantage. But in the case of rpc this seems like an order epsilon effect since the bulk of the time taken is the RPC message/response itself...?

[mexanick]

It is more than that especially when setting the channel settings. The settings themselves has to be stored as a block, otherwise you will have a lot of expensive seek-n-read operations to get the value (s) you want to write. So the block transactions looks like better organized way to set the things, which is faster, more reliable and easier to maintain. Also it won't break anything, just a certain rearrangement of the registers addresses is required.

[bdorney]

I see because for example in VFAT3 the "real" registers are CFG_CTRL_1 through CFG_CTRL_2 and our human readable registers are applying a mask to the bits sent to those addresses to only change a specific bit?

Same for VFAT2.