Skip to content

optoe: Add CMIS Bank support for transceivers with >8 lanes #473

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ishidawataru
Copy link

@ishidawataru ishidawataru commented Apr 2, 2025

This patch adds CMIS Bank support to the 'optoe3' device class in order
to enable access to CMIS transceivers with more than 8 lanes (e.g., OSFP-XD, CPO OEs).

  • Bank support is enabled only for the 'optoe3' dev class.
  • The default bank size is set to 4, and can be modified via the 'bank_size' sysfs entry.
  • For 'optoe3', the 'write_max' value is updated to 2 to comply with CMIS requirements,
    which mandate that both bank and page values be updated in a single WRITE operation.

Updated the behavior as below after discussing with @prgeor offline

  • 'bank_size' sysfs entry is added to enable and configure the size of the bank.
  • By default, bank size is set to 0.
  • When enabling bank by setting a value greater than 1, the 'write_max' value is
    automatically updated to 2 to comply with CMIS requirements,
    which mandate that both bank and page values be updated in a single WRITE operation.
  • If the 'write_max' value is already greater than 2, this module keeps the value as is.

Only tested with the SONiC VM + i2c-stub.

admin@sonic:~$ cat /sys/class/i2c-dev/i2c-1/device/1-0050/bank_size
4
admin@sonic:~$ hexdump /sys/class/i2c-dev/i2c-1/device/1-0050/eeprom 
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
001e880
admin@sonic:~$ echo 1 | sudo tee /sys/class/i2c-dev/i2c-1/device/1-0050/bank_size
1
admin@sonic:~$ hexdump /sys/class/i2c-dev/i2c-1/device/1-0050/eeprom 
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0008080

Snip of dmesg. See bank:3 is accessed.

[140341.928384] optoe 1-0050: eeprom write 2@126 --> 2 (4329977748)
[140341.928386] optoe 1-0050: sff_r/w: off 122880, len 2176, chunk_start_offset 124928, chunk_offset 124928, chunk_len 128, pending_len 128
[140341.928388] optoe 1-0050: optoe_eeprom_update_client off 124928 bank:3 page:255 phy_offset:128, count:128, opcode:0
[140341.928391] optoe 1-0050: eeprom write 2@126 --> 2 (4329977748)
[140341.928393] optoe 1-0050: eeprom read 32@128 --> 32 (4329977748)
[140341.928395] optoe 1-0050: eeprom read 32@160 --> 32 (4329977748)
[140341.928397] optoe 1-0050: eeprom read 32@192 --> 32 (4329977748)
[140341.928399] optoe 1-0050: eeprom read 32@224 --> 32 (4329977748)
[140341.928401] optoe 1-0050: eeprom write 2@126 --> 2 (4329977748)
[140341.928413] optoe 1-0050: optoe_read_write: off 125056  len:4096, opcode:r

Not tested with real hardware yet.

@ishidawataru ishidawataru requested a review from a team as a code owner April 2, 2025 07:23
@mssonicbld
Copy link

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ishidawataru
Copy link
Author

@prgeor PTAL

@ishidawataru
Copy link
Author

The optoe linear address space is extended as below to support Bank without changing how the user space interact with optoe.

Screenshot 2025-04-03 at 9 58 47

Copy link
Contributor

@paulmenzel paulmenzel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great overall.

+ * 2 byte writes are acceptable for PE and Vout changes per
+ * Application Note AN-2071.
+ *
+ * use 2 bytes for CMIS devices since CMIS 5.3 requires write both BankSelect and PageSlect in one WRITE acess.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrap the line?

- The default bank size is set to 4, and can be modified via the 'bank_size' sysfs entry.
- For 'optoe3', the 'write_max' value is updated to 2 to comply with CMIS requirements,
which mandate that both bank and page values be updated in a single WRITE operation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It’d be great if you could add a paragraph how to verify/test this to the commit message too.

@mssonicbld
Copy link

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@mssonicbld
Copy link

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ishidawataru
Copy link
Author

@prgeor I updated the patch based on our discussion PTAL.

@prgeor
Copy link
Contributor

prgeor commented Apr 21, 2025

@prgeor I updated the patch based on our discussion PTAL.

@ishidawataru Ack. reviewing.

@prgeor
Copy link
Contributor

prgeor commented Apr 21, 2025

@ishidawataru can you define what do you mean by bank_size ?

@mssonicbld
Copy link

/azp run

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@ishidawataru
Copy link
Author

@prgeor

bank_size represents the number of CMIS transceiver banks that the optoe driver assumes.
This setting is necessary because optoe maps CMIS 3D addresses into a linear address space.
The maximum of the linear address space is determined based on the bank_size, and address validity is checked accordingly.

For example, if bank_size is 0 or 1, the linear address space ends at 0x8080.
If bank_size is 2, the address space extends up to 0xf880.

admin@sonic:~$ echo 1 | sudo tee /sys/class/i2c-dev/i2c-1/device/1-0050/bank_size                                                                                      
1
admin@sonic:~$ hexdump /sys/class/i2c-dev/i2c-1/device/1-0050/eeprom                                                                                       
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
0008080
admin@sonic:~$ echo 2 | sudo tee /sys/class/i2c-dev/i2c-1/device/1-0050/bank_size                                                                                      
2
admin@sonic:~$ hexdump /sys/class/i2c-dev/i2c-1/device/1-0050/eeprom                                                                                       
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
000f880

I also added a commit that explains this in the comment.

1b2fb60

/* fundamental unit of addressing for EEPROM */
#define OPTOE_PAGE_SIZE 128
+
+#define OPTOE_DEFAULT_BANK_SIZE 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ishidawataru the default bank is 0, which means modules that support 8 lanes, the bank size = 1?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ishidawataru This probably can be removed if we implement full range of linear address for all bank pages. See my comments below

Comment on lines +32 to +34
+#define OPTOE_MAX_SUPPORTED_BANK_SIZE 8
+#define OPTOE_NON_BANKED_PAGE_SIZE 16 /* page 00h-0Fh are not banked */
+#define OPTOE_BANKED_PAGE_SIZE 240 /* page 10h-FFh are banked */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ishidawataru I am bit confused with respect to naming these by _SIZE. From the code use, it looks like these should be _COUNT?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these constants are still necessary after the discussion below, I’ll rename them as you suggested.

/* 0x80 places the offset in the top half, offset is last 7 bits */
*offset = OPTOE_PAGE_SIZE + (*offset & 0x7f);
-
- return page; /* note also returning client and offset */
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ishidawataru The computation does not consider full range of linear address for all banks. Its OK to use all page address for bank 0 i.e Assume that page 0x10, 0x11, 0x12... 0xFF exist for Bank 0 as well.

So that the getaddr() can be implemented as follows:-

    def getaddr(self, bank=0, page, offset, page_size=128):
          return ((uint32_t)bank * PAGES_PER_BANK + page) * page_size + offset;

In your case you are not wasting the address space for OPTOE_NON_BANKED_PAGE_SIZE but its OK to loose those many address range to keep the use space implementation of getaddr() simple

Here is the simple decoding of the above linear address that optoe needs to do:-

static uint8_t optoe_translate_offset(struct optoe_data *optoe,
		loff_t *offset, struct i2c_client **client, uint8_t *bank)
{
           unsigned int page = 0;
           
          *bank = *offset / (PAGES_PER_BANK * PAGE_SIZE);
           page = ((*offset / PAGE_SIZE) % PAGES_PER_BANK) - 1;
          *offset = PAGE_SIZE + linear_address % PAGE_SIZE;

         return page;
}

Assumption here is the user application will provide only the valid pages and the module will return IO error if incorrect Page address is selected.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prgeor This is the linear address map of the current implementation.

Are you suggesting to change it like below?

Screenshot 2025-04-24 at 18 04 36

Returning I/O errors for the grayed-out area will prevent us from using cat on the EEPROM file as shown below.

admin@sonic:~$ hexdump /sys/class/i2c-dev/i2c-1/device/1-0050/eeprom 
0000000 0000 0000 0000 0000 0000 0000 0000 0000
*
001e880

I suggest keeping the current implementation or returning the value from bank 0, rather than returning I/O errors.

What do you think?

+ * For CMIS transceivers that support Banked Pages, access to these pages
+ * is also supported. To access the banked pages, set the number of banks
+ * to access via the `bank_size` sysfs entry.
+ * By default, `bank_size` is set to 0, which disables this feature.
Copy link
Contributor

@prgeor prgeor Apr 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ishidawataru We probably don't need this. see my comments above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants