feat: Add content_layer property to items to address body, furniture and other roles

[cau-git]

Checklist:

• Documentation has been updated, if necessary.
• Examples have been added, if necessary.
• Tests have been added, if necessary.

[mergify]

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

🟢 Require two reviewer for test updates

Wonderful, this rule succeeded.

When test data is updated, we require two reviewers

• #approved-reviews-by >= 2

[dolfim-ibm]

One proposal to consider would also be:

furniture:
  children:
  - $ref: '#/texts/0'
  - $ref: '#/texts/1'
  label: unspecified
  name: _root_
  parent: null
  self_ref: '#/furniture'

body:
  children:
  - $ref: '#/furniture/0'
  - $ref: '#/texts/2'
  - $ref: '#/texts/3'
  - $ref: '#/furniture/1'
  label: unspecified
  name: _root_
  parent: null
  self_ref: '#/body'

This would require changes in how the DoclingDocument.add_text() works, and the document.iterate_items() would add an option to skip or not the #/furniture references.

[vagenas]

For reference, this can be related to #667.

[cau-git]

One proposal to consider would also be:

furniture:
  children:
  - $ref: '#/texts/0'
  - $ref: '#/texts/1'
  label: unspecified
  name: _root_
  parent: null
  self_ref: '#/furniture'

body:
  children:
  - $ref: '#/furniture/0'
  - $ref: '#/texts/2'
  - $ref: '#/texts/3'
  - $ref: '#/furniture/1'
  label: unspecified
  name: _root_
  parent: null
  self_ref: '#/body'

This would require changes in how the DoclingDocument.add_text() works, and the document.iterate_items() would add an option to skip or not the #/furniture references.

This would not work since there is no such thing as #/furniture/0 or #/furniture/1. It would only be the case if we change to something like furnitures as a field (similar to texts), which is a list. It would also make it unclear where to expect some content. E.g. a table in the furniture, should it be in tables or in furnitures (not both!)

[cau-git]

This is updated to use a new field in NodeItem called content_layer. It will be set as furniture if a page-header or page-footer is detected in the PDF pipeline.

See DS4SD/docling-core#148

[PeterStaar-IBM]

🚀