Skip to content

iOS VoiceOver issues in complex views: Flet generates empty accessibility nodes; Semantics workaround disables buttons #5417

Open
@Ambro86

Description

@Ambro86

Duplicate Check

Describe the bug

Describe the issue

I'm building an iOS app using Flet (compiled into .ipa, not web). I'm encountering serious accessibility issues when testing with VoiceOver on real iOS devices.

What happens:

  • When the UI is simple (e.g., 3–4 buttons), everything works fine.
  • But when the UI gets more complex (multiple buttons, columns, nested containers, text blocks), VoiceOver starts reading:
    • Empty groups
    • Or pauses and announces nothing before real content

This breaks the accessibility flow and confuses users relying on screen readers.


Semantics workaround (and its downside)

To remove the "empty" accessibility nodes, I tried wrapping parts of the UI in Semantics(container=True). This seems to help remove the extra elements...

BUT: Buttons inside a Semantics container become non-interactive with VoiceOver. They can’t be selected or activated.


Code sample

Code
### Example structure (simplified)

```python
# Title
ft.Text("AVVENTURA IN CORSO", style=ft.TextThemeStyle.HEADLINE_MEDIUM)

# Complex layout with multiple buttons
controls = [
    ft.Text("Welcome!"),
    ft.ElevatedButton(text="Start", on_click=...),
    ft.ElevatedButton(text="Shop", on_click=...),
    ft.ElevatedButton(text="Inventory", on_click=...),
    ft.ElevatedButton(text="Boss Fight", on_click=...),
    # etc.
]

# Optional workaround using Semantics
ft.Semantics(
    container=True,
    content=ft.Column(controls=controls)
)

### Full example with my attempt to mitigate this using semantics, but VoiceOver can't activate buttons
```python
    def crea_vista_gioco(self):
        """Crea la vista principale di gioco"""
        
        # Titolo con Semantics senza label
        titolo = ft.Semantics(
            container=True,
            content=ft.Text(
                "AVVENTURA IN CORSO", 
                size=24, 
                weight=ft.FontWeight.BOLD, 
                text_align=ft.TextAlign.CENTER,
                color=ft.Colors.AMBER_400,
                style=ft.TextThemeStyle.HEADLINE_MEDIUM
            )
        )
        
        # Ottieni i valori attuali dalle variabili globali se esistono
        valore_storia = "🎮 Benvenuto nell'Avventura Incrementale!\n Compagni gatti con abilità speciali\n Raccogli risorse e costruisci\n Combatti mostri e sali di livello\n🍽️ Gestisci cibo e acqua per energia\n🎵 Audio immersivo e feedback aptico\n\nPremi 'Inizia Avventura' per cominciare!"
        if hasattr(self, 'area_storia') and self.area_storia and hasattr(self.area_storia, 'value'):
            valore_storia = self.area_storia.value
        
        valore_stats = f" Statistiche Giocatore:\n Livello {self.livello}{self.vita}/{self.vita_massima} HP •  {self.monete} monete\n Attacco: {self.calcola_attacco_totale()} •  Difesa: {self.calcola_difesa_totale()}\n EXP: {self.esperienza}/{self.esperienza_necessaria}"
        if hasattr(self, 'area_stats') and self.area_stats and hasattr(self.area_stats, 'value'):
            valore_stats = self.area_stats.value
        
        # Area storia come gruppo semantico
        area_storia_locale = ft.Semantics(
            container=True,
            content=ft.Container(
                content=ft.Text(
                    valore_storia,
                    size=14,
                    color=ft.Colors.AMBER_100
                ),
                bgcolor=ft.Colors.DEEP_PURPLE_900,
                border_radius=5,
                padding=10,
                expand=True
            )
        )

        # Area statistiche come gruppo semantico con TextField
        area_stats_textfield = ft.TextField(
            value=valore_stats,
            multiline=True,
            read_only=True,
            min_lines=4,
            max_lines=6,
            text_size=14,
            color=ft.Colors.CYAN_100,
            border_color=ft.Colors.TRANSPARENT,
            focused_border_color=ft.Colors.TRANSPARENT,
            bgcolor=ft.Colors.TRANSPARENT
        )
        
        area_stats_locale = ft.Semantics(
            container=True,
            content=ft.Container(
                content=area_stats_textfield,
                bgcolor=ft.Colors.BLUE_GREY_900,
                border_radius=5,
                padding=10
            )
        )
        
        # Aggiorna i riferimenti globali per mantenere la sincronizzazione
        self.area_storia = area_storia_locale
        self.area_stats = area_stats_textfield  # Punta al TextField per aggiornamenti
        
        # Pulsanti di gioco
        pulsanti_gioco = []
        
        # Elemento decorativo per evitare elemento vuoto
        elemento_decorativo = ft.Semantics(
            container=True,
            content=ft.Container(height=1, bgcolor=ft.Colors.TRANSPARENT)
        )
        pulsanti_gioco.append(elemento_decorativo)
        
        # Azioni incrementali senza Semantics
        azioni_incrementali = self.azioni_incrementali_possibili()
        for i, (testo, funzione, tooltip) in enumerate(azioni_incrementali):
            pulsante_incrementale = ft.ElevatedButton(
                text=testo,
                on_click=funzione,
                width=280,
                height=50,
                bgcolor=ft.Colors.GREEN_600,
                color=ft.Colors.WHITE,
                tooltip=tooltip,
                key=f"azione_{i}_{testo.replace(' ', '_')}"
            )
            pulsanti_gioco.append(pulsante_incrementale)
        
        # Pulsante cambio area
        if len(self.aree_sbloccate) > 1:
            pulsante_aree = ft.ElevatedButton(
                text="Cambia Area",
                on_click=lambda e: self.page.go("/aree"),
                width=280,
                height=50,
                bgcolor=ft.Colors.BLUE_600,
                color=ft.Colors.WHITE,
                tooltip="Scegli area da esplorare",
                key="btn_cambia_area"
            )
            pulsanti_gioco.append(pulsante_aree)
        
        # Pulsanti di navigazione
        pulsante_combattimento = ft.ElevatedButton(
            text="Combattimento",
            on_click=lambda e: self.page.go("/combattimento"),
            width=280,
            height=50,
            bgcolor=ft.Colors.RED_600,
            color=ft.Colors.WHITE,
            tooltip="Combatti contro i mostri"
        )
        
        pulsante_negozio = ft.ElevatedButton(
            text="Negozio",
            on_click=lambda e: self.page.go("/negozio"),
            width=280,
            height=50,
            bgcolor=ft.Colors.ORANGE_600,
            color=ft.Colors.WHITE,
            tooltip="Visita il negozio"
        )
        
        pulsante_gatti = ft.ElevatedButton(
            text="Gatti",
            on_click=lambda e: self.page.go("/gatti"),
            width=280,
            height=50,
            bgcolor=ft.Colors.PINK_600,
            color=ft.Colors.WHITE,
            tooltip="Gestisci i tuoi gatti"
        )
        
        pulsanti_gioco.extend([pulsante_combattimento, pulsante_negozio, pulsante_gatti])
        
        # Pulsante boss
        if (self.area_attuale in self.boss_aree and 
            self.boss_aree[self.area_attuale]["nome"] not in self.boss_sconfitti and
            self.progressione_area.get(self.area_attuale, 0) >= 100):
            pulsante_boss = ft.ElevatedButton(
                text="Combatti Boss dell'Area!",
                on_click=self.combatti_boss,
                width=280,
                height=50,
                bgcolor=ft.Colors.DEEP_PURPLE_600,
                color=ft.Colors.WHITE,
                tooltip=f"Affronta il boss: {self.boss_aree[self.area_attuale]['nome']}"
            )
            pulsanti_gioco.append(pulsante_boss)
        
        # Pulsante salva
        pulsante_salva = ft.ElevatedButton(
            text="Salva Partita",
            on_click=self.salva_gioco,
            width=280,
            height=50,
            bgcolor=ft.Colors.PURPLE_600,
            color=ft.Colors.WHITE,
            tooltip="Salva il tuo progresso"
        )
        pulsanti_gioco.append(pulsante_salva)
        
        # Colonna pulsanti senza Semantics per permettere accesso individuale
        lista_pulsanti = ft.Column(
            controls=pulsanti_gioco,
            horizontal_alignment=ft.CrossAxisAlignment.CENTER,
            spacing=15,
            scroll=ft.ScrollMode.AUTO
        )
        
        # Controlli di gioco senza label per preservare il contenuto naturale
        gioco_controls = ft.Column([
            area_storia_locale,
            area_stats_locale,
            lista_pulsanti
        ], spacing=10, tight=False)
        
        # Pulsante menu come gruppo semantico
        pulsante_menu = ft.Semantics(
            container=True,
            content=ft.ElevatedButton(
                text="Torna al Menu",
                on_click=lambda e: self.page.go("/"),
                width=200,
                height=50,
                bgcolor=ft.Colors.GREY_600,
                color=ft.Colors.WHITE,
                tooltip="Torna al menu principale"
            )
        )
        
        # Content principale senza height fisso per evitare contenuto vuoto
        content = ft.Column([
            titolo,
            ft.Container(
                content=gioco_controls,
                bgcolor=ft.Colors.GREY_800,
                border_radius=10,
                padding=10,
                expand=True  # Usa expand invece di height
            ),
            pulsante_menu
        ], scroll=ft.ScrollMode.AUTO, spacing=30, expand=True)
        
        return ft.View(
            "/gioco",
            controls=[
                # Semantics senza label per evitare elemento vuoto ma preservare contenuto
                ft.Semantics(
                    container=True,
                    content=ft.Container(
                        content=content,
                        bgcolor=ft.Colors.GREY_900,
                        padding=20,
                        expand=True
                    )
                )
            ],
            bgcolor=ft.Colors.GREY_900
        )

---

To reproduce

  1. Build a moderately complex layout with:
    • Title text
    • A few text containers
    • 6–8 buttons (in a column or view)
  2. Compile the app to .ipa, install on iOS device
  3. Enable VoiceOver
  4. Navigate through the UI:
    • VoiceOver announces "empty group" or pauses where no content should be
  5. Wrap content in Semantics(container=True):
    • Empty elements may be gone
    • BUT VoiceOver can no longer activate buttons inside the Semantics block

Expected behavior

  • Complex views with multiple buttons and nested containers should not produce empty accessibility nodes by default.
  • VoiceOver should read only meaningful content, without announcing “empty group” or pausing unexpectedly.
  • Wrapping UI sections in Semantics(container=True) should help improve accessibility — but must not block interaction with child buttons.
  • Ideally, Flet should offer a way to structure accessible content for screen readers without breaking functionality.

Screenshots / Videos

Captures

[Upload media here]

Operating System

macOS

Operating system details

Flet version*: 0.28.3 - Platform: iOS (compiled to .ipa) - Device: iPhone (various models tested) - iOS version: 18.5 - Screen reader: VoiceOver ---

Flet version

0.28.3

Regression

No, it isn't

Suggestions

No response

Logs

Logs
[Paste your logs here]

Additional details

This only happens in complex views — basic UIs with a few buttons work fine. But when the layout includes nested containers, multiple controls, or grouped elements, VoiceOver starts announcing "empty group" or becomes inconsistent.

Trying to fix it using Semantics(container=True) helps reduce the noise, but then buttons inside become non-selectable.

If there's any temporary workaround or best practice to keep both accessibility and interactivity working, it would be very helpful in the meantime 🙏

I’m happy to help debug, share test builds, or provide screen recordings if needed.

Thanks again for the great work on Flet — this is the last roadblock before we can offer full accessibility on iOS.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingplatform: iosSpecific related to iOS

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions