Update: London Borough of Merton (#3317)

* reworked to add garden waste test case * merton yaml added * update_docu_links * Update names for README * update test case names * yaml instructions updated
mampfes · Dec 29, 2024 · 06699b5 · 06699b5
1 parent 2c59855
commit 06699b5
Show file tree

Hide file tree

Showing 10 changed files with 211 additions and 41 deletions.
diff --git a/README.md b/README.md
@@ -2128,7 +2128,8 @@ If your service provider is not listed, feel free to open a [source request issu
 - [London Borough of Harrow](/doc/source/harrow_gov_uk.md) / harrow.gov.uk
 - [London Borough of Hounslow](/doc/source/hounslow_gov_uk.md) / hounslow.gov.uk
 - [London Borough of Lewisham](/doc/source/lewisham_gov_uk.md) / lewisham.gov.uk
-- [London Borough of Merton](/doc/source/merton_gov_uk.md) / merton.gov.uk
+- [London Borough of Merton (New)](/doc/ics/merton_gov_uk.md) / merton.gov.uk
+- [London Borough of Merton (Old)](/doc/source/merton_gov_uk.md) / merton.gov.uk
 - [London Borough of Newham](/doc/source/newham_gov_uk.md) / newham.gov.uk
 - [Luton](/doc/source/apps_imactivate_com.md) / luton.gov.uk
 - [Maidstone Borough Council](/doc/source/maidstone_gov_uk.md) / maidstone.gov.uk

diff --git a/custom_components/waste_collection_schedule/sources.json b/custom_components/waste_collection_schedule/sources.json
@@ -13507,7 +13507,13 @@
       "id": "lewisham_gov_uk"
     },
     {
-      "title": "London Borough of Merton",
+      "title": "London Borough of Merton (New)",
+      "module": "ics",
+      "default_params": {},
+      "id": "ics_merton_gov_uk"
+    },
+    {
+      "title": "London Borough of Merton (Old)",
       "module": "merton_gov_uk",
       "default_params": {},
       "id": "merton_gov_uk"

diff --git a/custom_components/waste_collection_schedule/translations/de.json b/custom_components/waste_collection_schedule/translations/de.json
@@ -14835,6 +14835,48 @@
           "uprn": "Eine einfache Möglichkeit, Ihre Unique Property Reference Number (UPRN) zu finden, besteht darin, auf https://www.findmyaddress.co.uk/ zu gehen und Ihre Adressdaten einzugeben."
         }
       },
+      "args_ics_merton_gov_uk": {
+        "title": "Quelle konfigurieren",
+        "description": "Konfiguriere deinen Service Provider. \n\n- Go to https://fixmystreet.merton.gov.uk/waste and enter your post code, and on the following page select your address.\n- In the _Download your collection schedule_ panel, click `Add to your calendar`.\n- Click the `Copy` button in _Step 1: copy the link to the calendar_.\n- Use the copied link as the `url` parameter.\n\n\nMehr details: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/ics/merton_gov_uk.md",
+        "data": {
+          "calendar_title": "Kalender Titel",
+          "file": "Datei",
+          "headers": "Headers",
+          "method": "Methode",
+          "offset": "Offset",
+          "params": "Parameter",
+          "regex": "Regulärer Ausdruck",
+          "split_at": "Trennen bei",
+          "title_template": "Titelvorlage",
+          "url": "URL",
+          "verify_ssl": "SSL-Verifizierung aktivieren",
+          "version": "Version",
+          "year_field": "Jahresfeld"
+        },
+        "data_description": {
+          "calendar_title": "Ein lesbarerer oder benutzerfreundlicherer Name für den Müllkalender. Wenn nichts angegeben wird, wird der Name der Quelle verwendet."
+        }
+      },
+      "reconfigure_ics_merton_gov_uk": {
+        "title": "Quelle Neu Konfigurieren",
+        "description": "Konfiguriere deinen Service Provider. \n\n- Go to https://fixmystreet.merton.gov.uk/waste and enter your post code, and on the following page select your address.\n- In the _Download your collection schedule_ panel, click `Add to your calendar`.\n- Click the `Copy` button in _Step 1: copy the link to the calendar_.\n- Use the copied link as the `url` parameter.\n\n\nMehr details: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/ics/merton_gov_uk.md",
+        "data": {
+          "calendar_title": "Kalender Titel",
+          "file": "Datei",
+          "headers": "Headers",
+          "method": "Methode",
+          "offset": "Offset",
+          "params": "Parameter",
+          "regex": "Regulärer Ausdruck",
+          "split_at": "Trennen bei",
+          "title_template": "Titelvorlage",
+          "url": "URL",
+          "verify_ssl": "SSL-Verifizierung aktivieren",
+          "version": "Version",
+          "year_field": "Jahresfeld"
+        },
+        "data_description": {}
+      },
       "args_merton_gov_uk": {
         "title": "Quelle konfigurieren",
         "description": "Konfiguriere deinen Service Provider. Mehr details: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/source/merton_gov_uk.md",

diff --git a/custom_components/waste_collection_schedule/translations/en.json b/custom_components/waste_collection_schedule/translations/en.json
@@ -14865,6 +14865,48 @@
           "uprn": "An easy way to discover your Unique Property Reference Number (UPRN) is by going to https://www.findmyaddress.co.uk/ and entering in your address details."
         }
       },
+      "args_ics_merton_gov_uk": {
+        "title": "Configure Source",
+        "description": "Configure your service provider. \n\n- Go to https://fixmystreet.merton.gov.uk/waste and enter your post code, and on the following page select your address.\n- In the _Download your collection schedule_ panel, click `Add to your calendar`.\n- Click the `Copy` button in _Step 1: copy the link to the calendar_.\n- Use the copied link as the `url` parameter.\n\n\nMore details: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/ics/merton_gov_uk.md.",
+        "data": {
+          "calendar_title": "Calendar Title",
+          "file": "File",
+          "headers": "Headers",
+          "method": "Method",
+          "offset": "Offset",
+          "params": "Parameters",
+          "regex": "Regex",
+          "split_at": "Split At",
+          "title_template": "Title Template",
+          "url": "Url",
+          "verify_ssl": "Verify Ssl",
+          "version": "Version",
+          "year_field": "Year Field"
+        },
+        "data_description": {
+          "calendar_title": "A more readable, or user-friendly, name for the waste calendar. If nothing is provided, the name returned by the source will be used."
+        }
+      },
+      "reconfigure_ics_merton_gov_uk": {
+        "title": "Reconfigure Source",
+        "description": "Configure your service provider. \n\n- Go to https://fixmystreet.merton.gov.uk/waste and enter your post code, and on the following page select your address.\n- In the _Download your collection schedule_ panel, click `Add to your calendar`.\n- Click the `Copy` button in _Step 1: copy the link to the calendar_.\n- Use the copied link as the `url` parameter.\n\n\nMore details: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/ics/merton_gov_uk.md.",
+        "data": {
+          "calendar_title": "Calendar Title",
+          "file": "File",
+          "headers": "Headers",
+          "method": "Method",
+          "offset": "Offset",
+          "params": "Parameters",
+          "regex": "Regex",
+          "split_at": "Split At",
+          "title_template": "Title Template",
+          "url": "Url",
+          "verify_ssl": "Verify Ssl",
+          "version": "Version",
+          "year_field": "Year Field"
+        },
+        "data_description": {}
+      },
       "args_merton_gov_uk": {
         "title": "Configure Source",
         "description": "Configure your service provider. More details: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/source/merton_gov_uk.md.",

diff --git a/custom_components/waste_collection_schedule/translations/it.json b/custom_components/waste_collection_schedule/translations/it.json
@@ -14796,6 +14796,48 @@
           "uprn": "Un modo facile per scoprire il tuo Numero di Riferimento Proprietà Unica (UPRN) è andare su https://www.findmyaddress.co.uk/ e inserire i dettagli del tuo indirizzo."
         }
       },
+      "args_ics_merton_gov_uk": {
+        "title": "Configurazione Sorgente",
+        "description": "Compila i campi per ottenere le informazioni sul tuo servizio di raccolta. \n\n- Go to https://fixmystreet.merton.gov.uk/waste and enter your post code, and on the following page select your address.\n- In the _Download your collection schedule_ panel, click `Add to your calendar`.\n- Click the `Copy` button in _Step 1: copy the link to the calendar_.\n- Use the copied link as the `url` parameter.\n\n\nMaggiori informazioni: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/ics/merton_gov_uk.md.",
+        "data": {
+          "calendar_title": "Nome Calendario",
+          "file": "Nome File",
+          "headers": "Headers",
+          "method": "Metodo",
+          "offset": "Offset",
+          "params": "Parametri",
+          "regex": "Regex",
+          "split_at": "Split At",
+          "title_template": "Title Template",
+          "url": "Url",
+          "verify_ssl": "Verifica SSL",
+          "version": "Versione",
+          "year_field": "Year Field"
+        },
+        "data_description": {
+          "calendar_title": "Puoi cambiare il nome del calendario della raccolta dei rifiuti, altrimenti di default verra' utilizzato il nome del tuo fornitore di servizi."
+        }
+      },
+      "reconfigure_ics_merton_gov_uk": {
+        "title": "Riconfigurazione Sorgente",
+        "description": "Compila i campi per ottenere le informazioni sul tuo servizio di raccolta. \n\n- Go to https://fixmystreet.merton.gov.uk/waste and enter your post code, and on the following page select your address.\n- In the _Download your collection schedule_ panel, click `Add to your calendar`.\n- Click the `Copy` button in _Step 1: copy the link to the calendar_.\n- Use the copied link as the `url` parameter.\n\n\nPer maggiori informazioni: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/ics/merton_gov_uk.md.",
+        "data": {
+          "calendar_title": "Nome Calendario",
+          "file": "Nome File",
+          "headers": "Headers",
+          "method": "Metodo",
+          "offset": "Offset",
+          "params": "Parametri",
+          "regex": "Regex",
+          "split_at": "Split At",
+          "title_template": "Title Template",
+          "url": "Url",
+          "verify_ssl": "Verifica SSL",
+          "version": "Versione",
+          "year_field": "Year Field"
+        },
+        "data_description": {}
+      },
       "args_merton_gov_uk": {
         "title": "Configurazione Sorgente",
         "description": "Compila i campi per ottenere le informazioni sul tuo servizio di raccolta. Maggiori informazioni: https://github.com/mampfes/hacs_waste_collection_schedule/blob/master/doc/source/merton_gov_uk.md.",

diff --git a/...om_components/waste_collection_schedule/waste_collection_schedule/source/merton_gov_uk.py b/...om_components/waste_collection_schedule/waste_collection_schedule/source/merton_gov_uk.py
@@ -2,29 +2,33 @@
 
 import requests
 from bs4 import BeautifulSoup
-from waste_collection_schedule import Collection
-
-TITLE = "London Borough of Merton"
+from waste_collection_schedule import Collection  # type: ignore[attr-defined]
 
+TITLE = "London Borough of Merton (Old)"
 DESCRIPTION = "Source for www.merton.gov.uk services for London Borough of Merton, UK"
-
 URL = "https://www.merton.gov.uk/"
-
 TEST_CASES = {
-    "test 1": {"property": "25884617"},
-    "test 2": {"property": "25861170"},
+    "Without Garden Waste": {"property": "25929128"},
+    "With Garden Waste": {"property": "25937841"},
 }
-
 API_URL = "https://myneighbourhood.merton.gov.uk/Wasteservices/WasteServices.aspx"
-
-
+WASTE_CLASSES: list = [
+    "food-caddy",
+    "papercard-wheelie",
+    "plastics-boxes",
+    "rubbish-wheelie",
+    "textiles",
+    "batteries",
+    "garden",
+]
 ICON_MAP = {
     "Food waste": "mdi:food",
     "Paper and card": "mdi:newspaper",
     "Plastics, glass, cans and cartons": "mdi:glass-fragile",
     "Rubbish": "mdi:trash-can",
     "Textiles": "mdi:hanger",
     "Household batteries": "mdi:battery",
+    "Garden waste": "mdi:leaf",
 }
 
 
@@ -33,42 +37,38 @@ def __init__(self, property: str):
         self._property = property
 
     def fetch(self):
-        entries = []
         session = requests.Session()
+
         params = {"ID": self._property}
         r = session.get(API_URL, params=params)
         r.raise_for_status()
         soup = BeautifulSoup(r.text, features="html.parser")
-        soup.prettify()
 
-        # Search for the specific bin in the table using BS4
-        collections = soup.find("table", class_=("collectiondays")).find_all(
-            "tr",
-            class_=(
-                "food-caddy",
-                "papercard-wheelie",
-                "plastics-boxes",
-                "rubbish-wheelie",
-                "textiles",
-                "batteries",
-            ),
-        )
+        # Search for table containing bin schedule info
+        collections = soup.find("table", {"class": "collectiondays"})
         if not collections:
-            raise Exception("No collections found for given UPRN")
-        # Loop the collections
-        for collection in collections:
-            # Get all the cells
-            cells = collection.find_all("td")
+            raise Exception("No collections found for given property id")
+
+        entries = []
+        # iterate through waste types finding pick-up dates
+        for item in WASTE_CLASSES:
+            trs = collections.find("tr", {"class": item})
+            cells = trs.find_all("td")
             # First cell is the bin_type
             title = cells[0].get_text().strip()
-            # Date is on the second cell, second paragraph, wrapped in p
-            collectionDate = cells[1].select("p > b")[2].get_text(strip=True)
-            # Add data to the main JSON Wrapper
-            entries.append(
-                Collection(
-                    date=datetime.strptime(collectionDate, "%A %d %B %Y").date(),
-                    t=title,
-                    icon=ICON_MAP.get(title),
+            # Date is in the second cell, second paragraph, wrapped in p
+            # Not all properties have garden waste collections, so silently deal with the IndexError
+            try:
+                collectionDate = cells[1].select("p > b")[2].get_text(strip=True)
+            except IndexError:
+                pass
+            else:
+                entries.append(
+                    Collection(
+                        date=datetime.strptime(collectionDate, "%A %d %B %Y").date(),
+                        t=title,
+                        icon=ICON_MAP.get(title),
+                    )
                 )
-            )
+
         return entries
diff --git a/doc/ics/merton_gov_uk.md b/doc/ics/merton_gov_uk.md
@@ -0,0 +1,23 @@
+# London Borough of Merton (New)
+
+London Borough of Merton (New) is supported by the generic [ICS](/doc/source/ics.md) source. For all available configuration options, please refer to the source description.
+
+
+## How to get the configuration arguments
+
+- Go to <https://fixmystreet.merton.gov.uk/waste> and enter your post code, and on the following page select your address.
+- In the _Download your collection schedule_ panel, click `Add to your calendar`.
+- Click the `Copy` button in _Step 1: copy the link to the calendar_.
+- Use the copied link as the `url` parameter.
+
+## Examples
+
+### Address with Garden Waste subscription
+
+```yaml
+waste_collection_schedule:
+  sources:
+    - name: ics
+      args:
+        url: https://fixmystreet.merton.gov.uk/waste/4273584/calendar.ics
+```
diff --git a/doc/ics/yaml/merton_gov_uk.yaml b/doc/ics/yaml/merton_gov_uk.yaml
@@ -0,0 +1,13 @@
+---
+title: London Borough of Merton (New)
+url: https://www.merton.gov.uk
+howto:
+  en: |
+    - Go to <https://fixmystreet.merton.gov.uk/waste> and enter your post code, and on the following page select your address.
+    - In the _Download your collection schedule_ panel, click `Add to your calendar`.
+    - Click the `Copy` button in _Step 1: copy the link to the calendar_.
+    - Use the copied link as the `url` parameter.
+
+test_cases:
+  Address with Garden Waste subscription:
+    url: https://fixmystreet.merton.gov.uk/waste/4273584/calendar.ics
diff --git a/doc/source/ics.md b/doc/source/ics.md
@@ -311,6 +311,7 @@ This source has been successfully tested with the following service providers:
 - [Caerphilly, Wales](/doc/ics/recollect.md) / caerphilly.gov.uk
 - [Falkirk](/doc/ics/falkirk_gov_uk.md) / falkirk.gov.uk
 - [Gedling Borough Council (unofficial)](/doc/ics/gedling_gov_uk.md) / gbcbincalendars.co.uk
+- [London Borough of Merton (New)](/doc/ics/merton_gov_uk.md) / merton.gov.uk
 - [Middlesbrough Council](/doc/ics/recollect.md) / middlesbrough.gov.uk
 - [Westmorland & Furness Council, Barrow area](/doc/ics/barrowbc_gov_uk.md) / barrowbc.gov.uk
 - [Westmorland & Furness Council, South Lakeland area](/doc/ics/southlakeland_gov_uk.md) / southlakeland.gov.uk