Enhanced Offline Support with Multi-Layer Caching in OWASP BLT

[cicada0007]

User description

🚀 Offline-First Solution for OWASP BLT 🌍✨

We successfully implemented a robust offline-first architecture for the OWASP BLT application, ensuring seamless functionality even in network disruptions. This includes:

🔹 Server-Side API Caching – Middleware (NetworkStatusMiddleware) to track network status, compatible with Django 5.1's async handling, and adds X-Network-Status headers for client awareness.

🔹 API Response Caching & Fallback – A cache_api_response decorator that stores responses in memory & filesystem, auto-detects network availability, and serves cached data when offline. Includes ApiCacheFallbackMiddleware for resilient API handling.

🔹 Image Caching System – Prevents broken images offline by storing them locally and replacing API response URLs with cached versions.

🔹 Client-Side Enhancements – IndexedDB-based caching, a Service Worker for offline asset management, and frontend modifications to handle offline scenarios gracefully.

🔹 Resilient Architecture – Background network checks, clear offline status indicators, and auto-sync when connectivity is restored.

💡 Business Impact
Users can continue using the app seamlessly during network interruptions, ensuring a smooth experience with previously accessed content. This enhances reliability, especially for users with intermittent connectivity.

🔗 Check it out:

Screen.Recording.2025-03-13.145535.mp4

🎯 Closes #3357

---

PR Type

Enhancement, Bug fix, Tests

---

Description

• Introduced multi-layer caching for APIs and images to enhance offline support.

• Added middleware for network status tracking and fallback to cached data.

• Implemented client-side caching utilities for APIs and images using IndexedDB.

• Updated templates to dynamically load statistics and handle offline scenarios.

---

Changes walkthrough 📝

	Relevant files
Enhancement	10 files stats_dashboard.html Added dynamic stats loading with offline fallback.              +106/-101 header.html Added scripts for API and image caching.                                  +31/-0    views.py Integrated offline-first caching for API views.                    +220/-76 cache_utils.py Added utilities for API and image caching.                              +253/-0  cache.py Introduced decorators for API caching and offline resilience. +105/-0  cache_middleware.py Added middleware for API cache fallback.                                  +72/-0    middleware.py Introduced middleware for network status tracking.              +63/-0    image-cache.js Implemented client-side image caching utility.                      +268/-0  network-status.js Added network status detection and UI updates.                      +243/-0  api-cache.js Developed API caching utility for offline support.              +217/-0
Miscellaneous	2 files home.html Removed local mode panel and debug features.                          +3/-119  __init__.py Added package initialization for API functionality.            +1/-0
Configuration changes	1 files settings.py Configured caching settings and middleware integration.    +26/-0
Additional files	1 files app_logs.txt [link]

---

Need help?

Type /help how to ... in the comments thread for any questions about PR-Agent usage.

Check out the documentation for more information.

[github-actions]

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶 3357 - Partially compliant Compliant requirements: Cache API data and images to provide fallback content during network errors. Ensure users see some data instead of a blank screen when offline. Non-compliant requirements: [] Requires further human verification: []

⏱️ Estimated effort to review: 5 🔵🔵🔵🔵🔵

🧪 No relevant tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Possible Issue The fetchWithCache function in api-cache.js does not handle cases where the response is not JSON (e.g., HTML error pages). This could lead to runtime errors when parsing the response. const fetchWithCache = async (url, options = {}, expiration = DEFAULT_CACHE_EXPIRATION) => { const cacheKey = `${CACHE_PREFIX}${url}`; const cachedData = getFromCache(cacheKey); // Set default headers options.headers = options.headers || {}; // If we only want cached data (offline mode) if (options.offlineOnly) { if (cachedData) { return Promise.resolve(cachedData.data); } return Promise.reject(new Error('No cached data available for offline use')); } // Try to fetch fresh data try { const response = await fetch(url, options); if (!response.ok) { throw new Error(`API error: ${response.status}`); } const data = await response.json(); // Cache the successful response for future use setInCache(cacheKey, data, expiration); return data; } catch (error) { console.error('Network error:', error); // If we have cached data, return it if (cachedData) { // Add flag to indicate this is cached data return { ...cachedData.data, _fromCache: true, _cachedAt: cachedData.timestamp }; } // No cached data, rethrow the error throw error; Performance Concern The cacheVisibleImages function in image-cache.js processes all visible images on the page, which could lead to performance issues on pages with many images or frequent DOM mutations. function cacheVisibleImages() { // Find all image elements const imgElements = document.querySelectorAll('img'); // Find all elements with background images const elementsWithBackgroundImage = Array.from(document.querySelectorAll('*')) .filter(el => { const style = window.getComputedStyle(el); const backgroundImage = style.backgroundImage; return backgroundImage && backgroundImage !== 'none' && backgroundImage.includes('url('); }); // Extract image URLs from img elements const imgUrls = Array.from(imgElements) .map(img => img.src) .filter(src => src); // Filter out empty URLs // Extract image URLs from background images const backgroundImgUrls = elementsWithBackgroundImage .map(el => { const style = window.getComputedStyle(el); const backgroundImage = style.backgroundImage; // Extract URL from the url('...') pattern const urlMatch = /url\(['"]?([^'"()]+)['"]?\)/g.exec(backgroundImage); return urlMatch ? urlMatch[1] : null; }) .filter(src => src); // Filter out null values // Combine all URLs and remove duplicates const allImageUrls = [...new Set([...imgUrls, ...backgroundImgUrls])]; // Cache each image allImageUrls.forEach(url => { cacheImage(url); }); Scalability Issue The NetworkStatusMiddleware checks network status every 60 seconds for all requests, which might introduce unnecessary overhead in high-traffic environments. import time from django.utils.deprecation import MiddlewareMixin from website.cache_utils import check_network_status, is_network_available import asyncio class NetworkStatusMiddleware: """ Middleware to track network status and inject appropriate headers. This middleware will: 1. Periodically check network connectivity 2. Add response headers to indicate network status 3. Trigger offline mode behavior when network is unavailable """ # Set to 'both' to indicate this middleware supports both sync and async requests async_mode = 'both' def __init__(self, get_response): self.get_response = get_response self.last_check_time = 0 self.check_interval = 60 # Check network status every 60 seconds def __call__(self, request): """Handle synchronous requests""" # Check network status periodically self._check_network_if_needed() # Add a flag to the request indicating if we're offline request.is_offline = not is_network_available() # Get response from the next middleware/view response = self.get_response(request) # Add a header to indicate network status if hasattr(response, 'headers'): response.headers['X-Network-Status'] = 'offline' if not is_network_available() else 'online' return response async def __acall__(self, request): """Handle asynchronous requests""" # Check network status periodically self._check_network_if_needed() # Add a flag to the request indicating if we're offline request.is_offline = not is_network_available() # Get response from the next middleware/view response = await self.get_response(request) # Add a header to indicate network status if hasattr(response, 'headers'): response.headers['X-Network-Status'] = 'offline' if not is_network_available() else 'online' return response def _check_network_if_needed(self): """Check network status if enough time has passed since last check""" current_time = time.time() if current_time - self.last_check_time > self.check_interval: check_network_status() self.last_check_time = current_time

[github-actions]

PR Code Suggestions ✨

[DonnieBLT]

This would be for the flutter app.

[cicada0007]

This would be for the flutter app.

Ok sir