fix(http): allow HEAD method in file-server

[krugerLucas]

Summary

Fixes inconsistency where serveFile() supported HEAD requests but serveDir() returned 405 Method Not Allowed.

Problem

• serveFile() correctly handles HEAD requests (returns headers without body)
• serveDir() rejects HEAD requests with 405 Method Not Allowed
• This creates inconsistent behavior and breaks HTTP compliance

Solution

• Changed line 658 in serveDir() to allow HEAD requests alongside GET
• HEAD requests now properly delegate to serveFile() which already has correct implementation
• Added comprehensive tests for both file and directory HEAD requests

Changes

• http/file_server.ts: Allow HEAD method in serveDir validation
• http/file_server_test.ts: Add tests for serveDirHeadRequest scenarios

Test Plan

• New tests pass: serveDirHeadRequest and serveDirHeadRequestForDirectoryWithIndex
• Existing tests still pass
• HEAD returns correct headers without body
• POST/PUT/DELETE still properly rejected with 405

Fixes #6750

[CLAassistant]

All committers have signed the CLA.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.14%. Comparing base (7a77939) to head (a6cb2fa).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #6752   +/-   ##
=======================================
  Coverage   94.14%   94.14%           
=======================================
  Files         585      585           
  Lines       42367    42367           
  Branches     6683     6683           
=======================================
  Hits        39888    39888           
  Misses       2430     2430           
  Partials       49       49           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[kt3k]

This doesn't change createServeDirResponse and request body is still returned when the file server returns directory listing (You can check assert(!res.body) fails with the url http://localhost/nested-subdirs/subdir/). We need to modify createServeDirResponse to return null body response when the request method is HEAD (in a similar way as serveFile)