Skip to content

Add Files API endpoints#266

Merged
memoryz merged 9 commits into
Azure:masterfrom
Flavinou:flavinou/files
Jul 10, 2025
Merged

Add Files API endpoints#266
memoryz merged 9 commits into
Azure:masterfrom
Flavinou:flavinou/files

Conversation

@Flavinou
Copy link
Copy Markdown
Contributor

Added Files API endpoints based on documentation (https://docs.databricks.com/api/azure/workspace/files)

Tests

  • Added serialization/ deserialization unit tests
  • Added sample code + tested in real workspace environment

Resolves #235

@Flavinou
Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree company="Ubisoft"

@memoryz
Copy link
Copy Markdown
Collaborator

memoryz commented May 17, 2025

Hi @Flavinou , appreciate the PR. However, could you please remove all the changes that include only whitespace and format changes? Your PR has 215 changed files, and it's super time consuming for me to walk through them one by one.

@Flavinou
Copy link
Copy Markdown
Contributor Author

Sure ! Sorry about that, I blindly followed the contribution guidelines when I should have done it in another PR.

@Flavinou Flavinou force-pushed the flavinou/files branch 2 times, most recently from 967c844 to d7f0815 Compare May 18, 2025 15:58
@memoryz memoryz requested a review from Copilot July 5, 2025 18:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Adds support for Databricks Files API by introducing client-side models, interface, implementation, tests, and sample usage

  • Introduce DirectoryEntry and DirectoriesList models to represent file system entries and pagination
  • Define IFilesApi and implement it in FilesApiClient, extending ApiClient with HEAD support and wiring into DatabricksClient
  • Add comprehensive unit tests and sample program scenarios for all file and directory operations

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
Models/DirectoryEntry.cs New record for file/directory metadata
Models/DirectoriesList.cs New paginated directory listing model
IFilesApi.cs Interface defining Files API methods
FilesApiClient.cs Implements file and directory operations
ApiClient.cs Refactored to add HEAD support (HttpHead) and shared response helper
DatabricksClient.cs Exposes Files API on the root client and disposes it
FilesApiClientTest.cs Unit tests covering directory and file operations
SampleProgram.cs / SampleProgram.Files.cs Sample scenarios demonstrating Files API usage
Comments suppressed due to low confidence (2)

csharp/Microsoft.Azure.Databricks.Client/FilesApiClient.cs:31

  • Adding tests for ListDirectoryContents with non-default pageSize and pageToken would catch URL-building bugs and ensure pagination logic is correct.
        if (pageSize != null)

csharp/Microsoft.Azure.Databricks.Client/Models/DirectoryEntry.cs:30

  • The API returns last_modified as milliseconds since Unix epoch. Ensure you have a custom JSON converter registered (e.g. a JsonConverter for epoch milliseconds) otherwise deserialization to DateTimeOffset will fail or produce incorrect values.
    public DateTimeOffset? LastModified { get; set; }

Comment thread csharp/Microsoft.Azure.Databricks.Client/FilesApiClient.cs
Comment thread csharp/Microsoft.Azure.Databricks.Client/FilesApiClient.cs Outdated
Comment thread csharp/Microsoft.Azure.Databricks.Client/FilesApiClient.cs Outdated
@memoryz
Copy link
Copy Markdown
Collaborator

memoryz commented Jul 6, 2025

@Flavinou I added a commit to your PR to add support for pageable list and also addressed some of the Copilot code review comments. Could you please take a look and kindly test the ListDirectoryContentsPageable and Download functions?

@Flavinou
Copy link
Copy Markdown
Contributor Author

Flavinou commented Jul 9, 2025

@memoryz Of course, thank you for the follow up I didn't know about the Azure built-in pageable features !
I will test it shortly

@Flavinou
Copy link
Copy Markdown
Contributor Author

Flavinou commented Jul 9, 2025

Tested with a real workspace, it looks fine to me - I edited the sample program to list the directory contents based on the user input, it's minor

@memoryz memoryz merged commit f7a082c into Azure:master Jul 10, 2025
2 checks passed
@memoryz memoryz added this to the 2.9 milestone Jul 10, 2025
@Flavinou Flavinou deleted the flavinou/files branch July 10, 2025 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Microsoft.Azure.Databricks.Client library on NuGet doesn't expose '/fs/files' API

3 participants