Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add transactions #146

Open
wants to merge 11 commits into
base: master
Choose a base branch
from

Conversation

jsgalarraga
Copy link

This PR introduces Firestore transactions to the package.

Why

Transactions are a powerful feature to have in the backend. This is an effort to have a more complete feature set for a dart backend.

It has been requested by users in #63 and #126.

Inspiration

The work here has been inspired by:

Implementation notes

  • The new Transaction class, does not extend the existing Reference even though it needs some of its methods because a Transaction does not have a predefined path. The _fullPath and _encodeMap methods have been adapted.
  • The mutations in a Transaction have to be exposed for the FirestoreGateway to have access to them and run the transaction commit. To safely do this, it is kept as an UnmodifiableListView to prevent the end user from modifying it.
  • When multiple transaction operations are run in parallel, some of them fail to keep the database consistent. When this happens, a GrpcError with StatusCode.aborted is thrown. In this PR, the error handling is implemented to retry the transaction up to 5 times by default (as the other firestore SDKs do).
  • The document attribute in FirestoreGateway had the route to the /documents path, which is needed for most common operations in the database. However, for the transactions, the raw database route is needed, therefore this has been updated to have both database pointing to the root and documentDatabase which adds the /documents path.

Examples

To ensure transactions are working and keeping the database consistent with all the write operations performed in parallel, the following test has been run: Increase a value multiple times both with and without transactions. As we can see, when running with transactions, the numbers are increased correctly 3 times, but the number is only increased by 1 when ran without transactions.

transactions-example.mov

With transactions

void main(List<String> args) async {
  Firestore.initialize('project-id');

  await Future.wait([
    increaseValueWithTransaction(Firestore.instance),
    increaseValueWithTransaction(Firestore.instance),
    increaseValueWithTransaction(Firestore.instance),
  ]);

  firestore.close();
}

Future<void> increaseValueWithTransaction(Firestore firestore) async {
  await firestore.runTransaction(
    (transaction) async {
      final doc = await transaction.get('testing/test');
      final value = doc.map['key'];
      transaction.update('testing/test', {'key': value + 1});
    },
  );
}

Without transactions

void main(List<String> args) async {
  Firestore.initialize('project-id');

  await Future.wait([
    increaseValue(Firestore.instance),
    increaseValue(Firestore.instance),
    increaseValue(Firestore.instance),
  ]);

  firestore.close();
}

Future<void> increaseValue(Firestore firestore) async {
  final doc = await firestore.document('testing/test').get();
  final value = doc.map['key'];
  await firestore.document('testing/test').set({'key': value + 1});
}

@evandrobubiak
Copy link

Any predictions for this merge? We are really looking forward to this feature

@cachapa
Copy link
Owner

cachapa commented Nov 6, 2024

Hi @evandrobubiak, sorry for allowing this to stale, I must've missed the original PR email.

I'll review this now.

Copy link
Owner

@cachapa cachapa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

I would just like to discuss the use of recursion - if you agree that it should be removed or have good arguments to keep it.

@@ -150,6 +160,49 @@ class FirestoreGateway {
.toList();
}

Future<T> runTransaction<T>(
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate recursiveness as much as anyone but here it feels a bit unwarranted, especially since it exposes attempt and maxAttemps in a "public" method (the class isn't supposed to be called directly but still).

Since you have the transaction itself being run in a private method _runTransaction, would it make sense to wrap the retries in a loop instead of having the function calling itself?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your arguments make a lot of sense. @jsgalarraga , could you review this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @cachapa for the review. It's been some months since I implemented the solution, so I don't remember why I didn't use a loop instead of recursion.
From a quick review at the code, I believe that the catching the aborted error in the loop and continue with it would be the hard part. However, I will think about it and try to refactor to use a loop or come up with strong reasons on why recursion might be a better solution.

Unfortunately I won't be able to get into this next week, but I will work on it as soon as I can.

PS. @evandrobubiak feel free to also contribute with code. I appreciate you reviving this PR, but I would kindly ask to avoid posting a message a couple hours after the review just to put pressure on getting the feature out.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jsgalarraga thank you for getting back to me, and once again sorry for allowing this to go unreviewed for so long.

I appreciate you trying out a loop-based solution but if you think that doesn't work out I'm happy accepting the patch as-is.

@cachapa
Copy link
Owner

cachapa commented Nov 6, 2024

Also, would you mind writing a few tests?
It doesn't need to be comprehensive, but at least covering the basics, essentially the samples you wrote for the documentation.

@jsgalarraga
Copy link
Author

Oh, sure! Will also work on the tests

@kartikey321
Copy link

kartikey321 commented Feb 18, 2025

Hi @jsgalarraga , I was trying to implement this add transactions pr in my project for running transactions but I am facing issues in it regarding permission denied.

The error that I am facing:
gRPC Error (code: 7, codeName: PERMISSION_DENIED, message: Missing or insufficient permissions., details: [], rawResponse: null, trailers: {x-debug-tracking-id: 17790388099227199145;o=1})

This error hits as soon as the beginTransaction request is hit

What I tried:

  1. I made sure that my request is authenticated by signing in by firebase auth in firedart and logging the token at the time of request
  2. I tried regenerating the rpc proto files, but there was no change in the result
  3. I tried the official cloud_firestore package for running transactions in my same firebase project and it worked fine so I don't think there is any issue with permissions in my firebase project.
  4. I tried running any other function like getting all documents in a collection but all that were working fine.
  5. I tried to research the error and found this stackoverflow
    https://stackoverflow.com/questions/57138055/firestore-begintransaction-rest-api-returns-permission-denied-when-creating-a-re
    It talks about how readwrite in beginTransaction requires an auth but my requests were already authenticated as I tested it in point 1 to log the idToken getter

The reason why I can't move forward with just cloud_firestore is that it doesn't support windows and I plan to give support for windows too.

It would be really helpful if you or @cachapa could help me figure out the issue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants