-
-
Notifications
You must be signed in to change notification settings - Fork 757
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
File upload service #1052
Comments
@ranhsd thanks for your comments. It's working fine for me with Postman. I just can't figure out how to send files from the browser using socket-io as the transporter.
Any ideas about how it could be done? |
Hi @aessig Actually I didn't try it with Socket.IO |
Hi @aessig, |
Hi @sbsujith I think you don't need to use websockets for uploading files. For this specific operation you can use REST (this is what I did) FeathersJS gives you the ability to have 2 transports for your services (REST and Socket) so just use REST for this specific service |
Thanks @ranhsd for the response. I could access the file over socket-io in server at context.data.files instead of context.params.files |
Awesome @sbsujith |
for those who are looking this or facing the same issue. here is a solution u can use drop zone on socket.io or API to upload file to your database. you have to using hooks. `const dauria = require('dauria'); before: {
} after: { error: { on yourservice.hook.js then you are done for more detail check |
Hi,
I am building an app and want to use feathers server side only. I am planning to migrate my current server side implementation from another NodeJS framework (that I currently use) and use FeathersJS. The reason I'm doing it is because FetahersJS allows me to create multiple services that each one of them can use different database.
One of the features that I noticed that FeathersJS don't solve in the Framework is files upload. I also noticed that there are some questions about it on GitHub as well as on Stackoverflow.
After reading the questions, guides (like this one: https://github.com/feathersjs/docs/blob/master/guides/advanced/file-uploading.md) I came up with a solution and I really need you to share your thoughts about it.
Before I start describe the solution, here are some assumptions:
My solution goes like this:
The service
I started by generating a new Feathers service via the service CLI. the service name is files.
Next, I npm installed multer which allows me to upload one or multiple files and handle the multipart/form-data header
My files.service.js content looks like the following:
You can see that I use multer array because this solution allow users to upload multiple files. From the code above you can see that you can easily change the number of files via the FILES_SERVICE_MAX_ITEMS env variable (I am running feathers in docker + docker-compose locally and on Kuberentes remotely so it's very easy and straight forward to define env variable there)
You can also noticed that I didn't perform a lot of changes to the current service implementation and it's very basic.
Before create hook
I generated a before create hook again using the Feathers CLI and name it uploadFilesToGcs. This hook will get the file content and metadata and will use google cloud storage SDK to upload them to google cloud storage
This is the code inside the upload to gcs file:
In the code above you can see that I first intiialized the google cloud SDK with my project id and service account key (both values are saved in the env variables of my app), then I performed a very simple validation and check that I really have files to work with, then I created a promise array to upload all files to google cloud storage via the google cloud SDK. Finally, I pass the results to the hook context data.
The context data will contain an array of objects that will be stored in the database (in my case I use MongoDB but of course Feathers allows me to use any other SQL/noSQL database). The object that will be stored in the database will have the following structure:
name - is the name of the files that I generated to make sure it will be unique
originalName - is the original name of the file as sent by the client
contentType - is the mime type as sent from the client (with the help of multer)
bucket - the bucket name where the items are stored
provider - the provider who host it. In my case I use google
As you can see I don't store the item url here because I can calculate it later (in the after find and after get hooks). I also don't need to store the bucket and provider here but I decided to do so because this solution now allows me to use one service with multiple provider so from the client I can decide that I want to upload file to S3 and the only think that I will need to do is in the before create hook to check that payload and use upload to amazon S3 adapter instead of google cloud storage so this solution actually allows me to use multiple storage side by side.
After find + after get hooks
Each of the files must have a URL so users will be able to access it from within the app, browser etc.
Like I mentioned above I prefer to not store absolute urls in my database from various reasons like: to be storage agnostic and also I cannot count on the storage provider that they will not modify their storage URL in the future so that's why I decided to "calculate" the service URL in the application layer.
For that purpose I didn't generated any hook (like I did above) and go to the "quick and dirty" solution and write the code inside the files.hook.js file directly but you can definitely generate a new hook for that to create a cleaner solution.
Here is the content of files.hook.js file:
From the code above it's easy to understand that I build the file url manually by concatenate the bucket name and the file name to the storage URL. Also, I remove the bucket name and the provider name from the result because the user shouldn't care about it.
Don't forget that I can access to provider and bucket fields here and build the url according to the provider name so if it will be "aws" then I can simply use amazon S3 storage URL but because this solution is for google cloud storage only I don't even check for the provider name.
How to test it
To test this solution you first must have a running feathersJS app. Then you need to npm install multer and google cloud storage sdk and after your server is running use any REST client (I prefer to use Postman because it's nice and easy).
From the image above you can see that I pass a JWT token in the Authorization header but FeathersJS allows you to change it and expose it publicly.
Also please notice to the Content-Type header. Multer expect that it will be equal to multipart/form-data
This is how my request body looks like:
As you can see from the image above. I can upload 2 photos in one service call. Of course you can modify multer config and upload more than 2 but please notice that it is consume memory since photos are kept in memory during the request.
Finally, here is the response:
I use ** to mask the data :)
That's it. Please let me know what you think about this solution. I was thinking maybe we can also extract this solution to a different module something like feathers-storage or something else and this solution will define an interface that will be implemented by various providers (e.g. google, aws, azure, grid store and more)
Thanks in advance!
Ran.
The text was updated successfully, but these errors were encountered: