Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SERP API Integration? #84

Open
hartator opened this issue Nov 30, 2017 · 2 comments
Open

SERP API Integration? #84

hartator opened this issue Nov 30, 2017 · 2 comments

Comments

@hartator
Copy link

Do you know if we can integrate with another backend like ours SerpAPI to get the Google Results?

We have solved the issues related to proxies and captchas, that will be great to integrate seemly with tools like SERPS. I can work on a fork if you want.

@gsouf
Copy link
Member

gsouf commented Nov 30, 2017

Hi @hartator

Thank for your interest in the library. It's for sure possible to integrate with your backend, there are several ways to do that. As far as I can imagine for now:

  • deeply by providing a custom implementation for google url interface
  • in surface by providing a client that fetches data from your urls and uses the internal parser for the output

We have to take care about how the library will behave because of the urls being different from original google urls. Do your serpApi provide the final google url that was fetched? That might solve some problems.

I think it's better to do this within another package (let's say search-engine-google-serpAPI) that I would be happy to host within the organization.

@hartator
Copy link
Author

hartator commented Dec 1, 2017

Hi @gsouf!

Thank for your interest in the library. It's for sure possible to integrate with your backend, there are several ways to do that. As far as I can imagine for now:

  • deeply by providing a custom implementation for google url interface
  • in surface by providing a client that fetches data from your urls and uses the internal parser for the output

Thank you for the awesome work you have been putting in this. I guess whatever is simpler. I suppose we can use our JSON output to hash and be fine, or directly the raw html should be exactly the same as Google's.

We have to take care about how the library will behave because of the urls being different from original google urls. Do your serpApi provide the final google url that was fetched? That might solve some problems.

Not right now as we provide only raw html at the moment, but we can add this in the JSON output. We've tried to do a 1:1 mapping of each request params, so params already look similar. Does it help? For example: https://serpapi.com/search?q=Coffee&hl=fr&gl=fr&num=100 -> https://www.google.com/search?q=Coffee&oq=Coffee&hl=fr&gl=fr&num=100&sourceid=chrome&ie=UTF-8

I think it's better to do this within another package (let's say search-engine-google-serpAPI) that I would be happy to host within the organization.

Sure, whatever is the most convenient for you guys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants