Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with chromote package #159

Open
sale4cast opened this issue Jun 23, 2024 · 0 comments
Open

Problem with chromote package #159

sale4cast opened this issue Jun 23, 2024 · 0 comments

Comments

@sale4cast
Copy link

Dear community,

First of all, I would like to thanks everyone for your previous support. I want to extract "hotel prices" from google. From your previous suggestions, I can extract the first page prices perfectly. However, I want to extract also second page prices. Here I attach the code below:

library(shiny)
library(curl)
library(chromote)
library(pagedown)

ui <- fluidPage(
  textOutput("targetId"),
  tableOutput("result")
)

r <- Chromote$new()

server <- function(session, input, output) {
  driver <- r$new_session()
  output$targetId <- renderText(paste0("Target ID: ", driver$get_target_id()))
  p <- driver$Page$loadEventFired(wait_ = FALSE)
  driver$Page$navigate("https://google.com", wait_ = FALSE)
  
  p$then(function(value){
    googleSearchText <- "4 star hotel in barcelona"
    driver$Runtime$evaluate(paste0('document.querySelector("textarea").value = "', googleSearchText,'"'))
    
    {
      p2 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("input[aria-label=\'Google Search\']").click()', wait_ = FALSE)
    }
    
    p2
  })$then(function(value){
    driver$Runtime$evaluate('document.querySelector("div.R2w7Jd").click()')
    {
      p3 <- driver$Page$loadEventFired(wait_ = FALSE)
      driver$Runtime$evaluate('document.querySelector("div.JWXKNd").click()', wait_ = FALSE)
    }
    p3
  })$then(function(value){
    priceElement <- driver$Runtime$evaluate(
      'var elements = document.querySelectorAll(".K1smNd > c-wiz[jsrenderer=\'hAbFdb\'] .PwV1Ac");
                 var elementPrices = [];
                 elements.forEach(function(element) {
                   elementPrices.push(element.innerText);
                 });
                 elementPrices.join("@");'
    )
    driver$close()
    
    splittedPriceElements <- unlist(strsplit(priceElement$result$value, "@"))
    
    neighborHotelRoomPrices <- lapply(splittedPriceElements, function(aElement){
      ## Great Deal\n$80    Deal 5%\n$90
      roomPrice <- unlist(strsplit(aElement,"\n"))
      if(length(roomPrice) > 1){
        return(roomPrice[[2]])
      }
      return (aElement)
    })
    output$result <- renderTable(as.data.frame(unlist(neighborHotelRoomPrices)))
  })
}


shinyApp(ui, server)

Now, I want to extract second page prices as like the screenshot below:

Screenshot_368

Question: How can I extract second page prices in asynchronous way?

Note: Here I extract prices from a single page url and data updated in the particular results section only.

Best Regards,
SaleForecast

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant